Image segmentation method, medical image segmentation system and computer terminal

Abstract

The present application discloses an image segmentation method, a medical image segmentation system and a computer terminal. The method includes: obtaining an image to be processed, wherein the image to be processed contains multiple organ images of a biological object; using a pre-trained segmentation model with different low-rank adapter sets inserted, respectively segmenting the image to be processed to obtain segmentation results corresponding to different low-rank adapter sets, wherein the low-rank adapters contained in different low-rank adapter sets are obtained by continuous learning of the pre-trained segmentation model with the initial adapter inserted using different training data sets; merging the segmentation results corresponding to the different low-rank adapter sets to obtain the target segmentation result of the image to be processed, wherein the target segmentation result is used to characterize the positions of multiple organ images in the image to be processed. The present application solves the technical problem of low accuracy and efficiency in segmenting organ images of biological objects.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and in particular, to an image segmentation method, a medical image segmentation system and a computer terminal. Background Art

[0002] Accurately segment and identify different organs or tissues in the organ image of the biological object to obtain images of different organs or tissues, or position information of different organs or tissues in the original image, that is, segment the organ image based on computer technology. By segmenting the organs in the organ image of the biological object, for example, in the medical field, it can help doctors improve the efficiency and accuracy of medical work, and can be applied to medical imaging research, tumor image research and other fields.

[0003] At present, when the organ image segmentation of biological objects is performed based on deep learning technology in the relevant technology, due to the high plasticity of the deep learning model itself, when new organ segmentation tasks and new training data sets need to be added, the current deep learning-based organ segmentation method will produce more serious forgetting, or need to add a large number of new learning parameters, resulting in too fast parameter growth, and thus unable to efficiently and lightly adapt and update the training model while retaining the original segmentation task, resulting in low accuracy and efficiency in the segmentation of organ images of biological objects.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present application provide an image segmentation method, a medical image segmentation system and a computer terminal to at least solve the technical problem of low accuracy and efficiency in segmenting organ images of biological objects.

[0006] According to one aspect of an embodiment of the present application, an image segmentation method is provided, the method comprising: obtaining an image to be processed, wherein the image to be processed includes multiple organ images of a biological object; using a pre-trained segmentation model into which different low-rank adapter sets are inserted, respectively segmenting the image to be processed to obtain segmentation results corresponding to different low-rank adapter sets, wherein the low-rank adapters included in the different low-rank adapter sets are obtained by continuously learning the pre-trained segmentation model into which an initial adapter is inserted using different training data sets; merging the segmentation results corresponding to the different low-rank adapter sets to obtain a target segmentation result of the image to be processed, wherein the target segmentation result is used to characterize the positions of the multiple organ images in the image to be processed.

[0007] According to another aspect of an embodiment of the present application, a method for segmenting a medical image is also provided, the method comprising: acquiring a medical image, wherein the medical image includes multiple organ images of a biological object; segmenting the medical image separately using a pre-trained segmentation model into which different low-rank adapter sets are inserted, to obtain segmentation results corresponding to different low-rank adapter sets, wherein the low-rank adapters included in the different low-rank adapter sets are obtained by continuously learning the pre-trained segmentation model into which the initial adapter is inserted using different training data sets; merging the segmentation results corresponding to the different low-rank adapter sets to obtain an organ image segmentation result of the medical image.

[0008] According to another aspect of an embodiment of the present application, a medical image segmentation system is also provided, including: a medical imaging device, used to acquire medical image images, wherein the medical image images include multiple organ images of biological objects; a processing device, connected to the medical imaging device, used to use a pre-trained segmentation model inserted with different low-rank adapter sets to segment the medical image images separately, obtain segmentation results corresponding to different low-rank adapter sets, and merge the segmentation results corresponding to different low-rank adapter sets to obtain organ image segmentation results of the medical image images, wherein the low-rank adapters included in the different low-rank adapter sets are obtained by continuously learning the pre-trained segmentation model inserted with the initial adapter using different training data sets.

[0009] According to another aspect of an embodiment of the present application, an image segmentation method is also provided, the method comprising: responding to an input instruction acting on an operation interface, displaying an image to be processed on the operation interface, wherein the image to be processed includes multiple organ images of a biological object; responding to an image segmentation instruction acting on the operation interface, displaying a target segmentation result of the image to be processed on the operation interface, wherein the target segmentation result is used to characterize the positions of the multiple organ images in the image to be processed, the target segmentation result is a result obtained by merging the segmentation results corresponding to different low-rank adapter sets, the segmentation results corresponding to different low-rank adapter sets are the results obtained by segmenting the image to be processed respectively using pre-trained segmentation models inserted with different low-rank adapter sets, and the low-rank adapters contained in the different low-rank adapter sets are obtained by continuously learning the pre-trained segmentation model inserted with the initial adapter using different training data sets.

[0010] According to another aspect of an embodiment of the present application, an image segmentation method is also provided, the method comprising: obtaining an image to be processed by calling a first interface, wherein the first interface comprises a first parameter, the parameter value of the first parameter comprises the image to be processed, and the image to be processed comprises multiple organ images of a biological object; using a pre-trained segmentation model into which different low-rank adapter sets are inserted, respectively segmenting the image to be processed to obtain segmentation results corresponding to different low-rank adapter sets, wherein the low-rank adapters contained in the different low-rank adapter sets are obtained by continuously learning the pre-trained segmentation model into which the initial adapter is inserted using different training data sets; merging the segmentation results corresponding to the different low-rank adapter sets to obtain a target segmentation result of the image to be processed, wherein the target segmentation result is used to characterize the positions of the multiple organ images in the image to be processed; outputting the target segmentation result by calling a second interface, wherein the second interface comprises a second parameter, the parameter value of the second parameter comprises the target segmentation result.

[0011] According to another aspect of the embodiments of the present application, a computer terminal is further provided, including: a memory storing an executable program; and a processor for running the program, wherein the method in each embodiment of the present application is executed when the program is running.

[0012] According to another aspect of an embodiment of the present application, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored executable program, wherein when the executable program is running, the device where the computer-readable storage medium is located is controlled to execute the methods in each embodiment of the present application.

[0013] According to another aspect of the embodiments of the present application, a computer program product is also provided, including a computer program, and when the computer program is executed by a processor, the method in each embodiment of the present application is implemented.

[0014] According to another aspect of an embodiment of the present application, a computer program product is also provided, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method in each embodiment of the present application is implemented.

[0015] According to another aspect of the embodiments of the present application, a computer program is also provided, and when the computer program is executed by a processor, the methods in the various embodiments of the present application are implemented.

[0016] Through the scheme provided by the above-mentioned embodiment of the present application, an image to be processed is obtained, wherein the image to be processed includes multiple organ images of a biological object; a pre-trained segmentation model into which different low-rank adapter sets are inserted is used to segment the image to be processed respectively, to obtain segmentation results corresponding to different low-rank adapter sets, wherein the low-rank adapters included in the different low-rank adapter sets are obtained by continuously learning the pre-trained segmentation model into which the initial adapter is inserted using different training data sets; the segmentation results corresponding to the different low-rank adapter sets are merged to obtain a target segmentation result of the image to be processed, wherein the target segmentation result is used to characterize the positions of the multiple organ images in the image to be processed. It is easy to notice that the present application inserts different low-rank adapter sets into the pre-trained segmentation model. Since different low-rank adapter sets are obtained through continuous learning, different low-rank adapter sets can better adapt to the characteristics of specific organs in the image to be processed during the training process, thereby improving the accuracy of segmentation. The pre-trained segmentation model is inserted with different low-rank adapter sets. The pre-trained segmentation model can be fine-tuned and updated by efficiently and lightly adding a small amount of model parameters, thereby ensuring the extremely low parameter growth characteristics; at the same time, the parameters added by the new task will not affect the pre-trained segmentation model, thereby ensuring the no forgetting characteristic; by merging the segmentation results corresponding to different low-rank adapter sets, a more comprehensive and accurate target segmentation result can be obtained, different low-rank adapter sets can capture different characteristics of different organs in the image to be processed, and by merging the segmentation results corresponding to different low-rank adapter sets, different feature information can be comprehensively utilized to improve the accuracy and completeness of the segmentation results, which can effectively improve the segmentation accuracy and efficiency of biological object organ images, and provide an effective solution to the technical problems in the field of medical image segmentation, thereby solving the technical problems of low accuracy and efficiency in segmenting organ images of biological objects.

[0017] It is easy to notice that the above general description and the following detailed description are only for the purpose of exemplifying and explaining the present application, and do not constitute a limitation of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0019] Figure 1 is a schematic diagram of an application scenario of an image segmentation method according to an embodiment of the present application;

[0020] Figure 2 is a schematic diagram of a computer terminal as a computing node in a computing environment according to an embodiment of the present application;

[0021] Figure 3 is a flow chart of an image segmentation method according to an embodiment of the present application;

[0022] Figure 4 is a schematic diagram of an optional image to be processed according to an embodiment of the present application;

[0023] Figure 5 is a schematic diagram of an optional segmentation result of an image to be processed according to an embodiment of the present application;

[0024] Figure 6 is a schematic diagram of the structure of an optional pre-trained segmentation model according to an embodiment of the present application;

[0025] Figure 7 is a schematic structural diagram of an optional first embedding layer (Loco-pe) according to an embodiment of the present application;

[0026] Figure 8 is a schematic structural diagram of an optional first transformation layer (Loco-pvt) according to an embodiment of the present application;

[0027] Fig. 9 is a schematic diagram of an image segmentation device according to an embodiment of the present application;

[0028] Fig.10 is a flow chart of a medical image segmentation method according to an embodiment of the present application;

[0029] Fig.11 is a schematic diagram of a medical image segmentation system according to an embodiment of the present application;

[0030] Fig.12 is a flowchart of an optional image segmentation method according to an embodiment of the present application;

[0031] Fig.13 is a flowchart of another optional image segmentation method according to an embodiment of the present application;

[0032] Fig.14 It is a structural block diagram of a computer terminal according to an embodiment of the present application. DETAILED DESCRIPTION

[0033] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0034] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0035] First, some nouns or terms that appear in the description of the embodiments of the present application are subject to the following explanations:

[0036] Continue semantic segmentation (CSS) refers to the process of continuous semantic segmentation in a video sequence. Traditional semantic segmentation technology mainly performs pixel-level semantic segmentation on static images, while CSS extends this technology to the video field, realizing pixel-level semantic segmentation for each frame in a video sequence. Continuous semantic segmentation technology usually needs to take into account the correlation between consecutive frames in a video sequence in order to better capture the movement and changes of objects in the video.

[0037] The Pyramid Vision Transformer (PVT) can be an artificial intelligence model that combines the visual pyramid and the attention mechanism. PVT combines the multi-scale information extraction of the visual pyramid and the feature learning ability of the attention mechanism, and can better capture the details and overall information inside the image when processing visual tasks.

[0038] Low-Rank Adaptation (LoRA for short) can refer to a technical method in the field of artificial intelligence for adaptively adjusting the weights of neural networks during the training phase to improve the performance and generalization ability of the model; in deep learning, neural networks usually contain a large number of parameters, which need to be learned through a large amount of training data to adapt to specific tasks. However, the training data may be insufficient or incomplete, resulting in poor performance of the neural network on certain tasks. LoRA technology performs low-rank adaptation on the parameters of the neural network, that is, decomposing the parameter matrix into low-rank components, thereby reducing the number and complexity of parameters and improving the generalization ability of the model.

[0039] According to an embodiment of the present application, an image segmentation method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0040] Considering that the machine learning model consumes a lot of computing resources of the mobile terminal, the above method provided in the embodiment of the present application can be applied to Figure 1 The application scenarios shown are not limited to these. Figure 1 is a schematic diagram of an application scenario of an image segmentation method according to an embodiment of the present application. Figure 1 In the application scenario shown, the machine learning model is deployed in the server 10, and the server 10 can be connected to one or more clients 20 through a local area network connection, a wide area network connection, an Internet connection, or other types of data networks. The client 20 here may include but is not limited to: smart phones, tablet computers, laptops, PDAs, personal computers, smart home devices, vehicle-mounted devices, etc. The client 20 can interact with the user through a graphical user interface to call the large model, thereby implementing the method provided in the embodiment of the present application.

[0041] It should be noted that, when the operating resources of the client device can meet the deployment and operating conditions of the large model, the embodiments of the present application can be carried out in the client device.

[0042] Figure 2 is a schematic diagram of a computer terminal as a computing node in a computing environment according to an embodiment of the present application, such as Figure 2 As shown, computing environment 201 includes multiple computing nodes (such as servers) running on a distributed network (shown as 210-1, 210-2, ..., in the figure). Different computing nodes contain local processing and memory resources, and end users 202 can remotely run applications or store data in computing environment 201. Applications can be provided as multiple services 220-1, 220-2, 220-3 and 220-4 in computing environment 201, representing services "A", "D", "E" and "H" respectively.

[0043] The end user 202 can provide and access services through a web browser or other software application on the client, and in some embodiments, the end user 202's provision and / or request can be provided to the entry gateway 230. The entry gateway 230 can include a corresponding agent to handle the provision and / or request for the service (one or more services provided in the computing environment 201).

[0044] Services are provided or deployed based on various virtualization technologies supported by the computing environment 201. In some embodiments, services can be provided based on virtual machine (VM)-based virtualization, container-based virtualization, and / or similar methods. Virtual machine-based virtualization can be to simulate a real computer by initializing a virtual machine, and execute programs and applications without directly contacting any actual hardware resources. While the virtual machine virtualizes the machine, according to container-based virtualization, a container can be started to virtualize the entire operating system (OS) so that multiple workloads can run on a single operating system instance.

[0045] In an embodiment based on container virtualization, several containers of a service can be assembled into a Pod (e.g., a Kubernetes Pod). Figure 2 As shown, service 220-2 can be equipped with one or more Pods 240-1, 240-2, ..., 240-N (collectively referred to as Pods). Pods can include a proxy 245 and one or more containers 242-1, 242-2, ..., 242-M (collectively referred to as containers). One or more containers in a Pod process requests related to one or more corresponding functions of the service, and the proxy 245 generally controls network functions related to the service, such as routing, load balancing, etc. Other services can also be equipped with similar Pods.

[0046] During operation, executing a user request from end user 202 may require invoking one or more services in computing environment 201, and executing one or more functions of one service may require invoking one or more functions of another service. Figure 2 As shown, service "A" 220-1 receives a user request from end user 202 from ingress gateway 230, service "A" 220-1 may call service "D" 220-2, and service "D" 220-2 may request service "E" 220-3 to perform one or more functions.

[0047] The computing environment described above can be a cloud computing environment, where the allocation of resources is managed by the cloud service provider, allowing the development of functions without considering the implementation, adjustment or expansion of servers. The computing environment allows developers to execute code in response to events without building or maintaining complex infrastructure. Services can be divided into a set of functions that can be automatically and independently scaled, rather than expanding a single hardware device to handle potential loads.

[0048] Under the above operating environment, this application provides Figure 3 The image segmentation method shown. Figure 3 FIG. 1 is a flow chart of an image segmentation method according to an embodiment of the present application. Figure 3As shown, the following steps may be specifically included:

[0049] Step S302, obtaining an image to be processed.

[0050] The images to be processed include multiple organ images of biological objects.

[0051] The above-mentioned biological objects may refer to organisms such as animals with organs (including humans), plants, microorganisms, etc., and may include but are not limited to mammals, birds, reptiles, amphibians, fish, various plants with organs, fungi, bacteria, etc. The specific type of biological object can be determined according to actual needs and is not limited here.

[0052] The above-mentioned image to be processed may refer to an image showing an organ of a biological object. An organ may refer to a structural unit composed of multiple tissues that can perform a specific function. The image type of the image to be processed may include but is not limited to computed tomography (CT image), magnetic resonance imaging (MRI image), X-ray imaging (X-ray imaging), positron emission tomography (PET image) and nuclear medicine imaging (Nuclear Medicine Imaging, Nuclear Medicine image), etc. The image type of the image to be processed can be determined according to actual needs and is not limited here.

[0053] Figure 4 is a schematic diagram of an optional image to be processed according to an embodiment of the present application, such as Figure 4 As shown, an image containing multiple organs of the human body is displayed, and the image to be processed can also be determined according to actual needs, which is not limited here.

[0054] In an optional embodiment, image data containing multiple organs can be obtained from a medical image database or actual clinical data, and image data containing images of multiple organs of a biological object can be selected from the medical image data set, which can be used to segment and identify multiple organs of the biological object. The acquired original image data can be preprocessed, including denoising, grayscale standardization, image registration and other steps to improve image quality and accuracy. Image data containing multiple organs can be selected as needed to ensure the diversity and representativeness of the data set. Acquiring the images to be processed can provide accurate input data for subsequent organ segmentation and identification, and provide strong support for medical image research and analysis.

[0055] The image segmentation method proposed in the present application can be applied to multiple specific application scenarios such as organ segmentation of medical images and segmentation of tumor images. Organ segmentation of medical images can refer to the use of computer vision and image processing technology to automatically or semi-automatically segment and mark organs or tissues in medical images, so as to extract regions or structures of interest. Organ image segmentation can include the segmentation of organs such as the brain, heart, liver, and lungs, which can help doctors analyze image data more accurately. In particular, the image segmentation method proposed in the present application, in various application scenarios, belongs to image processing, and is not directly implemented on living human or animal bodies; the result generated by the proposed image segmentation method is the position information of the organ image in the image to be processed, and the position information of the organ image in the image to be processed itself cannot directly derive the diagnosis result of the disease or the health status, that is, it does not have the direct purpose of obtaining the diagnosis result of the disease or the health status.

[0056] Step S304, using the pre-trained segmentation model into which different low-rank adapter sets are inserted, the image to be processed is segmented respectively to obtain segmentation results corresponding to different low-rank adapter sets.

[0057] Among them, the low-rank adapters included in different low-rank adapter sets are obtained by continuously learning the pre-trained segmentation model with the initial adapter inserted using different training data sets.

[0058] The above-mentioned training data set may refer to a set of organ images of biological objects containing organ image segmentation results. The types of organs in the organ image set of biological objects may include organ types that have not been segmented by the initial adapter, and may also include organ types that have been segmented before by the initial adapter. The types of organs in the organ image set may be determined according to actual needs and are not limited here. The above-mentioned different training data sets may be used to indicate that the training data sets used each time are not exactly the same, and may include organs of the same category or organs of different categories. Each training data set is no longer accessed after the training is completed.

[0059] The above-mentioned initial adapter may refer to a model structure used to adapt a model to another task or field. The adapter may be a small network structure added on top of the pre-trained model, used to fine-tune the new task or field while keeping the pre-trained model parameters unchanged. The adapter can improve the performance of the model on the new task or field while reducing dependence on large-scale data sets. The specific type of the initial adapter can be determined according to actual needs and is not limited here.

[0060] The above-mentioned pre-trained segmentation model may refer to a pre-trained network model capable of performing image segmentation processing. The pre-trained segmentation model may be the low-rank continuous pyramid visual transformer network proposed in this application. The pre-trained segmentation model may be determined according to actual needs, and the pre-trained segmentation model is not specifically limited here.

[0061] The above-mentioned continuous learning can refer to the continuous learning method (continual learning) in the field of artificial intelligence, which enables the model to continuously learn and adapt to new knowledge while constantly receiving new data. Through continuous learning technology, the model can continuously learn new tasks, knowledge or data, while trying to retain previously learned knowledge, thereby continuously improving the generalization ability and adaptability of the model. In practical applications, continuous learning technology can help the model better adapt to the ever-changing environment and data, and thus be better applied to various scenarios in real life.

[0062] The above-mentioned segmentation results may refer to the segmentation results of the corresponding organ images under different segmentation tasks. The segmentation task may refer to a pre-set segmentation task that needs to be performed on the image to be processed, for example, segmenting a specified organ image, or segmenting each organ image contained in the image to be processed separately, etc. The segmentation task can be determined according to actual needs and is not limited here.

[0063] Figure 5 is a schematic diagram of an optional segmentation result of an image to be processed according to an embodiment of the present application, such as Figure 5 As shown, it shows the Figure 4 The segmentation results of different organ images obtained by segmenting the image to be processed are shown in FIG. Figure 5 The upper and lower images can represent the segmentation results of the corresponding organ images under two different segmentation tasks, and the segmentation results of different types of organs can be obtained. The segmentation results corresponding to the segmentation tasks can be determined according to actual needs and are not limited here.

[0064] In an optional embodiment, in deep learning, the parameters of the neural network are obtained through training with large-scale data sets. The parameters of the neural network may not be flexible enough to adapt to different tasks or environments. The low-rank adapter can adjust the parameters of the neural network to adapt to different tasks or environments by introducing additional adaptive parameters. In the field of image segmentation, a pre-trained deep learning model can be used to perform segmentation tasks. Due to the complexity and diversity of medical imaging data, different low-rank adapter sets can be inserted to segment the image to be processed. By inserting a low-rank adapter into the pre-trained model, the model can be personalized without destroying the original structure. Specifically, by continuously learning the pre-trained segmentation model with the initial adapter inserted using different training data sets, different low-rank adapter sets can be obtained. Each low-rank adapter set corresponds to a specific set of training data, which can better adapt to different types of medical images. Finally, these different low-rank adapter sets are applied to the processed images respectively, and multiple different segmentation results can be obtained, thereby improving the accuracy and robustness of the segmentation.

[0065] Step S306, merging the segmentation results corresponding to different low-rank adapter sets to obtain a target segmentation result of the image to be processed.

[0066] The target segmentation result is used to characterize the positions of multiple organ images in the image to be processed.

[0067] The above target segmentation result can be represented by coordinate data, or by marking the coordinate data on the segmentation result image, which can be determined according to actual needs. The representation form of the target segmentation result is not limited here.

[0068] In an optional embodiment, merging the segmentation results corresponding to different low-rank adapter sets refers to fusing multiple different segmentation results to obtain a final target segmentation result, so that the final segmentation result can better characterize the positions of multiple organs in the image. By using different low-rank adapter sets to segment the image to be processed, multiple different segmentation results can be obtained. A variety of merging strategies can be used to merge different segmentation results. During the merging process, the confidence or accuracy corresponding to each segmentation result can be considered, and different segmentation results can be weighted fused. High-confidence results can also be retained based on the confidence, while low-confidence results are discarded. The specific merging strategy can be determined according to actual needs and is not limited here, so as to obtain a more accurate final segmentation result. The final target segmentation result can be used to characterize the positions of multiple organs in the image to be processed, accurately segment the positions of different organs, and facilitate the research and analysis of medical images.

[0069] Through the scheme provided by the above-mentioned embodiment of the present application, an image to be processed is obtained, wherein the image to be processed includes multiple organ images of a biological object; a pre-trained segmentation model into which different low-rank adapter sets are inserted is used to segment the image to be processed respectively, to obtain segmentation results corresponding to different low-rank adapter sets, wherein the low-rank adapters included in the different low-rank adapter sets are obtained by continuously learning the pre-trained segmentation model into which the initial adapter is inserted using different training data sets; the segmentation results corresponding to the different low-rank adapter sets are merged to obtain a target segmentation result of the image to be processed, wherein the target segmentation result is used to characterize the positions of the multiple organ images in the image to be processed. It is easy to notice that the present application inserts different low-rank adapter sets into the pre-trained segmentation model. Since different low-rank adapter sets are obtained through continuous learning, different low-rank adapter sets can better adapt to the characteristics of specific organs in the image to be processed during the training process, thereby improving the accuracy of segmentation. The pre-trained segmentation model is inserted with different low-rank adapter sets. The pre-trained segmentation model can be fine-tuned and updated by efficiently and lightly adding a small amount of model parameters, thereby ensuring the extremely low parameter growth characteristics; at the same time, the parameters added by the new task will not affect the pre-trained segmentation model, thereby ensuring the no forgetting characteristic; by merging the segmentation results corresponding to different low-rank adapter sets, a more comprehensive and accurate target segmentation result can be obtained, different low-rank adapter sets can capture different characteristics of different organs in the image to be processed, and by merging the segmentation results corresponding to different low-rank adapter sets, different feature information can be comprehensively utilized to improve the accuracy and completeness of the segmentation results, which can effectively improve the segmentation accuracy and efficiency of biological object organ images, and provide an effective solution to the technical problems in the field of medical image segmentation, thereby solving the technical problems of low accuracy and efficiency in segmenting organ images of biological objects.

[0070] In the above embodiment of the present application, the pre-trained segmentation model inserted with any one of the low-rank adapter sets includes at least: an input module, an encoder inserted with a first low-rank adapter, a decoder inserted with a second low-rank adapter, and an output module; using the pre-trained segmentation model inserted with any one of the low-rank adapters, the image to be processed is segmented to obtain the segmentation result corresponding to any one of the low-rank adapters, including: using the input module to extract features of the image to be processed to obtain the original image features of the image to be processed; using the encoder inserted with the first low-rank adapter to encode the original image features to obtain the first image features; using the decoder inserted with the second low-rank adapter to decode the first image features to obtain the second image features; using the output module to reconstruct the second image features to obtain the segmentation result corresponding to any one of the low-rank adapters.

[0071] The above-mentioned input module may refer to the input convolution layer of the pre-trained segmentation model, which may be used to adjust the dimensions of the image and prediction and to extract image features.

[0072] The above-mentioned output module may refer to the output convolution layer of the pre-trained segmentation model, which may be used to adjust the dimensions of the image and prediction as well as image integration.

[0073] In an optional embodiment, the input module is used to perform feature extraction on the image to be processed to obtain the original image features of the image to be processed. The original image features may include information such as edges, textures, shapes, etc. in the image, which is convenient for subsequent image segmentation tasks; the original image features can be encoded using an encoder inserted with a first low-rank adapter to obtain first image features. The encoding process can compress and extract the original image features, making the feature representation more compact and effective; the first image features are decoded using a decoder inserted with a second low-rank adapter to obtain second image features. The decoder can decode the encoded features into the original feature representation for subsequent segmentation tasks. In the above steps, the original image features can be converted into feature representations with more abstract expression capabilities through the encoding process, thereby better capturing the information in the image and reducing the dimension of the features. degree, which helps to improve the generalization ability of the model and reduce the risk of overfitting. The decoding process can convert the abstract features after encoding processing back into the original image features, thereby realizing the reconstruction of image features. Through decoding processing, important information in the image can be retained, while noise and useless information can be removed to obtain clearer and more accurate image features, providing a better basis for subsequent segmentation results. Finally, the output module can be used to reconstruct the second image features to obtain the segmentation result corresponding to any low-rank adapter. The output module can map the decoded features back to the original image space to obtain the final segmentation result. The above process can help accurately segment different organs or structures in the image. The above steps realize the organ segmentation task of medical images through the process of feature extraction, encoding, decoding and reconstruction, and improve the accuracy and efficiency of segmentation.

[0074] In the above embodiment of the present application, an encoder with a first low-rank adapter inserted includes: a plurality of encoder modules connected in sequence, the encoder module including: a first embedding layer with a first sub-adapter inserted and a plurality of first transformation layers with a second sub-adapter inserted, wherein the first low-rank adapter includes a first sub-adapter and a second sub-adapter; using the encoder with the first low-rank adapter inserted to encode the original image features to obtain the first image features, including: using the first embedding layer with the first sub-adapter inserted in the current encoder module to process the features input by the current encoder module to obtain the first embedded features, wherein, when the current encoder module is the first encoder module among the multiple encoder modules, the features input by the current encoder module are the original image features, and when the current encoder module is not the first encoder module In the case of a sub-adapter module, the feature input to the current encoder module is the feature output by the previous encoder module; using any first transform layer in the current encoder module with the second sub-adapter inserted therein, the feature input to the first transform layer is transformed to obtain the feature output by the first transform layer, wherein, in the case that the first transform layer is the first first transform layer, the feature input to the first transform layer is the first embedded feature, in the case that the first transform layer is not the last first transform layer, the feature output by the first transform layer is used as the feature input to the next first transform layer, and in the case that the first transform layer is the last first transform layer, the feature output by the first transform layer is used as the feature output by the current encoder module; and the feature output by the last encoder module among the multiple encoder modules is determined to be the first image feature.

[0075] The encoder inserted with the first low-rank adapter in the present application may include multiple encoder modules (EncoderBlock). Optionally, the encoder inserted with the first low-rank adapter may include 4 encoder modules, and the number can also be determined according to actual needs, which is not limited here. The encoder module may include a first embedding layer and multiple first transformation layers connected in sequence, and the number of first transformation layers can be determined according to actual needs, which is not limited here.

[0076] In an optional embodiment, the encoder may include multiple encoder modules, each of which includes an embedding layer and multiple transformation layers. The embedding layer is used to process the input features to obtain embedded features; the transformation layer transforms the embedded features to obtain output features. Specifically, the first embedding layer in the current encoder module can be used to process the input features to obtain the first embedded features, that is, feature extraction is performed on the input image to extract useful information for subsequent processing; any first transformation layer in the current encoder module with a second sub-adapter inserted can be used to transform the input features of the first transformation layer to obtain the features output by the transformation layer, that is, the extracted features are further processed and converted so that the features can better express the information of the image. The number of encoder modules can be determined according to actual needs and is not limited here. Multiple encoder modules connected in sequence can be executed cyclically to realize the process of encoding the original image features to obtain the first image features. Specifically, the first encoder module can encode the original image features, and the input end of the second encoder module can be connected to the output end of the first encoder module to encode the output result of the first encoder module, and so on. Finally, the feature output by the last encoder module among the multiple encoder modules can be determined as the first image feature. The final high-level features extracted from the input image can be used for subsequent image segmentation tasks. The workflow of the encoder in image segmentation in the above steps, through multiple processing and conversion, finally obtains the first image features suitable for image segmentation, which can improve the accuracy and efficiency of segmentation.

[0077] In the above embodiment of the present application, the first embedding layer includes: a first convolutional layer, a first normalization layer, and a first sub-adapter inserted into the first convolutional layer; using the first embedding layer with the first sub-adapter inserted in the current encoder module, the features of the current encoder module input are processed to obtain the first embedded features, including: merging the parameters of the first convolutional layer and the parameters of the first sub-adapter to obtain first merged parameters; using the first merged parameters to convolve the features of the current encoder module input to obtain the first convolutional features; using the first normalization layer to normalize the first convolutional features to obtain the first embedded features.

[0078] In an optional embodiment, the first embedding layer can be used to extract features and perform feature processing to ultimately achieve image segmentation. Specifically, the first embedding layer may include three components: a first convolutional layer, a first normalization layer, and a first sub-adapter. The first convolutional layer is used to perform a convolution operation on the input image to extract features in the image. The first normalization layer is used to normalize the features obtained by convolution to make the distribution of the features more stable. The first sub-adapter can be used as a learnable parameter to adjust and adapt the parameters of the first convolutional layer to improve the performance of the network. The parameters of the first convolutional layer and the parameters of the first sub-adapter can be merged to obtain a first merged parameter. The parameters of the first convolutional layer and the parameters of the first sub-adapter are merged to obtain a first merged parameter. The parameters of the encoder are fused together to improve the expressiveness of the features and the performance of the network; the features of the current encoder module input are convolved using the first merging parameters to obtain the first convolutional features, that is, the input features are further extracted and processed to enhance the expressiveness and richness of the features; finally, the first convolutional features are normalized using the first normalization layer to obtain the first embedded features, which can further optimize the expression of the features and make the features more stable and easy to process. The processed features will be sent to the subsequent layers of the network for deeper feature extraction and image segmentation tasks. The above steps realize the extraction and processing of input image features and provide better feature representation and performance for subsequent image segmentation tasks.

[0079] In the above embodiment of the present application, the feature input by the current encoder module includes multiple sub-features, and the multiple sub-features are used to characterize the multiple image blocks obtained after the image to be processed is divided into blocks; the feature input by the current encoder module is convolved using the first merging parameter to obtain a first convolution feature, including: using the first merging parameter to convolve the multiple sub-features respectively to obtain multiple sub-convolution features, wherein the first convolution feature includes multiple sub-convolution features; using the first normalization layer to normalize the first convolution feature to obtain a first embedded feature, including: using the first normalization layer to normalize the multiple sub-convolution features respectively to obtain multiple sub-normalized features; summing the multiple sub-normalized features with the multiple position features respectively to obtain the first embedded feature, wherein the multiple position features are used to characterize the positions of the multiple image blocks in the image to be processed.

[0080] In an optional embodiment, the input feature can be processed by the encoder module to extract the first embedded feature. Specifically, the image to be processed can be divided into blocks to obtain multiple image blocks. Based on the input module, that is, the input convolution layer, feature extraction is performed on the multiple image blocks to obtain multiple sub-features. Blocking the image to be processed can help improve processing efficiency and accuracy. The multiple image blocks obtained after blocking can be processed separately, so that it is easier to identify and segment the organ area of ​​interest. In the process of feature extraction of multiple image blocks based on the input module, a convolutional neural network or a feature extractor can be used, which is not limited here. The input module can help identify different features in the image block, such as texture, shape, color, etc., so as to extract feature information of the image block. By extracting features from multiple image blocks, multiple sub-features can be obtained. The features input by the current encoder module include multiple Sub-features, multiple sub-features can be used to characterize multiple image blocks after the image to be processed is divided into blocks, and multiple sub-features can be image block features of different positions or scales; the input feature can be convolved with a first merging parameter to obtain a first convolution feature, and the first merging parameter can be used to convolve multiple sub-features respectively to obtain multiple sub-convolution features, and then these sub-convolution features are combined into the first convolution feature; the first convolution feature can be normalized with a first normalization layer to obtain a first embedded feature; the multiple sub-convolution features are normalized, and then the multiple sub-normalized features are summed with multiple position features to obtain a first embedded feature, and the position feature can be used to characterize the position information of multiple image blocks in the image to be processed. Through the above steps, the feature information of the input image block can be effectively extracted, and segmentation can be performed in combination with the position information, which is helpful to improve the accuracy and robustness of image segmentation.

[0081] In the above embodiment of the present application, the first transformation layer includes: an attention layer inserted with the first adapter parameters and a forward feedback layer inserted with the second adapter parameters; using any one of the first transformation layers inserted with the second sub-adapter, the features of the input to the first transformation layer are transformed to obtain the features of the output of the first transformation layer, including: using the attention layer inserted with the first adapter parameters to perform attention processing on the features of the input to the first transformation layer to obtain attention features; summing the features of the input to the first transformation layer and the attention features to obtain merged features; using the forward feedback layer inserted with the second adapter parameters to process the merged features to obtain the features output by the first transformation layer.

[0082] In an optional embodiment, the first transformation layer can improve the accuracy and effect of segmentation through an attention mechanism and a forward feedback layer. Specifically, the first transformation layer includes an attention layer and a forward feedback layer. The attention layer can perform attention processing on the input features, that is, select and emphasize important features through learned weight parameters, and suppress unimportant features, thereby obtaining attention features, which can help the network better focus on important areas and details and improve the accuracy of segmentation; the input features and the attention features can be summed to obtain merged features, and the attention features can be fused into the original features, so that the network can more effectively combine global and local information when processing features, and improve the ability to capture details and boundaries; the merged features are processed using the forward feedback layer to obtain the features output by the first transformation layer. The forward feedback layer may include a convolution layer, an activation function, etc., for extracting higher-level feature representations. Through the above steps, the network can further extract and learn features to obtain more accurate segmentation results. The design of the above steps utilizes the attention mechanism and the forward feedback layer to improve the accuracy and effect of image segmentation, so that the network can better capture important features and details, thereby achieving more accurate organ segmentation.

[0083] In the above embodiment of the present application, an attention layer into which the first adapter parameters are inserted is used to perform attention processing on the features of the first transformation layer input to obtain attention features, including: determining a query vector, a key vector and a value vector based on the features of the first transformation layer input; merging the weight parameters of the query vector and the query parameters in the first adapter parameters to obtain query merge parameters, and merging the weight parameters of the value vector and the value parameters in the first adapter parameters to obtain value merge parameters; determining the attention features based on the query merge parameters, the value merge parameters, the weight parameters of the key vector, the query vector, the key vector and the value vector.

[0084] The above attention processing can be used to achieve weighted attention to different inputs in a neural network in order to better capture the correlation between inputs. In the attention mechanism, there can be three important vectors: query vector, key vector and value vector. The query vector can be used to represent the input information at the current moment and to calculate the correlation with other input information. The query vector can be generated from the output of the previous layer and can be regarded as a vector representing the current input; the key vector can be used to represent the input information at other moments and to calculate the correlation with the query vector. The key vector can be generated from the output of the previous layer and can be regarded as a vector representing other inputs; the value vector can represent the specific content of the input information at the current moment. The value vector is also generated from the output of the previous layer and can be regarded as a vector representing the current input content. By calculating the correlation between the query vector and the key vector and taking the correlation as the weight, the value vector is weighted and summed to obtain the output of the attention mechanism, which can make the model pay more attention to the input information related to the query vector, thereby improving the performance of the model.

[0085] In an optional embodiment, an attention mechanism can be used to improve the model's understanding and segmentation effect of the image. Specifically, the query vector, key vector and value vector can be determined through the input features of the first transformation layer, and the three elements in the attention mechanism can be introduced: query vector, key vector and value vector; the weight parameter of the query vector and the query parameter in the first adapter parameter can be merged to obtain a query merge parameter; the weight parameter of the value vector and the value parameter in the first adapter parameter can be merged to obtain a value merge parameter, that is, information from different sources is integrated together for subsequent calculation of attention features. Finally, the attention feature can be determined based on the query merge parameter, the value merge parameter, the weight parameter of the key vector, the query vector, the key vector and the value vector, and the attention weight is obtained by calculating the similarity between the query vector and the key vector; then the value vector is weighted and summed using the attention weight to obtain the final attention feature. Through the above steps, the model can better understand the structure and features of the image in the image segmentation task, thereby improving the accuracy and effect of segmentation. The introduction of the attention mechanism can enable the model to pay more attention to important areas and features, and improve the expression and generalization capabilities of the model.

[0086] In the above embodiment of the present application, the forward feedback layer with the second adapter parameter inserted includes: a second normalization layer, at least one linear layer and a second adapter parameter inserted into the at least one linear layer; the merged features are processed using the forward feedback layer with the second adapter parameter inserted to obtain the features output by the first transformation layer, including: normalizing the merged features using the second normalization layer to obtain normalized features; merging the second adapter parameter and the parameters of the at least one linear layer to obtain at least one second merged parameter; linearly processing the normalized features using the at least one second merged parameter to obtain linear features; summing the merged features and the linear features to obtain the features output by the first transformation layer.

[0087] In an optional embodiment, the second normalization layer in the forward feedback layer can normalize the output of the previous layer to ensure the stability and consistency of the features. After the normalization, the normalized features can be input into at least one linear layer for linear transformation to further extract the features. After the linear layer, a second adapter parameter is inserted to further adjust and optimize the features. The second adapter parameter and the parameters of at least one linear layer can be merged to obtain at least one second merged parameter; the normalized features can be linearly processed using at least one second merged parameter to obtain linear features. Finally, the merged features and the linear features can be summed to obtain the features output by the first transformation layer. The features output by the first transformation layer will be used as the input of the next layer to continue the subsequent feature extraction and segmentation operations. Through the above steps, the forward feedback layer can effectively extract features in the image and perform segmentation, thereby achieving accurate medical image organ segmentation.

[0088] In the above embodiment of the present application, the decoder with the second low-rank adapter inserted includes: a first decoding layer and multiple decoder modules connected in sequence, the decoder module includes: multiple second transform layers and second decoding layers with the second low-rank adapter inserted; using the decoder with the second low-rank adapter inserted to decode the first image feature to obtain the second image feature, including: using the first decoding layer to decode the first image feature to obtain the first decoding feature; using any one of the second transform layers with the second low-rank adapter inserted in the current decoder module to transform the input feature of the second transform layer to obtain the output feature of the second transform layer, wherein, when the current decoder module is the first decoder module among multiple decoder modules and the second transform layer is the first second transform layer, the input feature of the first transform layer is the first decoding feature, and when the second transform layer is not the last second transform layer, the output feature of the second transform layer is used as the input feature of the next second transform layer; using the second decoding layer in the current decoder module to transform the output feature of the last first transform layer to obtain the output feature of the current decoder module; determining that the output feature of the last decoder module among multiple decoder modules is the second image feature.

[0089] In an optional embodiment, a decoder can be used to decode the feature map output by the encoder to generate a final segmentation result. The decoder may include a first decoding layer and multiple decoder modules. The decoder module includes multiple second transformation layers and second decoding layers. In the decoder, the first decoding layer can be used to decode the first image feature to obtain a first decoding feature. Then, the features input to the second transform layer are transformed by the second transform layer to obtain the features output by the second transform layer; the features output by the last first transform layer are transformed by the second decoding layer to obtain the features output by the current decoder module; the number of decoder modules can be determined according to actual needs and is not limited here. In multiple decoder modules, multiple decoder modules connected in sequence can be executed cyclically to realize the process of decoding the first image features and obtaining the second image features. Specifically, the first decoder module can decode the first image features, and the input end of the second decoder module can be connected to the output end of the first decoder module to decode the output result of the first decoder module. By analogy, the features output by the last decoder module are determined to be the second image features, that is, the final segmentation result, that is, the process of gradually decoding and reconstructing the original image is realized. Through the collaborative work of multiple decoder modules, an accurate segmentation result is finally obtained. The above decoding process can effectively extract feature information in the image, thereby realizing accurate segmentation of the image.

[0090] In the above embodiment of the present application, the input module includes: a second convolutional layer, a third normalization layer, and a third low-rank adapter inserted into the second convolutional layer; the input module is used to extract features of the image to be processed to obtain the original image features of the image to be processed, including: merging the parameters of the second convolutional layer and the parameters of the third low-rank adapter to obtain input merged parameters; using the input merged parameters to perform convolution processing on the image to be processed to obtain second convolution features; using the third normalization layer to normalize the second convolution features to obtain original image features.

[0091] In an optional embodiment, the second convolutional layer in the input module can be used to extract features in the image. In the organ segmentation task of medical images, the second convolutional layer will perform a convolution operation on the input medical image to extract features related to the organ. The third normalization layer is used to normalize the features in the convolutional neural network, which helps to improve the generalization ability and training speed of the network. The third normalization layer normalizes the features extracted by the second convolutional layer to reduce the differences between the features. The module that can use the third low-rank adapter to reduce the parameter dimension can be used to reduce the parameter dimension of the second convolutional layer and the third normalization layer. By combining the second convolutional layer and the third normalization layer The parameters of the three low-rank adapters are merged to obtain the input merged parameters; the parameters of the second convolutional layer and the third low-rank adapter are merged to obtain the input merged parameters, which can be used to perform convolution processing on the processed image to extract the second convolutional features; the input merged parameters can be used to perform convolution processing on the processed image to obtain the second convolutional features, which contain the image feature information extracted and processed by the second convolutional layer. Finally, the third normalization layer can be used to normalize the second convolutional features to obtain the original image features, which will be passed to the subsequent network modules for further image segmentation tasks.

[0092] In the above embodiment of the present application, the output module includes: a third convolutional layer and a target segmentation output layer, wherein the target segmentation output layer is used to segment the training data set corresponding to any low-rank adapter set; the second image feature is reconstructed using the output module to obtain a segmentation result corresponding to any low-rank adapter, including: convolution processing of the second image feature using the third convolutional layer to obtain a third convolutional feature; and the third convolutional feature is reconstructed using the target segmentation output layer to obtain a segmentation result corresponding to any low-rank adapter.

[0093] In an optional embodiment, the output module is used to generate a final segmentation result. Specifically, the output module may include a third convolutional layer and a target segmentation output layer. The third convolutional layer can receive feature images from the intermediate feature extraction layer and perform convolution processing on these feature images to further extract high-level features of the image, which helps the network to better understand the structure and content of the image; the target segmentation output layer can receive the feature image of the third convolutional layer and be used to segment the training data set corresponding to any low-rank adapter set; the target segmentation output layer can convert the third convolutional features into the final segmentation result by learning the parameters and weights of the network. The output module further processes the image features through the third convolutional layer, and then converts the processed features into the final segmentation result through the target segmentation output layer.

[0094] Optionally, in the segmentation of organ images, the present application can use a pre-trained segmentation model to perform new segmentation tasks. Since different segmentation tasks may involve different organs or structures, different segmentation output layers can be added to the pre-trained segmentation model during the training process to adapt to different task requirements. Adding different segmentation output layers can help the model learn different segmentation targets, such as different organs or structures, so that the model can learn multiple different tasks at the same time during the training process, thereby improving the generalization ability and applicability of the model. By adding different segmentation output layers, the training process of the model can be better controlled, so that the model can better adapt to specific segmentation tasks. That is, adding different segmentation output layers to the pre-trained segmentation model during the training process can help the model better adapt to different segmentation tasks and improve the performance and generalization ability of the model.

[0095] In the above-mentioned embodiment of the present application, the pre-trained segmentation model is obtained by training the pre-trained model using the initial training data, and different training data sets are prohibited from being accessed after continuous learning of the pre-trained segmentation model with the initial adapter inserted.

[0096] In an optional embodiment, since the pre-trained segmentation model refers to a model pre-trained on a large-scale dataset, a deep learning model can be pre-trained using a large amount of image data on a public dataset, and then fine-tuned using a dataset in a specific field to adapt to the needs of a specific task. In the organ segmentation task of medical imaging, the pre-trained segmentation model can help accelerate the convergence of the model and improve the performance of the model. By pre-training on a large-scale dataset, the model can learn a common feature representation, thereby improving the generalization ability on a specific task; in the continuous learning stage, a dataset in a specific field, such as a medical imaging dataset, can be used to fine-tune the pre-trained segmentation model with the initial adapter inserted to adapt to the requirements of the task. In this process, different training datasets will adjust the model to different degrees to make the model better adapt to the characteristics and requirements of the specific task; prohibiting access to different training datasets can avoid information leakage or confusion between different datasets. If different datasets are allowed to access each other, it may cause the performance of the model on a specific task to deteriorate, because the model may overfit the features of a dataset and ignore the features of other datasets. In order to ensure the performance of the model on a specific task, it is set to prohibit access to different training datasets in the continuous learning stage.

[0097] This application proposes a continuous whole-body organ segmentation model with lightly weighted low-rank adaptation. Specifically, a visual pyramid transformer (PVT) base segmentation model is trained and frozen on the initial task, and then lightweight trainable low-rank adaptive parameters are continuously added to the frozen model for each new learning task. Through a comprehensive exploration of architectural modifications, three important layers are identified, namely block embedding, query and value in multi-head attention, and feedforward layers, which are crucial for adapting to new segmentation tasks while keeping most of the pre-trained parameters fixed. The model proposed in this application can continuously segment new organs without causing catastrophic forgetting while maintaining a low parameter growth rate. It is continuously trained and tested on four datasets of different body parts covering a total of 121 organs. The number of organs can be determined according to actual needs and is not limited here. The results show that the model achieves high segmentation accuracy, close to the upper limit of visual-semantic prior converters and neural networks (NeuralNetwork U-Net, referred to as nnUNet), and significantly outperforms other regularization-based continuous semantic segmentation (Continuous Semantic Compared with the architecture-based continuous semantic segmentation method, the model proposed in this application has a lower parameter growth rate (16.7% vs. 96.7%) and achieves better performance.

[0098] Figure 6is a schematic diagram of the structure of an optional pre-trained segmentation model according to an embodiment of the present application, that is, the overall framework of the low-rank continuous pyramid visual transformer network proposed in the present application, such as Figure 6 As shown, the pre-trained segmentation model may include an input module 62, an encoder 64 inserted with a first low-rank adapter, a decoder 66 inserted with a second low-rank adapter, and an output module 68, wherein the encoder 64 inserted with the first low-rank adapter may include a plurality of encoder modules 602 connected in sequence, the encoder module includes a first embedding layer 6002 inserted with a first sub-adapter and a plurality of first transformation layers 6004 inserted with a second sub-adapter; the decoder 66 inserted with the second low-rank adapter may include: a first decoding layer 6006 and a plurality of decoder modules 604 connected in sequence, the decoder module may include: a plurality of second transformation layers 6008 and a second decoding layer 6010 inserted with a second low-rank adapter, and the structure of the pre-trained segmentation model may also be adjusted according to actual needs, which is not limited here.

[0099] The above low-rank continuous pyramid visual transformer network can use the pre-trained network model (3DPVT) as the backbone. The network model (3DPVT) can be further trained using the total dataset (TotalSegmentator). For the remaining training tasks, D = {D1, ..., D T} represents a dataset sequence, and the model follows the sequence in each D t training on t∈{1,…,T}, and D is not revisited after training. t , t th Dataset D t ={X t ,Y t}Nt fused with O t Organ type, the overall framework of the low-rank continuous pyramid visual transformer (Loco-pvt) network proposed in this application can be used for continuous whole-body organ segmentation. The network can be composed of a stack of encoder and decoder blocks. The embedding layer (Loco-pe) of the encoder can have a convolutional layer with a stride of 2 for downsampling. The decoder embedding layer (Deconv-pe) can use a deconvolutional layer for upsampling. Continuous LoRA is added to the linear layer for multi-head attention in Loco-pvt and Q / V projection in the feedforward network, and is also added to the convolutional layer (Loco-conv) in the embedding layer (Loco-pe). After training the initial task 0 (TotalSegmentator), the base network is frozen; in each subsequent continuous learning step, a set of trainable LoRA parameters and a new segmentation output layer are added to the corresponding layer of the network, and new tasks are learned from the current dataset. It can be assumed that (X t , Y t) indicates D t All input images and corresponding segmentation masks, voxel position i and organ category c j The prediction graph of is given by:

[0100] ;

[0101] Where f represents the trainable LoRA parameters in the initial task and the specific task Frozen parameters of previous training Transformer neural network, Can indicate the The low-rank matrix pair of the dataset, Represents the Sigmoid output of a specific trainable task, and the final prediction is the union of all previous predictions, possibly with overlapping classes.

[0102] The first transformation layer and the second transformation layer in the present application can adopt a continuous vision transformer layer (LoRAContinuous Vision Transformer Layer, referred to as Loco-pvt), which is a model layer for visual reasoning. The first transformation layer and the second transformation layer can also be determined according to actual needs, which is not limited here; the Loco-pvt framework inherits the key advantages of the two methods and is customized for continuous multi-organ segmentation, enabling low-rank adaptation in each transformer block and the block embedding module of the encoder; for each data set D, a small set of trainable LoRA parameters can be set to:

[0103] ;

[0104] Among them (A t , B t ) represents the LoRA matrix of the lth Loco-pvt block, M t represents the total number of trainable Loco-pvt blocks for the t-th dataset.

[0105] Regarding the modules (Loco-mha and loco-ffn), for the pre-trained weight matrix and the t-th dataset, the update can be constrained by representing the latter with a low-rank decomposition ,in, , , and rank , during training, W0 is frozen and does not receive gradient updates, while A t , B tcontains trainable parameters; both W0 and ∆W are multiplied by the same input, and the output vectors of W0 and ∆W are summed coordinately. The forward pass of h=W0x can be summarized as:

[0106] ;

[0107] Each A can be initialized with a random Gaussian t , initialize each B with a zero matrix t , at the beginning of training is zero, then, we can use Scaling ∆W t x, where α is a constant in r; although LoRA can be applied to any dense layer, it is shown in the related art that applying LoRA to the queries and values ​​of the MHA module produces the most significant performance improvement. Therefore, this application adopts similar design choices in each Loco-pvt block and uses higher r to accommodate the greater complexity of learning from visual signals.

[0108] The input module and the first embedding layer in this application can adopt a continuous image block embedding layer based on low-rank adaptation. layer, which can be referred to as Loco-pe for short), can be used to process sequence data. The input module and the first embedding layer can also be determined according to actual needs and are not limited here. The block embedding module projects the input block into a lower-dimensional implicit embedding space. Each encoder level of Loco-pvt is accompanied by a separate embedding module. The embedding module extracts feature maps of various resolutions. This spatial information is crucial for training robust continuous segmentation models across different datasets. Ideally, trainable parameters can be assigned to all convolution generators for each dataset. Compared with the dense layer, it is observed that the inclusion of LoRA in the convolutional layer leads to a significant increase in the number of parameters. To address this problem, the application of convolutional LoRA is specifically limited to the encoding PE layer, namely the Loco-pe layer. The Loco-pe layer combines LoRA-enabled 3D convolutions to project the input image blocks into the embedding space in each encoder level. In order to enhance the feature aggregation and projection of all scales of the segmentation output, LoRA is also enabled in the second to last convolutional layers of the decoder. In each convolution weight matrix, the D of the δ dataset t , a pair of matrices (A t ,B t ) have the same rank r, the forward pass of the convolution operation can be expressed as:

[0109] ;

[0110] Among them, δ0 can represent the frozen convolution weight W0 from pre-training.

[0111] The second decoding layer in the present application may adopt an embedding layer (Deconv-pe). The setting of the embedding layer (Deconv-pe) may be to replace the convolution layer in the above-mentioned embedding layer (Loco-pe) with a deconvolution layer. The second decoding layer may also be determined according to actual needs, which is not limited here.

[0112] Figure 7 is a schematic diagram of the structure of an optional first embedding layer (Loco-pe) according to an embodiment of the present application, such as Figure 7 As shown, the first embedding layer 6002 includes a first convolutional layer 702, a first normalization layer 706, and a first sub-adapter 704 inserted into the first convolutional layer. Based on the above structure, the first embedding layer can merge the parameters of the first convolutional layer and the parameters of the first sub-adapter to obtain first merged parameters; use the first merged parameters to convolve the features input by the current encoder module to obtain first convolution features; use the first normalization layer to normalize the first convolution features to obtain first embedded features.

[0113] Figure 8 is a schematic diagram of the structure of an optional first transformation layer (Loco-pvt) according to an embodiment of the present application, such as Figure 8 As shown, the first transformation layer 6004 includes: an attention layer 802 inserted with a first adapter parameter 8002 and a forward feedback layer 804 inserted with a second adapter parameter; the forward feedback layer inserted with the second adapter parameter includes: a second normalization layer 8012, at least one linear layer 8014 and a second adapter parameter 8016 inserted into at least one linear layer; based on the above structure, the first transformation layer can determine a query vector 8004, a key vector 8006 and a value vector 8008 based on the features input by the first transformation layer; merge the weight parameters of the query vector and the query parameters in the first adapter parameters to obtain a query merge parameter, and merge the weight parameters of the value vector and the first adapter parameters. The value parameters in the parameters are merged to obtain the value merged parameters; based on the query merged parameters, the value merged parameters, the weight parameters of the key vector, the query vector, the key vector and the value vector, the attention feature is determined; the feature 8010 input to the first transformation layer and the attention feature are summed to obtain a merged feature; the merged feature is normalized by the second normalization layer to obtain a normalized feature; the second adapter parameter and the parameter of at least one linear layer are merged to obtain at least one second merged parameter; the normalized feature is linearly processed by using at least one second merged parameter to obtain a linear feature; the merged feature and the linear feature are summed to obtain the feature output by the first transformation layer.

[0114] Optionally, in the model inference phase, in order to merge the output probability maps of all learning tasks, the preset body part-aware output merging method can be followed to pre-calculate the average body part distribution map of each data set, apply body part regression to the test scan to eliminate the out-of-distribution body part regions from the prediction of each task, and then use entropy-based integration to combine the predictions of all tasks; for a given CT image, the general process is as follows: use the Body Parts Regression (BPR) toolkit to locate and segment the body parts of the CT image in the longitudinal direction; input different body parts into the decoder head of the corresponding organ (group) to predict the segmentation of the corresponding organ (group); calculate the confidence of all voxels in the image based on the probability distribution and entropy of each voxel label predicted by each decoder head, which is used as a weight reference for the conflicting voxel labels when the predictions of all decoder heads are subsequently integrated; integrate the organ (group) prediction labels of all decoder heads. For voxels that appear in the prediction results of multiple decoder heads at the same time and have conflicting labels, decide which label to use based on the confidence of each decoder head on the voxel.

[0115] The dataset of this application can use a public dataset (TotalSegmentator.TotalSeg) as a task for basic model training, which can be used to evaluate the model proposed in this application. It consists of 1204 CT scans of different body parts with 103 labeled anatomical structures (face labels are removed). The number of labels and the number of CT scans can be determined according to actual needs and are not limited here. Three internal datasets can be continuously segmented. These datasets cover chest body parts, head and neck body parts, and esophageal datasets with tumors. The chest organ dataset (CHO) contains 153 chest CT scans with 16 labeled chest organs, including 7 organs overlapping with TotalSeg and 9 new organs; the head and neck organ dataset (HNO) includes 244 head and neck CT scans, in which 9 new organ maps are annotated. For example, an esophageal dataset (EsoTumor) contains 567 esophageal CT scans with tumors, which can be used for esophageal segmentation. 80%:20% segmentation can be used for training and testing. The ratio of training and testing can be determined according to actual needs and is not limited here. In the final stage, a total of 121 organs can be learned from all datasets. The median voxel resolution of the datasets (TotalSeg, HNO, CHO and EsoTumor) is 1.5×1.5×1.5mm, 1×1×2mm, 0.7×0.7×5mm and 1×1×5mm, respectively. Continuous semantic segmentation. In the continuous semantic segmentation experiment, the model can be updated according to a series of datasets. In each step, only the current dataset is used for training, and all previous datasets are inaccessible. Two continuous semantic segmentations are verified to demonstrate the robustness of the method of this application.

[0116] In the continuous semantic segmentation (CSS) test, the pre-trained segmentation model can be updated according to a series of datasets. In each step, the current dataset can be used for training, while all previous datasets are inaccessible. Two CSS orders are verified to prove the robustness of the method, including Order A (Order A: TotalSeg→CHO→HNO→EsoTumor) and Order B (Order B: TotalSeg→HNO→CHO→EsoTumor). As shown in Table 1, the method proposed in this application (Ours) can be compared with 4 major CSS methods, including 2 mainstream regularization-based baselines (MiB, PLOP), a regularization-based medical method (LISMO) and a new architecture-based method (SUN). All methods are implemented in the nnUNet data preprocessing and enhancement framework. The method proposed in this application uses PVT as the backbone, while the other 4 methods are based on CNN. In this study, the upper limits of PVT and nnUNet on each dataset are listed in Table In , the method proposed in this application reports the final continuous segmentation performance using the Dice similarity coefficient (DSC) and the 95% Hausdorff distance (HD95).

[0117] Table 1

[0118]

[0119] The pre-trained segmentation model (3DPVT base model) in this application can be the same as the network model (UniMISS), which can include 4 encoder modules [2, 3, 4, 3], each encoder module can include a PVT layer and 3 decoder blocks, each decoder block has [3, 4, 3] PVT layers, and the stride of convolution in the encoder PE and deconvolution in the decoder PE for down / up sampling purposes is 2. For the LoRA setting, this application sets the query / value layer and FFN levels to 64, 16 in Loco-pvt, and sets the convolution layer to 16 in Loco-pe / Loco-conv; LoRA alpha can be set to half of the corresponding level, the encoder can be initialized based on the self-supervised pre-training parameters (Unimiss), the batch size can be set according to the self-supervised pre-training parameters (Unimiss), the block size is 2, and the block size is 224×224×32 for all datasets and experiments. The ratio between the training set and the validation set can be 4:1. Each test experiment can be trained using the optimizer (AdamW). 6000 training rounds can be set for the training base model on the dataset (TotalSeg), 500 epochs can be set for the continuous training steps, the initial learning rate of PVT can be set to 1e-4, the weight decay can be set to 3e-5, and the model can be trained on a single dataset (NVIDIA A100 GPU).

[0120] The final continuous segmentation results (segmentation) of this application are shown in the table below in both order and single task upper limits. As shown, on the three previously learned datasets and all organs, the proposed method significantly outperforms three regularization-based methods (i.e., MiB, PLOP, and LISMO) of two orders, where the severe catastrophic forgetting of these methods may be caused by the large domain gap between different body parts. On the other hand, the proposed method belongs to the architecture-based method, so it will not forget past tasks and has order invariance when the basic training dataset (TotalSeg) is the same. The parameter increase rate of the pre-trained segmentation model proposed in this application is significantly lower than that of other methods. The proposed method adds A lightweight low-rank adapter was proposed, which only increased the performance of each task by 5.56%. In particular, there is also a small gap between the nnUNet upper limit of 90.49% and the PVT upper limit of 89.66% on all organs, which indicates that there is a potential capability difference between nnUNet and PVT. The method proposed in this application is close to reaching the PVT upper limit for all organs, with a tiny 0.4% decrease in DSC and a 0.9 mm increase in HD95, which demonstrates the efficiency and effectiveness of the well-trained frozen-PVT continuous LoRA. For detailed step-by-step DSC and forgetting rates of all CSS methods on each dataset, see Supplementary Table 2.

[0121] Table 2

[0122]

[0123] In terms of ablation studies, in order to demonstrate the importance of each continuous LoRA component in the proposed Loco-pvt network, this application also conducted two ablation studies using further continuous segmentation of the dataset (TotalSeg) to the dataset (CHO), as shown in Table 2. In the table on the left, various LoRA combinations were evaluated on three PVT components (Attn-QV, FFN, and PE-Conv), including the query and value projection layers in the multi-head attention (Attn-QV), the feedforward network in the transformer layer (FFN), and the 3D convolution in the image block embedding layer (PE-Conv). Compared with the base setting, adding an additional LoRA to FFN improves the CHO segmentation performance by 3.46%, from 76.38%. As high as 79.84%; adding an additional LoRA to pe-conv will lead to 80.36%, thereby obtaining 3.98%; this application adopts the complete LoCo design of LoRA in all three components to further improve the performance to 81.51%, and reduces HD95 from 7.96 mm to 5.77 mm. This ablation result shows that adding LoRA to FFN and pe-conv is effective and necessary. FFN provides additional capabilities to project and integrate new features from attention. Pe-conv adapts or locates at different block resolutions. In the right table of Table 2, this application further studies the effect of adding LoRA in the encoder or decoder. In the encoder-only (Encoder with frozen decoder) In the decoder only setting, the average DSC drops to 76.89%, while HD95 rises to 7.88 mm; similarly, in the decoder only setting with a frozen encoder, the average DSC decreases to 77.21%, while HD95 increases to 7.52 mm. The results show that LoRA in both the encoder (feature extraction) and decoder (organ localization) help enhance the network's adaptability to new tasks and are equally important for the Loco-pvt network to achieve performance comparable to the upper limit. This ablation study validates the necessity of each component in the network proposed in this application.

[0124] This application proposes a new Loco-pvt framework, which combines LoRA with Vision Transformer (ViT) for continuous whole-body organ segmentation. The PVT base model is trained and frozen on the initial task, and then lightweight trainable LoRA parameters are continuously added to the frozen base model, which avoids catastrophic forgetting and adapts the model to new tasks while maintaining a low parameter growth rate. Quantitative results on four datasets covering different body parts show that the method proposed in this application achieves very high accuracy on all 121 organs considered, with no specific limitation on the number of organs. It is close to reaching the PVT upper limit and outperforms other regularization-based methods. The final model proposed in this application shows a significantly lower parameter growth rate while maintaining a comparable performance level, highlighting the effectiveness of the method of this application in optimizing resource usage without affecting the quality of image segmentation.

[0125] This application enables whole-body organ segmentation for CT, and has the ability of continuous learning with no forgetting and low growth of model parameters, ensuring organ segmentation performance and model expansion efficiency, helping to solve the pain points of organ delineation in daily clinical practice for doctors. For whole-body organ segmentation, a new continuous learning framework with no forgetting and extremely low parameter growth is proposed to learn a single deep segmentation model for segmenting 121 whole-body organs. The number of organs is not specifically limited here. Using the network structure of visual pyramid transformer and continuous low-rank adaptation, it is proved that incrementally adding low-rank adaptation to a pre-trained and then frozen visual pyramid transformer can extract sufficiently representative image features, and additionally learn image features based on new data sets to subsequently effectively segment new classes. In addition, this application also extends low-rank adaptation to three-dimensional convolutional layers to better adapt to the resolution differences between different data sets.

[0126] This application proposes a new continuous segmentation framework with both zero forgetting and extremely low parameter growth, which is used to segment 121 organs and tumors throughout the body. This framework adopts the efficient parameter model fine-tuning technology based on LoRA. The model can be fine-tuned and updated by efficiently and lightly adding a small amount of model parameters, ensuring the extremely low parameter growth feature; at the same time, the parameters added by the new task will not have any impact on the old model, thus ensuring the zero forgetting feature; the model can segment all 121 organs at one time, because the model can continuously learn and update on different organ datasets through LoRA, thereby covering the 121 organs covered by multiple datasets. Integrated into one segmentation model, it can segment 121 organs. The continuous learning architecture and several multi-organ segmentation datasets containing 121 organs themselves bring about the benefits. The more datasets there are, the more organs can be segmented. This continuous segmentation framework adopts the efficient parameter model fine-tuning technology based on LoRA. The model can be fine-tuned and updated by efficiently and lightly adding a small number of model parameters, ensuring the extremely low parameter growth feature. At the same time, the parameters added by the new task will not have any impact on the old model, thus ensuring the no-forgetting feature. At the same time, through continuous learning on multiple multi-organ datasets, the ability of a single model to segment 121 organs is achieved.

[0127] This application proposes a network structure based on PVT and continuous LoRA. LoRA is an efficient parameter model fine-tuning technology. Here, LoRA technology is used to update the model on training data sets of continuous different segmentation tasks; the continuous here can mean combining LoRA technology with continuous learning. Here, "continual" can be understood as "continual learning", a classic machine learning task, that is, continuously learning and updating a model; a pre-trained and then frozen PVT is used to extract sufficiently representative and general image features, and by incrementally adding and training LoRA parameters to the training data sets of continuous different segmentation tasks, additional learning and updating of image features based on the new data set can be performed to efficiently complete downstream segmentation tasks; LoRA is extended to the three-dimensional convolutional layer in PVT to better adapt to the resolution differences between different data sets.

[0128] According to an embodiment of the present application, an image segmentation device for implementing the above-mentioned image segmentation method is also provided. Fig. 9 is a schematic diagram of an image segmentation device according to an embodiment of the present application, such as Fig. 9 As shown, the device includes: an acquisition module 902 , a segmentation module 904 and a merging module 906 .

[0129] Among them, the acquisition module 902 is used to acquire the image to be processed, wherein the image to be processed contains multiple organ images of a biological object; the segmentation module 904 is used to use a pre-trained segmentation model inserted with different low-rank adapter sets to segment the image to be processed respectively, and obtain segmentation results corresponding to different low-rank adapter sets, wherein the low-rank adapters contained in the different low-rank adapter sets are obtained by continuously learning the pre-trained segmentation model inserted with the initial adapter using different training data sets; the merging module 906 is used to merge the segmentation results corresponding to different low-rank adapter sets to obtain the target segmentation result of the image to be processed, wherein the target segmentation result is used to characterize the positions of multiple organ images in the image to be processed.

[0130] Among them, the pre-trained segmentation model inserted with any one of the low-rank adapter sets includes at least: an input module, an encoder inserted with a first low-rank adapter, a decoder inserted with a second low-rank adapter, and an output module; the segmentation module is also used to use the input module to extract features of the image to be processed to obtain the original image features of the image to be processed; use the encoder inserted with the first low-rank adapter to encode the original image features to obtain the first image features; use the decoder inserted with the second low-rank adapter to decode the first image features to obtain the second image features; use the output module to reconstruct the second image features to obtain the segmentation result corresponding to any one of the low-rank adapters.

[0131] Among them, the encoder with the first low-rank adapter inserted includes: a plurality of encoder modules connected in sequence, the encoder module includes: a first embedding layer with the first sub-adapter inserted and a plurality of first transformation layers with the second sub-adapter inserted, wherein the first low-rank adapter includes the first sub-adapter and the second sub-adapter; the segmentation module is also used to use the first embedding layer with the first sub-adapter inserted in the current encoder module to process the features input by the current encoder module to obtain the first embedded features, wherein, when the current encoder module is the first encoder module among the multiple encoder modules, the features input by the current encoder module are the original image features, and when the current encoder module is not the first encoder module, the features input by the current encoder module are the original image features. It is the feature output by the previous encoder module; using any first transformation layer in the current encoder module with the second sub-adapter inserted, the feature input to the first transformation layer is transformed to obtain the feature output by the first transformation layer, wherein, when the first transformation layer is the first first transformation layer, the feature input to the first transformation layer is the first embedded feature, when the first transformation layer is not the last first transformation layer, the feature output by the first transformation layer is used as the feature input to the next first transformation layer, and when the first transformation layer is the last first transformation layer, the feature output by the first transformation layer is used as the feature output by the current encoder module; determine that the feature output by the last encoder module among multiple encoder modules is the first image feature.

[0132] Among them, the first embedding layer includes: a first convolutional layer, a first normalization layer, and a first sub-adapter inserted into the first convolutional layer; a segmentation module is also used to merge the parameters of the first convolutional layer and the parameters of the first sub-adapter to obtain first merged parameters; convolution processing is performed on the features input by the current encoder module using the first merged parameters to obtain first convolution features; and the first convolution features are normalized using the first normalization layer to obtain first embedded features.

[0133] Among them, the features input by the current encoder module include multiple sub-features, and the multiple sub-features are used to characterize the multiple image blocks obtained after the image to be processed is divided into blocks; the segmentation module is also used to use the first merging parameter to convolve the multiple sub-features respectively to obtain multiple sub-convolution features, wherein the first convolution feature includes multiple sub-convolution features; the first convolution feature is normalized by the first normalization layer to obtain the first embedded feature, including: using the first normalization layer to normalize the multiple sub-convolution features respectively to obtain multiple sub-normalized features; summing the multiple sub-normalized features with the multiple position features respectively to obtain the first embedded feature, wherein the multiple position features are used to characterize the positions of the multiple image blocks in the image to be processed.

[0134] Among them, the first transformation layer includes: an attention layer inserted with the first adapter parameters and a forward feedback layer inserted with the second adapter parameters; a segmentation module, which is also used to use the attention layer inserted with the first adapter parameters to perform attention processing on the features input to the first transformation layer to obtain attention features; summing the features input to the first transformation layer and the attention features to obtain merged features; processing the merged features using the forward feedback layer inserted with the second adapter parameters to obtain the features output by the first transformation layer.

[0135] Among them, the segmentation module is also used to determine the query vector, key vector and value vector based on the features of the first transformation layer input; merge the weight parameters of the query vector and the query parameters in the first adapter parameters to obtain query merge parameters, and merge the weight parameters of the value vector and the value parameters in the first adapter parameters to obtain value merge parameters; determine the attention features based on the query merge parameters, the value merge parameters, the weight parameters of the key vector, the query vector, the key vector and the value vector.

[0136] Among them, the forward feedback layer with the second adapter parameters inserted includes: a second normalization layer, at least one linear layer and the second adapter parameters inserted into the at least one linear layer; a segmentation module, which is also used to use the second normalization layer to normalize the merged features to obtain normalized features; merge the second adapter parameters and the parameters of at least one linear layer to obtain at least one second merged parameter; use at least one second merged parameter to perform linear processing on the normalized features to obtain linear features; sum the merged features and the linear features to obtain the features output by the first transformation layer.

[0137] Among them, the decoder with the second low-rank adapter inserted includes: a first decoding layer and multiple decoder modules connected in sequence, and the decoder module includes: multiple second transform layers and second decoding layers with the second low-rank adapter inserted; a segmentation module, which is also used to use the first decoding layer to decode the first image feature to obtain the first decoding feature; use any second transform layer with the second low-rank adapter inserted in the current decoder module to transform the input feature of the second transform layer to obtain the output feature of the second transform layer, wherein, when the current decoder module is the first decoder module among multiple decoder modules, and the second transform layer is the first second transform layer, the input feature of the first transform layer is the first decoding feature, and when the second transform layer is not the last second transform layer, the output feature of the second transform layer is used as the input feature of the next second transform layer; use the second decoding layer in the current decoder module to transform the output feature of the last first transform layer to obtain the output feature of the current decoder module; determine that the output feature of the last decoder module among multiple decoder modules is the second image feature.

[0138] Among them, the input module includes: a second convolutional layer, a third normalization layer, and a third low-rank adapter inserted into the second convolutional layer; the segmentation module is also used to merge the parameters of the second convolutional layer and the parameters of the third low-rank adapter to obtain input merged parameters; the input merged parameters are used to convolve the image to be processed to obtain a second convolution feature; the second convolution feature is normalized using the third normalization layer to obtain the original image feature.

[0139] Among them, the output module includes: a third convolutional layer and a target segmentation output layer, wherein the target segmentation output layer is used to segment the training data set corresponding to any low-rank adapter set; the segmentation module is also used to use the third convolutional layer to convolve the second image feature to obtain the third convolutional feature; the target segmentation output layer is used to reconstruct the third convolutional feature to obtain the segmentation result corresponding to any low-rank adapter.

[0140] The pre-trained segmentation model is obtained by training the pre-trained model using the initial training data, and different training data sets are prohibited from being accessed after continuous learning of the pre-trained segmentation model with the initial adapter inserted.

[0141] According to an embodiment of the present application, a method for segmenting a medical image is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0142] Fig.10 is a flow chart of a medical image segmentation method according to an embodiment of the present application, such as Fig.10 As shown, the following steps may be specifically included:

[0143] Step S1002, obtaining a medical image.

[0144] Among them, the medical imaging images include multiple organ images of biological objects.

[0145] The above-mentioned medical imaging images may refer to images of multiple organs of biological objects obtained by using various medical imaging technologies. The image type of the medical imaging images may be determined according to actual needs and is not limited here. In an optional embodiment, a biological object may be scanned by a medical imaging device, such as a CT scanner, to obtain an image containing multiple organs of the biological object. CT scanning is capable of obtaining cross-sectional images of the inside of a biological object at different angles, showing the structures of various tissues and organs. When performing an electronic computer tomography scan, a cross-sectional scan of a patient may be performed by a medical imaging device scanner to generate a series of images. The images may be grayscale images, which may show different tissues and organs inside the biological object. Each organ has a specific shape, density and feature in the image. By analyzing these features, segmentation and recognition of different organs may be achieved. In the process of image segmentation, the acquired CT image may be preprocessed, including operations such as noise removal and contrast enhancement, and then the image may be processed using an image segmentation algorithm to segment the image into different regions or pixels, thereby achieving recognition and segmentation of different organs. The image segmentation algorithm may include methods such as threshold segmentation, edge detection, and region growing. By acquiring a medical image and segmenting the image, it may help accurately identify various organs inside the biological object, facilitating subsequent image segmentation processing of the medical image.

[0146] Step S1004, using the pre-trained segmentation model inserted with different low-rank adapter sets, segment the medical image separately to obtain segmentation results corresponding to different low-rank adapter sets.

[0147] Among them, the low-rank adapters included in different low-rank adapter sets are obtained by continuously learning the pre-trained segmentation model with the initial adapter inserted using different training data sets.

[0148] In an optional embodiment, a pre-trained segmentation model with different low-rank adapter sets inserted can be used. The pre-trained segmentation model is obtained by training on a large data set, and some common features and rules can be learned. The pre-trained segmentation model can be determined first. The pre-trained segmentation model has been trained on a certain task and has learned some common features; then, in order to adapt to specific medical images, different low-rank adapter sets can be inserted. The low-rank adapter can be obtained by continuously learning the pre-trained model on different training data sets. The model can be fine-tuned on a specific task to make the model better adapt to the characteristics and rules of the task; by using different low-rank adapter sets, different segmentation results can be obtained. Different segmentation results correspond to different training data sets and learning processes. By comparing different segmentation results, the low-rank adapter set that best suits the specific task can be selected, thereby improving the performance and accuracy of the segmentation model on medical images. The knowledge of the pre-trained model can be effectively utilized, and it can also adapt to the needs of specific tasks through transfer learning, thereby improving the effect of image segmentation.

[0149] Step S1006, merging the segmentation results corresponding to different low-rank adapter sets to obtain the organ image segmentation result of the medical image.

[0150] In an optional embodiment, a low-rank adapter can be used to segment medical images. The low-rank adapter can reduce the dimensionality of the input multi-dimensional data and extract key features to achieve the purpose of image segmentation; when performing organ segmentation, multiple low-rank adapters can be used to segment different organs respectively, and each low-rank adapter will generate a corresponding segmentation result, that is, the pixels in the image are assigned to different organ categories. These segmentation results may have certain errors or incompleteness; in order to improve the accuracy and completeness of the segmentation results, the segmentation results corresponding to different low-rank adapter sets can be merged. This means that multiple segmentation results are fused together to obtain more comprehensive and accurate organ segmentation results. By merging the segmentation results of different adapters, their respective deficiencies can be compensated and the overall segmentation accuracy can be improved; by merging the segmentation results corresponding to different low-rank adapter sets, the organ image segmentation results of the medical image can be obtained. This merging process usually undergoes certain algorithm processing, such as pixel-level weighted averaging or voting mechanism, to ensure that the final segmentation result is accurate and reliable, which can effectively improve the quality and reliability of the segmentation result.

[0151] According to an embodiment of the present application, a medical image segmentation system for implementing the above-mentioned medical image segmentation method is also provided. Fig.11 is a schematic diagram of a medical image segmentation system according to an embodiment of the present application, such as Fig.11As shown, the system includes: a medical imaging device 1102 and a processing device 1104 .

[0152] Among them, the medical imaging device 1102 is used to collect medical imaging images, wherein the medical imaging images include multiple organ images of biological objects; the processing device 1104 is connected to the medical imaging device, and is used to use a pre-trained segmentation model inserted with different low-rank adapter sets to segment the medical imaging images respectively, obtain segmentation results corresponding to different low-rank adapter sets, and merge the segmentation results corresponding to different low-rank adapter sets to obtain organ image segmentation results of the medical imaging images, wherein the low-rank adapters included in the different low-rank adapter sets are obtained by continuously learning the pre-trained segmentation model inserted with the initial adapter using different training data sets.

[0153] The above-mentioned medical imaging equipment may refer to equipment used to acquire medical imaging images, which may include but are not limited to X-ray machines, CT scanners, MRI magnetic resonance imaging devices, ultrasonic equipment and nuclear medicine equipment, etc. The medical imaging equipment can be determined according to actual needs and is not limited here. The medical imaging equipment can provide doctors with clear organ images and can be used to obtain high-quality organ images to support the research and practice of image segmentation algorithms.

[0154] The above-mentioned processing device may refer to a computer or other device used to perform organ image segmentation processing on medical images. The processing device may be determined according to actual needs and is not limited here.

[0155] According to an embodiment of the present application, an image segmentation method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0156] Fig.12 is a flowchart of an optional image segmentation method according to an embodiment of the present application. Fig.12 As shown, the following steps may be specifically included:

[0157] Step S1202, in response to an input instruction on the operation interface, displaying the image to be processed on the operation interface.

[0158] The images to be processed include multiple organ images of biological objects.

[0159] In an optional embodiment, responding to an input instruction on the operation interface means that the system receives an instruction input by a user on the operation interface, and can execute corresponding functions according to the user's operation. The user can interact with the system through buttons, sliders, menus, etc. on the operation interface to control the behavior of the system, and can display images to be processed on the operation interface according to the user's instructions. These images to be processed contain multiple organ images of biological objects. These images can be medical images, such as CT, MRI, etc., which contain image information of different organs inside the body of the biological object; displaying the images to be processed allows users to observe and analyze the images, helping users to understand the content and structure of the images. Users can use tools on the operation interface to zoom in, zoom out, move, etc. on the images to better view the detailed information in the images. At the same time, users can also select different image processing algorithms and parameters to perform image segmentation, thereby achieving accurate segmentation and identification of different organs of biological objects. Responding to input instructions on the operation interface and displaying the images to be processed are important steps in image segmentation. Through the above steps, user interaction with the system can be achieved, helping users to analyze and process images, thereby achieving accurate segmentation of organs of biological objects.

[0160] Step S1204, in response to the image segmentation instruction applied on the operation interface, the target segmentation result of the image to be processed is displayed on the operation interface.

[0161] Among them, the target segmentation result is used to characterize the positions of multiple organ images in the image to be processed. The target segmentation result is the result of merging the segmentation results corresponding to different low-rank adapter sets. The segmentation results corresponding to different low-rank adapter sets are obtained by using pre-trained segmentation models inserted with different low-rank adapter sets to segment the images to be processed respectively. The low-rank adapters contained in different low-rank adapter sets are obtained by continuously learning the pre-trained segmentation models inserted with the initial adapter using different training data sets.

[0162] In an optional embodiment, on the image segmentation operation interface, the user can perform target segmentation on the image to be processed by responding to instructions, and the target segmentation results are displayed on the operation interface. The target segmentation results are used to characterize the positions of multiple organs in the image to be processed. These results are obtained by merging the segmentation results corresponding to different low-rank adapter sets. Specifically, the segmentation results corresponding to different low-rank adapter sets are obtained by inserting pre-trained segmentation models of different low-rank adapter sets and segmenting the image to be processed respectively. The low-rank adapters contained in each low-rank adapter set are obtained by continuously learning the pre-trained segmentation model with the initial adapter inserted using different training data sets. The image to be processed is segmented using different low-rank adapter sets, and then these different segmentation results are merged to obtain more accurate organ segmentation results. This method can improve the accuracy and robustness of segmentation, making the organ segmentation of medical images more accurate and reliable.

[0163] According to an embodiment of the present application, an image segmentation method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0164] Fig.13 is a flowchart of another optional image segmentation method according to an embodiment of the present application. Fig.13 As shown, the following steps may be specifically included:

[0165] Step S1302: Acquire the image to be processed by calling the first interface.

[0166] The first interface includes a first parameter, a parameter value of the first parameter includes an image to be processed, and the image to be processed includes multiple organ images of a biological object.

[0167] In an optional embodiment, obtaining the image data to be processed can be achieved by calling the first interface, the first interface can include a first parameter, and the parameter value of the first parameter includes the image to be processed. The image to be processed usually includes images of multiple organs of a biological object, such as different organs in medical images such as the heart, lungs, liver, etc.; after obtaining the image to be processed by calling the first interface, the image segmentation process can be started. Image segmentation is the process of segmenting an image into regions or objects with independent meanings. For medical images, it is mainly to segment different organs or tissues in the image. In the organ segmentation of medical images, various image processing algorithms and techniques, such as threshold segmentation, edge detection, region growth, etc., can be used to process and segment the image. By properly processing and analyzing the image to be processed, the contours and boundaries of each organ can be accurately extracted, thereby realizing the segmentation and identification of different organs in the image. Acquiring the image to be processed by calling the first interface is a very important step in the image segmentation process. It lays the foundation for subsequent image processing and segmentation work and provides an effective data source for organ segmentation of medical images.

[0168] Step S1304, using the pre-trained segmentation model into which different low-rank adapter sets are inserted, the image to be processed is segmented respectively to obtain segmentation results corresponding to different low-rank adapter sets.

[0169] Among them, the low-rank adapters included in different low-rank adapter sets are obtained by continuously learning the pre-trained segmentation model with the initial adapter inserted using different training data sets.

[0170] In an optional embodiment, segmentation using a pre-trained segmentation model with different low-rank adapter sets inserted can improve the performance of the model on new data, especially in the organ segmentation task of medical images. Since the data set is usually small and the annotation cost is high, transfer learning can significantly reduce data requirements and annotation burden; since the pre-trained segmentation model is trained on a large-scale data set and learns a general feature representation, different low-rank adapter sets are inserted, and the adapters are obtained by further training the pre-trained model with the initial adapter inserted on different training data sets; by continuous learning on different data sets, the adapter set can learn the feature representation of a specific data set, thereby improving the performance of the model on the data set, and these different low-rank adapter sets can be used to segment the processed image to obtain multiple segmentation results, each result corresponding to an adapter set; by comparing the segmentation results of different adapter sets, we can select one or more adapter sets or fuse multiple results to obtain more accurate segmentation results, which can effectively improve the organ segmentation performance of medical images and have strong generalization ability.

[0171] Step S1306, merging the segmentation results corresponding to different low-rank adapter sets to obtain a target segmentation result of the image to be processed.

[0172] The target segmentation result is used to characterize the positions of multiple organ images in the image to be processed.

[0173] In an optional embodiment, when performing organ segmentation, a low-rank adapter set method can be used to implement it. The low-rank adapter set is a method based on machine learning. By learning the sample features in the training data set, a segmentation model suitable for different scenes can be generated. The image to be processed can be segmented to obtain segmentation results corresponding to different low-rank adapter sets. The segmentation results corresponding to each low-rank adapter set may have certain errors or incompleteness, but by combining multiple different segmentation results, a more accurate and complete target segmentation result can be obtained; the segmentation results corresponding to different low-rank adapter sets can be merged by voting or weighted averaging. By counting the number of times or weights that each pixel is marked as an organ in different segmentation results, a comprehensive target segmentation result can be obtained in the end, which can better characterize the positions of multiple organs in the image to be processed, improve the accuracy and robustness of the segmentation results, and by combining the results of multiple segmentation models, the limitations of a single model can be compensated and the accuracy of organ segmentation can be improved.

[0174] Step S1308: output the target segmentation result by calling the second interface.

[0175] The second interface includes a second parameter, and a parameter value of the second parameter includes a target segmentation result.

[0176] In an optional embodiment, deep learning and other technologies can be used to implement image segmentation. During this process, the second interface can be called to output the target segmentation result. Specifically, the second interface can include a second parameter, and the parameter value of this parameter includes the target segmentation result. The medical image can be input into the image segmentation system. After a series of processing and calculations, the image is analyzed and learned through deep learning and other technologies to accurately identify the target organ or structure in the medical image; after completing the segmentation and labeling, the system will call the second interface to output the target segmentation result; the second interface will return a parameter value containing the target segmentation result, which is usually a binary image containing the target organ or structure or a segmented contour image; the doctor can conduct further research by viewing this output result. Calling the second interface to output the target segmentation result is a very important step in the field of image segmentation, which improves the analysis and application efficiency of medical images.

[0177] The embodiment of the present application may provide a computer terminal, which may be any computer terminal in the electronic device group. Optionally, in this embodiment, the computer terminal may also be replaced by a terminal device such as a mobile terminal.

[0178] Optionally, in this embodiment, the computer terminal may be located in at least one network device among a plurality of network devices of the computer network.

[0179] In this embodiment, the computer terminal can execute the program code in the method.

[0180] Optionally, Fig.14 is a structural block diagram of a computer terminal according to an embodiment of the present application, such as Fig.14 As shown, the computer terminal A may include: one or more (only one is shown in the figure) processors 1402, a memory 1404, a storage controller, and a peripheral interface, wherein the peripheral interface is connected to a radio frequency module, an audio module, and a display.

[0181] Among them, the memory can be used to store software programs and modules, such as program instructions / modules corresponding to the methods and devices in the embodiments of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, implementing the methods in the above embodiments. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the terminal A via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0182] The processor can call the information and application programs stored in the memory through the transmission device to perform the following steps: obtaining an image to be processed, wherein the image to be processed includes multiple organ images of a biological object; using a pre-trained segmentation model inserted with different low-rank adapter sets to segment the image to be processed respectively, and obtaining segmentation results corresponding to different low-rank adapter sets, wherein the low-rank adapters included in the different low-rank adapter sets are obtained by continuously learning the pre-trained segmentation model inserted with the initial adapter using different training data sets; merging the segmentation results corresponding to the different low-rank adapter sets to obtain a target segmentation result of the image to be processed, wherein the target segmentation result is used to characterize the positions of the multiple organ images in the image to be processed.

[0183] Optionally, an input module, an encoder with a first low-rank adapter inserted, a decoder with a second low-rank adapter inserted, and an output module; the above-mentioned processor can also execute the program code of the following steps: using the input module to extract features of the image to be processed to obtain the original image features of the image to be processed; using the encoder with the first low-rank adapter inserted to encode the original image features to obtain the first image features; using the decoder with the second low-rank adapter inserted to decode the first image features to obtain the second image features; using the output module to reconstruct the second image features to obtain the segmentation result corresponding to any low-rank adapter.

[0184] Optionally, multiple encoder modules are connected in sequence, and the encoder module includes: a first embedding layer with a first sub-adapter inserted and multiple first transformation layers with second sub-adapters inserted, wherein the first low-rank adapter includes the first sub-adapter and the second sub-adapter; the above-mentioned processor can also execute the program code of the following steps: using the first embedding layer with the first sub-adapter inserted in the current encoder module, processing the features input by the current encoder module to obtain a first embedded feature, wherein, when the current encoder module is the first encoder module among the multiple encoder modules, the features input by the current encoder module are the original image features, and when the current encoder module is not the first encoder module, the features input by the current encoder module are the previous A feature output by an encoder module; using any first transformation layer in the current encoder module with a second sub-adapter inserted therein, transforming the feature input to the first transformation layer to obtain the feature output by the first transformation layer, wherein, when the first transformation layer is the first first transformation layer, the feature input to the first transformation layer is the first embedded feature, when the first transformation layer is not the last first transformation layer, the feature output by the first transformation layer is used as the feature input to the next first transformation layer, and when the first transformation layer is the last first transformation layer, the feature output by the first transformation layer is used as the feature output by the current encoder module; determining the feature output by the last encoder module among multiple encoder modules as the first image feature.

[0185] Optionally, the first embedding layer includes: a first convolutional layer, a first normalization layer, and a first sub-adapter inserted into the first convolutional layer; the above-mentioned processor can also execute the program code of the following steps: merging the parameters of the first convolutional layer and the parameters of the first sub-adapter to obtain first merged parameters; using the first merged parameters to convolve the features input by the current encoder module to obtain first convolution features; using the first normalization layer to normalize the first convolution features to obtain first embedded features.

[0186] Optionally, the feature input by the current encoder module includes multiple sub-features, and the multiple sub-features are used to characterize multiple image blocks obtained after the image to be processed is divided into blocks; the above-mentioned processor can also execute the program code of the following steps: using the first merging parameter to perform convolution processing on the multiple sub-features respectively to obtain multiple sub-convolution features, wherein the first convolution feature includes multiple sub-convolution features; using the first normalization layer to normalize the first convolution feature to obtain a first embedded feature, including: using the first normalization layer to normalize the multiple sub-convolution features respectively to obtain multiple sub-normalized features; summing the multiple sub-normalized features with the multiple position features respectively to obtain the first embedded feature, wherein the multiple position features are used to characterize the positions of the multiple image blocks in the image to be processed.

[0187] Optionally, the first transformation layer includes: an attention layer inserted with a first adapter parameter and a forward feedback layer inserted with a second adapter parameter; the above-mentioned processor can also execute the program code of the following steps: using the attention layer inserted with the first adapter parameter, performing attention processing on the features input to the first transformation layer to obtain attention features; summing the features input to the first transformation layer and the attention features to obtain merged features; using the forward feedback layer inserted with the second adapter parameter to process the merged features to obtain the features output by the first transformation layer.

[0188] Optionally, the processor may also execute program code for the following steps: determining a query vector, a key vector and a value vector based on the features input by the first transformation layer; merging the weight parameters of the query vector and the query parameters in the first adapter parameters to obtain query merging parameters, and merging the weight parameters of the value vector and the value parameters in the first adapter parameters to obtain value merging parameters; determining attention features based on the query merging parameters, the value merging parameters, the weight parameters of the key vector, the query vector, the key vector and the value vector.

[0189] Optionally, the forward feedback layer into which the second adapter parameters are inserted includes: a second normalization layer, at least one linear layer and a second adapter parameter inserted into at least one linear layer; the processor may also execute the following program code: normalizing the merged features using the second normalization layer to obtain normalized features; merging the second adapter parameters and the parameters of at least one linear layer to obtain at least one second merged parameter; performing linear processing on the normalized features using at least one second merged parameter to obtain linear features; and summing the merged features and the linear features to obtain features output by the first transformation layer.

[0190] Optionally, the decoder with the second low-rank adapter inserted includes: a first decoding layer and multiple decoder modules connected in sequence, the decoder module includes: multiple second transform layers and second decoding layers with the second low-rank adapter inserted; the above-mentioned processor can also execute the program code of the following steps: using the first decoding layer to decode the first image feature to obtain the first decoding feature; using any second transform layer in the current decoder module with the second low-rank adapter inserted to transform the feature input to the second transform layer to obtain the feature output by the second transform layer, wherein, when the current decoder module is the first decoder module among multiple decoder modules and the second transform layer is the first second transform layer, the feature input to the first transform layer is the first decoding feature, and when the second transform layer is not the last second transform layer, the feature output by the second transform layer is used as the feature input to the next second transform layer; using the second decoding layer in the current decoder module to transform the feature output by the last first transform layer to obtain the feature output by the current decoder module; determining that the feature output by the last decoder module among multiple decoder modules is the second image feature.

[0191] Optionally, the input module includes: a second convolutional layer, a third normalization layer, and a third low-rank adapter inserted into the second convolutional layer; the above-mentioned processor can also execute the program code of the following steps: merging the parameters of the second convolutional layer and the parameters of the third low-rank adapter to obtain input merged parameters; using the input merged parameters to perform convolution processing on the processed image to obtain second convolution features; using the third normalization layer to normalize the second convolution features to obtain original image features.

[0192] Optionally, the output module includes: a third convolutional layer and a target segmentation output layer, wherein the target segmentation output layer is used to segment the training data set corresponding to any low-rank adapter set; the above-mentioned processor can also execute the program code of the following steps: using the third convolutional layer to convolve the second image feature to obtain a third convolutional feature; using the target segmentation output layer to reconstruct the third convolutional feature to obtain a segmentation result corresponding to any low-rank adapter.

[0193] Optionally, the pre-trained segmentation model is obtained by training the pre-trained model using the initial training data, and different training data sets are prohibited from being accessed after continuous learning of the pre-trained segmentation model with the initial adapter inserted.

[0194] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0195] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0196] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, disk, CD), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0197] The embodiment of the present application further provides a computer-readable storage medium. Optionally, in this embodiment, the computer-readable storage medium can be used to store the program code executed by the method provided in the above embodiment.

[0198] Optionally, in this embodiment, the above storage medium may be located in any electronic device in a group of electronic devices in a computer network, or in any mobile terminal in a group of mobile terminals.

[0199] Optionally, in this embodiment, a computer-readable storage medium is configured to store program codes for executing the following steps: obtaining an image to be processed, wherein the image to be processed contains multiple organ images of a biological object; using a pre-trained segmentation model into which different low-rank adapter sets are inserted, respectively segmenting the image to be processed to obtain segmentation results corresponding to different low-rank adapter sets, wherein the low-rank adapters contained in the different low-rank adapter sets are obtained by continuously learning the pre-trained segmentation model into which the initial adapter is inserted using different training data sets; merging the segmentation results corresponding to the different low-rank adapter sets to obtain a target segmentation result of the image to be processed, wherein the target segmentation result is used to characterize the positions of the multiple organ images in the image to be processed.

[0200] Optionally, an input module, an encoder with a first low-rank adapter inserted, a decoder with a second low-rank adapter inserted, and an output module; the computer-readable storage medium is also configured to store program codes for executing the following steps: using the input module to extract features of the image to be processed to obtain the original image features of the image to be processed; using the encoder with the first low-rank adapter inserted to encode the original image features to obtain first image features; using the decoder with the second low-rank adapter inserted to decode the first image features to obtain second image features; using the output module to reconstruct the second image features to obtain the segmentation result corresponding to any low-rank adapter.

[0201] Optionally, multiple encoder modules are connected in sequence, and the encoder modules include: a first embedding layer with a first sub-adapter inserted and multiple first transformation layers with second sub-adapters inserted, wherein the first low-rank adapter includes the first sub-adapter and the second sub-adapter; the computer-readable storage medium is also configured to store program code for performing the following steps: using the first embedding layer with the first sub-adapter inserted in the current encoder module, processing the features input to the current encoder module to obtain a first embedded feature, wherein, when the current encoder module is the first encoder module among the multiple encoder modules, the features input to the current encoder module are original image features, and when the current encoder module is not the first encoder module, the features input to the current encoder module are original image features. The feature is the feature output by the previous encoder module; using any first transformation layer in the current encoder module with the second sub-adapter inserted, the feature input to the first transformation layer is transformed to obtain the feature output by the first transformation layer, wherein, when the first transformation layer is the first first transformation layer, the feature input to the first transformation layer is the first embedded feature, when the first transformation layer is not the last first transformation layer, the feature output by the first transformation layer is used as the feature input to the next first transformation layer, and when the first transformation layer is the last first transformation layer, the feature output by the first transformation layer is used as the feature output by the current encoder module; determine that the feature output by the last encoder module among multiple encoder modules is the first image feature.

[0202] Optionally, the first embedding layer includes: a first convolutional layer, a first normalization layer, and a first sub-adapter inserted into the first convolutional layer; the computer-readable storage medium is also configured to store program code for performing the following steps: merging the parameters of the first convolutional layer and the parameters of the first sub-adapter to obtain first merged parameters; convolving the features input by the current encoder module using the first merged parameters to obtain first convolutional features; and normalizing the first convolutional features using the first normalization layer to obtain first embedded features.

[0203] Optionally, the feature input by the current encoder module includes multiple sub-features, and the multiple sub-features are used to characterize multiple image blocks obtained after the image to be processed is divided into blocks; the computer-readable storage medium is also configured to store program codes for executing the following steps: using a first merging parameter to perform convolution processing on multiple sub-features respectively to obtain multiple sub-convolution features, wherein the first convolution feature includes multiple sub-convolution features; using a first normalization layer to normalize the first convolution feature to obtain a first embedded feature, including: using a first normalization layer to normalize multiple sub-convolution features respectively to obtain multiple sub-normalized features; summing the multiple sub-normalized features with multiple position features respectively to obtain a first embedded feature, wherein the multiple position features are used to characterize the positions of multiple image blocks in the image to be processed.

[0204] Optionally, the first transformation layer includes: an attention layer inserted with a first adapter parameter and a forward feedback layer inserted with a second adapter parameter; the computer-readable storage medium is also configured to store program codes for executing the following steps: using the attention layer inserted with the first adapter parameter, performing attention processing on the features input to the first transformation layer to obtain attention features; summing the features input to the first transformation layer and the attention features to obtain merged features; processing the merged features using the forward feedback layer inserted with the second adapter parameter to obtain the features output by the first transformation layer.

[0205] Optionally, the computer-readable storage medium is also configured to store program code for performing the following steps: determining a query vector, a key vector and a value vector based on features input by the first transformation layer; merging the weight parameters of the query vector and the query parameters in the first adapter parameters to obtain query merging parameters, and merging the weight parameters of the value vector and the value parameters in the first adapter parameters to obtain value merging parameters; determining attention features based on the query merging parameters, the value merging parameters, the weight parameters of the key vector, the query vector, the key vector and the value vector.

[0206] Optionally, the forward feedback layer into which the second adapter parameters are inserted comprises: a second normalization layer, at least one linear layer and a second adapter parameter inserted into at least one linear layer; the computer-readable storage medium is also configured to store program codes for executing the following steps: normalizing the merged features using the second normalization layer to obtain normalized features; merging the second adapter parameters and the parameters of at least one linear layer to obtain at least one second merged parameter; linearly processing the normalized features using at least one second merged parameter to obtain linear features; and summing the merged features and the linear features to obtain the features output by the first transformation layer.

[0207] Optionally, a decoder with a second low-rank adapter inserted includes: a first decoding layer and multiple decoder modules connected in sequence, the decoder module includes: multiple second transform layers and second decoding layers with second low-rank adapters inserted; the computer-readable storage medium is also configured to store program codes for executing the following steps: using the first decoding layer to decode the first image feature to obtain the first decoding feature; using any one of the second transform layers in the current decoder module with the second low-rank adapter inserted to transform the feature input to the second transform layer to obtain the feature output by the second transform layer, wherein, when the current decoder module is the first decoder module among multiple decoder modules and the second transform layer is the first second transform layer, the feature input to the first transform layer is the first decoding feature, and when the second transform layer is not the last second transform layer, the feature output by the second transform layer is used as the feature input to the next second transform layer; using the second decoding layer in the current decoder module to transform the feature output by the last first transform layer to obtain the feature output by the current decoder module; determining that the feature output by the last decoder module among multiple decoder modules is the second image feature.

[0208] Optionally, the input module includes: a second convolutional layer, a third normalization layer, and a third low-rank adapter inserted into the second convolutional layer; the computer-readable storage medium is also configured to store program codes for performing the following steps: merging the parameters of the second convolutional layer and the parameters of the third low-rank adapter to obtain input merged parameters; convolving the image to be processed using the input merged parameters to obtain second convolutional features; and normalizing the second convolutional features using the third normalization layer to obtain original image features.

[0209] Optionally, the output module includes: a third convolutional layer and a target segmentation output layer, wherein the target segmentation output layer is used to segment the training data set corresponding to any low-rank adapter set; the computer-readable storage medium is also configured to store program codes for executing the following steps: using the third convolutional layer to convolve the second image feature to obtain a third convolutional feature; using the target segmentation output layer to reconstruct the third convolutional feature to obtain a segmentation result corresponding to any low-rank adapter.

[0210] Optionally, the pre-trained segmentation model is obtained by training the pre-trained model using the initial training data, and different training data sets are prohibited from being accessed after continuous learning of the pre-trained segmentation model with the initial adapter inserted.

[0211] The embodiment of the present application further provides a computer program product. Optionally, in this embodiment, the computer program product may include a computer program, and the computer program implements the method provided in the embodiment when executed by a processor.

[0212] The above-mentioned computer program product may refer to a software program that has been written, tested and released and can be run on a computer or other device. The computer program product may include an application, an operating system, tool software, etc., which is used to implement specific functions or solve specific problems.

[0213] The embodiments of the present application also provide a computer program product. Optionally, the computer program product may include a non-volatile computer-readable storage medium, which may be used to store a computer program, and when the computer program is executed by a processor, the method provided in the embodiments is implemented.

[0214] The above-mentioned non-volatile computer-readable storage medium may refer to a medium for storing data. The non-volatile computer-readable storage medium can keep the data from being lost when the power is off, and can be used to store long-term data, such as operating systems, applications, and user files. The non-volatile storage medium may include hard disk drives, solid-state drives, optical disks, and flash memory storage devices, etc.

[0215] The embodiment of the present application further provides a computer program. Optionally, in this embodiment, when the computer program is executed by a processor, the method provided in the above embodiment is implemented.

[0216] The above-mentioned computer program may refer to a collection of instructions used to tell a computer to perform a specific task or operation. A computer program may be written by a programmer using a specific programming language and may include algorithms, data structures, logic, and control flows. A computer program may be used for a variety of purposes, including application software, operating systems, and the like.

[0217] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0218] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0219] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0220] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0221] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk, etc., which can store program code.

[0222] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. An image segmentation method, characterized in that: include: Acquire an image to be processed, wherein the image to be processed is used to represent an image showing an organ of a biological object, and the image to be processed includes multiple organ images of the biological object; Using a pre-trained segmentation model inserted with different low-rank adapter sets, the image to be processed is segmented respectively to obtain segmentation results corresponding to the different low-rank adapter sets, wherein the low-rank adapters contained in the different low-rank adapter sets are obtained by continuously learning the pre-trained segmentation model inserted with the initial adapter using different training data sets, and the different training data sets are used to indicate that the training data sets used each time are not completely the same, and the types of organs in the training data sets at least include the types of organs that have not been segmented and processed by the initial adapter, and the types of organs that have been segmented and processed before by the initial adapter; The segmentation results corresponding to the different low-rank adapter sets are merged to obtain a target segmentation result of the image to be processed, wherein the target segmentation result is used to characterize the positions of the multiple organ images in the image to be processed.

2. The method according to claim 1, characterized in that The pre-trained segmentation model inserted with any one of the low-rank adapter sets comprises at least: an input module, an encoder inserted with a first low-rank adapter, a decoder inserted with a second low-rank adapter, and an output module; Using the pre-trained segmentation model into which any one of the low-rank adapters is inserted, segmenting the image to be processed to obtain a segmentation result corresponding to any one of the low-rank adapters, including: Using the input module to extract features of the image to be processed to obtain original image features of the image to be processed; Encoding the original image feature by using the encoder inserted with the first low-rank adapter to obtain a first image feature; Decoding the first image feature by using the decoder into which the second low-rank adapter is inserted to obtain a second image feature; The output module is used to reconstruct the second image feature to obtain a segmentation result corresponding to any one of the low-rank adapters.

3. The method according to claim 2, characterized in that The encoder with the first low-rank adapter inserted comprises: a plurality of encoder modules connected in sequence, the encoder module comprising: a first embedding layer with the first sub-adapter inserted and a plurality of first transform layers with the second sub-adapter inserted, wherein the first low-rank adapter includes the first sub-adapter and the second sub-adapter; Encoding the original image feature by using the encoder inserted with the first low-rank adapter to obtain the first image feature, comprising: Using a first embedding layer in which a first sub-adapter is inserted in a current encoder module, a feature input to the current encoder module is processed to obtain a first embedding feature, wherein when the current encoder module is the first encoder module among the multiple encoder modules, the feature input to the current encoder module is the original image feature, and when the current encoder module is not the first encoder module, the feature input to the current encoder module is a feature output by a previous encoder module; Using any one of the first transformation layers in the current encoder module into which the second sub-adapter is inserted, transforming the input features of the first transformation layer to obtain the output features of the first transformation layer, wherein, when the first transformation layer is the first first transformation layer, the input features of the first transformation layer are the first embedded features, when the first transformation layer is not the last first transformation layer, the output features of the first transformation layer are used as the input features of the next first transformation layer, and when the first transformation layer is the last first transformation layer, the output features of the first transformation layer are used as the output features of the current encoder module; Determine that a feature output by a last encoder module among the multiple encoder modules is the first image feature.

4. The method according to claim 3, characterized in that The first embedding layer includes: a first convolutional layer, a first normalization layer, and the first sub-adapter inserted into the first convolutional layer; Using a first embedding layer in which a first sub-adapter is inserted in a current encoder module, a feature input by the current encoder module is processed to obtain a first embedding feature, including: Merging the parameters of the first convolutional layer and the parameters of the first sub-adapter to obtain first merged parameters; Using the first merging parameter to perform convolution processing on the feature input by the current encoder module to obtain a first convolution feature; The first convolution feature is normalized by using the first normalization layer to obtain the first embedded feature.

5. The method according to claim 4, characterized in that The feature input by the current encoder module includes a plurality of sub-features, and the plurality of sub-features are used to represent a plurality of image blocks obtained after dividing the image to be processed into blocks; Using the first merging parameter to perform convolution processing on the feature input by the current encoder module to obtain a first convolution feature, including: using the first merging parameter to perform convolution processing on the multiple sub-features respectively to obtain multiple sub-convolution features, wherein the first convolution feature includes the multiple sub-convolution features; The first convolution feature is normalized by using the first normalization layer to obtain the first embedded feature, including: using the first normalization layer to normalize the multiple sub-convolution features respectively to obtain multiple sub-normalized features; and summing the multiple sub-normalized features with multiple position features respectively to obtain the first embedded feature, wherein the multiple position features are used to characterize the positions of the multiple image blocks in the image to be processed.

6. The method according to claim 3, characterized in that The first transformation layer includes: an attention layer inserted with a first adapter parameter and a forward feedback layer inserted with a second adapter parameter; Using any first transformation layer into which the second sub-adapter is inserted, transforming the input features of the first transformation layer to obtain the output features of the first transformation layer, including: Using the attention layer into which the first adapter parameter is inserted, attention processing is performed on the features input by the first transformation layer to obtain attention features; Summing the input feature of the first transformation layer and the attention feature to obtain a merged feature; The merged features are processed using the forward feedback layer into which the second adapter parameters are inserted to obtain the features output by the first transformation layer.

7. The method according to claim 6, characterized in that Using the attention layer into which the first adapter parameter is inserted, attention processing is performed on the features input by the first transformation layer to obtain attention features, including: Determine a query vector, a key vector, and a value vector based on the features input by the first transformation layer; Merging the weight parameter of the query vector with the query parameter in the first adapter parameter to obtain a query merge parameter, and merging the weight parameter of the value vector with the value parameter in the first adapter parameter to obtain a value merge parameter; The attention feature is determined based on the query merging parameter, the value merging parameter, a weight parameter of the key vector, the query vector, the key vector and the value vector.

8. The method according to claim 6, characterized in that The forward feedback layer with the second adapter parameters inserted therein comprises: a second normalization layer, at least one linear layer and the second adapter parameters inserted into the at least one linear layer; Processing the merged features using the forward feedback layer into which the second adapter parameters are inserted to obtain the features output by the first transformation layer includes: Using the second normalization layer to normalize the merged features to obtain normalized features; Merging the second adapter parameter and the parameter of the at least one linear layer to obtain at least one second merged parameter; Using the at least one second merging parameter, linearly process the normalized feature to obtain a linear feature; The combined feature and the linear feature are summed to obtain the feature output by the first transformation layer.

9. The method according to claim 2, characterized in that: The decoder with the second low-rank adapter inserted comprises: a first decoding layer and a plurality of decoder modules connected in sequence, wherein the decoder module comprises: a plurality of second transform layers and a second decoding layer with the second low-rank adapter inserted; Decoding the first image feature using the decoder into which the second low-rank adapter is inserted to obtain a second image feature includes: Decoding the first image feature using the first decoding layer to obtain a first decoded feature; Using any one of the second transform layers in the current decoder module into which the second low-rank adapter is inserted, transform the features input to the second transform layer to obtain the features output by the second transform layer, wherein, when the current decoder module is the first decoder module among the multiple decoder modules and the second transform layer is the first second transform layer, the features input to the second transform layer are the first decoding features, and when the second transform layer is not the last second transform layer, the features output by the second transform layer are used as the features input to the next second transform layer; Using the second decoding layer in the current decoder module to transform the features output by the last first transformation layer, to obtain the features output by the current decoder module; Determine a feature output by a last decoder module among the multiple decoder modules as the second image feature.

10. The method according to claim 2, characterized in that The input module includes: a second convolutional layer, a third normalization layer, and a third low-rank adapter inserted into the second convolutional layer; Using the input module to extract features of the image to be processed to obtain original image features of the image to be processed includes: Merging the parameters of the second convolutional layer and the parameters of the third low-rank adapter to obtain input merged parameters; Performing convolution processing on the image to be processed using the input merging parameters to obtain a second convolution feature; The second convolutional features are normalized by using the third normalization layer to obtain the original image features.

11. The method according to claim 2, characterized in that The output module includes: a third convolutional layer and a target segmentation output layer, wherein the target segmentation output layer is used to segment the training data set corresponding to any one of the low-rank adapter sets; Reconstructing the second image feature by using the output module to obtain a segmentation result corresponding to any one of the low-rank adapters includes: Using the third convolution layer to perform convolution processing on the second image feature to obtain a third convolution feature; The target segmentation output layer is used to reconstruct the third convolutional feature to obtain a segmentation result corresponding to any one of the low-rank adapters.

12. The method according to any one of claims 1 to 11, characterized in that The pre-trained segmentation model is obtained by training the pre-trained model using initial training data, and the different training data sets are prohibited from being accessed after continuous learning of the pre-trained segmentation model with the initial adapter inserted.

13. A medical image segmentation system, characterized in that: include: A medical imaging device, used for acquiring medical imaging images, wherein the medical imaging images are used to represent images showing organs of biological objects, and the medical imaging images include images of multiple organs of the biological objects; A processing device is connected to the medical imaging device, and is used to use a pre-trained segmentation model inserted with different low-rank adapter sets to segment the medical image respectively, obtain segmentation results corresponding to the different low-rank adapter sets, and merge the segmentation results corresponding to the different low-rank adapter sets to obtain organ image segmentation results of the medical image, wherein the low-rank adapters included in the different low-rank adapter sets are obtained by continuously learning the pre-trained segmentation model inserted with the initial adapter using different training data sets, and the different training data sets are used to indicate that the training data sets used each time are not exactly the same, and the types of organs in the training data sets at least include organ types that have not been segmented by the initial adapter and organ types that have been segmented before by the initial adapter.

14. A computer terminal, characterized in that: include: A memory storing an executable program; A processor, configured to run the program, wherein the program executes the method according to any one of claims 1 to 12 when running.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored executable program, wherein when the executable program is executed, the device where the computer-readable storage medium is located is controlled to execute the method according to any one of claims 1 to 12.

16. A computer program product, characterized in that The invention comprises a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 12.

Images