Image segmentation method, and electronic device and storage medium

Abstract

Disclosed in the present disclosure are an image segmentation method, and an electronic device and a storage medium. The method comprises: acquiring a first image and a second image of a target part of a biological object, wherein the target part includes a focus area, the first image is an image that is obtained by means of scanning the target part by using electronic computed tomography technology, and the second image is an image that is obtained by means of scanning the target part by using magnetic resonance technology; clipping the second image on the basis of the area of the target part in the first image, so as to obtain a clipped image; and on the basis of the first image and the clipped image, performing image segmentation on the target part included in the second image, so as to obtain a target segmentation result. The present disclosure solves the technical problem of the accuracy of segmenting a focus area in a CT image and an MRI image in the related art.

Description

Image segmentation method, electronic device and storage medium

[0001] Cross-reference

[0002] This disclosure claims priority to a Chinese patent application filed with the Patent Office of China on May 29, 2023, with priority number 202310623869.0 and invention name “Image Segmentation Method, Electronic Device and Storage Medium,” the entire contents of which are incorporated by reference into this disclosure. Technical Field

[0003] The present disclosure relates to the field of data processing, and in particular to an image segmentation method, an electronic device, and a storage medium. Background Art

[0004] Currently, CT (Computed Tomography) and MRI (Magnetic Resonance Imaging) images of biological subjects can be registered, and then the lesion area in the images can be segmented based on the registration results. However, the current conditions for registering CT and MRI images are relatively difficult, resulting in large errors in the registration results, which in turn leads to low accuracy in segmenting the lesion area based on the registration results.

[0005] To address the above-mentioned problems, no effective solutions have been proposed so far.

[0006] Summary of the Invention

[0007] The embodiments of the present disclosure provide an image segmentation method, an electronic device, and a storage medium to at least solve the technical problem of low accuracy in segmenting lesion areas in CT images and MRI images in related technologies.

[0008] According to one aspect of an embodiment of the present disclosure, an image segmentation method is provided, comprising: acquiring a first image and a second image of a target part of a biological object, wherein the target part includes a lesion area, the first image is an image obtained by scanning the target part using electronic computed tomography technology, and the second image is an image obtained by scanning the target part using magnetic resonance technology; cropping the second image based on an area of ​​the target part in the first image to obtain a cropped image; and performing image segmentation on the target part included in the second image based on the first image and the cropped image to obtain a target segmentation result.

[0009] According to one aspect of an embodiment of the present disclosure, there is provided an image segmentation method, comprising: in response to an input instruction set as an action on an operation interface, displaying a first image and a second image of a target part of a biological object on the operation interface, wherein the target part includes a lesion area, the first image is an image obtained by scanning the target part using electronic computed tomography technology, and the second image is an image obtained by scanning the target part using magnetic resonance technology; in response to an image segmentation instruction set as the action on the operation interface, displaying a target segmentation result on the operation interface, wherein the target segmentation result is obtained by performing image segmentation on the target part included in the second image based on the first image and a cropped image, and the cropped image is obtained by cropping the second image based on an area of ​​the target part in the first image.

[0010] According to one aspect of an embodiment of the present disclosure, an image segmentation method is provided, comprising: displaying a first image and a second image of a target part of a biological object on a presentation screen of a virtual reality (VR) device or an augmented reality (AR) device, wherein the target part includes a lesion area, the first image is an image obtained by scanning the target part using electronic computed tomography technology, and the second image is an image obtained by scanning the target part using magnetic resonance technology; cropping the second image based on an area of ​​the target part in the first image to obtain a cropped image; performing image segmentation on the target part included in the second image based on the first image and the cropped image to obtain a target segmentation result; and driving the VR device or AR device to render and display the target segmentation result.

[0011] According to one aspect of an embodiment of the present disclosure, an image segmentation method is provided, comprising: acquiring a first image and a second image of a target part of a biological object by calling a first interface, wherein the first interface includes a first parameter, a parameter value of the first parameter is the first image and the second image, the target part includes a lesion area, the first image is an image obtained by scanning the target part using electronic computed tomography technology, and the second image is an image obtained by scanning the target part using magnetic resonance technology; based on the area of ​​the target part in the first image, the second image is cropped to obtain a cropped image; based on the first image and the cropped image, the target part included in the second image is image segmented to obtain a target segmentation result; and outputting the target segmentation result by calling a second interface, wherein the second interface includes a second parameter, and the parameter value of the second parameter is the target segmentation result.

[0012] According to another aspect of an embodiment of the present disclosure, an electronic device is provided, including: a memory storing an executable program; and a processor configured to run the program, wherein the program executes any one of the above-mentioned image segmentation methods when running.

[0013] According to another aspect of an embodiment of the present disclosure, a computer-readable storage medium is further provided, which includes a stored executable program, wherein when the executable program runs, the device where the computer-readable storage medium is located is controlled to execute any one of the above-mentioned image segmentation methods.

[0014] In an embodiment of the present disclosure, a method is adopted in which a first image and a second image of a target part of a biological object are acquired; the second image is cropped based on the area of ​​the target part in the first image to obtain a cropped image; and the target part contained in the second image is segmented based on the first image and the cropped image to obtain a target segmentation result. The second image is cropped by using the area of ​​the target part in the first image to determine the cropped image of the target part in the second image, so as to reduce the influence of other images other than the cropped image in the second image on the process of segmenting the lesion area. Finally, the first image and the cropped image are used to segment the second image to obtain the target segmentation result, thereby improving the clarity of local details in the lesion area and making the segmented target segmentation result undistorted, thereby improving the segmentation accuracy of the lesion area, and further solving the technical problem of the accuracy of segmenting the lesion area in CT images and MRI images in the related art.

[0015] It is easy to note that the above general description and the following detailed description are only for the purpose of exemplifying and explaining the present disclosure, and do not constitute a limitation of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings described herein are used to provide a further understanding of the present disclosure and constitute a part of the present disclosure. The exemplary embodiments of the present disclosure and their descriptions are provided to explain the present disclosure and do not constitute an improper limitation of the present disclosure. In the drawings:

[0017] FIG1 is a schematic diagram of a hardware environment of a virtual reality device according to an image segmentation method according to Example 1 of the present disclosure;

[0018] FIG2 is a structural block diagram of a computing environment of an image segmentation method according to Embodiment 1 of the present disclosure;

[0019] FIG3 is a flow chart of an image segmentation method according to Embodiment 1 of the present disclosure;

[0020] FIG4 is a schematic diagram of a candidate segmentation result determination process according to Embodiment 1 of the present disclosure;

[0021] FIG5 is a schematic diagram of a target segmentation result determination process according to Example 1 of the present disclosure;

[0022] FIG6 is a schematic diagram of a segmentation result comparison according to Example 1 of the present disclosure;

[0023] FIG7 is a flowchart of an image segmentation method according to Embodiment 2 of the present disclosure;

[0024] FIG8 is a flowchart of an image segmentation method according to Embodiment 3 of the present disclosure;

[0025] FIG9 is a flowchart of an image segmentation method according to Embodiment 4 of the present disclosure;

[0026] FIG10 is a structural block diagram of an image segmentation apparatus according to Embodiment 5 of the present disclosure;

[0027] FIG11 is a structural block diagram of an image segmentation apparatus according to Embodiment 6 of the present disclosure;

[0028] FIG12 is a structural block diagram of an image segmentation apparatus according to Embodiment 7 of the present disclosure;

[0029] FIG13 is a structural block diagram of an image segmentation apparatus according to Embodiment 8 of the present disclosure;

[0030] FIG14 is a structural block diagram of an electronic device according to Embodiment 9 of the present disclosure. DETAILED DESCRIPTION

[0031] In order to enable those skilled in the art to better understand the solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the embodiments described are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present disclosure.

[0032] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are set to distinguish similar objects, and are not necessarily set to describe a specific order or sequence. It should be understood that the numbers used in this way are interchangeable where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0033] First, some nouns or terms that appear in the description of the embodiments of the present disclosure are explained as follows:

[0034] Probabilistic U-Net model: Probabilistic U-Net is a model that processes images by gradually reducing and then increasing the image resolution. Compared with the traditional U-Net model, the present disclosure adds a new encoder to the model that is configured to modify and intervene in the image. Through this encoder, differences can be made between each processed image, thereby making the lesion area segmented using the image more accurate. The modification and intervention operations may include but are not limited to user data enhancement operations such as scaling, rotation, mirroring, and gamma transformation.

[0035] Magnetic Resonance Imaging (MRI) is a technique that effectively scans and images diseased organs or tissues at target locations in biological subjects, clearly demonstrating the current state of the affected area. The grayscale value of an MRI image captured using MRI technology represents MRI signal intensity, which reflects the length of relaxation time.

[0036] Computed tomography (CT) can also scan and image diseased organs or tissues, clearly demonstrating the current state of the affected area. Unlike magnetic resonance imaging (MRI), the grayscale values ​​of CT images typically indicate the density of tissues, organs, and lesions.

[0037] Example 1

[0038] According to an embodiment of the present disclosure, an image segmentation method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0039] FIG1 is a schematic diagram of the hardware environment of a virtual reality device for an image segmentation method according to Example 1 of the present disclosure. As shown in FIG1 , virtual reality device 104 is connected to terminal 106, which is connected to server 102 via a network. Virtual reality device 104 is not limited to virtual reality helmets, virtual reality glasses, or all-in-one virtual reality devices. Terminal 104 is not limited to PCs, mobile phones, or tablet computers. Server 102 can be a server corresponding to a media file operator. The network includes, but is not limited to, a wide area network, a metropolitan area network, or a local area network.

[0040] Optionally, the virtual reality device 104 of this embodiment includes: a memory, a processor, and a transmission device. The memory is configured to store an application, which can be configured to execute the following steps: acquiring a first image and a second image of a target portion of a biological object, wherein the target portion includes a lesion area, the first image being an image obtained by scanning the target portion using computed tomography technology, and the second image being an image obtained by scanning the target portion using magnetic resonance imaging technology; cropping the second image based on the area of ​​the target portion in the first image to obtain a cropped image; and performing image segmentation of the target portion included in the second image based on the first image and the cropped image to obtain a target segmentation result. This solves the technical problem of accurately segmenting lesion areas in CT and MRI images in related technologies, achieving the goal of accurately identifying lesion areas in images.

[0041] The terminal of this embodiment can be configured to display a first image and a second image of a target part of a biological object on a presentation screen of a virtual reality (VR) device or an augmented reality (AR) device, wherein the target part includes a lesion area, the first image is an image obtained by scanning the target part using electronic computed tomography technology, and the second image is an image obtained by scanning the target part using magnetic resonance technology. The terminal also sends a cropped image obtained by cropping the second image based on the area of ​​the target part in the first image, and a target segmentation result obtained by image segmentation of the target part included in the second image based on the first image and the cropped image to the virtual reality device 104. After receiving the cropped image and the target segmentation result, the virtual reality device 104 displays the cropped image and the target segmentation result at the target delivery position.

[0042] Optionally, the virtual reality device 104 of this embodiment includes an eye-tracking HMD (Head Mount Display) and an eye-tracking module that function similarly to those in the aforementioned embodiment. Specifically, the screen in the HMD is configured to display real-time images, and the eye-tracking module in the HMD is configured to obtain the real-time trajectory of the user's eye movements. The terminal of this embodiment obtains the user's position and movement information in real three-dimensional space through a tracking system, and calculates the three-dimensional coordinates of the user's head in virtual three-dimensional space, as well as the user's field of view in the virtual three-dimensional space.

[0043] The hardware structure block diagram shown in FIG1 can serve not only as an exemplary block diagram of the AR / VR device (or mobile device) described above, but also as an exemplary block diagram of the server described above. In an optional embodiment, FIG2 shows, as a block diagram, an embodiment using the AR / VR device (or mobile device) shown in FIG1 as a computing node in a computing environment 201. FIG2 is a structural block diagram of a computing environment for an image segmentation method according to Example 1 of the present disclosure. As shown in FIG2, the computing environment 201 includes multiple computing nodes (such as servers) (illustrated as 210-1, 210-2, ...) running on a distributed network. Different computing nodes include local processing and memory resources, and end users 202 can remotely run applications or store data in the computing environment 201. Applications can be provided as multiple services 220-1, 220-2, 220-3, and 220-4 in the computing environment 201, representing services "A," "D," "E," and "H," respectively.

[0044] End user 202 can provide and access services through a web browser or other software application on a client. In some embodiments, the provisioning and / or request of end user 202 can be provided to the ingress gateway 230. The ingress gateway 230 may include a corresponding agent to handle the provisioning and / or request for services (one or more services provided in the computing environment 201).

[0045] Services are provided or deployed based on various virtualization technologies supported by the computing environment 201. In some embodiments, services can be provided based on virtual machine (VM)-based virtualization, container-based virtualization, and / or similar methods. Virtual machine-based virtualization can be to simulate a real computer by initializing a virtual machine, executing programs and applications without directly contacting any actual hardware resources. While the virtual machine virtualizes the machine, according to container-based virtualization, a container can be started to virtualize the entire operating system (OS) so that multiple workloads can run on a single operating system instance.

[0046] In one embodiment based on container virtualization, several containers of a service can be assembled into a Pod (e.g., a Kubernetes Pod). For example, as shown in Figure 2, service 220-2 can be equipped with one or more Pods 240-1, 240-2, ..., 240-N (collectively referred to as Pods). The Pod may include a proxy 245 and one or more containers 242-1, 242-2, ..., 242-M (collectively referred to as containers). One or more containers in the Pod process requests related to one or more corresponding functions of the service, and the proxy 245 generally controls network functions related to the service, such as routing, load balancing, etc. Other services can also be equipped with similar Pods.

[0047] During operation, executing a user request from end user 202 may require invoking one or more services in computing environment 201. Executing one or more functions of one service may require invoking one or more functions of another service. As shown in FIG2 , service "A" 220-1 receives a user request from end user 202 from ingress gateway 230. Service "A" 220-1 may invoke service "D" 220-2, and service "D" 220-2 may request service "E" 220-3 to execute one or more functions.

[0048] This computing environment can be a cloud computing environment, where resource allocation is managed by the cloud service provider, allowing for feature development without having to worry about implementing, adjusting, or scaling servers. This computing environment allows developers to execute code in response to events without building or maintaining complex infrastructure. Services can be partitioned to perform a set of functions that can scale independently and automatically, rather than scaling a single hardware device to handle the potential load.

[0049] In the above operating environment, the present disclosure provides an image segmentation method as shown in Figure 3. It should be noted that the image segmentation method of this embodiment can be executed by the mobile terminal of the embodiment shown in Figure 1. Figure 3 is a flow chart of an image segmentation method according to Example 1 of the present disclosure. As shown in Figure 3, the method may include the following steps:

[0050] Step S302 : Acquire a first image and a second image of a target part of a biological object.

[0051] The target part includes a lesion area, the first image is an image obtained by scanning the target part using electronic computed tomography technology, and the second image is an image obtained by scanning the target part using magnetic resonance technology.

[0052] The above-mentioned lesion area may refer to a lesion area caused by a lesion in a tissue or organ of a biological object, such as an appendicitis lesion area, a nasopharyngeal carcinoma lesion area, etc. The above-mentioned target site may be a tissue or organ containing the lesion area. The above-mentioned first image may refer to an image obtained by tomography of the lesion area, such as a CT image. The above-mentioned second image may refer to an image obtained by magnetic resonance imaging of the lesion area, such as an MRI image. For ease of understanding, the following text uses a CT image to represent the first image and an MRI image to represent the second image.

[0053] Considering that some tissues and organs of biological objects are difficult to segment out the lesion area only through CT images or MRI images when they are diseased, for example, in the early stage, the lesion area of ​​nasopharyngeal carcinoma is generally in the top wall, back wall and other areas. When the area is displayed by simple tomographic imaging, it may not be possible to directly segment out the specific location of the lesion area. Since tomography technology and magnetic resonance imaging technology are commonly used in the medical field to determine the lesion area, in order to ensure the accuracy of the segmented lesion area, the image segmentation system can first obtain a first image and a second image containing the target area, namely the above-mentioned CT image and MRI image, and then segment the lesion area by combining the CT image and the MRI image to determine the specific location. Among them, the target area at least includes the lesion area where the disease occurs. The first image can be obtained by scanning the target area using electronic computed tomography technology, and the second image can be obtained by scanning the target area using magnetic resonance technology.

[0054] Step S304 : cropping the second image based on the area of ​​the target part in the first image to obtain a cropped image.

[0055] The target region in the first image may refer to the region of the organ or tissue where the disease occurs in the first image, i.e., the target region, which also includes the lesion region. Considering that the lesion region is generally located at the center of the region, and that the edge portion can provide a basis for distinguishing healthy tissue from lesioned tissue, the target region in the first image can be divided into at least two parts: the edge portion and the center portion of the target region. Alternatively, the target region may be located within a preset distance from the boundary of the lesion region. The specific portion of the target region in the first image can be set by the staff according to actual needs and is not limited here. The probability of the lesion region being located in the center portion is generally higher than that of the region being located in the edge portion. To avoid missed identification, both the edge portion and the center portion of the target region can be considered during the segmentation of the lesion region. The cropped image may refer to the image of the region corresponding to the target region in the second image. If the target region in the first image includes both the edge portion and the center portion, the corresponding cropped image also includes both the edge portion and the center portion of the target region in the second image.

[0056] At present, when combining CT images and MRI images to segment the lesion area, the position of the lesion area is usually segmented by cross-modal registration of the CT images and MRI images. The accuracy of cross-modal registration is one of the important factors affecting the accuracy of determining the segmentation of the lesion area. Since rigid or affine transformation is usually used to align CT images and MRI images, even if rigid transformation is performed on the MRI image to improve the registration accuracy of the MRI image and CT image, the final registration result may not be able to show important local details between the CT image or the MRI image that can effectively reflect the lesion area, resulting in severe distortion of the registered image.

[0057] Therefore, in an optional solution of this embodiment, in order to improve the accuracy of the segmented lesion area, the second image, that is, the MRI image, can be cropped to obtain the above-mentioned cropped area, and then the cropped area can be used to segment the lesion area.

[0058] Specifically, considering that both CT images and MRI images can reflect diseased organs or tissues in a biological subject, the aforementioned lesion area can be segmented. CT images are digitally converted, reconstructed analog images with high density resolution, capable of displaying the density of tissues, organs, and lesions in different grayscales. They can also be easily post-processed by a computer, such as performing two-dimensional or three-dimensional display analysis, to more clearly display the location of the lesion area in the image. MRI images, on the other hand, have multiple imaging sequences and can display multiple different high-resolution, multi-parameter images for the same lesion area. Therefore, when determining the cropped image, image recognition can first be performed on the first image, i.e., the CT image, to analyze and process the target area in the first image, i.e., the area in the first image where the organ or tissue lesion occurs. This area can then be used to crop the second image, obtaining images corresponding to the edge and center of the lesion area in the second image, i.e., the aforementioned cropped image.

[0059] Step S306 : performing image segmentation on the target part included in the second image based on the first image and the cropped image to obtain a target segmentation result.

[0060] The target segmentation result may refer to an image of the segmented lesion area.

[0061] In an optional scheme of this embodiment, as shown above, considering that the second image, i.e., the MRI image, has higher tissue resolution and no bone artifacts than the first image, i.e., the CT image, some lesion areas, such as the lesion area corresponding to the above-mentioned nasopharyngeal carcinoma, will not be affected by the neck bones. Therefore, after obtaining the above-mentioned cropped image, the first image and the cropped image can be used to perform image registration and image segmentation on the area containing the target part in the second image, that is, the above-mentioned first image and the cropped image can be registered, and then the area containing the above-mentioned target part in the second image can be segmented according to the registration result, so as to obtain the lesion area, and then obtain the above-mentioned target segmentation result.

[0062] In an optional solution of this embodiment, since errors generated when aligning CT images and MRI images are inevitable, when performing image segmentation on the target part to determine the above-mentioned target segmentation result, the same second image can be subjected to multiple image recognition and image segmentation to determine multiple first registration errors, or multiple substitute images can be determined from other images in the imaging sequence where the second image is located, and then image recognition and image segmentation are performed on the multiple substitute images to determine multiple second registration errors. Finally, the segmentation result with the smallest registration error is selected as the above-mentioned target segmentation result, thereby improving the accuracy of the determined target segmentation result, and thus solving the technical problem of low accuracy in segmenting the lesion area in the related art.

[0063] In an embodiment of the present disclosure, a method is adopted in which a first image and a second image of a target part of a biological object are acquired; the second image is cropped based on the area of ​​the target part in the first image to obtain a cropped image; and the target part contained in the second image is segmented based on the first image and the cropped image to obtain a target segmentation result. The second image is cropped by using the area of ​​the target part in the first image to determine the cropped image of the target part in the second image, so as to reduce the influence of other images other than the cropped image in the second image on the process of segmenting the lesion area. Finally, the first image and the cropped image are used to segment the second image to obtain the target segmentation result, thereby improving the clarity of local details in the lesion area and making the segmented target segmentation result undistorted, thereby improving the segmentation accuracy of the lesion area, and further solving the technical problem of the accuracy of segmenting the lesion area in CT images and MRI images in the related art.

[0064] In an embodiment of the present disclosure, the second image is cropped based on the area of ​​the target part in the first image to obtain a cropped image, including: performing image segmentation on the target part contained in the first image to obtain an image segmentation result of the first image, wherein the image segmentation result is set to represent the area of ​​the target part in the first image; and cropping the second image based on the image segmentation result to obtain a cropped image.

[0065] The above-mentioned image segmentation result may refer to the area of ​​the aforementioned target part in the first image, and may include the edge portion and the center portion of the lesion area.

[0066] When using the area of ​​the target part in the first image to crop the second image to obtain the above-mentioned cropped image, the first image can be first segmented to obtain the area that can represent the target part in the first image, that is, the above-mentioned image segmentation result, and then as shown above, the area can be used to crop the second image to obtain an image that can represent the area corresponding to the target part in the first image and the area corresponding to the target part in the second image, that is, the cropped image.

[0067] In an optional solution of this embodiment, a pre-trained first image segmentation model can be used to perform image segmentation on the first image to obtain the above-mentioned image segmentation result, so as to improve the accuracy of the image segmentation result and further improve the accuracy of the cropped image. The image segmentation result can be obtained by the following formula:

[0068] in, represents the image segmentation result, f ct () represents the first image segmentation model, X ct represents the first image, Y=1 represents the mask value of the true segmentation result in the second image is 1, W ct Represents the network parameters in the first segmentation model.

[0069] In an embodiment of the present disclosure, when the image segmentation result is set to represent the boundary area of ​​the edge of the target part in the first image, the second image is cropped based on the image segmentation result to obtain a cropped image, including: performing expansion processing on the image segmentation result based on preset expansion parameters to obtain an expansion processing result; performing erosion processing on the image segmentation result based on preset erosion parameters to obtain an erosion processing result; obtaining the difference between the expansion processing result and the erosion processing result to obtain the target boundary area; and cropping the second image based on the target boundary area to obtain a cropped image.

[0070] In an optional scheme of this embodiment, as shown above, the area of ​​the target part in the first image may include an edge part and a center part. In order to better segment the lesion area, corresponding cropped images can be generated for the above-mentioned edge part and center part respectively, and then the cropped images can be used to determine the above-mentioned target segmentation result.

[0071] Specifically, when the image segmentation result is a boundary region of the edge of the target part in the first image, considering that the first image, such as a CT image, has a high density resolution, the region where the target part is located and other regions in the first image can be represented in the form of 0s and 1s. Therefore, when cropping the second image based on the image segmentation result to obtain a cropped image, the image segmentation result can first be dilated based on a preset dilation parameter, such as a 5mm dilation, to obtain the above-mentioned dilation processing result. Then, the image segmentation result can be eroded based on a preset erosion parameter, such as a 5mm erosion, to obtain the above-mentioned erosion processing result. Finally, the difference between the dilation processing result and the erosion processing result is obtained to obtain the boundary region of the target part in the first image, i.e., the above-mentioned target boundary region. After obtaining the target boundary region, the second image can be cropped based on the target boundary region to obtain the boundary region of the target part, i.e., the edge portion, and the image corresponding to the region in the second image, i.e., the above-mentioned cropped image, thereby improving the accuracy of the cropped image. It should be noted that the specific image dilation and image erosion processes can be referred to in relevant literature and will not be repeated here.

[0072] In an embodiment of the present disclosure, when the image segmentation result is set to represent the center area of ​​the target part in the first image, the second image is cropped based on the image segmentation result to obtain a cropped image, including: eroding the image segmentation result based on preset erosion parameters to obtain the target center area; cropping the second image based on the target center area to obtain a cropped image.

[0073] In an optional solution of this embodiment, corresponding to the above, when the image segmentation result is the center area of ​​the edge of the target part in the first image, the image segmentation result can be directly eroded according to the preset erosion parameters, so as to quickly obtain the center area of ​​the target part in the first image, that is, the target center area. After obtaining the target center area, the second image can be cropped according to the target center area to obtain the center area of ​​the target part, that is, the center part, and the image corresponding to the area in the second image, that is, the cropped image, thereby improving the accuracy of the cropped image.

[0074] In an optional scheme of this embodiment, considering that the probability of the lesion area being in the central area of ​​the target part is greater than the probability of being in the edge area, the preset erosion parameter set for determining the central area may be different from the preset erosion parameter set for determining the edge area. The size of the specific preset erosion parameter may be determined according to the lesion type of the organ or tissue to be detected. For example, if the lesion type is such that the lesion is easy to segment out the location of the lesion area, such as appendicitis, then the corresponding preset erosion parameter set for determining the central area may be greater than the preset erosion parameter set for determining the edge area; if the lesion type is such that the lesion is not easy to segment out the lesion area, such as nasopharyngeal carcinoma, then the corresponding preset erosion parameter set for determining the central area may be less than the preset erosion parameter set for determining the edge area.

[0075] In an optional solution of this embodiment, in order to avoid missed recognition, the images corresponding to the above-mentioned edge area and center area are usually used as cropped images. Therefore, in order to improve the efficiency of determining the cropped image, in the process of determining the edge area, the area corresponding to the eroded image segmentation result can be directly used as the above-mentioned center area.

[0076] In an embodiment of the present disclosure, image segmentation is performed on a target part contained in a second image based on the first image and the cropped image to obtain a target segmentation result, including: performing image segmentation on the first image, and performing image segmentation on the target part contained in the second image based on the first image and the cropped image to obtain a plurality of candidate segmentation results; performing error prediction on the plurality of candidate segmentation results based on the first image and the second image to obtain segmentation errors of the plurality of candidate segmentation results, wherein different segmentation errors are set to characterize the errors between different candidate segmentation results and a preset segmentation result; and screening out a target segmentation result from the plurality of candidate segmentation results based on the segmentation errors of the plurality of candidate segmentation results.

[0077] The candidate segmentation results may include not only the segmentation results obtained by performing image segmentation on the second image using a pre-trained second image segmentation model, but also the image segmentation results obtained by performing image segmentation on the first image, i.e., the region corresponding to the target part segmented in the CT image. To ensure the applicability of image segmentation, the second image segmentation model may be a probabilistic U-Net model. When performing segmentation on the second image, cropped images corresponding to the edge and center regions may be used as prior knowledge, thereby improving the accuracy of the candidate segmentation results.

[0078] When determining the target segmentation result, as described above, since errors are inevitable when registering CT and MRI images, to ensure the accuracy of the determined target segmentation result, multiple segmentation methods can be used, with the segmentation result with the smallest error being used as the target segmentation result. In addition to performing multiple segmentations of the second image to determine the segmentation result with the smallest error as shown in step S306, the target region contained in the second image can also be segmented using the first image and the cropped image to obtain the aforementioned multiple candidate segmentation results. Then, corresponding to determining the registration error, the first image and the second image can be used to perform error prediction on the aforementioned multiple candidate segmentation results to determine the segmentation errors corresponding to the different candidate segmentation results. The segmentation error can be determined by comparing the candidate segmentation result with a preset standard segmentation result, or it can be determined by using a pre-trained error prediction model to perform error prediction on the multiple candidate segmentation results based on the first image and the second image. The specific error prediction model is described below. The standard segmentation error can be determined using data generated when a worker manually segmented the image. Finally, after obtaining the above-mentioned segmentation error, the target segmentation result can be determined from multiple candidate segmentation results according to the preset screening rules. The preset screening rules can mean that the candidate segmentation result with the smallest segmentation error is used as the above-mentioned target segmentation result, thereby improving the accuracy of the determined target segmentation result, or it can mean that the candidate segmentation results with a segmentation error less than a preset threshold are displayed to the staff, and then the staff can actively determine the target segmentation result from the screened candidate segmentation results, thereby facilitating the staff to check the specific pathological conditions of the organs or tissues.

[0079] In an embodiment of the present disclosure, image segmentation is performed on the first image, and image segmentation is performed on the target part contained in the second image based on the first image and the cropped image to obtain multiple candidate segmentation results, including: performing image segmentation on the target part contained in the first image to obtain a first candidate segmentation result; splicing the first image and the second image to obtain a spliced ​​image; performing image segmentation on the spliced ​​image to obtain a second candidate segmentation result; performing image segmentation on the spliced ​​image based on the cropped image to obtain a third candidate segmentation result; and obtaining multiple candidate segmentation results based on the first candidate segmentation result, the second candidate segmentation result, and the third candidate segmentation result.

[0080] The first candidate result may refer to an image segmentation result obtained by performing image segmentation on the first image.

[0081] In an optional solution of this embodiment, the edge area of ​​the first image and the center area of ​​the second image can be spliced ​​together by splicing the first image and the second image, or the center area of ​​the first image and the edge area of ​​the second image can be spliced ​​together to obtain multiple spliced ​​images, and then the spliced ​​images are segmented to obtain multiple candidate segmentation results, namely the second candidate segmentation result and the third candidate segmentation result. In an optional solution of this embodiment, in order to improve the efficiency of determining the candidate segmentation results, the cropped image determined above can be used as prior knowledge instead of being used as a priori knowledge. Instead, the spliced ​​image can be directly segmented to obtain the corresponding segmentation result, namely the second candidate segmentation result. In order to ensure the accuracy and rationality of the determined candidate segmentation results, the cropped image can be further used as prior knowledge to perform image segmentation on the spliced ​​image to obtain the corresponding segmentation result, namely the third candidate segmentation result.

[0082] After obtaining the above-mentioned second candidate segmentation result and the third candidate segmentation result, considering that the first candidate result obtained by segmenting the first image can also reflect the lesion area of ​​the target part, therefore, in order to improve the accuracy of the target segmentation result, the above-mentioned candidate segmentation result can also be determined based on the first candidate segmentation result, the second candidate segmentation result and the third segmentation result. In an optional scheme of this embodiment, the determined first candidate segmentation result, the second candidate segmentation result and the third segmentation result can be displayed to the user in a preset display interface, and the user can select a specific candidate segmentation result. Alternatively, based on the clarity of the obtained first candidate segmentation result, the second candidate segmentation result and the third segmentation result, the segmentation result with a clarity greater than a preset threshold can be used as the above-mentioned candidate segmentation result. Corresponding to the above-mentioned segmentation formula, the candidate segmentation result can also be expressed in the form of a segmentation formula, with the candidate segmentation result as For example, represents the first candidate segmentation result, represents the second candidate segmentation result, It represents the third candidate segmentation result. The specific calculation formula is as follows:

[0083] in, and Represents the second image X based on the boundary area t1 and X t2 The cropped image is and The second image X is based on the central area t1 and X t2 The cropped image.

[0084] In an embodiment of the present disclosure, image segmentation is performed on the spliced ​​image based on the cropped image to obtain a third candidate segmentation result, including: performing feature extraction on the spliced ​​image to obtain spliced ​​image features; performing feature extraction on the cropped image to obtain cropped image features; modifying the spliced ​​image features based on the cropped image features to obtain modified image features; and decoding the modified image features to obtain the third candidate segmentation result.

[0085] In an optional solution of this embodiment, in order to avoid the similarity of the determined candidate segmentation results being too high, which leads to the inability to improve the accuracy of the determined target segmentation results, when using the cropped image to segment the spliced ​​image, the image features of the spliced ​​image can be modified according to the image features of the cropped image to obtain multiple modified image features of the spliced ​​image, so that there are certain differences between the modified image features, and then the modified image features are decoded to obtain the above-mentioned third candidate segmentation result with lower similarity, that is, the aforementioned and

[0086] Specifically, when using a cropped image to segment a stitched image, image recognition and feature extraction can first be performed on the stitched image to determine image features of the stitched image, i.e., the stitched image features described above. Image recognition and feature extraction can also be performed on the cropped image to determine image features of the cropped image, i.e., the cropped image features described above. The cropped image features can include edge region features of edge regions of the cropped image and center region features of the center region. To ensure that the image features of the modified stitched image are different from the stitched image features, the stitched image features can be modified based on the cropped image features. For example, the color features, shape features, and texture features of the stitched image can be modified based on the cropped image features. During the modification, the stitched image features can be modified using the aforementioned edge region features, such that the features of the edge region of the modified stitched image differ from the features of the edge region in the stitched image features; alternatively, the stitched image features can be modified using the aforementioned center region features, such that the features of the center region of the modified stitched image differ from the features of the center region in the stitched image features; alternatively, the stitched image features can be modified using both edge region features and center region features, i.e., cropped image features, to simultaneously change the features of the edge region and the features of the center region in the stitched image features, thereby obtaining new image features of the stitched image. Finally, by decoding the modified image features, the aforementioned third candidate segmentation result can be obtained.

[0087] In an embodiment of the present disclosure, error prediction is performed on multiple candidate segmentation results based on the first image and the second image to obtain segmentation errors of the multiple candidate segmentation results, including: superimposing the multiple candidate segmentation results with the first image and the second image respectively to obtain multiple superimposed images; and using an error prediction model to perform error prediction on the multiple superimposed images respectively to obtain segmentation errors of the multiple candidate segmentation results.

[0088] The superimposed image may refer to a first superimposed image formed by superimposing the candidate segmentation result and the first image, and a second superimposed image formed by superimposing the candidate segmentation result and the second image.

[0089] In an optional scheme of this embodiment, multiple segmentation errors corresponding to the multiple candidate segmentation results can be determined by superimposing multiple candidate segmentation results with the first image and the second image, and using an error prediction model to perform error prediction on the superimposed images. Specifically, the first image and the second image can be superimposed according to a preset superposition rule, for example, according to the edge area of ​​the target part in the first image and the edge area of ​​the target part in the second image, so as to obtain multiple first superimposed images. At the same time, the multiple candidate segmentation results and the second image can be superimposed according to the preset superposition rule to obtain multiple second superimposed images.

[0090] After obtaining the above-mentioned multiple superimposed images, the pre-trained error prediction model can be used to perform error prediction on the multiple superimposed images respectively, so as to determine the segmentation errors corresponding to the multiple candidate segmentation results. In order to ensure the accuracy of the predicted segmentation error, the difference between the candidate segmentation results and the real data can be used, for example As the segmentation error corresponding to the candidate segmentation result, the segmentation error can be obtained by the following formula:

[0091] in, Represents the candidate segmentation result Corresponding segmentation errors. Corresponding to the aforementioned multiple candidate segmentation results, the segmentation errors may include:

[0092] In an embodiment of the present disclosure, based on the segmentation errors of multiple candidate segmentation results, a target segmentation result is screened out from the multiple candidate segmentation results, including: determining the volume of the target part in the multiple candidate segmentation results; based on the segmentation error and volume, screening out the target segmentation result from the multiple candidate segmentation results.

[0093] In an optional scheme of this embodiment, in addition to directly taking the candidate segmentation result with the smallest segmentation error as the target segmentation result, in order to ensure the accuracy of the determined target segmentation result, the volume of the target part in the candidate segmentation result can also be combined to determine the target segmentation result from multiple candidate segmentation results.

[0094] Specifically, after determining the segmentation errors corresponding to multiple candidate segmentation results, the volume occupied by the target part in the multiple candidate segmentation results can be further determined, and then the target segmentation result can be screened out from the multiple candidate segmentation results based on the ratio of the segmentation error to the volume. That is, the candidate segmentation result with the smallest ratio of the segmentation error to the volume can be used as the above-mentioned target segmentation result, thereby improving the rationality and accuracy of the determined target segmentation result.

[0095] In an optional scheme of this embodiment, multiple first segmentation results can be screened out from multiple candidate segmentation results based on the above-mentioned segmentation errors, for example, multiple segmentation results with segmentation errors less than a preset error threshold, or the first n segmentation results after sorting the segmentation errors in ascending order are used as the multiple first segmentation results. At the same time, multiple second segmentation results can be screened out from multiple candidate segmentation results based on the volume occupied by the above-mentioned target part, for example, the first m segmentation results with a volume greater than a preset volume threshold, or the first m segmentation results after sorting the volumes in descending order are used as the second segmentation results. Then, based on the intersection of the first segmentation result and the second segmentation result, the segmentation result included in both the first segmentation result and the second segmentation result is used as the target segmentation result.

[0096] In an optional scheme of this embodiment, after determining multiple segmentation errors, the segmentation error can also be adjusted according to the volume of the target part corresponding to each segmentation error in the candidate segmentation result. For example, a standard volume can be determined based on the aforementioned standard segmentation result, and then the segmentation error can be adjusted according to the ratio of the target part in the candidate segmentation result to the standard volume. Finally, the segmentation result corresponding to the smallest adjusted segmentation error is used as the above-mentioned target segmentation result.

[0097] To facilitate understanding of the above-mentioned image segmentation process, FIG4 is a schematic diagram of a candidate segmentation result determination process according to Example 1 of the present disclosure, and FIG5 is a schematic diagram of a target segmentation result determination process according to Example 1 of the present disclosure.

[0098] In Figure 4, CT represents a CT image, i.e., the first image; T1 and T2 represent MRI images, i.e., the second image; Candidate 1 refers to the lesion region obtained by segmenting the first image, i.e., the aforementioned first candidate segmentation result; Candidate 2 refers to the lesion region obtained by segmenting the spliced ​​image obtained by splicing the first and second images, i.e., the aforementioned second candidate segmentation result; Candidates 3 and 4 refer to the lesion regions obtained by modifying and decoding the image features of the spliced ​​image based on the image features of the cropped image, i.e., the aforementioned third candidate segmentation result; Model 1 refers to the aforementioned first image segmentation model; Model 2 refers to the aforementioned second image segmentation model, i.e., the probabilistic U-NET model; and Model 3 refers to a feature modification model configured to modify the image features of the spliced ​​image. To improve segmentation efficiency, Model 3 can be inserted into Model 2 as a modification module. Model 4 in Figure 5 refers to the aforementioned error prediction model. Errors 1, 2, 3, and 4 correspond to the segmentation errors corresponding to Candidates 1, 2, 3, and 4, respectively, when superimposed with different CT or MRI images.

[0099] Taking the segmentation of the primary lesion of nasopharyngeal carcinoma as an example, when the image of nasopharyngeal carcinoma is segmented to obtain the location of the primary lesion, the process shown in Figure 4 can be first executed to obtain multiple candidate segmentation results. Specifically, the process of obtaining multiple candidate segmentation results shown in Figure 4 can be divided into 5 steps, including: the image segmentation system can first obtain a biological object, such as a CT image and an MRI image of a human nasal cavity and neck, that is, CT, T1 and T2 in Figure 4; the image segmentation system can use a first image segmentation model, that is, model 1, to segment the CT image to obtain a first candidate segmentation result, that is, candidate 1, and use a second image segmentation model, that is, model 2, to segment the MRI image according to the position of the nasopharynx in the CT image to obtain a cropped image; the image segmentation system can splice the CT image and the MRI image, that is, splice CT and T1 or stitching CT and T2, or stitching CT, T1 and T2 at the same time to obtain a stitched image, and then continue to use the second image segmentation model, i.e., model 2, to segment the stitched image to obtain a second candidate segmentation result, i.e., candidate 2; the image segmentation system can extract the features of the cropped image, i.e., obtain the edge region features of the edge region and the center region features of the center region in the cropped image, as well as the stitching image features of the stitched image; the image segmentation system can modify the features of the stitched image according to the features of the cropped image, and then decode the modified stitching image features to obtain a third candidate segmentation result, i.e., candidate 3 and candidate 4. For example, the edge region features can be used to modify the stitching image features, and candidate 3 can be obtained after decoding; the center region features can be used to modify the stitching image features, and candidate 4 can be obtained after decoding.

[0100] After obtaining candidate 1, candidate 2, candidate 3, and candidate 4, the image segmentation system can execute the process shown in FIG5 to filter out the target segmentation result from the multiple candidate segmentation results, thereby obtaining the location of the primary lesion of nasopharyngeal carcinoma. Specifically, the process of filtering out the target segmentation result shown in FIG5 can be divided into three steps, including: the image segmentation system can first perform image recognition on the multiple candidate segmentation results to determine the multiple volumes of the target part in the multiple candidate segmentation results, and at the same time superimpose the multiple candidate segmentation results with the CT image and the MRI image to obtain multiple superimposed images; the image segmentation system can use the error prediction model, i.e., model 4, to perform error prediction on the multiple superimposed images to obtain multiple segmentation errors corresponding to the multiple candidate segmentation results, and determine a minimum error from the multiple segmentation errors as the target error corresponding to the multiple candidate segmentation results; the image segmentation system can filter out the target segmentation result, i.e., the target result in FIG5, from the multiple candidate segmentation results according to the target errors corresponding to the candidate segmentation results and their respective volumes, according to a preset screening method.

[0101] In this embodiment, a method for segmenting a primary lesion of nasopharyngeal carcinoma is also provided, which includes: obtaining a first image and a second image of the patient's head and neck, wherein the first image is an image obtained by scanning the head and neck using electronic computed tomography technology, and the second image is an image obtained by scanning the head and neck using magnetic resonance technology; based on the area of ​​the primary lesion of nasopharyngeal carcinoma in the first image, the second image is cropped to obtain a cropped image; based on the first image and the cropped image, the image segmentation of the primary lesion of nasopharyngeal carcinoma contained in the second image is performed to obtain a target segmentation result.

[0102] The above-mentioned nasopharyngeal carcinoma is a common cancer. The above-mentioned first image may refer to an image obtained by scanning the target part through electronic computed tomography technology, such as a CT image, and the second image may refer to an image obtained by scanning the target part through magnetic resonance technology, such as an MRI image. It should be noted that the above-mentioned nasopharyngeal carcinoma primary lesion segmentation method is not only suitable for nasopharyngeal carcinoma, but can also be set to other cancers whose primary lesions can be determined by equipment scanning, such as lung cancer, etc. The corresponding scanning parts of the first image and the second image can be the patient's chest area. Nasopharyngeal carcinoma is selected as the cancer to be detected to illustrate the method for segmenting the primary lesion of cancer for the sake of ease of understanding, not because the method can only segment the primary lesion of nasopharyngeal carcinoma.

[0103] The target segmentation result may refer to an image of a primary nasopharyngeal carcinoma lesion, and may be used to determine a specific location of the primary nasopharyngeal carcinoma lesion.

[0104] In an optional solution of this embodiment, in order to accurately segment the primary lesion of nasopharyngeal carcinoma and improve the accuracy of nasopharyngeal carcinoma diagnosis, the patient's head and neck can be scanned using electronic computed tomography technology and magnetic resonance imaging technology to obtain the above-mentioned first image and second image, that is, the CT image and MRI image of the patient's head and neck. Then, in order to further improve the accuracy of the primary lesion of nasopharyngeal carcinoma determined using the first image and the second image, image recognition can be performed on the first image, that is, the CT image, to analyze and process the above-mentioned head and neck area in the first image, and then the second image can be cropped using the area to obtain images corresponding to the edge and center parts of the patient's head and neck in the second image, that is, the above-mentioned cropped image. Finally, the first image and the cropped image are used to perform image segmentation on the head and neck area contained in the second image, thereby obtaining the above-mentioned target segmentation result. The specific cropping and segmentation process can be as described above and will not be repeated here.

[0105] In order to facilitate the understanding of the process of segmenting the primary lesion of nasopharyngeal carcinoma, the process of segmenting the primary lesion of nasopharyngeal carcinoma can be divided into at least three parts, wherein the first part is the data set collection and data annotation part, wherein the collected data set can be used to train the aforementioned multiple models, such as the first image segmentation model, the second image segmentation model, the error prediction model, etc. The second part is the detailed implementation part, which is the process of using multiple models to process the first image and the second image to determine the target segmented image, that is, the primary lesion of nasopharyngeal carcinoma; the third part is the comparison and evaluation part, which is the process of evaluating the above-mentioned primary lesion segmentation method of nasopharyngeal carcinoma.

[0106] Specifically, in the first part, CT images and MRI images generated by nasopharyngeal carcinoma patients during cancer diagnosis and treatment can be collected to construct the above-mentioned dataset, and the nasopharyngeal carcinoma primary lesions in these images can be annotated to implement data annotation operations. For example, when constructing the dataset, multiple CT images and MRI images generated by 124 nasopharyngeal carcinoma patients during radiotherapy or chemoradiotherapy can be obtained, and these images can be used to construct the above-mentioned dataset, wherein the MRI images include at least T1-weighted images and T2-weighted images. Then, when annotating the nasopharyngeal carcinoma primary lesions in the CT images and MRI images, multiple experienced radiation oncologists can manually annotate the nasopharyngeal carcinoma primary lesions in the CT images and MRI images, referring to the clinical information and relevant diagnostic documents of the above-mentioned 124 nasopharyngeal carcinoma patients, so as to reduce the annotation errors caused by annotation differences and ensure the accuracy of the annotation results. In an optional solution of this embodiment, in order to improve the convenience of manual annotation and the convenience of subsequent use of the annotated data, the collected different CT images and MRI images can be unified. For example, the scanning size of the CT image can be set to 513×513×145 voxels and the average resolution can be set to 0.98×0.98×2.89mm, and the scanning size of the MRI image can be set to 448×448×19 voxels and the average resolution can be set to 0.57×0.57×6.00mm. In addition, in order to improve the accuracy of the trained model when using the above dataset to train the aforementioned multiple models, the data in the dataset can be randomly grouped. For example, 60% of the data in the dataset can be randomly grouped into one group, 30% of the data can be randomly grouped into one group, and 10% of the data can be randomly grouped into one group. The grouped data can then be used to perform model training, verification, and testing.

[0107] In the second part, a probabilistic U-Net model can be used to segment the primary nasopharyngeal carcinoma lesions in CT images and MRI images. As shown above, in order to ensure the accuracy of the segmented primary nasopharyngeal carcinoma lesions, an encoder, namely the aforementioned adjustment and modification model, can be added to the probabilistic U-Net model to modify the spliced ​​image features using the aforementioned cropped image features to obtain modified image features, and then the decoder in the probabilistic U-Net model is used to decode the modified image features to obtain multiple candidate segmentation results. Finally, the segmentation errors of the multiple candidate segmentation results are used to determine the target segmentation result. In an optional solution of this embodiment, in order to improve the applicability of the modified probabilistic U-Net model, the dimension of the latent space of the modified probabilistic U-Net model can be set to 6 dimensions, the input size of the training input patch can be set to (64, 192, 192), and the data enhancement of the above encoder, that is, the parameter transformation method used when modifying the image features is set to at least include: rotation, scaling, mirroring and gamma transformation, and the batch size is set to 2. When training the probabilistic U-Net model, the initial learning rate of the model can be set to 0.01, and a polygonal decay plan with an exponent of 0.9 can be used to train the probabilistic U-Net model for 1000 training cycles, thereby improving the accuracy of nasopharyngeal carcinoma primary lesions segmented using the probabilistic U-Net model.

[0108] In the third part, the primary lesions manually marked by the radiation oncologists can be used to compare the above-mentioned nasopharyngeal carcinoma segmentation method with the traditional primary lesion segmentation method from the perspective of four quantitative indicators, and the above-mentioned nasopharyngeal carcinoma segmentation method can be evaluated, analyzed, and improved. The above-mentioned four quantitative indicators can be: Dice score (a medical indicator, abbreviated as DSC%), 95% Hausdorff distance (Hausdorff distance, abbreviated as HD95), average surface distance (Average Surf Distance, abbreviated as ASD) and median surface distance (MedianSD).

[0109] Table 1

[0110] The above Table 1 is an evaluation table of nasopharyngeal carcinoma segmentation methods according to an embodiment of the present disclosure, wherein EF stands for early fusion, TB stands for tumor boundary, i.e., the edge area of ​​the head and neck, and TC stands for tumor center, i.e., the central area of ​​the head and neck. Table 1 shows the quantitative index evaluation results of different segmentation methods, wherein the data in rows 2-5 of Table 1 are respectively for the evaluation results of different quantitative indicators under single-modality CT, multi-modality CT&MRI, multi-modality CT&MRITB with context-awareness, and MRITC. As shown in Table 1, the unimodal segmentation model achieved a DSC of 76.6%, while with the early fusion of CT and MRI, the multimodal segmentation model slightly improved the DSC to 77.5%, and the corresponding distance indicators (i.e., HD95, ASD, and MedianSD) all decreased slightly, and the segmentation performance was enhanced. The segmentation model with context-aware input (CT using MRI at the tumor boundary / center) was generally comparable to the early fusion model, with DSC of 77.3% and 77.8%, respectively, and the segmentation performance did not change much, which means that when there is potential registration error, the improvement that the multimodal segmentation model can achieve is limited compared with the unimodal segmentation model. The 6th row in Table 1 is the quantitative index evaluation result corresponding to the segmentation of the primary nasopharyngeal carcinoma lesion after the above-mentioned single modality and multimodality are fused into two streams. The data in rows 7-8 in Table 1 are the quantitative index evaluation results of determining the target segmentation result after constructing multiple candidate segmentation results as shown in the above disclosure. Among them, the 7th row is the evaluation result corresponding to when there are only two candidate segmentation results, and the 8th row is the evaluation result corresponding to when there are 4 candidate segmentation results. When there are only two candidate segmentation results, the DSC of the segmentation model, that is, the aforementioned probabilistic U-Net model, can be improved to 78.5%, which is 1.9% higher than the single modality segmentation model. When there are four candidate segmentation results, the DSC of the segmentation model is further improved to 79.6%, which is a significant improvement compared with the DSC under any modality. At the same time, the corresponding distance indicators are compared with the single modality segmentation model, HD95 is reduced by 1.08mm, ASD is reduced by 0.42mm, and compared with the multimodal segmentation model, HD95 is reduced by 0.92mm, ASD is reduced by 0.23mm, which is also significantly reduced. The last row in Table 1 represents the quantitative index evaluation results of the nasopharyngeal carcinoma segmentation method proposed in the present disclosure. It can be seen from Table 1 that the nasopharyngeal carcinoma primary lesion segmented by the lesion segmentation method proposed in the present disclosure has better effect.

[0111] For ease of understanding, Figure 6 is a schematic diagram of a comparison of segmentation results according to Example 1 of the present disclosure. As shown in Figure 6, the first column is the segmentation result of the unimodal segmentation model (CT), the second column is the segmentation result of the multimodal segmentation model (CT&MRI), the third column is the segmentation result of the multimodal segmentation model for early fusion of edge areas (CT&MRI TB), and the fourth column is the segmentation result of the multimodal segmentation model for early fusion of edge areas (CT&MRI TC). The first row represents the segmentation results predicted using CT images, the second row represents the segmentation results predicted using CT images and MRI images, and the third row represents the final target segmentation results displayed on the CT image. The image includes the range of the nasopharyngeal carcinoma primary lesion manually annotated by a radiation oncologist, as well as the range of the nasopharyngeal carcinoma primary lesion predicted by the above-mentioned different segmentation models. Among them, the range corresponding to the black irregular box in the middle of each image is the predicted range of the nasopharyngeal carcinoma primary lesion, and the range corresponding to the gray irregular box next to the black irregular box is the manually annotated range of the nasopharyngeal carcinoma primary lesion. The DSC scores of different images can be shown in the upper right corner of the image.

[0112] From the above, it can be seen that the method for segmenting nasopharyngeal carcinoma primary lesions proposed in the present invention, by constructing a context-aware multimodal selection framework using CT images and MRI images, has higher segmentation accuracy and better segmentation effect than traditional primary lesion segmentation methods. Compared with traditional single-modal segmentation models and multimodal segmentation models, the DSC scores are improved by 3.0% and 2.1% respectively. Compared with nasopharyngeal carcinoma primary lesions manually annotated by many radiation oncologists, it has extremely high similarity, is more in line with the annotation capabilities of radiation oncologists, and has a good nasopharyngeal carcinoma primary lesion segmentation effect.

[0113] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data set for analysis, stored data, displayed data, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0114] It should be noted that for the aforementioned method embodiments, for simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present disclosure is not limited by the order of the actions described, because according to the present disclosure, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present disclosure.

[0115] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of the present disclosure.

[0116] Example 2

[0117] According to an embodiment of the present disclosure, an image segmentation method is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0118] FIG7 is a flow chart of an image segmentation method according to Embodiment 2 of the present disclosure. As shown in FIG7 , the method may include the following steps:

[0119] Step S702 : In response to the action being set as an input instruction on the operation interface, a first image and a second image of the target part of the biological object are displayed on the operation interface.

[0120] The target part includes a lesion area, the first image is an image obtained by scanning the target part using electronic computed tomography technology, and the second image is an image obtained by scanning the target part using magnetic resonance technology.

[0121] The above-mentioned input instruction may refer to an instruction set to display the first image and the second image in a preset operation interface. The above-mentioned first image may refer to an image obtained by performing a tomographic scan of the lesion area, such as a CT image, and the above-mentioned second image may refer to an image obtained by performing magnetic resonance imaging of the lesion area, such as an MRI image.

[0122] In an optional solution of this embodiment, after receiving the above-mentioned input instructions, the image segmentation system can first obtain a first image and a second image containing the target part, namely the above-mentioned CT image and MRI image, and display the first image and the second image in a preset operation interface to facilitate staff viewing.

[0123] In step S704, in response to the action being set as an image segmentation instruction on the operation interface, the target segmentation result is displayed on the operation interface, wherein the target segmentation result is obtained by performing image segmentation on the target part contained in the second image based on the first image and the cropped image, and the cropped image is obtained by cropping the second image based on the area of ​​the target part in the first image.

[0124] The above-mentioned image segmentation instruction may refer to an instruction for segmenting the target segmentation result from the second image, that is, an image of the area where the lesion area is located. The area of ​​the above-mentioned target part in the first image may refer to the area where the organ or tissue where the lesion occurs in the first image is located, that is, the area where the above-mentioned target part is located, and this area also contains the lesion area.

[0125] In an optional solution of this embodiment, after receiving the image segmentation instruction, the image segmentation system can first perform image recognition on the first image, that is, the CT image, analyze and process the area of ​​the target part in the first image, that is, the area where the organ or tissue lesion occurs in the first image, and then use this area to crop the second image to obtain images corresponding to the edge and center parts of the lesion area in the second image, that is, the above-mentioned cropped image. Then, the first image and the cropped image are used to perform image registration and image segmentation on the area containing the target part in the second image, that is, the first image and the cropped image can be registered, and finally, the area containing the target part in the second image can be segmented based on the registration result, so as to determine the position of the primary lesion in the lesion area, and then determine the above-mentioned target segmentation result. After determining the target segmentation result, the target segmentation result can also be output in a preset operation interface for easy viewing by staff.

[0126] It should be noted that the preferred implementation scheme involved in the above embodiments of the present disclosure is the same as the solution provided in Example 1, as well as the application scenario and implementation process, but is not limited to the solution provided in Example 1.

[0127] Example 3

[0128] According to an embodiment of the present disclosure, an image segmentation method is also provided in a virtual reality scenario such as a virtual reality VR device, an augmented reality AR device, etc. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that here.

[0129] FIG8 is a flow chart of an image segmentation method according to Embodiment 3 of the present disclosure. As shown in FIG8 , the method may include the following steps:

[0130] Step S802: Display the first image and the second image of the target part of the biological object on a presentation screen of a virtual reality (VR) device or an augmented reality (AR) device.

[0131] The target part includes a lesion area, the first image is an image obtained by scanning the target part using a computer tomography technique, and the second image is an image obtained by scanning the target part using a magnetic resonance imaging technique;

[0132] In an optional solution of this embodiment, in order to make the determined target segmentation result more accurate, a first image and a second image containing the target part of the biological object can be obtained through a virtual reality (VR) device or an augmented reality (AR) device, and the first image and the second image can be displayed in the VR or AR device to facilitate staff to observe the lesion area in the image. The first image can refer to an image obtained by tomography of the lesion area, such as a CT image, and the second image can refer to an image obtained by magnetic resonance imaging of the lesion area, such as an MRI image.

[0133] Step S804 : cropping the second image based on the area of ​​the target part in the first image to obtain a cropped image.

[0134] In an optional solution of this embodiment, in order to improve the accuracy of the position of the primary lesion in the segmented lesion area, the first image and the second image, i.e., the CT image and the MRI image, can be cropped to obtain the above-mentioned cropped area, and the cropped area can be used to determine the position of the primary lesion in the lesion area.

[0135] Specifically, considering that both CT images and MRI images can reflect the organs or tissues in the body of a biological object where lesions occur, the above-mentioned lesion area can be determined, and the CT image is a reconstructed analog image that has undergone digital conversion, with a high density resolution, and can display the density of tissue organs and lesions in different grayscales, and can be easily post-processed by a computer, such as performing two-dimensional display analysis, three-dimensional display analysis, etc., so as to more clearly display the position of the lesion area in the image. MRI images have multiple imaging sequences, and can display a variety of different high-tissue-resolution, multi-parameter images for the same lesion area. Therefore, when determining the cropped image, the first image, i.e., the CT image, can be firstly subjected to image recognition, and the area of ​​the above-mentioned target part in the first image can be analyzed and processed, i.e., the area where the organ or tissue lesions occur in the first image, and then the second image can be cropped using this area to obtain images corresponding to the edge and center parts of the lesion area in the second image, i.e., the above-mentioned cropped image.

[0136] Step S807, performing image segmentation on the target part included in the second image based on the first image and the cropped image to obtain a target segmentation result;

[0137] In an optional solution of this embodiment, since errors generated when aligning CT images and MRI images are inevitable, when performing image segmentation on the target part to determine the above-mentioned target segmentation result, the same second image can be subjected to multiple image recognition and image segmentation to determine multiple first registration errors, or multiple substitute images can be determined from other images in the imaging sequence where the second image is located, and then image recognition and image segmentation are performed on the multiple substitute images to determine multiple second registration errors. Finally, the segmentation result with the smallest registration error is selected as the above-mentioned target segmentation result, thereby improving the accuracy of the determined target segmentation result, and further solving the technical problem of low accuracy in segmenting the position of the primary lesion in the lesion area in the related art.

[0138] Step S809: driving the VR device or AR device to render and display the target segmentation result.

[0139] In an optional solution of this embodiment, a VR device or an AR device can be driven to render the target segmentation result so that the staff can more intuitively observe the relevant information of the primary lesion in the target area.

[0140] Optionally, in this embodiment, the image segmentation method may be configured in a hardware environment consisting of a server and a virtual reality device. The first image, the second image, the cropped image, and the target segmentation result of the target part of the biological object are displayed on a display screen of the virtual reality (VR) device or augmented reality (AR) device. The server may be a server corresponding to a media file operator. The network includes, but is not limited to, a wide area network, a metropolitan area network, or a local area network. The virtual reality device is not limited to a virtual reality helmet, virtual reality glasses, or an all-in-one virtual reality device.

[0141] In an embodiment of the present disclosure, a virtual reality device includes a memory, a processor, and a transmission device. The memory is configured to store an application program, which can be configured to execute: cropping a second image based on an area of ​​a target portion in a first image to obtain a cropped image, including: performing image segmentation on the target portion contained in the first image to obtain an image segmentation result of the first image, wherein the image segmentation result is configured to represent the area of ​​the target portion in the first image; and cropping the second image based on the image segmentation result to obtain a cropped image.

[0142] In an embodiment of the present disclosure, the application is further configured to execute: when the image segmentation result is set to represent the boundary area of ​​the edge of the target part in the first image, the second image is cropped based on the image segmentation result to obtain a cropped image, including: dilating the image segmentation result based on preset dilation parameters to obtain a dilation processing result; eroding the image segmentation result based on preset erosion parameters to obtain an erosion processing result; obtaining the difference between the dilation processing result and the erosion processing result to obtain the target boundary area; cropping the second image based on the target boundary area to obtain a cropped image.

[0143] In an embodiment of the present disclosure, the application is further configured to execute: when the image segmentation result is set to represent the center of the target part as the central area in the first image, the second image is cropped based on the image segmentation result to obtain a cropped image, including: eroding the image segmentation result based on preset erosion parameters to obtain the target central area; cropping the second image based on the target central area to obtain a cropped image.

[0144] In an embodiment of the present disclosure, the application is further configured to execute: performing image segmentation on a target portion contained in the second image based on the first image and the cropped image to obtain a target segmentation result, including: performing image segmentation on the first image, and performing image segmentation on the target portion contained in the second image based on the first image and the cropped image to obtain a plurality of candidate segmentation results; performing error prediction on the plurality of candidate segmentation results based on the first image and the second image to obtain segmentation errors of the plurality of candidate segmentation results, wherein different segmentation errors are set to characterize the errors between different candidate segmentation results and a preset segmentation result; and screening out a target segmentation result from the plurality of candidate segmentation results based on the segmentation errors of the plurality of candidate segmentation results.

[0145] In an embodiment of the present disclosure, the application is further configured to execute: performing image segmentation on the first image, and performing image segmentation on the target part contained in the second image based on the first image and the cropped image to obtain multiple candidate segmentation results, including: performing image segmentation on the target part contained in the first image to obtain a first candidate segmentation result; splicing the first image and the second image to obtain a spliced ​​image; performing image segmentation on the spliced ​​image to obtain a second candidate segmentation result; performing image segmentation on the spliced ​​image based on the cropped image to obtain a third candidate segmentation result; and obtaining multiple candidate segmentation results based on the first candidate segmentation result, the second candidate segmentation result, and the third candidate segmentation result.

[0146] In an embodiment of the present disclosure, the application is further configured to execute: performing image segmentation on the spliced ​​image based on the cropped image to obtain a third candidate segmentation result, including: performing feature extraction on the spliced ​​image to obtain spliced ​​image features; performing feature extraction on the cropped image to obtain cropped image features; modifying the spliced ​​image features based on the cropped image features to obtain modified image features; and decoding the modified image features to obtain the third candidate segmentation result.

[0147] In an embodiment of the present disclosure, the application is further configured to perform: performing error prediction on multiple candidate segmentation results based on the first image and the second image to obtain segmentation errors of the multiple candidate segmentation results, including: superimposing the multiple candidate segmentation results with the first image and the second image respectively to obtain multiple superimposed images; using the error prediction model to perform error prediction on the multiple superimposed images respectively to obtain segmentation errors of the multiple candidate segmentation results.

[0148] In an embodiment of the present disclosure, the application is further configured to perform: filtering out a target segmentation result from multiple candidate segmentation results based on the segmentation errors of the multiple candidate segmentation results, including: determining the volume of the target part in the multiple candidate segmentation results; filtering out the target segmentation result from the multiple candidate segmentation results based on the segmentation error and volume.

[0149] It should be noted that the above-mentioned image segmentation method applied in a VR device or an AR device in this embodiment may include the method of the embodiment shown in Figure 8, so as to achieve the purpose of driving the VR device or AR device to display the first image, the second image, the cropped image and the target segmentation result of the target part of the biological object.

[0150] Optionally, the processor of this embodiment can call the application stored in the memory to perform the above steps through a transmission device. The transmission device can receive media files sent by a server through a network, or can be configured to transmit data between the processor and the memory.

[0151] Optionally, in a virtual reality device, a head-mounted display with eye tracking is provided, wherein the screen in the HMD is configured to display the displayed video images, the eye tracking module in the HMD is configured to obtain the real-time movement trajectory of the user's eyeballs, the tracking system is configured to track the user's position information and motion information in the real three-dimensional space, and the computing processing unit is configured to obtain the user's real-time position and motion information from the tracking system, and calculate the three-dimensional coordinates of the user's head in the virtual three-dimensional space, as well as the user's field of view direction in the virtual three-dimensional space, etc.

[0152] In the embodiments of the present disclosure, a virtual reality device may be connected to a terminal, and the terminal and the server may be connected via a network. The virtual reality device is not limited to a virtual reality helmet, virtual reality glasses, an all-in-one virtual reality machine, etc. The terminal is not limited to a PC, a mobile phone, a tablet computer, etc. The server may be a server corresponding to a media file operator, and the network includes but is not limited to a wide area network, a metropolitan area network, or a local area network.

[0153] It should be noted that the preferred implementation scheme involved in the above embodiments of the present disclosure is the same as the solution provided in Example 1, as well as the application scenario and implementation process, but is not limited to the solution provided in Example 1.

[0154] Example 4

[0155] According to an embodiment of the present disclosure, an image segmentation method is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0156] FIG9 is a flow chart of an image segmentation method according to Embodiment 4 of the present disclosure. As shown in FIG9 , the method may include the following steps:

[0157] Step S902 : acquiring a first image and a second image of a target part of a biological object by calling a first interface.

[0158] Among them, the first interface includes a first parameter, the parameter value of the first parameter is a first image and a second image, the target part includes a lesion area, the first image is an image obtained by scanning the target part using electronic computed tomography technology, and the second image is an image obtained by scanning the target part using magnetic resonance technology.

[0159] The above-mentioned lesion area may refer to a lesion area caused by a tissue or organ lesion of a biological object, such as an appendicitis lesion area, a nasopharyngeal carcinoma lesion area, etc. The above-mentioned first image may refer to an image obtained by tomography of the lesion area, such as a CT image. The above-mentioned second image may refer to an image obtained by magnetic resonance imaging of the lesion area, such as an MRI image. For ease of understanding, the first image is represented by a CT image and the second image is represented by an MRI image.

[0160] In an optional solution of this embodiment, to ensure the accuracy of determining the location of the primary lesion in the lesion area, the image segmentation system may first obtain a first image and a second image containing the target area, namely the CT image and MRI image mentioned above, and then analyze the lesion area by combining the CT image and the MRI image to determine the specific location of the primary lesion. The target area includes at least the lesion area where the disease occurs. The first image can be obtained by scanning the target area using electronic computed tomography technology, and the second image can be obtained by scanning the target area using magnetic resonance imaging technology.

[0161] Step S904 : cropping the second image based on the area of ​​the target part in the first image to obtain a cropped image.

[0162] Considering that both CT images and MRI images can reflect the organs or tissues with lesions in the body of a biological object, the above-mentioned lesion area can be determined, and the CT image is a reconstructed analog image after digital conversion, with a high density resolution, and can display the density of tissue organs and lesions in different grayscales, and can be easily post-processed by computer, such as two-dimensional display analysis, three-dimensional display analysis, etc., so as to more clearly show the position of the lesion area in the image. MRI images have multiple imaging sequences, and can display a variety of different high-tissue-resolution, multi-parameter images for the same lesion area. Therefore, when determining the cropped image, the first image, that is, the CT image, can be firstly subjected to image recognition, and the area of ​​the above-mentioned target part in the first image can be analyzed and processed, that is, the area where the organ or tissue lesion occurs in the first image, and then the second image can be cropped using this area to obtain images corresponding to the edge and center parts of the lesion area in the second image, that is, the above-mentioned cropped image.

[0163] Step S907 : performing image segmentation on the target part included in the second image based on the first image and the cropped image to obtain a target segmentation result.

[0164] The target segmentation result may refer to a segmented image that can represent information of the primary lesion in the lesion area.

[0165] In an optional scheme of this embodiment, as shown above, considering that the second image, i.e., the MRI image, has higher tissue resolution and no bone artifacts than the first image, i.e., the CT image, some lesion areas, such as the lesion area corresponding to the above-mentioned nasopharyngeal carcinoma, will not be affected by the neck bones. Therefore, after obtaining the above-mentioned cropped image, the first image and the cropped image can be used to perform image registration and image segmentation on the area containing the target part in the second image, that is, the above-mentioned first image and the cropped image can be registered, and then the area containing the above-mentioned target part in the second image can be segmented according to the registration result, so as to determine the position of the primary lesion in the lesion area, and then determine the above-mentioned target segmentation result.

[0166] Step S909: Output the target segmentation result by calling the second interface.

[0167] The second interface includes a second parameter, and the parameter value of the second parameter is the target segmentation result.

[0168] After obtaining the target segmentation result, the image segmentation system can output the target segmentation to a preset operation interface through a preset second interface, so that the staff can view the target segmentation result conveniently.

[0169] It should be noted that the preferred implementation scheme involved in the above embodiments of the present disclosure is the same as the solution provided in Example 1, as well as the application scenario and implementation process, but is not limited to the solution provided in Example 1.

[0170] Example 5

[0171] According to an embodiment of the present disclosure, a device configured to implement the above-described image segmentation method is also provided. The device can be deployed in a target client. FIG10 is a block diagram of an image segmentation device according to Embodiment 5 of the present disclosure. As shown in FIG10 , the device 1000 includes: an acquisition module 1002 , a cropping module 1004 , and a segmentation module 1007 .

[0172] Among them, the acquisition module 1002 is configured to acquire a first image and a second image of a target part of a biological object, wherein the target part includes a lesion area, the first image is an image obtained by scanning the target part using electronic computed tomography technology, and the second image is an image obtained by scanning the target part using magnetic resonance technology; the cropping module 1004 is configured to crop the second image based on the area of ​​the target part in the first image to obtain a cropped image; the segmentation module 1007 is configured to perform image segmentation on the target part included in the second image based on the first image and the cropped image to obtain a target segmentation result.

[0173] In the above embodiment of the present disclosure, the cropping module 1004 includes: a first segmentation unit, configured to perform image segmentation on the target part contained in the first image to obtain an image segmentation result of the first image, wherein the image segmentation result is configured to represent the area of ​​the target part in the first image; an image cropping unit, configured to crop the second image based on the image segmentation result to obtain a cropped image.

[0174] In the above-mentioned embodiment of the present disclosure, when the image segmentation result is set to represent the boundary area of ​​the edge of the target part in the first image, the image cropping unit is further configured to: perform expansion processing on the image segmentation result based on preset expansion parameters to obtain an expansion processing result; perform erosion processing on the image segmentation result based on preset erosion parameters to obtain an erosion processing result; obtain the difference between the expansion processing result and the erosion processing result to obtain the target boundary area; and crop the second image based on the target boundary area to obtain a cropped image.

[0175] In the above embodiment of the present disclosure, when the image segmentation result is set to represent the central area of ​​the target part in the first image, the image cropping unit is also set to: erode the image segmentation result based on preset erosion parameters to obtain the target central area; crop the second image based on the target central area to obtain a cropped image.

[0176] In the above embodiment of the present disclosure, the segmentation module 1004 includes: a second segmentation unit, configured to perform image segmentation on the first image, and perform image segmentation on the target part contained in the second image based on the first image and the cropped image, to obtain multiple candidate segmentation results; an error prediction unit, configured to perform error prediction on the multiple candidate segmentation results based on the first image and the second image, to obtain segmentation errors of the multiple candidate segmentation results, wherein different segmentation errors are configured to characterize the errors between different candidate segmentation results and preset segmentation results; a result screening unit, configured to screen out the target segmentation result from the multiple candidate segmentation results based on the segmentation errors of the multiple candidate segmentation results.

[0177] In the above embodiment of the present disclosure, the second segmentation unit is further configured to: perform image segmentation on the target part contained in the first image to obtain a first candidate segmentation result; splice the first image and the second image to obtain a spliced ​​image; perform image segmentation on the spliced ​​image to obtain a second candidate segmentation result; perform image segmentation on the spliced ​​image based on the cropped image to obtain a third candidate segmentation result; obtain multiple candidate segmentation results based on the first candidate segmentation result, the second candidate segmentation result and the third candidate segmentation result.

[0178] In the above embodiment of the present disclosure, the second segmentation unit is further configured to: perform feature extraction on the stitched image to obtain stitched image features; perform feature extraction on the cropped image to obtain cropped image features; modify the stitched image features based on the cropped image features to obtain modified image features; and decode the modified image features to obtain a second candidate segmentation result.

[0179] In the above embodiment of the present disclosure, the error prediction unit is further configured to: superimpose multiple candidate segmentation results with the first image and the second image respectively to obtain multiple superimposed images; use the error prediction model to perform error prediction on the multiple superimposed images respectively to obtain segmentation errors of multiple candidate segmentation results.

[0180] In the above embodiment of the present disclosure, the result screening unit is further configured to: determine the volume of the target part in multiple candidate segmentation results; and screen the target segmentation result from the multiple candidate segmentation results based on the segmentation error and the volume.

[0181] It should be noted that the acquisition module 1002, cropping module 1004, and segmentation module 1007 described above correspond to steps S302 to S307 in Example 1. The examples and application scenarios implemented by the three modules and the corresponding steps are the same, but are not limited to the contents disclosed in Example 1. It should be noted that the above modules or units can be hardware components or software components stored in a memory and processed by one or more processors. The above modules can also be run as part of the device in the AR / VR device provided in Example 1.

[0182] It should be noted that the preferred implementation scheme involved in the above embodiments of the present disclosure is the same as the solution provided in Example 1, as well as the application scenario and implementation process, but is not limited to the solution provided in Example 1.

[0183] Example 6

[0184] According to an embodiment of the present disclosure, a device configured to implement the above-mentioned image segmentation method is also provided. The device can be deployed in a target client. FIG11 is a block diagram of an image segmentation device according to Embodiment 7 of the present disclosure. As shown in FIG11 , the device 1100 includes: a first display module 1102 and a second display module 1104.

[0185] Among them, the first display module 1102 is configured to respond to an input instruction set as an operation interface, and display a first image and a second image of a target part of a biological object on the operation interface, wherein the target part includes a lesion area, the first image is an image obtained by scanning the target part using electronic computed tomography technology, and the second image is an image obtained by scanning the target part using magnetic resonance technology; the second display module 1104 is configured to respond to an image segmentation instruction set as an operation interface, and display a target segmentation result on the operation interface, wherein the target segmentation result is obtained by performing image segmentation on the target part included in the second image based on the first image and the cropped image, and the cropped image is obtained by cropping the second image based on the area of ​​the target part in the first image.

[0186] It should be noted that the first display module 1102 and the second display module 1104 described above correspond to steps S702 to S704 in Example 2. The examples and application scenarios implemented by the two modules and the corresponding steps are the same, but are not limited to the contents disclosed in Example 2. It should be noted that the above modules or units can be hardware components or software components stored in a memory and processed by one or more processors. The above modules can also be run as part of the device in the AR / VR device provided in Example 1.

[0187] It should be noted that the preferred implementation scheme involved in the above embodiments of the present disclosure is the same as the solution provided in Example 1, as well as the application scenario and implementation process, but is not limited to the solution provided in Example 1.

[0188] Example 7

[0189] According to an embodiment of the present disclosure, a device configured to implement the above-described image segmentation method is also provided. This device can be deployed in a target client. Figure 12 is a block diagram of an image segmentation device according to Embodiment 8 of the present disclosure. As shown in Figure 12, the device 1200 includes: a first display module 1202, a cropping module 1204, a segmentation module 1207, and a second display module 1204.

[0190] Among them, the first display module 1202 is configured to display a first image and a second image of a target part of a biological object on a presentation screen of a virtual reality VR device or an augmented reality AR device, wherein the target part includes a lesion area, the first image is an image obtained by scanning the target part using electronic computed tomography technology, and the second image is an image obtained by scanning the target part using magnetic resonance technology; the cropping module 1204 is configured to crop the second image based on the area of ​​the target part in the first image to obtain a cropped image; the segmentation module 1207 is configured to perform image segmentation on the target part contained in the second image based on the first image and the cropped image to obtain a target segmentation result; the second display module 1204 is configured to drive the VR device or AR device to render and display the target segmentation result.

[0191] It should be noted that the first display module 1202, cropping module 1204, segmentation module 1207, and second display module 1204 described above correspond to steps S802 to S809 in Example 3. The examples and application scenarios implemented by the four modules and the corresponding steps are the same, but are not limited to the contents disclosed in Example 3. It should be noted that the above modules or units can be hardware components or software components stored in a memory and processed by one or more processors. The above modules can also be run as part of the device in the AR / VR device provided in Example 1.

[0192] It should be noted that the preferred implementation scheme involved in the above embodiments of the present disclosure is the same as the solution provided in Example 1, as well as the application scenario and implementation process, but is not limited to the solution provided in Example 1.

[0193] Example 8

[0194] According to an embodiment of the present disclosure, a device configured to implement the above-described image segmentation method is also provided. This device can be deployed on a target client. Figure 13 is a block diagram of an image segmentation device according to Embodiment 9 of the present disclosure. As shown in Figure 13, the device 1300 includes an acquisition module 1302, a cropping module 1304, a segmentation module 1307, and an output module 1309.

[0195] Among them, the acquisition module 1302 is configured to acquire a first image and a second image of a target part of a biological object by calling a first interface, wherein the first interface includes a first parameter, the parameter value of the first parameter is the first image and the second image, the target part includes a lesion area, the first image is an image obtained by scanning the target part using electronic computed tomography technology, and the second image is an image obtained by scanning the target part using magnetic resonance technology; the cropping module 1304 is configured to crop the second image based on the area of ​​the target part in the first image to obtain a cropped image; the segmentation module 1307 is configured to perform image segmentation on the target part contained in the second image based on the first image and the cropped image to obtain a target segmentation result; the output module 1309 is configured to output the target segmentation result by calling a second interface, wherein the second interface includes a second parameter, and the parameter value of the second parameter is the target segmentation result.

[0196] It should be noted that the acquisition module 1302, cropping module 1304, segmentation module 1307, and output module 1309 described above correspond to steps S902 to S909 in Example 4. The examples and application scenarios implemented by the four modules and the corresponding steps are the same, but are not limited to the contents disclosed in Example 4. It should be noted that the above modules or units can be hardware components or software components stored in a memory and processed by one or more processors. The above modules can also be run as part of the device in the AR / VR device provided in Example 1.

[0197] It should be noted that the preferred implementation scheme involved in the above embodiments of the present disclosure is the same as the solution provided in Example 1, as well as the application scenario and implementation process, but is not limited to the solution provided in Example 1.

[0198] Example 9

[0199] The embodiment of the present disclosure may provide an electronic device, which may be any electronic device in a group of electronic devices. Optionally, in this embodiment, the electronic device may also be replaced by a terminal device such as a mobile terminal.

[0200] Optionally, in this embodiment, the electronic device may be located in at least one grid device among a plurality of grid devices of a computer grid.

[0201] In this embodiment, the above-mentioned electronic device can execute the program code of the following steps in the image segmentation method: obtaining a first image and a second image of a target part of a biological object, wherein the target part includes a lesion area, the first image is an image obtained by scanning the target part using electronic computed tomography technology, and the second image is an image obtained by scanning the target part using magnetic resonance technology; based on the area of ​​the target part in the first image, the second image is cropped to obtain a cropped image; based on the first image and the cropped image, the target part contained in the second image is segmented to obtain a target segmentation result.

[0202] Alternatively, Figure 14 is a block diagram of an electronic device according to Embodiment 10 of the present disclosure. As shown, the electronic device A may include: a processor 1402 and a memory 1404, wherein an executable program is stored; the processor is configured to execute the program, wherein when the program is executed, the method for generating a virtual object described in Embodiment 1 is executed.

[0203] Optionally, as shown in FIG14 , the electronic device A may further include: a storage controller and a peripheral interface, wherein the peripheral interface is connected to the radio frequency module, the audio module and the display.

[0204] The memory may be configured to store software programs and modules, such as the program instructions / modules corresponding to the image segmentation method and apparatus in the embodiments of the present disclosure. The processor executes the software programs and modules stored in the memory to perform various functional applications and data processing, thereby implementing the above-mentioned image segmentation method. The memory may include a high-speed random access memory and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely located relative to the processor, and these remote memories may be connected to the terminal A via a grid. Examples of the above-mentioned grid include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0205] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: cropping the second image based on the area of ​​the target part in the first image to obtain a cropped image, including: performing image segmentation on the target part contained in the first image to obtain an image segmentation result of the first image, wherein the image segmentation result is set to represent the area of ​​the target part in the first image; cropping the second image based on the image segmentation result to obtain a cropped image.

[0206] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: when the image segmentation result is set to represent the boundary area of ​​the edge of the target part in the first image, the second image is cropped based on the image segmentation result to obtain a cropped image, including: performing expansion processing on the image segmentation result based on preset expansion parameters to obtain an expansion processing result; performing erosion processing on the image segmentation result based on preset erosion parameters to obtain an erosion processing result; obtaining the difference between the expansion processing result and the erosion processing result to obtain the target boundary area; and cropping the second image based on the target boundary area to obtain a cropped image.

[0207] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: when the image segmentation result is set to the central area in the first image representing the center of the target part, the second image is cropped based on the image segmentation result to obtain a cropped image, including: eroding the image segmentation result based on preset erosion parameters to obtain the target center area; cropping the second image based on the target center area to obtain a cropped image.

[0208] The processor can call the information and application stored in the memory through the transmission device to execute the following steps: performing image segmentation on the target part contained in the second image based on the first image and the cropped image to obtain a target segmentation result, including: performing image segmentation on the first image, and performing image segmentation on the target part contained in the second image based on the first image and the cropped image to obtain multiple candidate segmentation results; performing error prediction on the multiple candidate segmentation results based on the first image and the second image to obtain segmentation errors of the multiple candidate segmentation results, wherein different segmentation errors are set to characterize the errors between different candidate segmentation results and preset segmentation results; based on the segmentation errors of the multiple candidate segmentation results, screening out the target segmentation result from the multiple candidate segmentation results.

[0209] The processor can call the information and application stored in the memory through the transmission device to execute the following steps: perform image segmentation on the first image, and perform image segmentation on the target part contained in the second image based on the first image and the cropped image to obtain multiple candidate segmentation results, including: perform image segmentation on the target part contained in the first image to obtain a first candidate segmentation result; splice the first image and the second image to obtain a spliced ​​image; perform image segmentation on the spliced ​​image to obtain a second candidate segmentation result; perform image segmentation on the spliced ​​image based on the cropped image to obtain a third candidate segmentation result; obtain multiple candidate segmentation results based on the first candidate segmentation result, the second candidate segmentation result and the third candidate segmentation result.

[0210] The processor can call the information and application stored in the memory through the transmission device to execute the following steps: perform image segmentation on the spliced ​​image based on the cropped image to obtain a third candidate segmentation result, including: extracting features from the spliced ​​image to obtain spliced ​​image features; extracting features from the cropped image to obtain cropped image features; modifying the spliced ​​image features based on the cropped image features to obtain modified image features; and decoding the modified image features to obtain the third candidate segmentation result.

[0211] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: based on the first image and the second image, error prediction is performed on multiple candidate segmentation results to obtain segmentation errors of the multiple candidate segmentation results, including: superimposing the multiple candidate segmentation results with the first image and the second image respectively to obtain multiple superimposed images; using the error prediction model to perform error prediction on the multiple superimposed images respectively to obtain segmentation errors of the multiple candidate segmentation results.

[0212] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: based on the segmentation errors of multiple candidate segmentation results, filter out the target segmentation result from multiple candidate segmentation results, including: determining the volume of the target part in multiple candidate segmentation results; based on the segmentation error and volume, filter out the target segmentation result from multiple candidate segmentation results.

[0213] In an embodiment of the present disclosure, a method is adopted in which a first image and a second image of a target part of a biological object are acquired; the second image is cropped based on the area of ​​the target part in the first image to obtain a cropped image; and the target part contained in the second image is segmented based on the first image and the cropped image to obtain a target segmentation result. The second image is cropped by using the area of ​​the target part in the first image to determine the cropped image of the target part in the second image, so as to reduce the influence of other images other than the cropped image in the second image on the process of segmenting the lesion area. Finally, the first image and the cropped image are used to segment the second image to obtain the target segmentation result, thereby improving the clarity of local details in the lesion area and making the segmented target segmentation result undistorted, thereby improving the segmentation accuracy of the lesion area, and further solving the technical problem of the accuracy of segmenting the lesion area in CT images and MRI images in the related art.

[0214] Those skilled in the art will appreciate that the structure shown in FIG14 is merely illustrative, and that the computer terminal may also be a smartphone (e.g., an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile internet device (MID), a PAD, or other terminal device. FIG14 does not limit the structure of the aforementioned electronic devices. For example, the computer terminal A may include more or fewer components (e.g., a grid interface, a display device, etc.) than those shown in FIG14 , or may have a configuration different from that shown in FIG14 .

[0215] A person skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer-readable storage medium, which may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0216] Example 10

[0217] The embodiment of the present disclosure further provides a computer-readable storage medium. Optionally, in this embodiment, the storage medium can be configured to store the program code executed by the image segmentation method provided in the first embodiment.

[0218] In this embodiment, the storage medium may be located in any computer terminal in a computer terminal group in a computer grid, or in any mobile terminal in a mobile terminal group.

[0219] In this embodiment, the storage medium is configured to store program codes configured to execute the following steps: obtaining a first image and a second image of a target part of a biological object, wherein the target part includes a lesion area, the first image is an image obtained by scanning the target part using electronic computed tomography technology, and the second image is an image obtained by scanning the target part using magnetic resonance technology; based on the area of ​​the target part in the first image, the second image is cropped to obtain a cropped image; and based on the first image and the cropped image, the target part included in the second image is segmented to obtain a target segmentation result.

[0220] In this embodiment, the storage medium is configured to store program code configured to execute the following steps: cropping the second image based on the area of ​​the target part in the first image to obtain a cropped image, including: performing image segmentation on the target part contained in the first image to obtain an image segmentation result of the first image, wherein the image segmentation result is set to represent the area of ​​the target part in the first image; cropping the second image based on the image segmentation result to obtain a cropped image.

[0221] In this embodiment, the storage medium is configured to store program codes configured to execute the following steps: when the image segmentation result is set to represent the boundary area of ​​the edge of the target part in the first image, the second image is cropped based on the image segmentation result to obtain a cropped image, including: performing expansion processing on the image segmentation result based on preset expansion parameters to obtain an expansion processing result; performing erosion processing on the image segmentation result based on preset erosion parameters to obtain an erosion processing result; obtaining the difference between the expansion processing result and the erosion processing result to obtain the target boundary area; and cropping the second image based on the target boundary area to obtain a cropped image.

[0222] In this embodiment, the storage medium is configured to store program codes configured to execute the following steps: when the image segmentation result is set to represent the center area of ​​the target part in the first image, the second image is cropped based on the image segmentation result to obtain a cropped image, including: eroding the image segmentation result based on preset erosion parameters to obtain the target center area; cropping the second image based on the target center area to obtain a cropped image.

[0223] In this embodiment, the storage medium is configured to store program codes configured to execute the following steps: performing image segmentation on the target part contained in the second image based on the first image and the cropped image to obtain a target segmentation result, including: performing image segmentation on the first image, and performing image segmentation on the target part contained in the second image based on the first image and the cropped image to obtain multiple candidate segmentation results; performing error prediction on the multiple candidate segmentation results based on the first image and the second image to obtain segmentation errors of the multiple candidate segmentation results, wherein different segmentation errors are set to characterize the errors between different candidate segmentation results and a preset segmentation result; based on the segmentation errors of the multiple candidate segmentation results, screening out the target segmentation result from the multiple candidate segmentation results.

[0224] In this embodiment, the storage medium is configured to store program codes configured to execute the following steps: performing image segmentation on the first image, and performing image segmentation on the target part contained in the second image based on the first image and the cropped image to obtain multiple candidate segmentation results, including: performing image segmentation on the target part contained in the first image to obtain a first candidate segmentation result; splicing the first image and the second image to obtain a spliced ​​image; performing image segmentation on the spliced ​​image to obtain a second candidate segmentation result; performing image segmentation on the spliced ​​image based on the cropped image to obtain a third candidate segmentation result; and obtaining multiple candidate segmentation results based on the first candidate segmentation result, the second candidate segmentation result, and the third candidate segmentation result.

[0225] In this embodiment, the storage medium is configured to store program code configured to execute the following steps: performing image segmentation on the spliced ​​image based on the cropped image to obtain a third candidate segmentation result, including: performing feature extraction on the spliced ​​image to obtain spliced ​​image features; performing feature extraction on the cropped image to obtain cropped image features; modifying the spliced ​​image features based on the cropped image features to obtain modified image features; and decoding the modified image features to obtain the third candidate segmentation result.

[0226] In this embodiment, the storage medium is configured to store program codes configured to execute the following steps: performing error prediction on multiple candidate segmentation results based on the first image and the second image to obtain segmentation errors of the multiple candidate segmentation results, including: superimposing the multiple candidate segmentation results with the first image and the second image respectively to obtain multiple superimposed images; using the error prediction model to perform error prediction on the multiple superimposed images respectively to obtain segmentation errors of the multiple candidate segmentation results.

[0227] In this embodiment, the storage medium is configured to store program code configured to perform the following steps: based on the segmentation errors of multiple candidate segmentation results, filtering out a target segmentation result from multiple candidate segmentation results, including: determining the volume of the target part in the multiple candidate segmentation results; based on the segmentation error and volume, filtering out the target segmentation result from the multiple candidate segmentation results.

[0228] In an embodiment of the present disclosure, a method is adopted in which a first image and a second image of a target part of a biological object are acquired; the second image is cropped based on the area of ​​the target part in the first image to obtain a cropped image; and the target part contained in the second image is segmented based on the first image and the cropped image to obtain a target segmentation result. The second image is cropped by using the area of ​​the target part in the first image to determine the cropped image of the target part in the second image, so as to reduce the influence of other images other than the cropped image in the second image on the process of segmenting the lesion area. Finally, the first image and the cropped image are used to segment the second image to obtain the target segmentation result, thereby improving the clarity of local details in the lesion area and making the segmented target segmentation result undistorted, thereby improving the segmentation accuracy of the lesion area, and further solving the technical problem of the accuracy of segmenting the lesion area in CT images and MRI images in the related art.

[0229] The serial numbers of the above-mentioned embodiments of the present disclosure are for description only and do not represent the advantages or disadvantages of the embodiments.

[0230] In the above embodiments of the present disclosure, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0231] In the several embodiments provided in this disclosure, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0232] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0233] In addition, the functional units in the various embodiments of the present disclosure may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0234] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the various embodiments of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0235] The above is only a preferred embodiment of the present disclosure. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present disclosure. These improvements and modifications should also be regarded as within the scope of protection of the present disclosure. Industrial Applicability

[0236] The solution provided by the embodiment of the present disclosure can be set as a method of acquiring a first image and a second image of a target part of a biological object during image processing; cropping the second image based on the area of ​​the target part in the first image to obtain a cropped image; performing image segmentation on the target part contained in the second image based on the first image and the cropped image to obtain a target segmentation result. By cropping the second image using the area of ​​the target part in the first image, a cropped image of the target part in the second image is determined to reduce the influence of other images other than the cropped image in the second image on the process of segmenting the lesion area. Finally, the first image and the cropped image are used to segment the second image to obtain a target segmentation result, thereby improving the clarity of local details in the lesion area and ensuring that the segmented target segmentation result is not distorted, thereby improving the segmentation accuracy of the lesion area, and further solving the technical problem of the accuracy of segmenting the lesion area in CT images and MRI images in related technologies.

Claims

1. A method for image segmentation, wherein: include: Acquire a first image and a second image of a target part of a biological object, wherein the target part includes a lesion area, the first image is an image obtained by scanning the target part using electronic computer tomography technology, and the second image is an image obtained by scanning the target part using magnetic resonance technology; Based on the area of ​​the target part in the first image, cropping the second image to obtain a cropped image; Image segmentation is performed on the target part included in the second image based on the first image and the cropped image to obtain a target segmentation result.

2. The method according to claim 1, wherein: Cropping the second image based on the area of ​​the target part in the first image to obtain a cropped image, including: Performing image segmentation on the target part contained in the first image to obtain an image segmentation result of the first image, wherein the image segmentation result is set to represent a region of the target part in the first image; The second image is cropped based on the image segmentation result to obtain the cropped image.

3. The method according to claim 2, wherein: In a case where the image segmentation result is set to represent a boundary region of an edge of the target part in the first image, cropping the second image based on the image segmentation result to obtain the cropped image includes: Performing expansion processing on the image segmentation result based on preset expansion parameters to obtain an expansion processing result; Performing erosion processing on the image segmentation result based on preset erosion parameters to obtain an erosion processing result; Obtaining a difference between the dilation processing result and the erosion processing result to obtain a target boundary area; The second image is cropped based on the target boundary area to obtain the cropped image.

4. The method according to claim 2, wherein: In a case where the image segmentation result is set to represent that the center of the target part is in the central area of ​​the first image, cropping the second image based on the image segmentation result to obtain the cropped image includes: Performing erosion processing on the image segmentation result based on preset erosion parameters to obtain a target center area; The second image is cropped based on the target central area to obtain the cropped image.

5. The method according to claim 1, wherein: Performing image segmentation on the target part included in the second image based on the first image and the cropped image to obtain a target segmentation result includes: Performing image segmentation on the first image, and performing image segmentation on the target part included in the second image based on the first image and the cropped image to obtain a plurality of candidate segmentation results; Performing error prediction on the multiple candidate segmentation results based on the first image and the second image to obtain segmentation errors of the multiple candidate segmentation results, wherein different segmentation errors are set to represent errors between different candidate segmentation results and a preset segmentation result; The target segmentation result is selected from the plurality of candidate segmentation results based on segmentation errors of the plurality of candidate segmentation results.

6. The method according to claim 5, wherein: Performing image segmentation on the first image, and performing image segmentation on the target part included in the second image based on the first image and the cropped image, to obtain a plurality of candidate segmentation results, including: Performing image segmentation on the target part included in the first image to obtain a first candidate segmentation result; splicing the first image and the second image to obtain a spliced ​​image; Performing image segmentation on the stitched image to obtain a second candidate segmentation result; Performing image segmentation on the stitched image based on the cropped image to obtain a third candidate segmentation result; The plurality of candidate segmentation results are obtained based on the first candidate segmentation result, the second candidate segmentation result and the third candidate segmentation result.

7. The method according to claim 6, wherein: Performing image segmentation on the stitched image based on the cropped image to obtain a third candidate segmentation result includes: Extracting features from the stitched image to obtain stitched image features; Performing feature extraction on the cropped image to obtain cropped image features; Modifying the stitched image features based on the cropped image features to obtain modified image features; The modified image features are decoded to obtain the third candidate segmentation result.

8. The method according to claim 5, wherein: Performing error prediction on the plurality of candidate segmentation results based on the first image and the second image to obtain segmentation errors of the plurality of candidate segmentation results includes: Superimposing the plurality of candidate segmentation results with the first image and the second image respectively to obtain a plurality of superimposed images; The error prediction model is used to perform error prediction on the multiple superimposed images respectively to obtain segmentation errors of the multiple candidate segmentation results.

9. The method according to claim 5, wherein: Based on the segmentation errors of the multiple candidate segmentation results, screening out the target segmentation result from the multiple candidate segmentation results includes: Determining the volume of the target part in the plurality of candidate segmentation results; The target segmentation result is selected from the plurality of candidate segmentation results based on the segmentation error and the volume.

10. A method for image segmentation, wherein: include: In response to the action being set as an input instruction on an operation interface, a first image and a second image of a target part of a biological object are displayed on the operation interface, wherein the target part includes a lesion area, the first image is an image obtained by scanning the target part using electronic computer tomography technology, and the second image is an image obtained by scanning the target part using magnetic resonance technology; In response to the action being set as an image segmentation instruction on the operation interface, a target segmentation result is displayed on the operation interface, wherein the target segmentation result is obtained by performing image segmentation on the target part contained in the second image based on the first image and the cropped image, and the cropped image is obtained by cropping the second image based on the area of ​​the target part in the first image.

11. [Corrected 24.05.2024 in accordance with Rule 91] A method for image segmentation, wherein: include: Displaying a first image and a second image of a target part of a biological object on a presentation screen of a virtual reality (VR) device or an augmented reality (AR) device, wherein the target part includes a lesion area, the first image is an image obtained by scanning the target part using electronic computed tomography technology, and the second image is an image obtained by scanning the target part using magnetic resonance technology; Based on the area of ​​the target part in the first image, cropping the second image to obtain a cropped image; Performing image segmentation on the target part included in the second image based on the first image and the cropped image to obtain a target segmentation result; The VR device or the AR device is driven to render and display the target segmentation result.

12. [Corrected 24.05.2024 in accordance with Rule 91] A method for image segmentation, wherein: include: Acquire a first image and a second image of a target part of a biological object by calling a first interface, wherein the first interface includes a first parameter, a parameter value of the first parameter is the first image and the second image, the target part includes a lesion area, the first image is an image obtained by scanning the target part using electronic computer tomography technology, and the second image is an image obtained by scanning the target part using magnetic resonance technology; Based on the area of ​​the target part in the first image, cropping the second image to obtain a cropped image; Performing image segmentation on the target part included in the second image based on the first image and the cropped image to obtain a target segmentation result; The target segmentation result is output by calling a second interface, wherein the second interface includes a second parameter, and a parameter value of the second parameter is the target segmentation result.

13. [Corrected 24.05.2024 in accordance with Rule 91] An electronic device, wherein: include: A memory storing an executable program; A processor, configured to run the program, wherein the program executes the method according to any one of claims 1 to 12 when running.

14. [Corrected 24.05.2024 in accordance with Rule 91] A computer-readable storage medium, wherein The computer-readable storage medium includes a stored executable program, wherein when the executable program is executed, the device where the computer-readable storage medium is located is controlled to execute the method according to any one of claims 1 to 12.

Images