Method for processing an orbital image, image processing device, medium and system

Abstract

This application relates to the field of image processing technology, and discloses a method, image processing device, medium, and system for processing orbital images. The method includes: acquiring a first orbital image; segmenting a target of interest (ROI) in the first orbital image to obtain a first segmented image labeled with the region where the ROI is located; acquiring multiple orbital structure location images, each corresponding to a type of orbital structure; calculating the degree of overlap between the region where the ROI is located in the first segmented image and the region where the orbital structure is located in each of the orbital structure location images; and using the degree of overlap, determining the spatial positional relationship between the ROI and each orbital structure to obtain the image processing result corresponding to the first orbital image. By implementing the method of this application, the spatial positional relationship between the ROI and the orbital structure can be automatically and accurately determined.

Description

Technical Field

[0001] This application relates to the field of image processing technology, specifically to methods, equipment, media, and systems for processing orbital images. Background Technology

[0002] The orbit is a bony cavity that houses the visual organs, containing structures such as the eyeball, optic nerve, lacrimal gland, extraocular muscles, blood vessels, and fat. However, existing image segmentation techniques can only segment various orbital structures, but cannot automatically and accurately determine the spatial relationship between the target of interest, such as an orbital mass, and various orbital structures. Summary of the Invention

[0003] This application provides a method, image processing device, medium, and system for processing orbital images to solve the problem in related technologies that it is impossible to automatically and accurately determine the spatial positional relationship between the target of interest and the orbital structure.

[0004] In a first aspect, this application provides a method for processing an orbital image, the method comprising: acquiring a first orbital image; segmenting a target of interest in the first orbital image to obtain a first segmented image labeled with the region where the target of interest is located; acquiring multiple orbital structure location images, each orbital structure location image corresponding to a type of orbital structure; calculating the degree of overlap between the region where the target of interest is located in the first segmented image and the region where the orbital structure is located in each orbital structure location image; and using the degree of overlap to determine the spatial positional relationship between the target of interest and each orbital structure, thereby obtaining an image processing result corresponding to the first orbital image.

[0005] The orbital image processing method provided in this embodiment first segments the region of interest from the first orbital structure to obtain a first segmented image labeled with the region of interest. Then, by measuring the degree of overlap between the region of interest in the first segmented image and the regions of orbital structures in each orbital structure location image, the spatial positional relationship between the target of interest and each orbital structure is determined. Since the spatial relationship of the corresponding orbital structures is known in each orbital structure location image, the spatial positional relationship between the target of interest and each orbital structure can be cleverly and flexibly determined by measuring the degree of overlap, achieving automatic and accurate determination of spatial positional relationships with a relatively small overall computational load.

[0006] In one optional implementation, the process of constructing the orbital structure location image includes: acquiring a second orbital image, the second orbital image including a target orbital region, the target orbital region not including the target of interest; segmenting the orbital structure in the target orbital region in the second orbital image to obtain a second segmented image labeled with the regions where each orbital structure is located, the second segmented image including segmented regions corresponding to the target orbital region; mirroring the second segmented image to obtain a third segmented image; generating a fourth segmented image based on the segmented regions corresponding to the target orbital region in the third segmented image and the second segmented image; and using the fourth segmented image to generate the orbital structure location image.

[0007] By segmenting the target orbital region that does not include the target of interest on one side, a first segmented image is obtained. Then, by mirroring and flipping, a fourth segmented image can be cleverly constructed, in which both orbital regions are segmented regions. This solves the problem of not being able to collect orbital images that do not include the target of interest on both sides. Finally, the fourth segmented image can be used to generate images of the location of each orbital structure in a relatively clever way.

[0008] In one optional implementation, there are multiple fourth segmented images, and each fourth segmented image has label information corresponding to each orbital structure. Generating an orbital structure location image using the fourth segmented images includes: obtaining a first reference space; registering the image space of the fourth segmented images to the first reference space to obtain a first registered image; determining the probability of each orbital structure at a voxel location based on the label information corresponding to the same voxel location in all first registered images; and for each orbital structure, generating an orbital structure location image corresponding to the orbital structure based on the probability of the orbital structure at all voxel locations.

[0009] By registering the image space of the fourth segmented image to the first reference space, multiple fourth segmented images from different sources, spatial locations, poses, or sizes can be unified into the same spatial coordinate system. This ensures that the same voxel coordinates in different fourth segmented images correspond to the same orbital structure. Consequently, accurate spatial alignment is guaranteed when performing statistical and probability calculations on label information at the same voxel location, avoiding probability statistics errors caused by spatial mismatches and improving the accuracy of orbital structure location images.

[0010] In one optional implementation, obtaining a first orbital image includes: obtaining a first initial orbital image, the first initial orbital image including at least a first sequence of images; in response to the first initial orbital image being missing at least one of a second sequence of images and a third sequence of images, inputting the first sequence of images into an image generation model, so that the image generation model generates at least one of a second sequence of images and a third sequence of images based on the first sequence of images, obtaining at least one of a second sequence of images and a third sequence of images; obtaining the first orbital image based on at least one of the second sequence of images and the third sequence of images and the first initial orbital image.

[0011] By using an image generation model to complete the missing sequence of images in the first initial orbital image, the imaging information of the first orbital image can be ensured to be complete. Subsequently, based on the first orbital image, the target of interest can be segmented, which can further make the first segmented image more accurate, and further ensure that the positional relationship between the identified target of interest and each orbital structure is more accurate.

[0012] In one optional implementation, segmenting the target of interest in the first orbital image to obtain a first segmented image with the region where the target of interest is located includes: determining the image space corresponding to the first sequence of images as a second reference space; registering the image spaces of the second sequence of images and the third sequence of images in the first orbital image to the second reference space to obtain a registered first orbital image; and inputting the registered first orbital image into an image segmentation model so that the image segmentation model segments the target of interest in the registered first orbital image to obtain the first segmented image.

[0013] By registering the second and third image sequences to a second reference space, the image spaces of the first, second, and third image sequences can be unified under the same spatial coordinate system, achieving precise spatial alignment. Then, inputting the registered first orbital image into an image segmentation model for target of interest segmentation further avoids segmentation errors caused by spatial misalignment and voxel coordinate mismatch between different image sequences.

[0014] In one optional implementation, the degree of overlap between the region of interest in the first segmented image and the region of the orbital structure in each orbital structure position image is calculated, including: determining the image space corresponding to the first sequence of images as the second reference space; registering the image space of the orbital structure position image to the second reference space to obtain a second registered image; and calculating the degree of overlap between the region of interest in the first segmented image and the region of the orbital structure in each second registered image.

[0015] Registering all images of the orbital structures to a unified second reference space ensures spatial consistency between the second registered images and the first segmented image, further improving the accuracy of overlap calculation.

[0016] In one optional implementation, the spatial relationship between the target of interest and each orbital structure is determined by utilizing the various degrees of overlap, and the image processing result corresponding to the first orbital image is obtained. This includes: inputting the registered first orbital image, the first segmented image, and each degree of overlap into an image classification model, so that the image classification model crops the registered first orbital image based on the first segmented image to obtain bilateral orbital region images, and extracts features from the bilateral orbital region images and each degree of overlap to obtain the image processing result.

[0017] The image classification model can crop the registered first orbital image based on the first segmented image to obtain bilateral orbital region images. In this way, only the bilateral orbital regions in the bilateral orbital region images need to be considered, without having to consider other regions, thereby reducing computational complexity.

[0018] Secondly, this application provides an image processing apparatus, including: a memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to perform the orbital image processing method described in the first aspect or any corresponding embodiment.

[0019] Thirdly, this application provides a computer-readable storage medium storing computer instructions for causing a computer to execute the orbital image processing method described in the first aspect or any corresponding embodiment.

[0020] Fourthly, this application provides an image processing system, including: a storage device for storing a first orbital image; and an image processing device communicatively connected to the storage device. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the specific embodiments of this application or the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0022] Figure 1 This is a schematic diagram illustrating an application scenario according to an embodiment of this application; Figure 2 This is a schematic flowchart of a first method for processing orbital images according to an embodiment of this application; Figure 3 This is a schematic diagram of a second process for processing an orbital image according to an embodiment of this application; Figure 4 A flowchart of a specific method for processing an orbital image according to an embodiment of this application; Figure 5 This is a structural block diagram of an orbital image processing apparatus according to an embodiment of this application; Figure 6 This is a schematic diagram of the hardware structure of an image processing device according to an embodiment of this application. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0024] It is understood that before using the technical solutions disclosed in the various embodiments of this application, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this application in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0025] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0026] The orbit is a bony cavity that houses the visual organs, containing structures such as the eyeball, optic nerve, lacrimal gland, extraocular muscles, blood vessels, and fat. However, existing image segmentation techniques can only segment various orbital structures, but cannot automatically and accurately determine the spatial relationship between the target of interest, such as an orbital mass, and various orbital structures.

[0027] In view of this, this application segments the disease-free orbital region in the second orbital image and constructs a segmented image with no disease on both sides by mirroring and flipping it left and right; then, based on the label information in the segmented image with no disease on both sides, the position information of each orbital structure in the second orbital image is determined, thereby obtaining the position image corresponding to each orbital structure; finally, based on the degree of overlap between the target of interest, such as the orbital mass in the first orbital image and the orbital structure in each position image, the spatial positional relationship between the target of interest and the orbital structure is determined, thus realizing the automatic and accurate determination of the spatial positional relationship between the target of interest and the orbital structure.

[0028] As one optional application scenario in the embodiments of this application, such as Figure 1 As shown, the image processing system may include a storage device 101 and an image processing device 102. The storage device 101 and the image processing device 102 can establish a data connection, which may include a communication connection or a direct access connection.

[0029] Storage device 101 can be a tablet computer, laptop computer, handheld computer, desktop computer, game console, smart TV, smart wearable device, vehicle terminal, VR (Virtual Reality) device, AR (Augmented Reality) device, hard drive, USB flash drive, SD card, etc.

[0030] The image processing device 102 can be a tablet computer, a laptop computer, a handheld computer, or a desktop computer, a game console, a smart TV, a smart wearable device, an in-vehicle terminal, a VR device, an AR device, etc.

[0031] The storage device 101 and the image processing device 102 can also establish a communication connection through a network, wherein the network can be a wired network or a wireless network, and examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, wide area networks, mobile communication networks, and combinations thereof.

[0032] According to an embodiment of this application, an embodiment of a method for processing orbital images is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0033] This embodiment provides a method for processing orbital images, which can be used in image processing devices. Figure 2 This is a flowchart of a method for processing orbital images according to an embodiment of this application, such as... Figure 2 As shown, the process includes the following steps: Step S201: Obtain the first orbital image.

[0034] The first orbital image can be a medical image of both orbital regions to be processed. As a specific example, the first orbital image can be an MRI image, CT image, ultrasound image, or other image that reflects the internal tissue structure of the orbit.

[0035] The first orbital image can include multiple modalities of image sequences, such as plain scan T1-weighted imaging (T1WI), plain scan T2-weighted imaging (T2WI), and plain scan T1-weighted contrast-enhanced imaging (T1C).

[0036] As a specific example, when the storage device has independent communication capabilities, such as a server, personal computer, etc., the image processing device can establish a communication connection with the storage device. Therefore, the image processing device can initiate an image acquisition request to the storage device. The storage device can respond to the image acquisition request sent by the image processing device and send the first orbital image to the image processing device. Accordingly, the image processing device can acquire the first orbital image.

[0037] As another specific example, when the storage device does not have independent communication capabilities, such as a USB flash drive, hard drive, etc., the image processing device can be directly connected to the storage device. Therefore, the image processing device can directly access the storage device to obtain the first orbital image.

[0038] Step S202: Segment the target of interest in the first orbital image to obtain a first segmented image with the region where the target of interest is located marked.

[0039] The target of interest (ROI) can be a predefined segmentation object in the first orbital image that requires region identification, segmentation, and location analysis. The region of ROI can be the area occupied by the ROI in the first orbital image. As a specific example, the region of ROI can be a local region in the first orbital image with independent grayscale, signal, or morphological features, including but not limited to abnormal signal regions, high-density regions, low-density regions, locally protruding regions, locally thickened regions, regions with abnormal textures, regions with grayscale differences, occupied regions, foreign body regions, and marked regions. For example, the ROI can be an orbital mass, and the region of ROI can be the area occupied by the orbital mass in the first orbital image.

[0040] Here, we can first identify the first orbital image to obtain the region where the target of interest is located in the first orbital image, and then mark the region where the target of interest is located in the first orbital image to obtain the first segmented image with the region where the target of interest is located marked.

[0041] In addition, in some cases, while segmenting the target of interest in the first orbital image, the orbital structures in the first orbital image, such as the eyeball, medial rectus muscle, lateral rectus muscle, optic nerve and optic nerve sheath, lacrimal gland, intraconical space (excluding the optic nerve region), extraconical space (excluding the lacrimal gland region), and other regions within the orbit, can also be segmented. Label information is added to the region where each orbital structure is located, thus obtaining a first segmented image labeled with the regions where the orbital structures and the target of interest are located. The label information can be a numerical value such as 1-8, or other numerical values ​​or characters; this application does not specifically limit this.

[0042] Step S203: Obtain multiple orbital structure location images, each orbital structure location image corresponding to a type of orbital structure.

[0043] Orbital structure location images can be images that label the regions containing a single type of orbital structure. As a specific example, multiple orbital structure location images may include, but are not limited to, images of the eyeball, medial rectus muscle, lateral rectus muscle, optic nerve and optic nerve sheath, lacrimal gland, intraconical space (excluding the optic nerve region), extraconical space (excluding the lacrimal gland region), and other regions within the orbit.

[0044] Furthermore, each voxel location in the orbital structure location image can also be labeled with the probability that the voxel location is the corresponding orbital structure. For example, each voxel location in the eyeball location image can also be labeled with the probability that the voxel location is the eyeball.

[0045] As a concrete example, an image processing device may include a database that stores images of the locations of various orbital structures. Therefore, the image processing device can access the database to obtain images of the locations of these orbital structures.

[0046] As a specific example, images of the locations of each orbital structure can also be stored in a storage device, from which the image processing device can retrieve the images. For details on how the image processing device retrieves the first orbital image from the storage device, please refer to the method described here; it will not be repeated here.

[0047] Furthermore, it should be understood that the first segmented image and the images of each orbital structure location are in the same reference space. Therefore, the first segmented image and the images of each orbital structure location have the same image coordinate system, size and spatial correspondence. Thus, the first segmented image and the images of each orbital structure location are spatially comparable.

[0048] Step S204: Calculate the degree of overlap between the region of interest in the first segmented image and the region of the orbital structure in each orbital structure location image.

[0049] The degree of overlap can be used to characterize the spatial overlap between the region of interest in the first segmented image and the region of the orbital structure in each orbital structure location image.

[0050] For example, the degree of overlap can be the intersection-union ratio between the region where the target of interest is located and the region where the orbital structures are located.

[0051] For example, the degree of overlap can be the ratio of the overlap volume to the image volume, where the overlap volume can be the volume of the region where the target of interest is located and the region where the orbital structure is located overlaps with each other, and the image volume can be the volume corresponding to the first segmented image or the image of the orbital structure location.

[0052] For example, the degree of overlap can be the ratio of the number of first voxels to the number of second voxels, where the number of first voxels can be the number of voxels corresponding to the region where the target of interest is located in the first segmented image, and the number of second voxels can be the number of voxels corresponding to the region where the orbital structure is located in the orbital structure location image.

[0053] Step S205: Using the degree of overlap, determine the spatial positional relationship between the target of interest and each orbital structure, and obtain the image processing result corresponding to the first orbital image.

[0054] Spatial positional relationships can be defined as the relative positions, inclusion relationships, adjacency relationships, or overlap relationships between the target of interest and various orbital structures in the first segmented image.

[0055] As a specific example, multiple overlap levels can be sorted from largest to smallest to obtain a sorting result. The orbital structure corresponding to the first overlap level in the sorting result can be identified as the target orbital structure, and the image processing result of the target of interest being located above the target orbital structure can be obtained.

[0056] The orbital image processing method provided in this embodiment first segments the region of interest from the first orbital structure to obtain a first segmented image labeled with the region of interest. Then, by measuring the degree of overlap between the region of interest in the first segmented image and the regions of orbital structures in each orbital structure location image, the spatial positional relationship between the target of interest and each orbital structure is determined. Since the spatial relationship of the corresponding orbital structures is known in each orbital structure location image, the spatial positional relationship between the target of interest and each orbital structure can be cleverly and flexibly determined by measuring the degree of overlap, achieving automatic and accurate determination of spatial positional relationships with a relatively small overall computational load.

[0057] This embodiment provides a method for processing orbital images, which can be used in image processing devices. Figure 3 This is a flowchart of a method for processing orbital images according to an embodiment of this application, such as... Figure 3 As shown, the process includes the following steps: Step S301: Obtain the first orbital image.

[0058] Specifically, step S301 includes: Step S3011: Obtain a first initial orbital image, which includes at least a first sequence of images.

[0059] The first sequence image here can be a plain scan T1-weighted sequence image. Furthermore, it can be detected whether the first initial orbital image simultaneously contains a plain scan T1WI sequence image, a plain scan T2-weighted sequence image, and a plain scan T1-weighted enhanced sequence image. If the first initial orbital image includes a plain scan T1WI sequence image, a plain scan T2WI sequence image, and a T1C sequence image, then the first initial orbital image can be directly identified as the first orbital image.

[0060] In step S3012, in response to the absence of at least one of the second sequence image and the third sequence image in the first initial orbital image, the first sequence image is input into the image generation model so that the image generation model generates at least one of the second sequence image and the third sequence image based on the first sequence image, thereby obtaining at least one of the second sequence image and the third sequence image.

[0061] If the first initial orbital image is missing a plain scan T2WI sequence image and / or a T1C sequence image, then an image generation model can be used to generate a plain scan T2WI sequence image and / or a T1C sequence image based on the T1WI sequence image in the first initial orbital image.

[0062] The image generation model can be a pre-trained adversarial model. As a specific example, its training process can be as follows: an initial image generation model is built based on the CycleGAN framework; the first dataset is input into the initial image generation model, which learns the mapping relationship from T1WI sequence images to T2WI sequence images, and from T1WI sequence images to T1C sequence images, and continuously adjusts the model parameters of the initial image generation model to obtain the final image generation model.

[0063] The first dataset includes multiple third orbital images, and each third orbital image includes a T1WI sequence image, a T2WI sequence image, and a T1C sequence image, with all three sequences within the same image space. For example, the image spaces of the T2WI and T1C sequences in the third orbital image are registered to the T1WI sequence image, thus ensuring that all three sequences in the third orbital image reside in the same image space.

[0064] In addition, during the training of the initial image generation model, the generator can adopt the U-Net structure, the discriminator can adopt 70×70 PatchGAN, be trained for 200 training epochs, and use the Adam optimizer.

[0065] Step S3013: Based on at least one of the second sequence image and the third sequence image and the first initial orbital image, a first orbital image is obtained.

[0066] Here, at least one of the second sequence image and the third sequence image output by the image generation model can be combined with the first initial orbital image to obtain the first orbital image.

[0067] Since different image sequences have different focuses, if any one of the first, second, or third sequence images is missing from the first orbital image, the imaging information will be incomplete. Here, the missing sequence images in the first initial orbital image are supplemented by an image generation model, which ensures the integrity of the imaging information in the first orbital image. Subsequent segmentation of the target of interest based on the first orbital image can further improve the accuracy of the first segmented image, and further ensure that the positional relationship between the identified target of interest and each orbital structure is more accurate.

[0068] Step S302: Segment the target of interest in the first orbital image to obtain a first segmented image with the region where the target of interest is located marked.

[0069] Specifically, step S302 includes: Step S3021: Determine the image space corresponding to the first sequence of images as the second reference space.

[0070] As mentioned earlier, the first orbital image includes T1WI, T2WI, and T1C sequences. Since different sequences have different image spaces, it is necessary to register the T1WI, T2WI, and T1C sequences to the same image space. In some cases, the image space corresponding to the first sequence can be designated as the second reference space. In other cases, a second reference space can be directly selected, and the first, second, and third sequences can all be registered to this selected second reference space.

[0071] Step S3022: Register the image spaces of the second sequence image and the third sequence image in the first orbital image to the second reference space to obtain the registered first orbital image.

[0072] Here, both the second and third sequence images are registered to the second reference space, so that the first, second, and third sequence images in the first orbital image are all in the same image space.

[0073] In addition, the first orbital image after registration here may include a first sequence image after registration, a second sequence image after registration, and a third sequence image after registration.

[0074] Step S3023: Input the registered first orbital image into the image segmentation model so that the image segmentation model can segment the target of interest in the registered first orbital image to obtain the first segmented image.

[0075] As a specific example, after the registered first orbital image is input into the image segmentation model, the image segmentation model can also segment the orbital structures in the bilateral orbital regions of the registered first orbital image to obtain a first segmented image labeled with the region where the orbital structure is located and the region where the target of interest is located.

[0076] The image segmentation model can be a pre-trained deep learning model based on nnU-Net. The training process of the image segmentation model is similar to that of the image generation model described earlier, and will not be repeated here.

[0077] Additionally, the first segmented image here may include a first sequence of images labeled with the region where the target of interest is located, a second sequence of images labeled with the region where the target of interest is located, and a third sequence of images labeled with the region where the target of interest is located.

[0078] By registering the second and third image sequences to a second reference space, the image spaces of the first, second, and third image sequences can be unified under the same spatial coordinate system, achieving precise spatial alignment. Then, inputting the registered first orbital image into an image segmentation model for target of interest segmentation further avoids segmentation errors caused by spatial misalignment and voxel coordinate mismatch between different image sequences.

[0079] Step S303: Obtain multiple orbital structure location images, each orbital structure location image corresponding to a type of orbital structure.

[0080] Specifically, the process of constructing each orbital structure location image may include: Step S3031: Obtain a second orbital image. The second orbital image includes the target orbital region, but the target orbital region does not include the target of interest.

[0081] The second orbital image can be a pre-collected sample orbital image, and there can be multiple second orbital images. Furthermore, the second orbital image is similar to the first orbital image, both being medical images of the bilateral orbital regions. For the bilateral orbits in the second orbital image, one side can be the target orbital region excluding the target of interest, while the other side is the non-target orbital region including the target of interest.

[0082] As a specific example, if the target of interest is an orbital mass, then the target orbital region is the area that does not include the orbital mass. Therefore, in the second orbital image, one side of the orbital region contains the orbital mass, while the other side does not. Furthermore, the process of acquiring the second orbital image is similar to that of acquiring the first orbital image; please refer to the previous text for details, which will not be repeated here.

[0083] Step S3032: In the second orbital image, the target orbital region is segmented into orbital structures to obtain a second segmented image labeled with the regions where each orbital structure is located. The second segmented image includes the segmented regions corresponding to the target orbital region.

[0084] Here, we can first identify each orbital structure in the target orbital region of the second orbital image, segment the region where each orbital structure is located in the target orbital region, and then mark the region where each orbital structure is located in the second orbital image to obtain a second segmented image marked with the region where each orbital structure is located.

[0085] In addition, orbital structure segmentation is performed only on the target orbital region that does not contain the target of interest. This avoids affecting the accuracy of orbital structure segmentation of the target of interest, thus making the second segmented image more accurate.

[0086] Here, a target orbit region segmentation model can be pre-trained, and then the second orbit image can be input into the target orbit region segmentation model so that the target orbit region segmentation model can identify, segment and label the target orbit region in the second orbit image to obtain the second segmented image.

[0087] Step S3033: Mirror the second segmented image to obtain the third segmented image.

[0088] Since the aforementioned steps only segment the target orbital region, that is, only the orbital region on the side that does not contain the region of interest, the other orbital region containing the region of interest remains unsegmented. Here, mirroring the second segmented image flips the target orbital region with the segmented region to the orbital region on the other side containing the region of interest, and vice versa. This ensures that both the second and third segmented images contain the target orbital region with a segmented region on one side.

[0089] For example, in the second orbital image, the left orbit includes an orbital mass, while the right orbit does not. Therefore, orbital structure segmentation is performed on the right orbit to obtain the second segmented image. Then, the second segmented image is mirrored. Thus, the right orbit containing the segmented region can be flipped to the region where the left orbit is located, and the region where the left orbit is located can be flipped to the region where the right orbit is located, resulting in the third segmented image. Furthermore, both the second and third segmented images can contain segmented regions.

[0090] As a specific example, mirror flipping can be a left-right mirror flip, where a left-right mirror flip can be performed along the vertical midline corresponding to the second segmented image, swapping the left and right sides to obtain the third segmented image.

[0091] Step S3034: Generate a fourth segmentation image based on the segmentation region corresponding to the target orbital region in the third segmentation image and the second segmentation image.

[0092] Since both the second and third segmented images contain segmented regions, that is, both the second and third segmented images contain segmented regions corresponding to the target eye socket region, the second and third segmented images are merged and the relevant content that does not have segmented regions is removed, thus obtaining a fourth segmented image with segmented regions on both sides.

[0093] Step S3035: Use the fourth segmented image to generate an image of the location of the orbital structure.

[0094] Here, structural features such as edges, contours, corners, and connected regions of each orbital structure can be extracted from the fourth segmented image; then, based on the extracted structural features of each orbital structure, an orbital structure location image corresponding to each orbital structure can be generated.

[0095] By segmenting the target orbital region that does not include the target of interest on one side, a first segmented image is obtained. Then, by mirroring and flipping, a fourth segmented image can be cleverly constructed, in which both orbital regions are segmented regions. This solves the problem of not being able to collect orbital images that do not include the target of interest on both sides. Finally, the fourth segmented image can be used to generate images of the location of each orbital structure in a relatively clever way.

[0096] In some cases, there can be multiple fourth segmentation images, and each fourth segmentation image includes label information corresponding to each orbital structure. The label information can be used to specifically represent the corresponding orbital structure. For example, label 1 can be used to represent the eyeball, label 2 can be used to represent the extraocular muscles, and so on.

[0097] In an optional implementation, step S3035 includes: Step a1: Obtain the first reference space.

[0098] Step a2: Register the image space of the fourth segmented image to the first reference space to obtain the first registered image.

[0099] Step a3: Based on the label information corresponding to the same voxel position in all first registration images, determine the probability of each orbital structure at the voxel position.

[0100] Step a4: For each orbital structure, generate an orbital structure location image corresponding to the orbital structure based on the probability of the orbital structure at all voxel locations.

[0101] Here, the first reference space is a pre-defined or selected standard spatial coordinate system used to provide a unified spatial reference for multiple fourth segmentation images. As a specific example, the image space of any one of the multiple fourth segmentation images can also be determined as the first reference space.

[0102] By registering the image space of the fourth segmented image to the first reference space, multiple fourth segmented images from different sources, spatial locations, poses, or sizes can be unified into the same spatial coordinate system. This ensures that the same voxel coordinates in different fourth segmented images correspond to the same orbital structure. Consequently, accurate spatial alignment is guaranteed when performing statistical and probability calculations on label information at the same voxel location, avoiding probability statistics errors caused by spatial mismatches and improving the accuracy of orbital structure location images.

[0103] Furthermore, it should be understood that the second orbital image may include T1WI sequence images, T2WI sequence images, and T1-enhanced sequence images. Therefore, the fourth segmentation image obtained based on the second orbital image also includes the segmentation images corresponding to the T1WI sequence images, the T2WI sequence images, and the T1-enhanced sequence images.

[0104] Furthermore, if the second orbital image lacks at least one of the T2WI sequence image and the T1 enhanced sequence image, the T1WI sequence image included in the second orbital image can be input into the image generation model so that the image generation model can generate at least one of the corresponding T2WI sequence image and the T1 enhanced sequence image based on the T1WI sequence image included in the second orbital image.

[0105] Step S304: Calculate the degree of overlap between the region of interest in the first segmented image and the region of the orbital structure in each orbital structure location image.

[0106] Specifically, step S304 includes: Step S3041: The image space corresponding to the first sequence of images is determined as the second reference space. Please refer to the previous text for details.

[0107] Step S3042: Register the image space of the orbital structure position image to the second reference space to obtain the second registered image.

[0108] As mentioned earlier, before segmenting the target of interest in the first orbital image, the second and third sequence images in the first orbital image have been registered to the second reference space. That is, all three sequence images in the first orbital image are in the second reference space. Correspondingly, the first segmentation image is also in the second reference space, while the images of each orbital structure location are in the first reference space. Therefore, it is necessary to register the image space of the orbital structure location images to the second reference space to obtain the second registered image. This ensures that the second registered image and the first segmentation image are in the same reference space, i.e., the second reference space.

[0109] Step S3043: Calculate the overlap between the region of interest in the first segmented image and the region of the orbital structure in each of the second registered images. See the previous text for details.

[0110] Registering all images of the orbital structures to a unified second reference space ensures spatial consistency between the second registered images and the first segmented image, further improving the accuracy of overlap calculation.

[0111] Step S305: Using the different degrees of overlap, determine the spatial relationship between the target of interest and each orbital structure, and obtain the image processing result corresponding to the first orbital image.

[0112] Specifically, step S305 includes: Step S3051: The registered first orbital image, the first segmented image, and the various degrees of overlap are input into the image classification model, so that the image classification model can crop the registered first orbital image based on the first segmented image to obtain bilateral orbital region images, and extract features from the bilateral orbital region images and the various degrees of overlap to obtain the image processing result.

[0113] As a specific example, the various degrees of overlap can be saved into a table to obtain target table data. The registered first orbital image, the target table data, and the first segmented image are then input into an image classification model to obtain the image processing result.

[0114] The image classification model can crop the registered first orbital image based on the first segmented image to obtain bilateral orbital region images. In this way, only the bilateral orbital regions in the bilateral orbital region images need to be considered, without having to consider other regions, thereby reducing computational complexity.

[0115] In some cases, an image classification model may include a multilayer perceptron module, a convolutional module, and a fully connected layer. Specifically, the convolutional module can extract features from bilateral orbital region images to obtain a first feature vector; the multilayer perceptron module can extract features from multiple overlap levels to obtain a second feature vector; then, the first and second feature vectors can be concatenated to obtain a concatenated feature vector; finally, the fully connected layer processes the concatenated feature vector to obtain the final image processing result.

[0116] The training process for the image classification model is similar to that for the image generation model, and will not be repeated here. As a specific example, the image processing result can also include the location of each orbital structure in the first orbital image and the degree of overlap between them.

[0117] The orbital image processing method provided in this embodiment obtains a second segmented image by segmenting the target orbital region in a second orbital image. Then, by mirroring the second segmented image, a fourth segmented image is cleverly constructed, which includes the segmented regions corresponding to the target orbital region on both sides. Furthermore, the location images of each orbital structure are constructed using the fourth segmented image. Subsequently, by measuring the degree of overlap between the region of interest in the first segmented image and the regions of each orbital structure location image, the spatial relationship between the target of interest and each orbital structure can be automatically and accurately determined.

[0118] As a specific embodiment of this application, such as Figure 4 The image shown illustrates a specific method for processing an orbital image, comprising steps S401 to S407. Wherein, Step S401: Obtain a first initial orbital image, which includes at least a first sequence of images.

[0119] Step S402: Detect whether the first initial orbital image is missing at least one of the second sequence image and the third sequence image. If missing, use an image generation model to generate at least one of the second sequence image and the third sequence image based on the first sequence image to obtain the first orbital image.

[0120] Step S403: Register the image spaces of the second sequence image and the third sequence image in the first orbital image to the image space of the first sequence image to obtain the registered first orbital image.

[0121] Step S404: Use an image segmentation model to segment the target of interest in the registered first orbital image to obtain the first segmented image.

[0122] Step S405: Register the image space of each orbital structure location image to the image space of the first sequence of images to obtain multiple second registered images.

[0123] Step S406: Calculate the degree of overlap between the region of interest in the first segmented image and the region of the orbital structure in each of the second registration images.

[0124] Step S407: Input the registered first orbital image, the first segmented image, and the various overlap levels into the image classification model to obtain the image processing results.

[0125] This embodiment also provides an image processing device for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0126] This embodiment provides an image processing device, including a memory and a processor. The memory and the processor are communicatively connected. The memory stores computer instructions, and the processor executes the computer instructions to perform a method for processing orbital images.

[0127] In some cases, the image processing apparatus may also include means for processing orbital images, which is implemented by a processor. For example... Figure 5As shown, the processing device for the orbital image includes: The first acquisition module 510 is configured to acquire a first orbital image.

[0128] The segmentation module 520 is configured to segment the target of interest in the first orbital image to obtain a first segmented image with the region where the target of interest is located marked.

[0129] The second acquisition module 530 is configured to acquire multiple orbital structure location images, each orbital structure location image corresponding to a type of orbital structure.

[0130] The calculation module 540 is configured to calculate the degree of overlap between the region of interest in the first segmented image and the region of the orbital structure in each orbital structure location image.

[0131] The determination module 550 is configured to use various degrees of overlap to determine the spatial positional relationship between the target of interest and each orbital structure, and obtain the image processing result corresponding to the first orbital image.

[0132] In some optional implementations, the second acquisition module 530 is further configured to acquire a second orbital image, the second orbital image including a target orbital region, the target orbital region not including the target of interest; segment the target orbital region in the second orbital image to obtain a second segmented image labeled with the regions where each orbital structure is located, the second segmented image including segmented regions corresponding to the target orbital region; mirror the second segmented image to obtain a third segmented image; generate a fourth segmented image based on the segmented regions corresponding to the target orbital region in the third segmented image and the second segmented image; and generate an orbital structure location image using the fourth segmented image.

[0133] In some optional implementations, there are multiple fourth segmented images, and each fourth segmented image includes label information corresponding to each orbital structure; the second acquisition module 530 is further configured to acquire a first reference space; register the image space of the fourth segmented image to the first reference space to obtain a first registered image; determine the probability of each orbital structure at the voxel position based on the label information corresponding to the same voxel position in all first registered images; and for each orbital structure, generate an orbital structure position image corresponding to the orbital structure based on the probability of the orbital structure at all voxel positions.

[0134] In some optional implementations, the first acquisition module 510 is further configured to acquire a first initial orbital image, the first initial orbital image including at least a first sequence image; in response to the first initial orbital image being missing at least one of a second sequence image and a third sequence image, the first sequence image is input to an image generation model, so that the image generation model generates at least one of a second sequence image and a third sequence image based on the first sequence image, thereby obtaining at least one of a second sequence image and a third sequence image; and based on at least one of the second sequence image and the third sequence image and the first initial orbital image, a first orbital image is obtained.

[0135] In some optional implementations, the segmentation module 520 is further configured to determine the image space corresponding to the first sequence of images as the second reference space; register the image spaces of the second sequence of images and the third sequence of images in the first orbital image to the second reference space to obtain the registered first orbital image; and input the registered first orbital image into the image segmentation model so that the image segmentation model segments the target of interest in the registered first orbital image to obtain the first segmented image.

[0136] In some optional implementations, the calculation module 540 is further configured to determine the image space corresponding to the first sequence of images as the second reference space; register the image space of the orbital structure location image to the second reference space to obtain a second registered image; and calculate the degree of overlap between the region occupied by the target of interest in the first segmented image and the region where the orbital structure is located in each of the second registered images.

[0137] In some optional implementations, the determining module 550 is further configured to input the registered first orbital image, the first segmented image, and each degree of overlap into the image classification model, so that the image classification model crops the registered first orbital image based on the first segmented image to obtain bilateral orbital region images, and extracts features from the bilateral orbital region images and each degree of overlap to obtain image processing results.

[0138] Figure 6 This is a schematic diagram of the structure of an image processing device provided in an embodiment of this application.

[0139] The following is a detailed reference. Figure 6This diagram illustrates a suitable structure for implementing an image processing device according to embodiments of this application. The image processing device may include a processor (e.g., a central processing unit, graphics processing unit, etc.) 601, which can perform various appropriate actions and processes based on a program stored in read-only memory (ROM) 602 or a program loaded from memory 608 into random access memory (RAM) 603. RAM 603 also stores various programs and data required for the operation of the image processing device. The processor 601, ROM 602, and RAM 603 are interconnected via bus 604. Input / output (I / O) interface 605 is also connected to bus 604.

[0140] Typically, the following devices can be connected to I / O interface 605: input devices 606 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 607 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; memory devices 608 including, for example, magnetic tapes, hard disks, etc.; and communication devices 609. Communication device 609 allows the image processing device to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 6 An image processing apparatus with various devices is shown, but it should be understood that it is not required to implement or have all of the devices shown, and more or fewer devices may be implemented or have instead.

[0141] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 609, or installed from a memory 608, or installed from a ROM 602. When the computer program is executed by the processor 601, it performs the functions defined in the orbital image processing method of embodiments of this application.

[0142] Figure 6 The image processing device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0143] This application also provides a computer-readable storage medium. The methods described in this application can be implemented in hardware or firmware, or implemented as recordable on a storage medium, or implemented as computer code downloaded over a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the orbital image processing method shown in the above embodiments is implemented.

[0144] A portion of this application can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to this application through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0145] Although embodiments of this application have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of this application, and all such modifications and variations fall within the scope defined by the appended claims.

Claims

1. A method for processing an orbital image, characterized in that, The method includes: Obtain the first orbital image; The target of interest in the first orbital image is segmented to obtain a first segmented image labeled with the region where the target of interest is located; Multiple orbital structure location images are acquired, and each orbital structure location image corresponds to a type of orbital structure. Calculate the degree of overlap between the region of interest in the first segmented image and the region of the orbital structure in each of the orbital structure location images; By utilizing the degree of overlap, the spatial positional relationship between the target of interest and each of the orbital structures is determined, and the image processing result corresponding to the first orbital image is obtained.

2. The method according to claim 1, characterized in that, The process of constructing the orbital structure location image includes: A second orbital image is acquired, the second orbital image including a target orbital region, wherein the target orbital region does not include the target of interest; In the second orbital image, the target orbital region is segmented into orbital structures to obtain a second segmented image labeled with the regions where each orbital structure is located. The second segmented image includes segmented regions corresponding to the target orbital region. The second segmented image is mirrored to obtain the third segmented image; A fourth segmentation image is generated based on the segmentation regions corresponding to the target orbital region in the third segmentation image and the second segmentation image; The location image of the orbital structure is generated using the fourth segmented image.

3. The method according to claim 2, characterized in that, There are multiple fourth segmented images, and each fourth segmented image has label information corresponding to each of the orbital structures; The step of generating the orbital structure location image using the fourth segmented image includes: Obtain the first reference space; The image space of the fourth segmented image is registered to the first reference space to obtain the first registered image; Based on the label information corresponding to the same voxel position in all the first registration images, the probability of each orbital structure at the voxel position is determined; For each of the orbital structures, an orbital structure position image corresponding to the orbital structure is generated based on the probability of the orbital structure at all voxel positions.

4. The method according to claim 1, characterized in that, The acquisition of the first orbital image includes: Acquire a first initial orbital image, wherein the first initial orbital image includes at least a first sequence of images; In response to the absence of at least one of the second sequence image and the third sequence image in the first initial orbital image, the first sequence image is input into the image generation model so that the image generation model generates at least one of the second sequence image and the third sequence image based on the first sequence image, thereby obtaining at least one of the second sequence image and the third sequence image; The first orbital image is obtained based on at least one of the second sequence image and the third sequence image, as well as the first initial orbital image.

5. The method according to claim 4, characterized in that, The step of segmenting the target of interest in the first orbital image to obtain a first segmented image labeled with the region where the target of interest is located includes: The image space corresponding to the first sequence of images is determined as the second reference space; The image spaces of the second sequence image and the third sequence image in the first orbital image are registered to the second reference space to obtain the registered first orbital image; The registered first orbital image is input into the image segmentation model so that the image segmentation model can segment the target of interest in the registered first orbital image to obtain the first segmented image.

6. The method according to claim 5, characterized in that, The step of calculating the overlap between the region of interest in the first segmented image and the region of the orbital structure in each of the orbital structure location images includes: The image space corresponding to the first sequence of images is determined as the second reference space; The image space of the orbital structure location image is registered to the second reference space to obtain the second registered image; The degree of overlap between the region of interest in the first segmented image and the region of the orbital structure in each of the second registration images is calculated.

7. The method according to claim 5, characterized in that, The step of determining the spatial relationship between the target of interest and each of the orbital structures by utilizing the degree of overlap, and obtaining the image processing result corresponding to the first orbital image, includes: The registered first orbital image, the first segmented image, and each degree of overlap are input into an image classification model, so that the image classification model can crop the registered first orbital image based on the first segmented image to obtain bilateral orbital region images, and extract features from the bilateral orbital region images and each degree of overlap to obtain the image processing result.

8. An image processing device, characterized in that, include: A memory and a processor are communicatively connected, the memory storing computer instructions, and the processor executing the computer instructions to perform the method for processing orbital images according to any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a computer to perform the method for processing an orbital image as described in any one of claims 1 to 7.

10. An image processing system, characterized in that, include: Storage device for storing the first orbital image; The image processing device of claim 8 is communicatively connected to the storage device.

Images