Image processing method, device, electronic device and storage medium

Abstract

The present disclosure provides an image processing method that relates to the fields of artificial intelligence technology, particularly deep learning technology and image processing technology, and can be applied in image processing scenarios. A specific implementation scheme comprises: determining multiple target subregions in a target image based on the detection results and segmentation results of the target image; fusing the multiple target subregions based on the distances between them to obtain a target region; and obtaining target information based on the target region and the segmentation results. The present disclosure also provides an image processing device, an electronic device, and a storage medium.

Description

Technical Field

[0001] The present disclosure relates to the fields of artificial intelligence technology, particularly deep learning technology and image processing technology, and can be applied to image processing scenarios. More specifically, the present disclosure provides an image processing method, apparatus, electronic device, and storage medium. Background Art

[0002] With the development of artificial intelligence technology, the use of artificial intelligence technology to process two-dimensional or three-dimensional images has been widely used. Summary of the Invention

[0003] The present disclosure provides an image processing method, apparatus, device, and storage medium.

[0004] According to one aspect of the present disclosure, an image processing method is provided, the method comprising: determining a plurality of target sub-regions in the target image based on a detection result of the target image and a segmentation result of the target image; fusing the plurality of target sub-regions based on distances between the plurality of target sub-regions to obtain a target region; and obtaining target information based on the target region and the segmentation result.

[0005] According to another aspect of the present disclosure, an image processing device is provided, which includes: a determination module for determining multiple target sub-regions in the target image based on a detection result of the target image and a segmentation result of the target image; a fusion module for fusing the multiple target sub-regions based on distances between the multiple target sub-regions to obtain a target region; and an acquisition module for obtaining target information based on the target region and the segmentation result.

[0006] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method provided according to the present disclosure.

[0007] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided. The computer instructions are used to cause a computer to execute the method provided according to the present disclosure.

[0008] According to another aspect of the present disclosure, a computer program product is provided, comprising a computer program, which implements the method provided according to the present disclosure when executed by a processor.

[0009] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The accompanying drawings are provided to facilitate a better understanding of the present invention and do not constitute a limitation of the present disclosure.

[0011] Figure 1 is a schematic diagram of an exemplary system architecture to which the image processing method and apparatus according to one embodiment of the present disclosure can be applied;

[0012] Figure 2 is a flowchart of an image processing method according to an embodiment of the present disclosure;

[0013] Figure 3 is a flowchart of an image processing method according to another embodiment of the present disclosure;

[0014] Figure 4 is a schematic diagram of an image processing method according to an embodiment of the present disclosure;

[0015] Figure 5 is a flowchart of an image processing method according to another embodiment of the present disclosure;

[0016] Figure 6 is a schematic diagram of an image processing method according to another embodiment of the present disclosure;

[0017] Figure 7 is a block diagram of an image processing apparatus according to an embodiment of the present disclosure; and

[0018] Figure 8 is a block diagram of an electronic device to which an image processing method can be applied according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0019] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0020] Semantic segmentation of a 3D image can yield at least one mask. For example, based on a CT (Computed Tomography) image, it is possible to determine whether a lesion exists within the liver. In one example, semantic segmentation can be performed on a CT image of a liver with a lesion present to yield at least one lesion region. However, semantic segmentation alone may result in missed detections.

[0021] Figure 1 is a schematic diagram of an exemplary system architecture to which the image processing method and apparatus according to one embodiment of the present disclosure can be applied. It should be noted that, Figure 1 The examples shown are merely examples of system architectures to which the embodiments of the present disclosure may be applied, to help those skilled in the art understand the technical content of the present disclosure, but do not mean that the embodiments of the present disclosure may not be used in other devices, systems, environments or scenarios.

[0022] like Figure 1 As shown, the system architecture 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used as a medium for providing communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0023] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Terminal devices 101, 102, and 103 can be various electronic devices with display screens and support web browsing, including but not limited to smartphones, tablet computers, laptop computers, and desktop computers, etc.

[0024] The server 105 may be a server that provides various services, such as a background management server (for example only) that supports websites browsed by users using the terminal devices 101, 102, and 103. The background management server may analyze and process received data such as user requests, and feed back processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0025] It should be noted that the image processing method provided in the embodiments of the present disclosure can generally be executed by the server 105. Accordingly, the image processing apparatus provided in the embodiments of the present disclosure can generally be set in the server 105. The image processing method provided in the embodiments of the present disclosure can also be executed by a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105. Accordingly, the image processing apparatus provided in the embodiments of the present disclosure can also be set in a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105.

[0026] Figure 2 is a flowchart of an image processing method according to an embodiment of the present disclosure.

[0027] like Figure 2 As shown, the method 200 may include operations S210 to S230.

[0028] In operation S210 , a plurality of object sub-regions are determined in the object image according to a detection result of the object image and a segmentation result of the object image.

[0029] For example, the target image may be a CT image of the liver. It is understood that the target image may also be a CT image or a fluorescence imaging image of other target objects, and the present disclosure is not limited thereto.

[0030] For example, target detection can be performed on a target image to obtain a detection result. In one example, a lesion detection model can be used to perform target detection on a target image. The lesion detection model can be a deep learning model built based on a CNN (Convolutional Neural Network) model.

[0031] For another example, according to the detection result, at least one target sub-region may be determined in the target image.

[0032] For example, image segmentation (e.g., semantic segmentation) can be performed on the target image to obtain a segmentation result. In one example, the target image can be segmented using a lesion segmentation model. The lesion segmentation model can be a deep learning model built based on a U-Net (U-net) model.

[0033] For another example, according to the segmentation result, at least one target sub-region may be determined in the target image.

[0034] For example, multiple target subregions may be obtained according to at least one target subregion determined by the detection result and at least one target subregion determined by the segmentation result.

[0035] In operation S220 , the plurality of target sub-regions are merged according to the distances between the plurality of target sub-regions to obtain a target region.

[0036] For example, there may be K target areas, where K is an integer greater than or equal to 1.

[0037] For example, the Euclidean distance between the center points of multiple target subregions can be determined as the distance between the multiple target subregions. Multiple target subregions can be fused using various methods to obtain K target regions. In one example, at least two target subregions whose distance is less than a preset distance threshold can be fused to obtain a single target region.

[0038] In operation S230 , target information is obtained according to the target region and the segmentation result.

[0039] For example, the target information may include mask information corresponding to each target region. In one example, the segmentation result may correspond to at least one mask information. When determining a target subregion in the target image based on the segmentation result, the mask information may be correspondingly added to the target subregion determined by the segmentation result. Accordingly, among the K target regions, at least a portion of the K target regions may have mask information.

[0040] In an example, for a target region without mask information among the K target regions, its mask information may be determined in various ways, which is not limited in the present disclosure.

[0041] By integrating object detection and semantic segmentation into the two image processing methods, the disclosed embodiments can accurately identify the target region within the target image. For example, the region where a liver lesion is located in a CT image containing the liver can be accurately determined, thus preventing some lesions from being missed.

[0042] In some embodiments, determining multiple target sub-regions in the target image based on the detection results of the target image and the segmentation results of the target image includes: performing target detection on the target image to obtain a detection result, wherein the detection result includes at least one sub-detection box; and determining multiple target sub-regions in the target image based on the at least one sub-detection box and the segmentation result.

[0043] In some embodiments, determining multiple target sub-regions in the target image based on the detection results of the target image and the segmentation results of the target image includes: performing a first image segmentation process on the target image to obtain at least one first mask information; obtaining a segmentation result based on the at least one first mask information, wherein the segmentation result includes at least one sub-bounding box determined based on the at least one first mask information; and determining multiple target sub-regions in the target image based on the detection result and the at least one sub-bounding box.

[0044] The following will be combined Figure 3 Provide detailed explanation.

[0045] Figure 3 is a flowchart of an image processing method according to another embodiment of the present disclosure.

[0046] like Figure 3 As shown, the method 310 can determine multiple target sub-regions in the target image based on the detection result and the segmentation result of the target image. Detailed description will be given below in conjunction with operations S311 to S314.

[0047] In operation S311 , target detection is performed on the target image to obtain a detection result.

[0048] For example, the detection result includes at least one sub-detection box.

[0049] For example, as described above, the lesion detection model can be used to perform target detection on the target image.

[0050] For example, the lesion detection model can be constructed based on a 3D CNN model. The lesion detection model can be constructed according to, for example, [s z , s y , s x ] has a step size and size of [w z , w y , w x ] stereo window to perform target detection on the target image. The detection result may include at least one sub-detection frame. Each sub-detection frame has a center point (z_D, y_D, x_D). Furthermore, the distance d_D from the center point to the vertex of the sub-detection frame can be determined. Each sub-detection frame can be represented by [z_D, y_D, x_D, d_D, p]. p is the confidence of each sub-detection frame. In one example, the confidence p of each sub-detection frame output by the lesion detection model is greater than 0.5.

[0051] In operation S312 , a first image segmentation process is performed on the target image to obtain at least one first mask information.

[0052] For example, as described above, the lesion segmentation model may be used to perform a first image segmentation process on the target image.

[0053] For example, the lesion segmentation model can be constructed based on the 3D UNet model. The lesion segmentation model can be constructed according to, for example, [s′ z , s′ y , s′ x The step size and size of ] is [w′ z , w′ y , w′ x] performs semantic segmentation on the target image through a stereo window to obtain at least one mask information.

[0054] In operation S313 , a segmentation result is obtained according to the at least one first mask information.

[0055] For example, the segmentation result includes at least one sub-bounding box determined according to at least one first mask information.

[0056] For another example, at least one sub-bounding box can be determined as a segmentation result based on at least one mask information. That is, the segmentation result includes at least one sub-bounding box. Each sub-bounding box has a center point (z_S, y_S, x_S). Furthermore, the distance d_S from the center point to the vertex of the sub-bounding box can be determined. Each sub-bounding box can be represented by [z_S, y_S, x_S, d_S]. In one example, w′ z =w′ y =w′ x =3.

[0057] In operation S314, a plurality of object sub-regions are determined in the object image based on the at least one sub-detection frame and the at least one sub-bounding frame.

[0058] For example, at least one sub-detection frame and at least one sub-bounding frame can be mapped to the image space where the target image resides. The area defined by the at least one sub-detection frame is used as the at least one target sub-region. The area defined by the at least one sub-bounding frame is also used as the at least one target sub-region. Furthermore, multiple target sub-regions can be determined in the target image.

[0059] It should be noted that operation S311 can be performed first, and then operations S312 to S313 are performed, but the embodiments of the present disclosure are not limited to this. These two groups of operations can also be performed in other orders. For example, operation S311 and operations S312 to S313 can be performed in parallel, or operations S312 to S313 can be performed first, and then operation S311 is performed.

[0060] In some embodiments, as described above, when semantic segmentation is performed using the lesion segmentation model, it is possible to use, for example, [s′ z , s′ y , s′ x The step size and size of ] is [w′ z , w′ y , w′ x The target image is semantically segmented using a stereoscopic window of ] to determine the category (lesion or background) of each voxel point within the stereoscopic window. Connected domains are calculated based on the voxel points classified as lesions to obtain at least one connected domain, thereby obtaining at least one first mask information.

[0061] In some embodiments, unlike method 310, the lesion detection model can also be constructed based on a 2D CNN model. The target image can be divided into multiple 2D slice images along the depth direction. The lesion detection model is used to perform target detection on each slice image to obtain a detection result for the slice image. The depth information of each slice image within the target image is then used to obtain a detection result for the target image based on the detection results of the multiple slice images.

[0062] In other embodiments, determining multiple target sub-regions in the target image based on the detection results and at least one sub-enclosing box includes: for each target sub-region in at least one target sub-region determined by at least one sub-enclosing box, adding each first mask information corresponding to each sub-enclosing box to each target sub-region determined by each sub-enclosing box.

[0063] Alternatively, in other embodiments, determining multiple target sub-regions in the target image based on at least one sub-detection frame and at least one sub-enclosing frame includes: mapping each sub-enclosing frame and the first mask information corresponding to each sub-enclosing frame to the image space where the target image is located, to obtain each target sub-region determined by each sub-enclosing frame.

[0064] According to the embodiment of the present disclosure, the target sub-region determined by the sub-bounding box may include mask information.

[0065] Figure 4 is a schematic diagram of an image processing method according to an embodiment of the present disclosure.

[0066] like Figure 4 As shown, the target image 401 can be input into the lesion segmentation model 410 to obtain the segmentation result. The target image 401 can also be input into the lesion detection model 420 to obtain the detection result. According to the detection result and the segmentation result, the target information 402 can be obtained. Figure 4 As shown, the target image may be an image containing the liver.

[0067] In some embodiments, there are K target areas, where K is an integer greater than or equal to 1. Based on the distances between the multiple target sub-areas, multiple target sub-areas are fused to obtain the target area, including: determining K target sub-area groups based on the distances between the multiple target sub-areas, wherein the distance between each target sub-area in each target sub-area group and the target sub-area in other target sub-area groups is greater than a preset distance threshold; and fusing each target sub-area group in the K target sub-area groups to obtain the K target areas.

[0068] For example, as described above, the Euclidean distance between the center points of multiple target subregions can be determined as the distance between the multiple target subregions. Furthermore, K target subregion groups can be determined, each of which can include at least one target subregion. The distance between each target subregion in each target subregion group and a target subregion in another target subregion group is greater than a preset distance threshold D. In one example, for a target subregion group including at least two target subregions, the distance between at least two target subregions within the group is less than another preset distance threshold d.

[0069] For another example, the target sub-region may be a small cube-shaped region.

[0070] For another example, when a target sub-region group including at least two target sub-regions is fused, the center point (z, y, x) of the target sub-region group can be determined based on the center points of at least two target sub-regions in the group. A point P with the largest distance to the center point (z, y, x) of the target sub-region group can be determined from at least two target sub-regions in the group. Based on the point P and the center point (z, y, x) of the target sub-region group, a distance value d_ori is determined. Based on the coordinates (z, y, x) of the center point and the distance value d_ori, a large cube-shaped area is constructed as a target area. In one example, a maximum value d_max can be selected from the distance value d_ori of at least one target area. Based on the maximum value d_max and the center point of each target sub-region group, each cube area is constructed as each target area.

[0071] In some embodiments, there are K target areas, where K is an integer greater than or equal to 1. According to the target areas and the segmentation results, obtaining the target information includes: adding the preset mask information to the N target areas among the K target areas that do not contain the first mask information to obtain N target sub-information; and obtaining the target information based on the N target sub-information and at least one first mask information.

[0072] For example, as described above, a target subregion defined by a sub-bounding frame may contain the first mask information. However, a target subregion defined by a sub-detection frame may not contain the first mask information. Furthermore, a target region formed by fusing at least one target subregion defined by a sub-detection frame may also not contain the first mask information. There may be N target regions without the first mask information, where N is an integer greater than or equal to 1, and N is an integer less than or equal to K.

[0073] For another example, a preset cube region can be constructed based on the center point of the target region and the preset distance value d_preset. The center point of the cube region is the center point of the target region. The distance from the vertex of the cube region to the center point is the preset distance value d_preset described above. In one example, the preset distance value d_preset is less than or equal to the maximum value d_max described above.

[0074] Next, the values ​​of the voxels within the cube region may be set as preset mask values, so as to add the preset mask information to a target region that does not contain the first mask information.

[0075] In another example, a sphere may be constructed based on the center point of the target area and the preset distance value d_preset. The center point of the sphere is the center point of the target area, and the radius of the sphere is the preset distance value d_preset described above.

[0076] In some embodiments, the number of target areas is K, where K is an integer greater than or equal to 1. Obtaining target information based on the target areas and the segmentation results includes: classifying the K target areas to obtain classification results; determining, based on the classification results, M target areas of a preset category from the K target areas, where M is an integer greater than or equal to 1; and obtaining target information based on the M target areas and the segmentation results. Figure 5 Provide detailed explanation.

[0077] Figure 5 is a flowchart of an image processing method according to another embodiment of the present disclosure.

[0078] like Figure 5 As shown, the method 530 can obtain target information according to the target area and the segmentation result, which will be described in detail below in conjunction with operations S531 to S536.

[0079] In operation S531 , the i-th target area is classified to obtain an i-th category value.

[0080] For example, a classification model may be used to classify the i-th target region to obtain the i-th category value. In one example, the classification model may be constructed based on a 3D CNN model.

[0081] In operation S532 , it is determined whether the i-th category value is greater than or equal to a preset category threshold.

[0082] In an embodiment of the present disclosure, in response to determining that the i-th category value is greater than or equal to the preset category threshold, operation S533 is performed.

[0083] In an embodiment of the present disclosure, in response to determining that the i-th category value is less than the preset category threshold, operation S536 is performed to delete the i-th target area.

[0084] For example, the preset category threshold is 0.5.

[0085] The following will take the i-th category value being greater than the preset category threshold as an example and describe in detail in combination with operations S534 and S535.

[0086] In operation S533 , it is determined whether the i-th target region includes first mask information.

[0087] In an embodiment of the present disclosure, in response to determining that the i-th target region includes the first mask information, operation S534 is performed.

[0088] For example, if there is a target sub-region determined by a sub-bounding box in at least one target sub-region fused to form the i-th target region, it can be determined that the i-th target region contains at least one first mask information.

[0089] In an embodiment of the present disclosure, in response to determining that the i-th target region does not include the first mask information, operation S535 is performed.

[0090] For example, if at least one target sub-region fused to form the i-th target region is a target sub-region determined by a sub-detection frame, it can be determined that the i-th target region does not include the first mask information.

[0091] In operation S534 , the i-th target sub-information is obtained according to the at least one first mask information included in the i-th target region.

[0092] For example, when the i-th target region includes the first mask information, the i-th target sub-information may be determined according to the first mask information included in the i-th target region.

[0093] In operation S535 , the preset mask information is added to the i-th target region to obtain the i-th target sub-information.

[0094] For example, as described above, a cube region can be constructed based on the center point of the i-th target region and the preset distance value d_preset. The center point of the cube region is the center point of the i-th target region. The distance from the vertex of the cube region to the center point is the preset distance value d_preset described above.

[0095] Next, the values ​​of the voxels within the cube region may be set as preset mask values, so as to add the preset mask information to the i-th target region.

[0096] Next, the i-th target sub-information may be determined based on the preset mask information contained in the i-th target area.

[0097] In some embodiments, i=1, ... K. For example, the method 530 may be performed on K target regions in sequence to obtain M target sub-information.

[0098] In some embodiments, based on the detection result of the target image and the segmentation result of the target image, determining multiple target sub-regions in the target image includes: determining a local image based on the grayscale value of each voxel in the original image; performing a second image segmentation process on the local image to obtain second mask information; determining a target bounding box based on the second mask information; and determining the target image based on the target bounding box and the original image. Figure 6 Provide detailed explanation.

[0099] Figure 6 is a schematic diagram of an image processing method according to an embodiment of the present disclosure.

[0100] For example, the original image may include an original CT image of all organs of the subject, and the range of the grayscale value of the voxel of each organ in the original image is different.

[0101] For example, the original image is a three-dimensional image with a size of Z ori , Y ori , X ori In one example, Z ori =Y ori =X ori =512.

[0102] The range of grayscale values ​​corresponding to the liver is [HU min , HU max ], the original image can be cropped according to the range, the image area corresponding to the range is retained, and the grayscale value in the image area is normalized to determine the local image 603 containing the liver. The size of the local image 603 is Z ls , Y ls , X ls .

[0103] In one example, each grayscale value HU within the image region may be normalized by the following formula:

[0104]

[0105] HU' is the grayscale value after normalization. min is the minimum grayscale value corresponding to the liver, HU max is the maximum grayscale value corresponding to the liver.

[0106] The local image 603 can be input into the liver extraction model 600 to obtain the second mask information 604. An original bounding box is then determined based on the second mask information 604 so that the second mask information 604 can be contained in the original bounding box. The original bounding box is then expanded. In one example, the original bounding box is a cubic bounding box. The lower right vertex of the original bounding box is [z0, y0, x0], and the upper left vertex of the original bounding box is [z1, y1, x1]. The expansion coefficient can be used And the expansion parameter w is expanded. The lower right vertex after expansion is The upper left vertex after expansion is Based on the lower right vertex and the upper left vertex after expansion, a cubic target bounding box can be determined.

[0107] The original image can be cropped according to the target bounding box to obtain an image area. The image area can be adjusted so that the size of the adjusted image area is Z ls , Y ls , X ls In one example, the image region may be cropped or supplemented with voxels with a grayscale value of 0, so that the size of the adjusted image region is Z ls , Y ls , X ls According to the second mask information, the grayscale values ​​of the voxels in the adjusted image region that do not correspond to the liver are adjusted to 0 to obtain a target image 601 .

[0108] It should be noted that the target image in each embodiment described above is an image containing the liver as an example. However, the target image in the present disclosure is not limited thereto, and the target image may also be an image containing organs such as the lungs, kidneys, or heart.

[0109] Figure 7 is a block diagram of an image processing apparatus according to an embodiment of the present disclosure.

[0110] like Figure 7 As shown, the apparatus 700 may include a determination module 710 , a fusion module 720 and an acquisition module 730 .

[0111] The determination module 710 is configured to determine a plurality of target sub-regions in the target image according to a detection result of the target image and a segmentation result of the target image.

[0112] The fusion module 720 is configured to fuse the multiple target sub-regions according to the distances between the multiple target sub-regions to obtain a target region.

[0113] The obtaining module 730 is configured to obtain target information according to the target area and the segmentation result.

[0114] In some embodiments, the determination module includes: a target detection submodule, used to perform target detection on the target image to obtain the detection result, wherein the detection result includes at least one sub-detection frame; and a first determination submodule, used to determine the multiple target sub-areas in the target image based on the at least one sub-detection frame and the segmentation result.

[0115] In some embodiments, the determination module includes: a first image segmentation submodule, used to perform first image segmentation processing on the target image to obtain at least one first mask information; a first acquisition submodule, used to obtain the segmentation result based on the at least one first mask information, wherein the segmentation result includes at least one sub-bounding box determined based on the at least one first mask information; and a second determination submodule, used to determine the multiple target sub-areas in the target image based on the detection result and the at least one sub-bounding box.

[0116] In some embodiments, there are K target areas, where K is an integer greater than or equal to 1, and the acquisition module includes: an adding submodule, used to add the preset mask information to N target areas among the K target areas that do not contain the first mask information, to obtain N target sub-information, where N is an integer greater than or equal to 1 and N is an integer less than or equal to K; and a second acquisition submodule, used to obtain the target information based on the N target sub-information and the at least one first mask information.

[0117] In some embodiments, there are K target areas, where K is an integer greater than or equal to 1, and the fusion module includes: a third determination submodule, configured to determine K target subarea groups based on the distances between the multiple target subareas, wherein the distance between each target subarea in each target subarea group and the target subarea in other target subarea groups is greater than a preset distance threshold; and a fusion submodule, configured to fuse each target subarea group in the K target subarea groups to obtain K target areas.

[0118] In some embodiments, there are K target areas, where K is an integer greater than or equal to 1, and the acquisition module includes: a classification submodule, used to classify the K target areas to obtain a classification result; a fourth determination submodule, used to determine, based on the classification result, M target areas of a preset category from the K target areas, where M is an integer greater than or equal to 1; and a third acquisition submodule, used to obtain the target information based on the M target areas and the segmentation result.

[0119] In some embodiments, the determination module includes: a fifth determination submodule, used to determine the local image based on the grayscale value of each voxel in the original image; a second image segmentation submodule, used to perform a second image segmentation process on the local image to obtain second mask information; a sixth determination submodule, used to determine the target bounding box based on the second mask information; and a seventh determination submodule, used to determine the target image based on the target bounding box and the original image.

[0120] In the technical solutions disclosed herein, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0121] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0122] Figure 8 A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are provided as examples only and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0123] like Figure 8 As shown, the device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. Various programs and data required for the operation of the device 800 can also be stored in the RAM 803. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0124] Various components in device 800 are connected to I / O interface 805, including an input unit 806, such as a keyboard, mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a magnetic disk, optical disk, etc.; and a communication unit 809, such as a network card, modem, wireless communication transceiver, etc. The communication unit 809 allows device 800 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0125] The computing unit 801 can be a variety of general-purpose and / or specialized processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as the image processing method. For example, in some embodiments, the image processing method can be implemented as a computer software program that is tangibly contained in a machine-readable medium, such as the storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed on the device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the image processing method described above can be performed. Alternatively, in other embodiments, the computing unit 801 can be configured to perform the image processing method by any other appropriate means (e.g., by means of firmware).

[0126] Various embodiments of the systems and techniques described above can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0127] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0128] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0129] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0130] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0131] Computer systems may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The client and server relationship arises through computer programs running on the respective computers and having a client-server relationship to each other.

[0132] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not a limitation herein.

[0133] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. An image processing method, comprising: Determining a plurality of target subregions in the target image according to a detection result of the target image and a segmentation result of the target image, comprising: performing a first image segmentation process on the target image to obtain at least one first mask information; obtaining the segmentation result according to the at least one first mask information, wherein the segmentation result includes at least one sub-bounding box determined according to the at least one first mask information; and determining the plurality of target subregions in the target image according to the detection result and the at least one sub-bounding box; Merging the multiple target sub-regions according to distances between the multiple target sub-regions to obtain target regions, where the number of target regions is K, where K is an integer greater than or equal to 1; and Obtaining target information based on the target area and the segmentation result, including: adding preset mask information to N target areas among the K target areas that do not contain the first mask information, to obtain N target sub-information, where N is an integer greater than or equal to 1 and N is an integer less than or equal to K; and obtaining the target information based on the N target sub-information and the at least one first mask information.

2. The method according to claim 1, wherein The determining of a plurality of target sub-regions in the target image according to the detection result of the target image and the segmentation result of the target image further comprises: Performing target detection on the target image to obtain the detection result, wherein the detection result includes at least one sub-detection frame; and The plurality of target sub-regions are determined in the target image according to the at least one sub-detection frame and the segmentation result.

3. The method according to claim 1, wherein There are K target areas, where K is an integer greater than or equal to 1. The step of fusing the multiple target sub-regions according to the distances between the multiple target sub-regions to obtain the target region includes: Determining K target subregion groups based on the distances between the multiple target subregions, wherein the distance between each target subregion in each target subregion group and the target subregions in other target subregion groups is greater than a preset distance threshold; and Each target sub-region group in the K target sub-region groups is fused to obtain K target regions.

4. The method according to claim 1, wherein There are K target areas, where K is an integer greater than or equal to 1. The obtaining target information according to the target area and the segmentation result further includes: Classify K target areas and obtain classification results; Determining, according to the classification result, M target areas of a preset category from the K target areas, where M is an integer greater than or equal to 1; and The target information is obtained according to the M target regions and the segmentation result.

5. The method according to claim 1, wherein Determining a plurality of target sub-regions in the target image according to the detection result of the target image and the segmentation result of the target image includes: Determine the local image according to the gray value of each voxel in the original image; performing a second image segmentation process on the local image to obtain second mask information; Determining a target bounding box according to the second mask information; and The target image is determined according to the target bounding box and the original image.

6. An image processing device comprising: a determination module, configured to determine a plurality of target subregions in the target image based on a detection result of the target image and a segmentation result of the target image, comprising: a first image segmentation submodule, configured to perform a first image segmentation process on the target image to obtain at least one first mask information; a first obtaining submodule, configured to obtain the segmentation result based on the at least one first mask information, wherein the segmentation result includes at least one sub-bounding box determined based on the at least one first mask information; and a second determination submodule, configured to determine the plurality of target subregions in the target image based on the detection result and the at least one sub-bounding box; a fusion module, configured to fuse the multiple target sub-regions according to the distances between the multiple target sub-regions to obtain a target region, wherein the number of the target regions is K, where K is an integer greater than or equal to 1; and An acquisition module, used to obtain target information based on the target area and the segmentation result, including: an adding submodule, used to add preset mask information to N target areas of the K target areas that do not contain the first mask information, to obtain N target sub-information, wherein N is an integer greater than or equal to 1, and N is an integer less than or equal to K; and a second acquisition submodule, used to obtain the target information based on the N target sub-information and the at least one first mask information.

7. The device according to claim 6, wherein The determination module includes: a target detection submodule, configured to perform target detection on the target image to obtain the detection result, wherein the detection result includes at least one sub-detection frame; and The first determination submodule is configured to determine the plurality of target sub-regions in the target image according to the at least one sub-detection frame and the segmentation result.

8. The device according to claim 6, wherein There are K target areas, where K is an integer greater than or equal to 1. The fusion module includes: a third determining submodule, configured to determine K target subregion groups based on the distances between the plurality of target subregions, wherein the distance between each target subregion in each target subregion group and the target subregions in other target subregion groups is greater than a preset distance threshold; and The fusion submodule is configured to fuse each target sub-region group in the K target sub-region groups to obtain K target regions.

9. The device according to claim 6, wherein There are K target areas, where K is an integer greater than or equal to 1. The acquisition module includes: The classification submodule is used to classify K target areas and obtain classification results; a fourth determining submodule, configured to determine, based on the classification result, M target areas of a preset category from the K target areas, where M is an integer greater than or equal to 1; and The third obtaining submodule is configured to obtain the target information according to the M target regions and the segmentation result.

10. The device according to claim 6, wherein The determination module includes: a fifth determination submodule, configured to determine a local image based on the grayscale value of each voxel in the original image; a second image segmentation submodule, configured to perform a second image segmentation process on the local image to obtain second mask information; a sixth determining submodule, configured to determine a target bounding box according to the second mask information; and A seventh determination submodule is configured to determine the target image according to the target bounding box and the original image.

11. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 5.

12. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 5.

13. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 5.

Images