Target object determination method and apparatus, storage medium, and electronic device

Abstract

This invention provides a method, apparatus, storage medium, and electronic device for determining a target object. The method includes: segmenting an original image to obtain target image blocks corresponding to the target object, where each target image block includes a first target image block and a second target image block, the first target image block having a higher resolution than the second target image block; extracting features from the first target image block using a first branch network in a target neural network to obtain a first feature; extracting features from the second target image block using a second branch network at different resolutions to obtain a second set of features; and predicting the target image block based on the first and second features to obtain a prediction result for the target image block. The prediction result is used to characterize the recognition result for the target object within the target image block. This invention solves the problem of low accuracy in determining target objects in original images in related technologies.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and more specifically, to a method, apparatus, storage medium, and electronic device for determining a target object. Background Technology

[0002] When analyzing the original image of a target area, different analysts often bring subjective judgments, leading to various different analysis results. Furthermore, related techniques, when analyzing the multiple types of objects included in the original image, generally only perform coarse segmentation of local regions, making it impossible to accurately determine the target objects included in the original image. In other words, related techniques suffer from low accuracy in determining target objects in the original image.

[0003] There is currently no effective solution to the problem of low accuracy in identifying target objects in the original image in related technologies. Summary of the Invention

[0004] This invention provides a method, apparatus, storage medium, and electronic device for determining a target object, in order to at least solve the problem of low accuracy in determining target objects in original images in related technologies.

[0005] According to an embodiment of the present invention, a method for determining a target object is provided, comprising: acquiring an original image of a target region; segmenting the original image to obtain a target image block corresponding to the target object, wherein the target image block includes a first target image block and a second target image block, the first target image block having a higher resolution than the second target image block; extracting features from the first target image block using a first branch network in a target neural network to obtain a first feature, and extracting features from the second target image block using a second branch network in the target neural network at different resolutions to obtain a second set of features, the second set of features including multiple second features at different resolutions, the image block corresponding to the second feature having a lower resolution than the second target image block; predicting the target image block based on the first feature and the second set of features to obtain a prediction result for the target image block, the prediction result being used to characterize the recognition result for the target object in the target image block.

[0006] In an exemplary embodiment, segmenting the original image to obtain a target image block corresponding to the target object includes: segmenting the original image to obtain a first target image block; scaling the original image and then segmenting it to obtain a second target image block, wherein the center point coordinates and size of the second target image block are the same as those of the first target image block.

[0007] In one exemplary embodiment, the j-th tile corresponding to the j-th second feature among the plurality of second features included in the second set of features has the same center point coordinates as the second target tile, and the j-th tile has the same size as the second target tile; the resolution of the j-th tile is 1 / N of the resolution of the second target tile, where N=2. j j is a positive integer greater than or equal to 1.

[0008] In an exemplary embodiment, the step of predicting the target image patch pair based on the first feature and the second set of features to obtain the prediction result of the target image patch includes: fusing each of the second features in the second set of features with the first feature to obtain a plurality of third features; obtaining a target feature map based on the plurality of third features; and determining the prediction result of the target image patch based on the target feature map.

[0009] In an exemplary embodiment, segmenting the original image to obtain target image blocks corresponding to the target object includes: sliding the original image into blocks according to a predetermined sliding window size and a predetermined step size to obtain P first image blocks, where P is a positive integer greater than or equal to 2; sliding the first image into blocks according to the predetermined sliding window size and the predetermined step size to obtain P second image blocks, wherein the first image is an image obtained by scaling the original image; and combining the P first image blocks and the P second image blocks into P image blocks, wherein the P image blocks include the target image block corresponding to the target object.

[0010] In an exemplary embodiment, the step of sliding and slicing the original image according to a predetermined sliding window size and a predetermined step size to obtain P first image blocks includes: when the difference between the width W of the original image and the width w of the predetermined sliding window is not equal to an integer multiple of the predetermined step size b in the width direction, filling the boundary in the width direction of the original image so that the difference between the width W' of the filled image and w is equal to an integer multiple of b; and sliding and slicing the filled image according to the predetermined sliding window size and the predetermined step size to obtain the P first image blocks.

[0011] In an exemplary embodiment, the method further includes: acquiring P-1 image blocks other than the target image block from the P image blocks; extracting features from the first patch of each image block in the P-1 image blocks using the first branch network in the target neural network to obtain P-1 fourth features; and extracting features from the second patch of each image block in the P-1 image blocks using different resolutions using the second branch network in the target neural network to obtain P-1 sets of features, wherein the i-th set of features in the P-1 sets includes multiple fifth features with different resolutions, the resolution of the patch corresponding to the fifth feature is less than the resolution of the second patch of the i-th image block in the P-1 image blocks, and i is a positive integer greater than or equal to 1 and less than P-1; and obtaining P-1 prediction results based on the P-1 fourth features and the P-1 sets of features, wherein the P-1 prediction results are used to characterize the recognition of the P-1 image blocks. The results are then analyzed: The P-1 prediction results are combined with the prediction results of the target image block to obtain a set of prediction results for the original image; P center blocks of the predicted image blocks are obtained based on the set of prediction results, resulting in P center blocks, wherein each of the P prediction blocks includes the first image block in the corresponding image block and the prediction result of the corresponding image block; the center block has the same center point as the prediction image block, and the length and width of the center block are both half of the corresponding prediction image block; the P center blocks are stitched together to obtain a prediction mask image; if a first invalid region exists in the prediction mask image, a first elimination process is performed on the prediction mask image to obtain a target mask image, wherein the first elimination process is used to eliminate the invalid region in the prediction mask image; based on the target mask image, objects of a specified type identified in the original image and associated with the target region are determined.

[0012] In an exemplary embodiment, determining, based on the target mask image, an object of a specified type identified in the original image and associated with the target region, includes: converting the original image to a grayscale image; performing binarization processing on the grayscale image to obtain a binary image; performing a second elimination processing on the binary image if a second invalid region exists in the binary image to obtain a target processed image, wherein the second elimination processing is used to eliminate the second invalid region in the binary image; performing clustering processing on the pixels in the target processed image to obtain a target clustering image; and obtaining a target type image based on the target mask image and the target clustering image, wherein the target type image is used to identify the object of the specified type identified in the original image.

[0013] In an exemplary embodiment, obtaining a target type image based on the target mask image and the target cluster image includes: performing an XOR operation on the target mask image and the target cluster image to obtain the target type image.

[0014] In an exemplary embodiment, the method further includes: when the set of prediction results includes the P predicted patches and each of the P predicted patches identifies an identified object, concatenating the P predicted patches with the target type image to obtain a target segmentation map, wherein the target segmentation map is used to identify multiple objects identified in the original image, the multiple objects including an identified object identified in each of the P predicted patches and the specified type of object.

[0015] In an exemplary embodiment, the target site includes the intestine, the original image includes a pathological histological map of the intestine, and the target object includes at least one or any combination of the following: crypts, crypt tortuosity, crypt fusion, pyloric gland metaplasia, villi, villi changes, subvilli, lymphoid tissue, muscularis mucosae, compression area, ulcer, granuloma, epithelioid nodules, lymphatic vessels, stroma, and background.

[0016] According to another embodiment of the present invention, a target object determination device is also provided, comprising: a first acquisition module, configured to acquire an original image collected for a target region; a first obtaining module, configured to segment the original image to obtain a target image block corresponding to the target object, wherein the target image block includes a first target image block and a second target image block, the resolution of the first target image block being higher than the resolution of the second target image block; a second obtaining module, configured to extract features from the first target image block through a first branch network in a target neural network to obtain a first feature, and to extract features from the second target image block at different resolutions through a second branch network in the target neural network to obtain a second set of features, the second set of features including multiple second features at different resolutions, the resolution of the image block corresponding to the second feature being lower than the resolution of the second target image block; and a third obtaining module, configured to predict the target image block based on the first feature and the second set of features to obtain a prediction result of the target image block, the prediction result being used to characterize the recognition result of the target object in the target image block.

[0017] According to yet another embodiment of the present invention, a computer-readable storage medium is also provided, wherein a computer program is stored therein, wherein the computer program is configured to perform the steps in any of the above method embodiments when executed.

[0018] According to yet another embodiment of the present invention, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0019] This invention involves acquiring an original image of the target region, segmenting the original image to obtain target image blocks corresponding to the target object, wherein each target image block includes a first target image block and a second target image block, with the first target image block having a higher resolution than the second target image block. A first branch of a target neural network is used to extract features from the first target image block to obtain a first feature, and a second branch of the target neural network is used to extract features from the second target image block at different resolutions to obtain a second set of features. This second set of features includes multiple second features at different resolutions, with each second feature corresponding to an image block at a lower resolution than the aforementioned second target image block. The target image blocks are then predicted based on the first and second features to obtain a prediction result for the target image blocks. In other words, the first feature of the first target image block in the target image block is extracted by the first branch of the target neural network, and the second feature of the second target image block in the target image block is extracted at multiple resolutions using the second branch network to obtain a second set of features. Finally, a prediction result corresponding to the target image block is obtained based on the first and second features. By extracting features from the first target image patch to obtain local details of the target object, and by extracting features from the second target image patch at different resolutions to obtain surrounding information of the target object, the recognition result of the target object in the target image patch can be more accurately determined. Therefore, this solves the problem of low accuracy in determining the target object in the original image in related technologies, achieving the effect of improving the accuracy of target object determination. Attached Figure Description

[0020] Figure 1 This is a mobile terminal hardware structure block diagram of the target object determination method according to an embodiment of the present invention;

[0021] Figure 2 This is a flowchart of a method for determining a target object according to an embodiment of the present invention;

[0022] Figure 3 This is a flowchart of image segmentation according to an embodiment of the present invention;

[0023] Figure 4 This is an example diagram of the original image according to an embodiment of the present invention;

[0024] Figure 5 This is an example grayscale image according to an embodiment of the present invention;

[0025] Figure 6This is an example diagram of a binary image according to an embodiment of the present invention;

[0026] Figure 7 This is an example diagram of a filled diagram according to an embodiment of the present invention;

[0027] Figure 8 This is an example diagram of a clustering graph according to an embodiment of the present invention;

[0028] Figure 9 This is an example diagram of the original image cut according to an embodiment of the present invention;

[0029] Figure 10 This is a schematic diagram of a multi-scale high-resolution neural network structure according to an embodiment of the present invention;

[0030] Figure 11 This is an example diagram of the prediction results according to an embodiment of the present invention;

[0031] Figure 12 This is an example diagram of a prediction mask according to an embodiment of the present invention;

[0032] Figure 13 This is a schematic diagram comparing the mask images before and after optimization according to an embodiment of the present invention;

[0033] Figure 14 This is an example diagram of an interstitial membrane image according to an embodiment of the present invention;

[0034] Figure 15 This is a structural block diagram of a target object determination device according to an embodiment of the present invention. Detailed Implementation

[0035] The embodiments of the present invention will be described in detail below with reference to the accompanying drawings and examples.

[0036] It should be noted that the terms "first," "second," etc., in the specification, claims, and drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0037] The methods and embodiments provided in this application can be executed on a mobile terminal, computer terminal, or similar computing device. Taking running on a mobile terminal as an example, Figure 1 This is a mobile terminal hardware structure block diagram of the target object determination method according to an embodiment of the present invention. Figure 1 As shown, a mobile terminal may include one or more ( Figure 1Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 104 for storing data are also shown. The mobile terminal may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the mobile terminal described above. For example, the mobile terminal may also include components that are more... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0038] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the target object determination method in this embodiment of the invention. The processor 102 executes various functional applications and data processing by running the computer programs stored in the memory 104, thereby implementing the above-described method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the mobile terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0039] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the mobile terminal's communication provider. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0040] This embodiment provides a method for determining a target object. Figure 2 This is a flowchart of a method for determining a target object according to an embodiment of the present invention, such as... Figure 2 As shown, the process includes the following steps:

[0041] Step S202: Obtain the original image of the target area;

[0042] Step S204: The original image is segmented to obtain target image blocks corresponding to the target object, wherein the target image blocks include a first target image block and a second target image block, and the resolution of the first target image block is higher than that of the second target image block;

[0043] Step S206: Through the first branch network in the target neural network, feature extraction is performed on the first target image patch to obtain a first feature; and through the second branch network in the target neural network, feature extraction is performed on the second target image patch at different resolutions to obtain a second set of features. The second set of features includes multiple second features with different resolutions, and the resolution of the image patch corresponding to the second feature is smaller than the resolution of the second target image patch.

[0044] Step S208: Predict the target image block based on the first feature and the second set of features to obtain the prediction result of the target image block. The prediction result is used to characterize the recognition result of the target object in the target image block.

[0045] Through the above steps, the original image of the target area is acquired and segmented to obtain target image blocks corresponding to the target object. Each target image block includes a first target image block and a second target image block, with the first target image block having a higher resolution than the second target image block. The first branch of the target neural network is used to extract features from the first target image block, yielding a first feature. The second branch of the target neural network is used to extract features from the second target image block at different resolutions, yielding a second set of features. This second set of features includes multiple second features with different resolutions, and the resolution of the image block corresponding to each second feature is lower than that of the second target image block. The target image blocks are then predicted based on the first and second features to obtain a prediction result. In other words, the first feature of the first target image block in the target image block is extracted using the first branch of the target neural network, and the second feature of the second target image block in the target image block is extracted at multiple resolutions using the second branch network to obtain a second set of features. Finally, the prediction result corresponding to the target image block is obtained based on the first and second features. By extracting features from the first target image patch to obtain local details of the target object, and by extracting features from the second target image patch at different resolutions to obtain surrounding information of the target object, the recognition result of the target object in the target image patch can be more accurately determined. Therefore, this solves the problem of low accuracy in determining the target object in the original image in related technologies, achieving the effect of improving the accuracy of target object determination.

[0046] The entity performing the above steps may be an image processing system, an image processor, a terminal, a processor with human-computer interaction capabilities configured on a storage device, or a processing device or processing unit with similar processing capabilities, but is not limited to these.

[0047] In the above embodiments, the original image of the target area is acquired; the original image is segmented to obtain target image blocks corresponding to the target object. Each target image block includes a first target image block and a second target image block. For example, the first target image block is obtained by segmenting the original image, and the second target image block is obtained by scaling the original image and then segmenting it. Optionally, the center point coordinates and dimensions of the second target image block and the first target image block can be the same or different. The original image can be an image of a part of the human body or other images. The original image may include one or more different types of objects. In practical applications, the original image can be segmented to obtain multiple target image blocks. The first target image block is one of multiple first image blocks. For example, the original image can be segmented using a sliding window method with a sliding window size of 512*512, meaning the size of one first image block is 512*512. Other sliding window sizes can also be used, i.e., other scales can be selected for segmentation. Optionally, the sliding step size can be set to 256 (other step sizes can be set as needed). This results in multiple first image blocks after segmenting the original image. Optionally, the target image block can also be obtained by enlarging the original image before segmenting it using the sliding window method; and the second target image block can be obtained by scaling the original image before segmenting it. For example, the second target image patch is obtained by scaling the original image to 1 / 2 (or 1 / 4, or other) of the original image and then segmenting it using the sliding window method described above. The resolution of the first target image patch is higher than that of the second target image patch. That is, the first target image patch can see the local details of the target object, while the second target image patch can see the local and surrounding information corresponding to the target object. In other words, the field of view of the second target image patch is relatively larger than that of the first target image patch. For example, the first target image patch is a high-resolution patch, while the second target image patch has the same center point coordinates and size as the first target image patch, and its resolution is 1 / 4 (or 1 / 2, or other) of the first image patch. The first target image patch has a relatively low resolution. Features are extracted from the first target image patch using the first branch network of the target neural network to obtain the first feature. Then, features are extracted from the second target image patch using the second branch network of the target neural network at different resolutions to obtain the second set of features. This second set of features includes multiple second features with different resolutions. The resolution of the image patch corresponding to each second feature is lower than the resolution of the second target image patch. For example, a set of images that satisfy a predetermined relationship with the second target image patch can be obtained. This set of images can include one or more images, and the resolution of each image patch in the set is half the resolution of the second target image patch. jHere, j is a positive integer greater than or equal to 1. The second feature of each patch in a set of image patches is extracted, resulting in multiple second features with different resolutions. Then, the target image patch is predicted based on the first feature and the second set of features to obtain the prediction result. Specifically, the first feature of the first target patch in the target image patch is extracted through the first branch network of the target neural network, and the second feature of the second target patch in the target image patch is extracted at multiple resolutions to obtain the second set of features. The prediction result corresponding to the target image patch is then obtained based on the first and second features. By extracting features from the first target patch to obtain local details of the target object, and by extracting features from the second target patch at different resolutions to obtain surrounding information of the target object, the recognition result of the target object in the target image patch can be more accurately determined. Therefore, this solves the problem of low accuracy in determining the target object in the original image in related technologies, achieving the effect of improving the accuracy of target object determination.

[0048] In an optional embodiment, segmenting the original image to obtain target image blocks corresponding to the target object includes: segmenting the original image to obtain the first target image block; scaling the original image and then segmenting it to obtain the second target image block, wherein the center point coordinates and size of the second target image block are the same as those of the first target image block. In this embodiment, the first target image patch is obtained by switching the original image. For example, the original image can be divided into multiple first images, and the aforementioned first target image patch is one of these multiple first images. In practical applications, the original image can be divided into patches using a sliding window method. For example, the sliding window size is 512*512, meaning the size of one first image patch is 512*512. Of course, other sliding window sizes can also be used, i.e., other scales can be selected for dicing. Optionally, the sliding step size can be set to 256 (other step sizes can be set as needed). The second target image patch can be obtained by scaling the original image before dicing it. For example, the second target image patch is obtained by scaling the original image to 1 / 2 (or 1 / 4, or other) and then dicing it using the aforementioned sliding window method. Optionally, the center point coordinates and size of the second target image patch are the same as those of the first target image patch. Through this embodiment, the purpose of dicing the original image to obtain the first target image patch and scaling the original image before dicing it to obtain the second target image patch are achieved.

[0049] In an optional embodiment, the j-th tile corresponding to the j-th second feature among the plurality of second features included in the second set of features has the same center point coordinates as the second target tile, and the j-th tile has the same size as the second target tile; the resolution of the j-th tile is 1 / N of the resolution of the second target tile, where N=2. j , where j is a positive integer greater than or equal to 1. In this embodiment, for example, the second set of features includes n second features, where n is a positive integer greater than or equal to 1. The n second features are features corresponding to the n tiles, and the center point coordinates and dimensions of each of the n tiles are the same as those of the second target tile. The resolution of the j-th tile among the n tiles is equal to half the resolution of the second target tile. j This allows us to obtain multiple second features of multiple patches associated with the second target patch, i.e., multiple second features with different resolutions.

[0050] In an optional embodiment, the step of predicting the target image patch pair based on the first feature and the second set of features to obtain the prediction result of the target image patch includes: fusing each of the second features in the second set of features with the first feature to obtain a plurality of third features; obtaining a target feature map based on the plurality of third features; and determining the prediction result of the target image patch based on the target feature map. In this embodiment, a set of patches associated with the second target image patch can be determined. For example, the set of patches includes n (e.g., n=4, or other values) patches, and each patch in the set of patches has the same center point coordinates as the second target image patch, the same size as the second target image patch, and a resolution that satisfies a predetermined relationship with the resolution of the second target image patch. For example, the center point coordinates of the j-th patch in the set of patches (i.e., the above n patches) are the same as the center point coordinates of the second target image patch, the size of the j-th patch is the same as the size of the second target image patch, and the resolution of the j-th patch is 1 / 2 of the resolution of the second target image patch. jThe above-mentioned set of image patches includes n patches of different resolutions. A second branch network extracts the second features of each patch (e.g., the n patches) in the above-mentioned set of image patches to obtain a second set of features. Then, each second feature in the second set of features is fused with a first feature to obtain n third features. Based on these n third features, a target feature map corresponding to the target image patch is obtained. For example, the n third features are convolved using a 1x1 method and then combined to obtain the target feature map. Based on this target feature map, the prediction result of the target image patch can be determined, i.e., the recognition result of the predicted target image patch. Through this embodiment, the goal of integrating feature information from patches of different resolutions and exchanging information between multi-resolution representations is achieved, thereby improving the expressive power of high-resolution and low-resolution representations and achieving the effect of more accurately obtaining object information in image patches.

[0051] In an optional embodiment, segmenting the original image to obtain target image blocks corresponding to the target object includes: sliding the original image into blocks according to a predetermined sliding window size and a predetermined step size to obtain P first image blocks, where P is a positive integer greater than or equal to 2; sliding the first image into blocks according to the predetermined sliding window size and the predetermined step size to obtain P second image blocks, where the first image is an image obtained by scaling the original image; and combining the P first image blocks and the P second image blocks into P image blocks, wherein the P image blocks include the target image block corresponding to the target object. In this embodiment, the original image can be slicing into P first image blocks according to a predetermined sliding window size and a predetermined step size. In practical applications, a redundancy method can be used for slicing, for example, with a predetermined sliding window size of 512*512 and a predetermined step size of 256. Similarly, the first image can be slicing into P second image blocks using the same method. The first image is an image obtained by scaling the original image. Then, the P first image blocks and the P second image blocks are combined into P image blocks, including the target image block corresponding to the target object. Through this embodiment, the purpose of obtaining target image blocks through redundant sliding window slicing is achieved.

[0052] In an optional embodiment, the step of sliding and slicing the original image according to a predetermined sliding window size and a predetermined step size to obtain P first image blocks includes: when the difference between the width W of the original image and the width w of the predetermined sliding window is not equal to an integer multiple of the predetermined step size b in the width direction, filling the boundary of the original image in the width direction so that the difference between the width W' of the filled image and w is equal to an integer multiple of b; and sliding and slicing the filled image according to the predetermined sliding window size and the predetermined step size to obtain the P first image blocks. In this embodiment, when (Ww) is not equal to an integer multiple of b, it is necessary to fill the boundary of the original image in the width direction so that the difference between the width W' of the filled image and w is equal to an integer multiple of b, i.e., (W'-w) = kb; similarly, the boundary filling in the height direction of the original image can also be performed in the same way as described above.

[0053] In an optional embodiment, the method further includes: acquiring P-1 image blocks other than the target image block from the P image blocks; extracting features from the first patch of each image block in the P-1 image blocks using the first branch network in the target neural network to obtain P-1 fourth features; and extracting features from the second patch of each image block in the P-1 image blocks using the second branch network in the target neural network at different resolutions to obtain P-1 sets of features, wherein the i-th set of features in the P-1 sets includes multiple fifth features with different resolutions, the resolution of the patch corresponding to the fifth feature is less than the resolution of the second patch of the i-th image block in the P-1 image blocks, and i is a positive integer greater than or equal to 1 and less than P-1; and obtaining P-1 prediction results based on the P-1 fourth features and the P-1 sets of features, wherein the P-1 prediction results are used to characterize the recognition of the P-1 image blocks. The results are then analyzed: The P-1 prediction results are combined with the prediction results of the target image block to obtain a set of prediction results for the original image; P center blocks of the predicted image blocks are obtained based on the set of prediction results, resulting in P center blocks, wherein each of the P prediction blocks includes the first image block in the corresponding image block and the prediction result of the corresponding image block; the center block has the same center point as the prediction image block, and the length and width of the center block are both half of the corresponding prediction image block; the P center blocks are stitched together to obtain a prediction mask image; if a first invalid region exists in the prediction mask image, a first elimination process is performed on the prediction mask image to obtain a target mask image, wherein the first elimination process is used to eliminate the invalid region in the prediction mask image; based on the target mask image, objects of a specified type identified in the original image and associated with the target region are determined.In this embodiment, P-1 image blocks other than the target image block can be obtained from P image blocks. For each of the P-1 image blocks, the same method as for the target image block is used to extract the fourth feature of the first patch of each image block, resulting in P-1 fourth features. Additionally, features of the second patch of each image block are extracted at different resolutions, resulting in P-1 sets of features. Then, based on the P-1 fourth features and P-1 sets of features, P-1 prediction results corresponding to each of the P-1 image blocks are obtained. These P-1 prediction results are combined with the prediction results of the target image block to obtain a set of prediction results for the original image. Finally, based on this set of prediction results, P center blocks of the predicted image blocks are obtained, resulting in P center blocks. Each predicted image block includes the first patch of the corresponding image block and the prediction result of that image block. The center block can be a prediction of the image block. The central region, for example, if the predicted patch is 512*512, the central block can be a 256*256 region at the very center of the predicted patch. P central blocks are then stitched together to obtain the predicted mask image. Because the central block region has the highest confidence, the predicted edges are discarded, which improves the stitching artifacts. In other words, a complete predicted mask image is obtained through redundant stitching. Optionally, the predicted mask image can be optimized. When a first invalid region exists in the predicted mask image, a first elimination process is performed, such as hole filling, island elimination, and category merging, to obtain an optimized predicted mask image, which is the target mask image. Based on the target mask image, objects of a specified type associated with the target region identified in the original image can be determined. For example, the original image may contain multiple target objects, and the target mask image can identify objects of a specified type among these multiple target objects.

[0054] In an optional embodiment, determining the specified type of object associated with the target region in the original image based on the target mask image includes: converting the original image to a grayscale image; performing binarization processing on the grayscale image to obtain a binary image; performing a second elimination processing on the binary image if a second invalid region exists in the binary image to obtain a target processed image, wherein the second elimination processing is used to eliminate the second invalid region in the binary image; performing clustering processing on the pixels in the target processed image to obtain a target clustering image; and obtaining a target type image based on the target mask image and the target clustering image, wherein the target type image is used to identify the specified type of object identified in the original image. In this embodiment, the original image is transformed into a grayscale image by converting the RGB color space to the Gray color space. The grayscale image is then binarized to obtain a binary image. When a second invalid region exists in the binary image, a second elimination process is performed to obtain the target processed image. For example, island elimination and hole filling are performed on the binary image to eliminate invalid interference regions. Opening and closing operations are then performed to completely separate two closely spaced different parts. Density clustering is then performed on the target processed image. For example, the dbscan density clustering algorithm is used to label and distinguish each different block. This method is robust against noise and can handle clusters of arbitrary shapes and sizes, resulting in a target clustered image. Based on the target mask image and the target clustered image, a target type image is obtained. This target type image is used to identify objects of a specified type. For example, the target mask image and the target clustered image are XORed to obtain a median image of different object clusters. This embodiment achieves the goal of identifying objects of a specified type in the original image.

[0055] In an optional embodiment, obtaining the target type image based on the target mask image and the target cluster image includes: performing an XOR operation on the target mask image and the target cluster image to obtain the target type image. In this embodiment, performing an XOR operation on the target mask image and the target cluster image to obtain the target type image is equivalent to obtaining a median image of cluster images of different types of objects. Optionally, further optimization and elimination of isolated image tiles in the median image can be performed.

[0056] In an optional embodiment, the method further includes: when the set of prediction results includes the P predicted patches, and each of the P predicted patches identifies an identified object, concatenating the P predicted patches with the target type image to obtain a target segmentation map. The target segmentation map is used to identify multiple objects identified in the original image, including the identified object identified in each of the P predicted patches, and the specified type of object. In this embodiment, the P predicted patches included in the aforementioned set of prediction results are integrated with the target type image to obtain the target segmentation map. Thus, the obtained target segmentation map includes the identified objects in each of the aforementioned P predicted patches and the specified type of object, i.e., it identifies all types of objects included in the original image.

[0057] In an optional embodiment, the target site includes the intestine, and the original image includes a pathological histological map of the intestine. The target object includes at least one or any combination of the following: crypts, crypt tortuosity, crypt fusion, pyloric gland metaplasia, villi, villi changes, subvilli, lymphoid tissue, muscularis mucosae, compression area, ulcer, granuloma, epithelioid nodules, lymphatic vessels, stroma, and background. Taking an original image including a pathological histological map of the intestine as an example, this embodiment can achieve the purpose of identifying one or more types of objects included in the original image.

[0058] Obviously, the embodiments described above are only some embodiments of the present invention, and not all embodiments. The present invention will now be specifically described in conjunction with the embodiments.

[0059] The following description uses a tissue segmentation scheme for pathological images of inflammatory bowel disease as an example. This is merely an example and does not constitute a limitation on the scheme. This invention is also applicable to the segmentation of other images to determine the target objects included in the image. This invention innovatively proposes a multi-scale high-resolution semantic segmentation network to predict tissue blocks in pathological images. A 10000*10000-size pathological image (corresponding to the aforementioned original image) is sampled and segmented using a sliding window method, and predicted patches are stitched together using a redundancy method. Density clustering is used to distinguish individual tissue units on a pathological slice. Using deep learning and traditional image processing techniques, all elements of the entire pathological image are extracted. The following is a detailed description of this invention.

[0060] This scheme mainly implements a tissue segmentation method for pathological images of inflammatory bowel disease. Figure 3 This is an image segmentation flowchart according to an embodiment of the present invention, which is divided into three steps: first, tissue region clustering, see step 1; second, tissue segmentation, see step 2; and third, tissue matrix extraction, see step 3.

[0061] Step 1: Use image processing, density clustering and other techniques to process the original pathological tissue image and extract different tissue regions.

[0062] Step 1-1: Use the original pathological tissue image as input, such as... Figure 4 As shown, the image is transformed into a grayscale image by converting the RGB color space to the Gray color space. Figure 5 As shown.

[0063] Step 1-2: Binarize the grayscale image of the pathological tissue from Step 1-1 to obtain a binary image of the pathological tissue, such as... Figure 6 As shown.

[0064] Steps 1-3 to 1-4: Perform island removal and hole filling on the binary image of the pathological tissue from Step 1-2 to eliminate invalid interference areas; then perform opening and closing operations to completely separate two closely spaced different tissue blocks, obtaining a pathological tissue filled image, such as... Figure 7 As shown.

[0065] Steps 1-5: Density clustering is performed on the pathological tissue filling images from Steps 1-4. This technical solution uses the dbscan density clustering algorithm to label and distinguish each different tissue block. This method is robust against noise and can handle clusters of arbitrary shapes and sizes, resulting in a pathological tissue clustering map (corresponding to the aforementioned target clustering image), such as... Figure 8 As shown.

[0066] The clustering results of pathological tissue blocks can be obtained through the above steps 1-1 to 1-5, which can be used for subsequent extraction of interstitial elements.

[0067] Step 2: Use techniques such as sliding window redundancy, multi-scale high-resolution neural networks and image processing to segment and predict the original pathological tissue images, and extract various tissue elements.

[0068] Inflammatory bowel disease histological types are classified into 16 categories, namely: (1) crypts, (2) crypt tortuosity, (3) crypt fusion, (4) pyloric gland metaplasia, (5) villi, (6) villi changes, (7) subvilli, (8) lymphoid tissue, (9) muscularis mucosae, (10) compression area, (11) ulcer, (12) granuloma, (13) epithelioid nodules, (14) lymphatic vessels, (15) stroma, and (16) background.

[0069] Step 2-1: Using the original pathological tissue image as input, perform sliding window segmentation. The sliding window size is 512*512, and the sliding step length is 256, resulting in several 512*512 patches. Segment the image at different scales according to the required magnification of the network. For images whose width and height are not multiples of 512, boundary padding is required. Figure 9 As shown.

[0070] Step 2-2: Perform semantic segmentation network prediction on the patches from Step 2-1. This scheme innovatively proposes a multi-scale high-resolution neural network MHRNet (Multi High Resolution Network), the network structure of which is as follows: Figure 10 As shown.

[0071] This network has the following two characteristics:

[0072] (1) Multi-scale: Multi-scale high-resolution network training samples require patch pairs (corresponding to the aforementioned image patches) that are concentric, of the same size, but of different resolutions, such as Figure 10 As shown. Typically, mpp (micrometers per pixel) is set to 0.5, 1, 2, or 4. This configuration mimics the process of a pathologist diagnosing a whole slide. To diagnose a slide, pathologists often need to magnify details while simultaneously zooming out to observe the overall area and surrounding regions. Multi-scale image groups play a crucial role in acquiring local details and surrounding information.

[0073] The MHRNet multi-scale process is achieved through the interaction of the backbone network and branch networks. It has two branches: the Encoder-decoder context branch (corresponding to the second branch network mentioned above) is an auxiliary branch that encodes low-resolution, large-scale image patches to generate low-resolution, large-scale feature maps (corresponding to the second set of features mentioned above), serving as a surrounding information extractor; the High-resolution context branch (corresponding to the first branch network mentioned above) is the backbone branch that encodes high-resolution, small-scale image patches to generate high-resolution, small-scale feature maps (corresponding to the first set of features mentioned above), serving as a local information extractor. The feature maps from this branch are used as the final network output. During decoding in the auxiliary branch, the feature maps at each scale are cropped and resized, then concatenated with the corresponding size feature maps from the backbone branch via skip connections. Finally, the information from the two branches is fused using a 1x1 convolution.

[0074] (2) High resolution: Unlike previous classification-based convolutional neural networks that use low-resolution representations to recover high-resolution representations, MHRNet learns high-resolution representations to build new network structures. It maintains high-resolution representations throughout the entire network process and exchanges information between high and low resolution representations multiple times, thereby learning sufficiently rich high-resolution representations.

[0075] MHRNet's high resolution is achieved through the simultaneous forward pass of feature maps at four scales, gradually introducing low-resolution convolutions and connecting these convolutions in parallel. Simultaneously, by continuously exchanging information between these multi-resolution representations, the expressive power of both high-resolution and low-resolution representations is enhanced, allowing them to better complement each other. This is crucial for capturing detailed image information.

[0076] Semantic segmentation network predictions were performed on the patch pairs of different scales in step 2-1, and the results are as follows: Figure 11 As shown.

[0077] Steps 2-3 to 2-5: For each prediction result obtained in step 2-2, redundant stitching is performed. Redundant stitching means retaining the 256*256 region at the center of the 512*512 pixel patch prediction, as this region has the highest confidence, while discarding the prediction edges to improve stitching artifacts. A complete prediction mask is obtained through redundant stitching. The prediction mask is then subjected to hole filling, island removal, and category merging to obtain the optimized prediction mask image, as shown below. Figure 12 As shown.

[0078] Comparison of the effects before and after optimization using the redundancy splicing algorithm: Figure 13 As shown, the boundary effect is significantly improved.

[0079] It should be noted that there is no strict order between steps 1 and 2 above. Step 2 can be performed first and then step 1, or steps 1 and 2 can be performed simultaneously.

[0080] Step 3: Using the pathological tissue cluster map from Step 1 and the predicted mask map from Step 2, obtain the tissue stroma map through image processing techniques.

[0081] Steps 3-1 to 3-2: XOR the pathological tissue clustering images from steps 1-5 and the mask images from steps 2-5 to obtain stroma maps of different pathological tissue clustering images (corresponding to the aforementioned target type images). Elimination and optimization are then performed on isolated patches in the stroma maps, such as... Figure 14 As shown.

[0082] By integrating the pathological tissue segmentation results from step 2 and the pathological tissue stroma from step 3, we can obtain the full output of 16 categories of inflammatory bowel disease.

[0083] In this embodiment of the invention, a multi-scale high-resolution network is used to extract tissue from 512*512 image patches. Compared to the approach of recovering high-resolution representations from low-resolution representations generated by classification-based convolutional neural networks, the high-resolution network can maintain high-resolution representations throughout the entire process, exchanging information multiple times between high and low-resolution representations to learn sufficiently rich high-resolution representations. Simultaneously, it simulates the diagnostic process of pathologists, allowing the multi-scale network to fully utilize local details and global information, resulting in more accurate tissue extraction. A sliding window method is used to sample and cut a 10000*10000-size pathological image into several 512*512-size patches, leaving some overlapping areas during cutting. A redundancy method is used to extract 256*256-size regions from within each 512*512 patch for stitching. The redundancy sliding window method significantly improves the edge effects caused by patch stitching. Density clustering is used to distinguish various tissue units on the pathological slides, and an XOR operation is performed on tissue blocks and tissue regions to extract the stroma, thus completing the extraction of all elements from the entire pathological image.

[0084] Compared with related technologies, the technical solution of this invention has the following advantages: 1) It uses a multi-scale high-resolution neural network for tissue prediction, which makes the tissue segmentation effect more accurate; 2) It uses a sliding window method for slicing and a redundant method for stitching, which greatly improves the edge effect brought about by the stitching of the image pieces; 3) It uses density clustering to segment tissue pieces and separate the stroma, thus completing the extraction of all elements of the entire pathological image.

[0085] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0086] This embodiment also provides a device for determining a target object. Figure 15 This is a structural block diagram of a target object determination device according to an embodiment of the present invention, such as... Figure 15 As shown, the device includes:

[0087] The first acquisition module 1502 is used to acquire the original image of the target area;

[0088] The first obtaining module 1504 is used to segment the original image to obtain target image blocks corresponding to the target object, wherein the target image blocks include a first target image block and a second target image block, and the resolution of the first target image block is higher than the resolution of the second target image block;

[0089] The second obtaining module 1506 is used to extract features from the first target image patch through the first branch network in the target neural network to obtain a first feature, and to extract features from the second target image patch at different resolutions through the second branch network in the target neural network to obtain a second set of features. The second set of features includes multiple second features at different resolutions, and the resolution of the image patch corresponding to the second feature is smaller than the resolution of the second target image patch.

[0090] The third obtaining module 1508 is used to predict the target image block based on the first feature and the second set of features to obtain the prediction result of the target image block, and the prediction result is used to characterize the recognition result of the target object in the target image block.

[0091] In an optional embodiment, the first obtaining module 1504 includes: a first slicing unit, used to slice the original image to obtain the first target image patch; and a second slicing unit, used to scale the original image and slice it to obtain the second target image patch, wherein the center point coordinates and size of the second target image patch are the same as those of the first target image patch.

[0092] In an optional embodiment, the j-th tile corresponding to the j-th second feature among the plurality of second features included in the second set of features has the same center point coordinates as the second target tile, and the j-th tile has the same size as the second target tile; the resolution of the j-th tile is 1 / N of the resolution of the second target tile, where N=2. j j is a positive integer greater than or equal to 1.

[0093] In an optional embodiment, the third obtaining module 1508 includes: a fusion unit, configured to fuse each of the second features in the second group of features with the first feature respectively to obtain a plurality of third features; a first obtaining unit, configured to obtain a target feature map based on the plurality of third features; and a first determining unit, configured to determine the prediction result of the target image block based on the target feature map.

[0094] In an optional embodiment, the first obtaining module 1504 includes: a second obtaining unit, configured to slide and slice the original image according to a predetermined sliding window size and a predetermined step size to obtain P first image blocks, wherein P is a positive integer greater than or equal to 2; a third obtaining unit, configured to slide and slice the first image according to the predetermined sliding window size and the predetermined step size to obtain P second image blocks, wherein the first image is an image obtained by scaling the original image; and a compositing unit, configured to combine the P first image blocks and the P second image blocks into P image blocks, wherein the P image blocks include the target image block corresponding to the target object.

[0095] In an optional embodiment, the second obtaining unit includes: a filling subunit, configured to fill the boundary of the original image in the width direction when the difference between the width W of the original image and the width w of the predetermined sliding window is not equal to an integer multiple of the predetermined step size b in the width direction, so that the difference between the width W' of the filled image and w is equal to an integer multiple of b; and an obtaining subunit, configured to slide and cut the filled image into blocks according to the predetermined sliding window size and the predetermined step size to obtain the P first blocks.

[0096] In an optional embodiment, the above apparatus further includes: a second acquisition module, configured to acquire P-1 image blocks other than the target image block from the P image blocks after obtaining the set of prediction results; an extraction module, configured to extract features from the first patch of each image block in the P-1 image blocks through the first branch network in the target neural network to obtain P-1 fourth features, and to extract features from the second patch of each image block in the P-1 image blocks at different resolutions through the second branch network in the target neural network to obtain P-1 sets of features, wherein the i-th set of features in the P-1 sets of features includes multiple fifth features with different resolutions, the resolution of the patch corresponding to the fifth feature is less than the resolution of the second patch of the i-th image block in the P-1 image blocks, and i is a positive integer greater than or equal to 1 and less than P-1; and a fourth acquisition module, configured to obtain P-1 prediction results based on the P-1 fourth features and the P-1 sets of features, wherein the P-1 prediction results are used to characterize the recognition results of the P-1 image blocks. The system comprises: a result-combining module for combining the P-1 prediction results with the prediction results of the target image block to obtain a set of prediction results for the original image; a third acquisition module for acquiring the center blocks of P prediction blocks based on the set of prediction results, thereby obtaining P center blocks, wherein each of the P prediction blocks includes the first image block in the corresponding image block and the prediction result of the corresponding image block, the center block has the same center point as the prediction block, and the length and width of the center block are both half of the corresponding prediction block; a first stitching module for stitching the P center blocks to obtain a prediction mask image; a first processing module for performing a first elimination process on the prediction mask image to obtain a target mask image when a first invalid region exists in the prediction mask image, wherein the first elimination process is used to eliminate the invalid region in the prediction mask image; and a first determination module for determining, based on the target mask image, a specified type of object identified in the original image that is associated with the target region.

[0097] In an optional embodiment, the first determining module includes: a conversion unit for converting the original image into a grayscale image; a first processing unit for performing binarization processing on the grayscale image to obtain a binary image; a second processing unit for performing a second elimination processing on the binary image to obtain a target processed image when a second invalid region exists in the binary image, wherein the second elimination processing is used to eliminate the second invalid region in the binary image; a clustering unit for performing clustering processing on the pixels in the target processed image to obtain a target clustered image; and a fourth obtaining unit for obtaining a target type image based on the target mask image and the target clustered image, wherein the target type image is used to identify the specified type of object identified in the original image.

[0098] In an optional embodiment, the fourth obtaining unit includes a processing subunit, configured to perform an XOR operation on the target mask image and the target clustering image to obtain the target type image.

[0099] In an optional embodiment, the above apparatus further includes: a second stitching module, configured to stitch the P predicted image patches with the target type image to obtain a target segmentation map when the set of prediction results includes the P predicted image patches and each of the P predicted image patches identifies an identified object, wherein the target segmentation map is used to identify multiple objects identified in the original image, the multiple objects including an identified object identified in each of the P predicted image patches and the specified type of object.

[0100] In an optional embodiment, the target site includes the intestine, the original image includes a pathological histological map of the intestine, and the target object includes at least one or any combination of the following: crypts, crypt tortuosity, crypt fusion, pyloric gland metaplasia, villi, villi changes, subvilli, lymphoid tissue, muscularis mucosae, compression area, ulcer, granuloma, epithelioid nodules, lymphatic vessels, stroma, and background.

[0101] It should be noted that the above modules can be implemented by software or hardware. For the latter, they can be implemented in the following ways, but are not limited to: all the above modules are located in the same processor; or, the above modules are located in different processors in any combination.

[0102] Embodiments of the present invention also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to perform the steps in any of the above method embodiments when executed.

[0103] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0104] Embodiments of the present invention also provide an electronic device including a memory and a processor, the memory storing a computer program and the processor being configured to run the computer program to perform the steps in any of the above method embodiments.

[0105] In one exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0106] Specific examples in this embodiment can be found in the examples described in the above embodiments and exemplary implementations, and will not be repeated here.

[0107] It is obvious to those skilled in the art that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. They can be implemented using computer-executable program code, and thus can be stored in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those described herein, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0108] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for determining a target object, characterized in that, include: Acquire raw images of the target area; The original image is segmented to obtain target image blocks corresponding to the target object, wherein the target image blocks include a first target image block and a second target image block, and the resolution of the first target image block is higher than that of the second target image block; The first feature is obtained by extracting features from the first target image patch through the first branch network in the target neural network, and the second feature is obtained by extracting features from the second target image patch at different resolutions through the second branch network in the target neural network. The second set of features includes multiple second features with different resolutions, and the resolution of the image patch corresponding to the second feature is smaller than the resolution of the second target image patch. The target image patch is predicted based on the first feature and the second set of features to obtain the prediction result of the target image patch. The prediction result is used to characterize the recognition result of the target object in the target image patch. The step of segmenting the original image to obtain target image blocks corresponding to the target object includes: segmenting the original image to obtain the first target image block; scaling the original image and then segmenting it to obtain the second target image block, wherein the center point coordinates and size of the second target image block are the same as those of the first target image block.

2. The method according to claim 1, characterized in that, The j-th tile corresponding to the j-th second feature among the multiple second features included in the second group of features has the same center point coordinates as the second target tile, and the j-th tile has the same size as the second target tile; The resolution of the j-th image patch is 1 / N of the resolution of the second target image patch, where N=2. j j is a positive integer greater than or equal to 1.

3. The method according to claim 2, characterized in that, The step of predicting the target image patch pair based on the first feature and the second set of features to obtain the prediction result of the target image patch includes: Each of the second features in the second group of features is fused with the first feature to obtain multiple third features; Based on the multiple third features, a target feature map is obtained; The prediction result of the target image patch is determined based on the target feature map.

4. The method according to claim 1, characterized in that, The step of segmenting the original image to obtain target image blocks corresponding to the target object includes: The original image is slidably sliced ​​into blocks according to a predetermined sliding window size and a predetermined step size to obtain P first image blocks, where P is a positive integer greater than or equal to 2; The first image is slidably sliced ​​into blocks according to the predetermined sliding window size and the predetermined step size to obtain P second image blocks, wherein the first image is an image obtained by scaling the original image; P first image tiles and P second image tiles are combined into P image blocks, wherein the P image blocks include the target image block corresponding to the target object.

5. The method according to claim 4, characterized in that, The step of sliding and slicing the original image according to a predetermined sliding window size and a predetermined step size to obtain P first image patches includes: If the difference between the width W of the original image and the width w of the predetermined sliding window is not equal to an integer multiple of the predetermined step size b in the width direction, the boundary in the width direction of the original image is filled so that the difference between the width W' of the filled image and w is equal to an integer multiple of b. The filled image is slid-cut into blocks according to the predetermined sliding window size and the predetermined step size to obtain the P first image blocks.

6. The method according to claim 4, characterized in that, The method further includes: Obtain P-1 image blocks from the P image blocks, excluding the target image block; The first branch network in the target neural network extracts features from the first patch of each of the P-1 image blocks to obtain P-1 fourth features. The second branch network in the target neural network extracts features from the second patch of each of the P-1 image blocks at different resolutions to obtain P-1 sets of features. The i-th set of features in the P-1 sets includes multiple fifth features with different resolutions. The resolution of the patch corresponding to the fifth feature is less than the resolution of the second patch of the i-th image block in the P-1 image blocks. i is a positive integer greater than or equal to 1 and less than P-1. Based on the P-1 fourth features and the P-1 group of features, P-1 prediction results are obtained, wherein the P-1 prediction results are used to characterize the recognition results of the P-1 image blocks respectively; The P-1 prediction results are combined with the prediction results of the target image patch to obtain a set of prediction results for the original image; Based on the set of prediction results, P center blocks of prediction map patches are obtained, resulting in P center blocks. Each of the P prediction map patches includes the first map patch in the corresponding image patch and the prediction result of the corresponding image patch. The center block has the same center point as the prediction map patch, and the length and width of the center block are both half of the corresponding prediction map patch. The P central blocks are stitched together to obtain a predicted mask image; If a first invalid region exists in the predicted mask image, a first elimination process is performed on the predicted mask image to obtain a target mask image, wherein the first elimination process is used to eliminate the invalid region in the predicted mask image; Based on the target mask image, determine the objects of a specified type that are identified in the original image and associated with the target region.

7. The method according to claim 6, characterized in that, The step of determining, based on the target mask image, an object of a specified type identified in the original image and associated with the target region, includes: Convert the original image to a grayscale image; The grayscale image is binarized to obtain a binary image; If a second invalid region exists in the binary image, a second elimination process is performed on the binary image to obtain a target processed image, wherein the second elimination process is used to eliminate the second invalid region in the binary image; Clustering is performed on the pixels in the target image to obtain a target clustered image; Based on the target mask image and the target clustering image, a target type image is obtained, wherein the target type image is used to identify the specified type of object identified in the original image.

8. The method according to claim 7, characterized in that, The step of obtaining a target type image based on the target mask image and the target clustering image includes: The target mask image and the target clustering image are XORed to obtain the target type image.

9. The method according to claim 7, characterized in that, The method further includes: If the set of prediction results includes the P predicted image patches, and each of the P predicted image patches identifies an identified object, the P predicted image patches are stitched together with the target type image to obtain a target segmentation map. The target segmentation map is used to identify multiple objects identified in the original image. The multiple objects include an identified object identified in each of the P predicted image patches, as well as objects of the specified type.

10. The method according to any one of claims 1 to 9, characterized in that, The target site includes the intestine, the original image includes a pathological histological image of the intestine, and the target object includes at least one or any combination of the following: crypts, crypt tortuosity, crypt fusion, pyloric gland metaplasia, villi, villi changes, subvillous layer, lymphoid tissue, muscularis mucosae, compression area, ulcer, granuloma, epithelioid nodule, lymphatic vessels, stroma, background.

11. A device for determining a target object, characterized in that, include: The first acquisition module is used to acquire the original image of the target area; The first obtaining module is used to segment the original image to obtain target image blocks corresponding to the target object, wherein the target image blocks include a first target image block and a second target image block, and the resolution of the first target image block is higher than the resolution of the second target image block; The second obtaining module is used to extract features from the first target image patch through the first branch network in the target neural network to obtain a first feature, and to extract features from the second target image patch at different resolutions through the second branch network in the target neural network to obtain a second set of features. The second set of features includes multiple second features at different resolutions, and the resolution of the image patch corresponding to the second feature is smaller than the resolution of the second target image patch. The third obtaining module is used to predict the target image patch based on the first feature and the second set of features to obtain the prediction result of the target image patch, and the prediction result is used to characterize the recognition result of the target object in the target image patch; The device is used to segment the original image to obtain target image blocks corresponding to the target object in the following manner: the original image is segmented to obtain the first target image block; the original image is scaled and then segmented to obtain the second target image block, and the center point coordinates and size of the second target image block are the same as those of the first target image block.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of the method described in any one of claims 1 to 10.

13. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method described in any one of claims 1 to 10.

Images