Image processing method and device, electronic device and storage medium

Abstract

The present disclosure relates to an image processing method and device, an electronic device and a storage medium. The image processing method comprises: obtaining a target image including a target object; segmenting the target image to obtain the name of at least one pixel in the target image as a segmentation result; and determining the name of at least one target sub-object in the target object according to the segmentation result. Through the above process, the difficulty of implementing the naming process can be effectively reduced and the accuracy of naming can be improved, thereby improving the robustness of the image processing process.

Description

[0001] This application is a divisional application of the Chinese patent application submitted to the China Patent Office on July 14, 2020, with application number 202010674675.X and application name “Image processing method and device, electronic device and storage medium”. Technical Field

[0002] The present disclosure relates to the field of image processing technology, and in particular to an image processing method and device, an electronic device, and a storage medium. Background Art

[0003] Cardiovascular and cerebrovascular diseases are one of the diseases with the highest mortality rate at present, among which coronary heart disease has the highest incidence rate. Coronary heart disease is caused by atherosclerosis leading to coronary artery lumen stenosis, resulting in insufficient blood supply to the myocardium and causing a series of clinical symptoms.

[0004] In the process of analyzing coronary heart disease, the location of the lesion and the analysis of the description in the medical report often rely on the naming of the coronary centerline. Therefore, how to effectively name the segments in the coronary centerline has become a problem that needs to be solved urgently. Summary of the invention

[0005] The present disclosure proposes an image processing technology solution.

[0006] According to one aspect of the present disclosure, there is provided an image processing method, comprising:

[0007] Acquire a target image including a target object; segment the target image to obtain a name of at least one pixel in the target image as a segmentation result; and determine a name of at least one target sub-object in the target object according to the segmentation result.

[0008] In a possible implementation, segmenting the target image to obtain the name of at least one pixel in the target image as the segmentation result includes: inputting the target image into a neural network; determining the name of at least one pixel in the target image as the segmentation result based on the output of the neural network; wherein the neural network is trained by using a training image including the target object, and the target object in the training image is labeled by the name of at least one target sub-object.

[0009] In one possible implementation, the neural network is trained using a training image that includes the target object, including: determining labels of at least some pixels in the training image based on a label of the target object in the training image; and training the neural network using a training image that includes the labels of at least some pixels.

[0010] In one possible implementation, the neural network is trained using a training image that includes the target object, including: determining a target sub-object that is closest to a first target pixel point within the target object in the training image, wherein the first target pixel point is at least one pixel point in the training image other than the target object; using the determined annotation of the target sub-object as a label of the first target pixel point; and training the neural network based on the training image that includes the label of the first target pixel point and the annotation of the target object.

[0011] In one possible implementation, determining the name of at least one target sub-object in the target object based on the segmentation result includes: determining the name of at least one second target pixel point based on the segmentation result, wherein the second target pixel point is a pixel point contained in the target sub-object; counting the names of the second pixel points in the target sub-object to obtain a statistical result, and taking the name with the largest number in the statistical result as the name of the target sub-object.

[0012] In one possible implementation, determining the name of the second target pixel point based on the segmentation result includes: using the segmentation result corresponding to the second target pixel point as the name of the second target pixel point; or determining the name of the second target pixel point based on the segmentation result of at least one pixel point within a preset range of the second target pixel point.

[0013] In a possible implementation manner, the method further includes: processing the target object according to the name of at least one of the target sub-objects to obtain a processing result.

[0014] In a possible implementation, the processing of the target object according to the name of at least one of the target sub-objects includes: extracting target sub-objects with the same name in the target object; and / or, modifying the name of at least one target sub-object in the target object according to the names of adjacent target sub-objects.

[0015] In a possible implementation, the target image includes: a coronary artery mask image, or a coronary artery mask image and a heart mask image; and the target object includes a coronary artery centerline.

[0016] According to one aspect of the present disclosure, there is provided an image processing apparatus, comprising:

[0017] A target image acquisition module is used to acquire a target image including a target object; a segmentation module is used to segment the target image to obtain the name of at least one pixel in the target image as a segmentation result; and a naming module is used to determine the name of at least one target sub-object in the target object based on the segmentation result.

[0018] In one possible implementation, the segmentation module is used to: input the target image into a neural network; determine the name of at least one pixel in the target image as the segmentation result based on the output of the neural network; wherein the neural network is trained by a training image including the target object, and the target object in the training image is labeled by the name of at least one target sub-object.

[0019] In one possible implementation, the neural network is trained using a training image that includes the target object, including: determining labels of at least some pixels in the training image based on a label of the target object in the training image; and training the neural network using a training image that includes the labels of at least some pixels.

[0020] In one possible implementation, the neural network is trained using a training image that includes the target object, including: determining a target sub-object that is closest to a first target pixel point within the target object in the training image, wherein the first target pixel point is at least one pixel point in the training image other than the target object; using the determined annotation of the target sub-object as a label of the first target pixel point; and training the neural network based on the training image that includes the label of the first target pixel point and the annotation of the target object.

[0021] In one possible implementation, the naming module is used to: determine the name of at least one second target pixel point based on the segmentation result, wherein the second target pixel point is a pixel point contained in the target sub-object; count the names of the second pixel points in the target sub-object to obtain a statistical result, and use the name with the largest number in the statistical result as the name of the target sub-object.

[0022] In a possible implementation, the naming module is further used to: use the segmentation result corresponding to the second target pixel point as the name of the second target pixel point; or determine the name of the second target pixel point based on the segmentation result of at least one pixel point within a preset range of the second target pixel point.

[0023] In a possible implementation manner, the device further includes a processing module, and the processing module is used to: process the target object according to the name of at least one of the target sub-objects to obtain a processing result.

[0024] In a possible implementation, the processing module is further used to: extract target sub-objects having the same name in the target object; and / or, modify the name of at least one target sub-object in the target object according to the names of adjacent target sub-objects.

[0025] In a possible implementation, the target image includes: a coronary artery mask image, or a coronary artery mask image and a heart mask image; and the target object includes a coronary artery centerline.

[0026] According to one aspect of the present disclosure, there is provided an electronic device, including:

[0027] A processor; a memory for storing processor executable instructions; wherein the processor is configured to call the instructions stored in the memory to execute the above-mentioned image processing method.

[0028] According to one aspect of the present disclosure, a computer-readable storage medium is provided, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the above-mentioned image processing method is implemented.

[0029] In the disclosed embodiment, a target image including a target object is acquired, and the target image is segmented to obtain the name of at least one pixel in the target image as a segmentation result, thereby determining the name of at least one target sub-object in the target object according to the segmentation result. Through the above process, the naming process of the target object including multiple target sub-objects in the target image can be converted into a pixel-level segmentation process, effectively reducing the difficulty of the naming process and improving the accuracy of the naming, thereby improving the robustness of the image processing process.

[0030] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

[0031] Further features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and are used to illustrate the technical solutions of the present disclosure together with the specification.

[0033] Figure 1 A flowchart of an image processing method according to an embodiment of the present disclosure is shown.

[0034] Figure 2 A schematic diagram showing extraction of a target object according to an embodiment of the present disclosure is shown.

[0035] Figure 3 A schematic diagram showing an application example according to the present disclosure.

[0036] Figure 4 A block diagram of an image processing apparatus according to an embodiment of the present disclosure is shown.

[0037] Figure 5 A block diagram of an electronic device according to an embodiment of the present disclosure is shown.

[0038] Figure 6 A block diagram of an electronic device according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0039] Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0040] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0041] The term "and / or" herein is only a description of the association relationship of the associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the term "at least one" herein represents any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C can represent including any one or more elements selected from the set consisting of A, B, and C.

[0042] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present disclosure.

[0043] Figure 1A flowchart of an image processing method according to an embodiment of the present disclosure is shown, and the method can be applied to an image processing device, which can be a terminal device, a server or other processing device, etc. The terminal device can be a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc.

[0044] In some possible implementations, the image processing method may be implemented by a processor calling computer-readable instructions stored in a memory.

[0045] like Figure 1 As shown, the image processing method may include:

[0046] Step S11, acquiring a target image including a target object.

[0047] Step S12, segmenting the target image to obtain the name of at least one pixel in the target image as a segmentation result.

[0048] Step S13: determining the name of at least one target sub-object in the target object according to the segmentation result.

[0049] The target object may be any object with a naming requirement. Furthermore, in a possible implementation, the target object may be an object that needs to be named by region or segment. Since the target object may contain multiple segments or regions that need to be named, in a possible implementation, the target object may be divided into multiple target sub-objects according to the naming requirements of multiple segments or regions.

[0050] Specifically, in a possible implementation, the image processing method proposed in the embodiment of the present disclosure can be used to name each segment of the centerline of the coronary artery of the heart. Therefore, in one example, the target object can be the centerline of the coronary artery of the heart, that is, the coronary artery centerline. In this case, the target object can contain multiple target sub-objects, each of which can be one of the branches of the tree-like coronary artery centerline structure or one of the segments in the branch, that is, the coronary artery centerline segment; in one example, the target object can also be other parts of the coronary artery of the heart with naming requirements, or a partial area of ​​the centerline of the coronary artery of the heart, etc., which can be flexibly selected according to actual conditions. In a possible implementation, the method proposed in the embodiment of the present disclosure can also be applied to naming other organs or tissues containing multiple segments or multiple regions. Accordingly, in some possible implementations, the target object can be an organ or tissue with multiple regions or multiple segments naming requirements. The subsequent disclosed embodiments are all described by taking the target object as the centerline of the coronary artery of the heart as an example. When the target object is in other forms, its processing method can be flexibly expanded according to the method proposed in the embodiment of the present disclosure, and no further examples are given.

[0051] The name of the pixel in the target image may be the name of the target sub-object corresponding to the pixel. As the target object is implemented differently, the name of the target sub-object in the target object may be flexibly determined according to actual conditions. In one possible implementation, when the target object is the coronary centerline, the name of the target sub-object may be the name of a plurality of coronary centerline segments into which the coronary centerline may be divided, such as right coronary artery (RCA), right coronary origin posterior descending branch (R-PDA), left coronary origin posterior descending branch (L-PDA), right coronary origin posterior lateral branch (R-PLB), left coronary origin posterior lateral branch (L-PLB), left main trunk (LM), mid-distal circumflex branch (LCX), first and second obtuse marginal branches (OM1-2), left anterior descending branch (LAD), first and second diagonal branches (D1-2), or others (Others).

[0052] The target image may be any image containing the target object. In a possible implementation, the target image may be an image containing all the target objects. In a possible implementation, the target image may also be an image containing part of the target object. It can be flexibly selected according to the actual situation. It can be seen from the above disclosed embodiments that in a possible implementation, the target image may be a mask image. Specifically, the type of mask image can be flexibly determined according to the actual situation. As described in the above disclosed embodiments, in a possible case, the target object may include the coronary centerline. Accordingly, the target image may be an image containing the coronary centerline or an image that can be used to extract the coronary centerline. Therefore, in a possible implementation, the target image may include a coronary mask image. In one example, the coronary mask image may be a binary image. For example, the two values ​​0 and 1 may be used to distinguish whether the pixel in the image belongs to the coronary artery or is a background unrelated to the coronary artery.

[0053] In a possible implementation, the target image may include a coronary artery mask image and a heart mask image. The form of the coronary artery mask image may be as described in the above-mentioned disclosed embodiments, which will not be repeated here. The heart mask image may be a mask image formed by the heart to which the coronary arteries belong, and the various parts or chambers of the heart are divided by different values. In one example, the target image may be a mask image formed by superimposing the coronary artery mask image and the heart mask image. During the superposition process, the pixels in the heart mask image that do not belong to the coronary arteries can be marked as background values ​​in the coronary artery mask image. In the subsequent disclosed embodiments, the target image is a mask image formed by superimposing the heart mask image and the coronary artery mask image as an example for explanation. The remaining possible implementations can be expanded with reference to the subsequent disclosed embodiments, and will not be listed one by one.

[0054] When the target image contains both a coronary artery mask image and a heart mask image, segmentation can be performed based on the mask image. Compared with segmentation directly based on the original coronary artery image, the input environment of the subsequent segmentation process is simplified and the noise in the image processing process is reduced. Furthermore, the heart mask image can be used to retain information about each chamber of the heart, thereby facilitating the retention of the tree structure information and blood supply position information of the coronary arteries as much as possible, reducing the difficulty of subsequent segmentation and improving the accuracy of the segmentation results and the final determined name.

[0055] After obtaining the target image including the target object, the target image including the target object can be segmented through step S12 to obtain the name of at least one pixel in the target image as a segmentation result. In one possible implementation, segmentation can be to classify each pixel in the target image, thereby determining the category of each pixel in the target image; in one possible implementation, segmentation can also be to classify each foreground pixel in the target image (such as the pixel belonging to the coronary artery in the coronary artery mask image), thereby determining the category of each foreground pixel in the target image, etc. In the embodiments disclosed herein, the specific implementation method of segmentation can be flexibly determined according to actual conditions, and can refer to the following disclosed embodiments, which will not be expanded here.

[0056] Since segmentation can classify the pixels in the target image, further, the names of the various target sub-objects of the target object in the target image can be determined according to the classification results of each pixel in the target image. In one possible implementation, the name of each target sub-object in the target object can be determined according to the segmentation results; in a possible implementation, only the names of some target sub-objects in the target object can be determined according to the needs. The specific number of target sub-objects to be determined and the target sub-objects to be selected for name determination can be flexibly selected according to the actual situation, and are not limited in the embodiments disclosed herein. The specific implementation method of how to determine the name of the target sub-object based on the classification results of these pixels can be flexibly selected according to the actual situation. Please refer to the subsequent disclosed embodiments for details, which will not be expanded here.

[0057] In the disclosed embodiment, a target image including a target object is acquired, and the target image is segmented to obtain a segmentation result, thereby determining the name of at least one target sub-object in the target object according to the segmentation result. Through the above process, the naming process of the target object including multiple target sub-objects in the target image can be converted into a mask segmentation process, effectively reducing the difficulty of the naming process and improving the accuracy of the naming, thereby improving the robustness of the image processing process.

[0058] In a possible implementation, since the target image may include a mask image, segmentation may be performed based on the mask image, thereby simplifying the input environment of segmentation and reducing noise in the image processing process.

[0059] As described in the above-mentioned disclosed embodiments, the implementation form of the target image can be flexibly determined according to the actual situation. Therefore, the method of obtaining the target image including the target object can also be flexibly determined according to the actual situation. In a possible implementation, the method of obtaining the target image can be: performing vascular segmentation on the original image of the heart coronary artery to obtain a heart coronary artery mask image, then performing centerline extraction on the heart coronary artery mask image to obtain a coronary artery centerline image as the target object, and then superimposing the coronary artery centerline with the heart coronary artery mask image to obtain a target image containing the target object. In a possible implementation, the method of obtaining the target image can also be: performing vascular segmentation on the original image of the heart coronary artery to obtain a heart coronary artery mask image, then performing centerline extraction on the heart coronary artery mask image to obtain a coronary artery centerline image as the target object, and at the same time performing vascular segmentation on the original image of the heart to obtain a heart mask image, and then superimposing the coronary artery centerline, the heart coronary artery mask image and the heart mask image to obtain a target image containing the target object. The remaining possible implementation methods of step S11 can be flexibly selected according to the actual situation of the target image, and will not be illustrated one by one.

[0060] After obtaining the target image including the target object, the target image can be segmented in step S12 to obtain a segmentation result. In a possible implementation, the name of the target sub-object to which at least part of the pixels in the target image are closest or belong can be determined as the segmentation result based on the relative position of the pixels in the target image and the target object. In a possible implementation, the segmentation result of the target image can also be obtained by a neural network. In this case, step S12 can include:

[0061] Input the target image into the neural network;

[0062] According to the output of the neural network, determine the name of at least one pixel in the target image as a segmentation result;

[0063] The neural network is trained by a training image including a target object, and the target object in the training image is labeled by the name of at least one target sub-object.

[0064] The name of the pixel point output by the neural network can be the name of each pixel point in the target image, or the name of some pixels in the target image, which can be flexibly determined according to the actual situation. In a possible implementation, the neural network output can be the name of each foreground pixel point in the target image as the segmentation result.

[0065] The implementation form of the training image can refer to the implementation form of the target image mentioned above, and will not be repeated here. The way of labeling the target object of the training image is not limited in the embodiment of the present disclosure, and can be flexibly determined according to the function implemented by the neural network. In a possible implementation method, each pixel in the target object in the training image can also be labeled separately, or each pixel in the training image can be labeled separately. In a possible implementation method, multiple target sub-objects in the training image can be labeled separately. In one example, the target object can be divided into multiple target sub-objects, and then the names of some of the target sub-objects or each target sub-object are labeled, so that the pixels covered by the target sub-objects can be uniformly labeled. Which specific labeling method to choose can be flexibly selected according to actual conditions, and is not limited in the embodiment of the present disclosure.

[0066] The specific implementation form and training method of the neural network are not limited in the embodiments of the present disclosure and can be flexibly selected according to the actual situation. The initial model of the neural network can be arbitrarily selected according to the actual situation. In one example, Res-Vnet can be used as the implementation form of the neural network. The loss function adopted in the training can also be flexibly selected according to the actual situation. In one example, dice loss can be used as the loss function to train the neural network.

[0067] It can be seen from the above disclosed embodiments that in a possible implementation, the target image can be segmented by a neural network, because the neural network for segmenting the target image can be trained by a training image including the target object, and the target object in the training image is labeled by the name of at least one target sub-object. Therefore, based on the neural network obtained from the above training image, the pixel points belonging to the target object in the target image can be segmented, so as to obtain the segmentation result of each segmented pixel point, and the segmentation result can be the name of the target sub-object corresponding to the pixel point contained in the target image. The specific possible name types can refer to the above disclosed embodiments, which will not be repeated here.

[0068] By using training images labeled with the names of target sub-objects, the neural network is trained, and the segmentation of the target image is achieved using the trained neural network to obtain the segmentation result. Through the above process, the neural network can be effectively used to achieve pixel-level segmentation of the target image, and the segmentation result can be obtained more conveniently, thereby effectively reducing the difficulty of image processing and improving the practicality and generalization ability of image processing.

[0069] Furthermore, the way in which the neural network is trained using the training images including the target object can be flexibly determined according to the actual situation. In one possible implementation, the training of the neural network using the training images including the target object can include:

[0070] Determining labels of at least some pixels in the training image according to the annotations of the target object in the training image;

[0071] The neural network is trained using training images that include labels for at least some of the pixels.

[0072] As described in the above-mentioned disclosed embodiments, the annotation form of the target object in the training image can be flexibly selected according to the actual situation. Accordingly, as the annotation form is different, the method of determining at least part of the pixel labels in step S21 may also change accordingly. In one possible implementation, the annotation of the target object in the training image can be directly used as the label of the corresponding pixel in the target image. As described above, in one example, the target object can be divided into multiple segments of target sub-objects and each segment of the target sub-object can be named to achieve annotation. Therefore, accordingly, the annotation of the pixels covered by the named target sub-objects can be used as the labels of these pixels.

[0073] After determining the labels of at least some of the pixels in the training image, the neural network can be trained based on the training image containing at least some of the pixel labels. Through the above process, the neural network can be effectively trained based on the labeled training image to obtain a neural network with segmentation and naming functions, thereby improving the convenience of image processing.

[0074] As described in the above-mentioned disclosed embodiments, in a possible implementation, the target object may be the coronary centerline, and for the heart coronary mask image, the coronary centerline is only a part of the heart coronary mask, and there are also some pixels in the heart coronary mask image, which do not belong to the target object, i.e., the coronary centerline, but belong to the coronary artery, so the names corresponding to these pixels can help determine the name of the coronary centerline. Further, as described in the above-mentioned disclosed embodiments, in the process of labeling the target object in the training image, each pixel in the training image may be labeled, or each pixel in the target object of the training image may be labeled. Therefore, these pixels that help determine the name of the coronary centerline may not include labels. In a possible implementation, in order to improve the accuracy of the trained neural network, it is possible to consider determining the labels of these pixels based on the labels.

[0075] Therefore, in one possible implementation, the neural network is trained by training images including the target object, which may include:

[0076] Determine, within the target object in the training image, a target sub-object that is closest to a first target pixel point, wherein the first target pixel point is at least one pixel point in the training image other than the target object;

[0077] Using the determined target sub-object's annotation as the label of the first target pixel;

[0078] The neural network is trained based on the training image including the label of the first target pixel and the annotation of the target object.

[0079] Among them, the first target pixel point can be at least one pixel point in the training image that does not belong to the target object. The specific selection of which pixels to use as the first target pixel point can be flexibly determined according to the actual situation. In one possible implementation, the pixel point in the training image that is located outside the target object but can assist in naming the target object can be used as the first target pixel point. Taking the target object as the coronary artery centerline as an example, in one example, the pixel point in the training image that is located outside the coronary artery centerline and belongs to the coronary artery of the heart can be used as the first target pixel point; further, since the implementation form of the training image can be the same as the implementation form of the target image, and the above-mentioned disclosed embodiments have mentioned that the target image can be a mask image, therefore, in one example, the pixel points in the training image that do not belong to the background and do not belong to the target object can all or part of them be used as the first target pixel points.

[0080] In a possible implementation, the label of the first target pixel can be determined based on the label of at least part of the pixels in the training image. The specific method for determining the first target pixel can be flexibly selected according to the actual situation and is not limited to the embodiments of the present disclosure. It can be seen from the above that in a possible implementation, the annotation of the target sub-object closest to the first target pixel in the target object of the training image can be used as the label of the first target pixel. Furthermore, in an example, the target sub-object closest to the first target pixel can be the target sub-object to which the pixel closest to the first target pixel in the target object belongs.

[0081] After obtaining the label of the first target pixel point, the annotation of the target object in the training image can be directly used as the label of the corresponding pixel point in the target image in the manner described in the above-mentioned disclosed embodiment, thereby obtaining a training image with dense labels and training the neural network.

[0082] The above process is explained by taking the target object as the coronary artery centerline as an example. In one example, in the training image, the annotations of the pixels on the coronary artery centerline are first used as labels of these pixels, and then the pixels that belong neither to the background nor to the coronary artery centerline are used as first target pixels. For each first target pixel, the annotation of the pixel on the coronary artery centerline closest to the first target pixel is used as the label of the first target pixel. Then, a training image containing dense labels is obtained. By training the neural network with this training image containing dense labels, better training results can be obtained.

[0083] Through the above process, the training images with dense labels and complete information can be obtained based on the training images containing the target object annotations to train the neural network, thereby improving the accuracy of the trained neural network without increasing the difficulty of annotation, and then improving the accuracy of the segmentation results and the names of the target sub-objects obtained in the end. Improve the accuracy and convenience of image processing.

[0084] After obtaining the segmentation result of the target image through any of the above disclosed embodiments, the name of at least one target sub-object in the target object can be determined based on the segmentation result through step S13. The implementation method of step S13 is not limited. In a possible implementation method, the segmentation result of any pixel point contained in the target sub-object can be used as the name of the target sub-object. In a possible implementation method, step S13 may include:

[0085] Step S131, determining the name of at least one second target pixel point according to the segmentation result, wherein the second target pixel point is a pixel point included in the target sub-object;

[0086] Step S132, counting the names of the second pixel points in the target sub-object to obtain a statistical result, and taking the name with the largest number in the statistical result as the name of the target sub-object.

[0087] The second target pixel point may be a pixel point included in the target sub-object, that is, a pixel point covered by the target sub-object. In a possible implementation, since the segmentation result obtained may not be completely accurate, for the same target sub-object, the segmentation result of the second target pixel point contained therein may be accurate, that is, the same as the name of the corresponding target sub-object, or may be inaccurate, that is, different from the name of the corresponding target sub-object; in this case, if the segmentation result of any second target pixel point contained in the target sub-object is used as the name of the target sub-object, an inaccurate naming result is likely to be obtained.

[0088] Based on the above reasons, in a possible implementation, for any target sub-object whose name needs to be determined, the name of at least one second target pixel in the target sub-object can be obtained according to the segmentation result through step S131.

[0089] Specifically, the number of second target pixels in the target sub-object obtained in step S131 can be flexibly determined according to actual conditions. In a possible implementation, each second target pixel included in the target sub-object can be obtained. In a possible implementation, the second target pixel included in the target sub-object can be randomly sampled to obtain some second pixels. The specific sampling method and the number of samples can be flexibly determined according to actual conditions and are not limited in the embodiments of the present disclosure.

[0090] Furthermore, no matter whether part of the second target pixel points or all of the second target pixel points are obtained, the names of the obtained second target pixel points need to be determined according to the segmentation result. The method of determining the names can be flexibly determined according to the actual situation. In a possible implementation method, determining the names of the second target pixel points according to the segmentation result may include:

[0091] The segmentation result corresponding to the second target pixel is used as the name of the second target pixel.

[0092] Based on the segmentation result of at least one pixel within the preset range of the second target pixel, a name of the second target pixel is determined.

[0093] It can be seen from the above disclosed embodiments that in a possible implementation manner, the segmentation result of the second target pixel point can be directly used as the name of the second target pixel point.

[0094] The above-mentioned disclosed embodiments have proposed that the segmentation result of the pixels in the target image may not be completely accurate. In this case, if the segmentation result of the second target pixel is used as the name of the second target pixel, errors may also occur. Therefore, in a possible implementation method, the name of the second target pixel can be determined based on the segmentation result of at least one pixel within a preset range of the second target pixel.

[0095] Among them, the preset range of the second target pixel point can be flexibly set according to actual conditions, such as 9 neighborhoods, 16 neighborhoods, etc. around the second target pixel point, which is not limited in the embodiment of the present disclosure. Generally speaking, the pixel points near the second target pixel point have a high probability of belonging to the same target sub-object as the second target pixel point, so through these pixel points, the more accurate name of the second target pixel point can be determined. In an example, the segmentation result of each pixel point included in the preset range of the second target pixel point can be obtained, and then from these acquired segmentation results, the segmentation result with the highest proportion or the largest number is selected, or the segmentation result with a proportion or a number exceeding a set threshold (such as a proportion exceeding 50%) is selected as the name of the second target pixel point; in an example, the pixel points included in the preset range of the second target pixel point can also be sampled, and then from the segmentation results corresponding to the sampled pixel points, the segmentation result with the highest proportion or the largest number is selected, or the segmentation result with a proportion or a number exceeding a set threshold (such as a proportion exceeding 50%) is selected as the name of the second target pixel point.

[0096] As described in the above-mentioned disclosed embodiments, there may be some pixels in the target image, which, although not belonging to the target object, can assist in determining the name of the target sub-object. Therefore, by determining the name of the second target pixel based on the segmentation result of at least one pixel within a preset range of the second target pixel, the segmentation results of these pixels with auxiliary functions can be effectively introduced to determine the name of the second target pixel, thereby improving the accuracy of the determined name of the second target pixel, and then improving the accuracy of the determined name of the target sub-object, thereby improving the accuracy of image processing.

[0097] After determining the name of at least one second target pixel point by any of the above methods, the name of the target sub-object can be obtained based on the statistical results of the second pixel point names by step S132. The implementation method of step S132 can be flexibly determined according to the actual situation. For example, after counting the name of each second pixel point or part of the second pixel points in the target sub-object, the name with the largest number of these statistical names can be used as the name of the target sub-object, or the name with the highest appearance ratio among these statistical names can be used as the name of the target sub-object, or the name with a number exceeding a preset threshold or a ratio exceeding a preset threshold among these statistical names can be used as the name of the target sub-object. The value of the preset threshold is not limited in the present embodiment of the disclosure and can be flexibly set according to the actual situation.

[0098] The above process is explained by taking the target object as the coronary artery centerline as an example. In one example, the name of each coronary artery centerline segment of the coronary artery centerline can be determined in turn. Specifically, for each coronary artery centerline segment, each pixel point contained therein, i.e., the second target pixel point, can be traversed first to determine the names of these second target pixel points. The determination method can be directly determined based on the segmentation result, or it can be determined based on the segmentation results of the surrounding pixel points. Please refer to the above-mentioned disclosed embodiments and will not be repeated here. After determining the names of these second target pixel points, the names with the largest number or the highest proportion in the coronary artery centerline segment can be counted as the name of the coronary artery centerline segment.

[0099] By determining the name of at least one second target pixel point according to the segmentation result, and counting the names of the determined second pixel points, the name of the target sub-object is obtained according to the statistical results. Through the above process, even if the segmentation results of some pixels in the target image are inaccurate, a more accurate name of the target sub-object can be obtained based on the proportion, thereby improving the accuracy of the name of the target sub-object finally determined, and then improving the accuracy and precision of image processing.

[0100] In a possible implementation, the image processing method proposed in the embodiment of the present disclosure may further include:

[0101] Step S14: Process the target object according to the name of at least one target sub-object to obtain a processing result.

[0102] It can be seen from the above disclosed embodiments that after obtaining the name of at least one target sub-object in the target image, the target object in the target image can be further processed and optimized based on the determined name according to actual needs to obtain the final image processing result. How to process according to the name of the target sub-object and what kind of processing result to obtain can be flexibly determined according to the actual image processing needs, as detailed in the following disclosed embodiments, which will not be expanded here.

[0103] By processing the target object according to the name of at least one target sub-object to obtain a processing result, the target object can be further optimized based on the name of the target sub-object. Even if the accuracy of the segmentation result obtained by previously segmenting the target image is low, resulting in a certain error in the determined name, it can be corrected through processing to obtain a more accurate processing result, thereby improving the accuracy and robustness of image processing.

[0104] In a possible implementation, step S14 may include:

[0105] Extract the target sub-object with the same name in the target object. And / or,

[0106] According to the names of the adjacent target sub-objects, the name of at least one target sub-object in the target object is modified.

[0107] The method of extracting the target sub-objects with the same name in the target object is not limited. In a possible implementation, when the target object is the coronary centerline, since the coronary centerline can be a tree structure, for the coronary centerline segments with the same name, due to inaccurate segmentation, the coronary centerline segments may contain multiple branches. In order to obtain more accurate coronary centerline segments, in one example, the coronary centerline segments with multiple branches under the same name can be extracted, and the most complete or longest one can be selected as the extraction result. Figure 2 A schematic diagram of extracting a target object according to an embodiment of the present disclosure is shown. It can be seen from the figure that the coronary artery centerline segment named R-PLB has two branches before extraction. After extraction, the shorter branch is deleted and only the longer branch is retained as the extraction result.

[0108] The manner of modifying the target object according to the name of the target sub-object is also not limited in the embodiments of the present disclosure. In one possible implementation, when the target object is the coronary artery centerline, some of the coronary artery centerline segments in the coronary artery centerline may be in the following situation: that is, the coronary artery centerline segment may be located between two other coronary artery centerline segments with the same name, and the name of the coronary artery centerline segment is different from the names of the two other coronary artery centerline segments. Due to the continuity of the coronary artery centerline, the name of the coronary artery centerline segment is likely to be inaccurate. Therefore, the name of the current coronary artery centerline segment may be modified based on the names of other adjacent coronary artery centerline segments to maintain the continuity of the coronary artery centerline.

[0109] During the implementation process of step S14, the above-mentioned extraction process and correction process may be included at the same time, or only one of the processes may be included according to the actual situation, or other correction processes may be included. The execution order of these processes during the implementation process can also be flexibly selected according to the actual situation. The specific implementation method can be flexibly determined according to the actual situation and is not limited to the above-mentioned disclosed embodiments.

[0110] By extracting target sub-objects with the same name in the target object, and / or correcting the name of at least one target sub-object in the target object according to the names of adjacent target sub-objects, the occurrence of inaccurate naming results due to inaccurate segmentation can be further reduced, the accuracy of the final processing results can be improved, and the precision and robustness of image processing can be improved.

[0111] Application scenario examples

[0112] Cardiovascular and cerebrovascular diseases are among the diseases with the highest mortality rate at present, among which coronary heart disease has the highest incidence rate. Coronary heart disease is caused by atherosclerosis leading to coronary artery lumen stenosis, resulting in insufficient blood supply to the myocardium and causing a series of clinical symptoms, including angina pectoris, myocardial infarction, myocardial failure, arrhythmia and sudden death. Therefore, the results of coronary stenosis and plaque detection have guiding significance for diagnosis and subsequent treatment. In the analysis of coronary heart disease, the location of lesions and the description analysis in medical reports often need to rely on the naming of the coronary centerline. Therefore, how to effectively name the segments in the coronary centerline has become a problem that needs to be solved urgently.

[0113] Figure 3 A schematic diagram according to an application example of the present disclosure is shown. As shown in the figure, an image processing method is proposed in an embodiment of the present disclosure. This processing method can implement segmented naming of the coronary artery centerline through image processing. The image processing process can be:

[0114] As shown in the figure, the image processing process can be roughly divided into four steps.

[0115] In the first step, the coronary artery mask image and the heart mask image are obtained respectively, and the two mask images are merged to form a merged mask image. At the same time, the centerline of the coronary artery mask image is extracted to obtain the coronary artery centerline composed of multiple coronary artery centerline segments (target sub-objects) as the target object. Then the merged mask image is superimposed with the target object to obtain a target image including the target object (coronary artery centerline).

[0116] In the second step, the target image is segmented through the trained neural network to obtain the segmentation result of each foreground pixel (i.e., the pixel belonging to the coronary artery of the heart) in the target image.

[0117] In the third step, according to the segmentation result obtained in the second step, the name of each coronary artery centerline segment in the target image is determined by voting and naming. The specific voting and naming process can be:

[0118] For each coronary artery center line segment, the pixels contained therein are determined as second target pixels, and then the names corresponding to the segmentation results of these second target pixels are counted, and the names with the largest number are selected as the name of the coronary artery center line segment.

[0119] In the fourth step, after each coronary artery centerline segment is named, the most complete or longest branch of the coronary artery centerline segments with multiple branches of the same name can be retained as the final result through post-processing, and the discontinuous coronary artery centerline segments can be corrected. For example, if the parent node (parent segment) and child node (child segment) of a centerline segment have the same name, but the coronary artery centerline segment is inconsistent with their names, the coronary artery centerline segment can be modified to a name consistent with its parent node and child node. Through the above post-processing, the final processing result is obtained.

[0120] Furthermore, the training process of the neural network used for segmenting the target image in the second step can be:

[0121] First, a training image is prepared. The training image can be in the same form as the target image, that is, a coronary artery mask image and an image obtained by superimposing the heart mask image and the coronary artery centerline are used as the training image.

[0122] Since the training images are used to train the neural network, supervised groundtruth data is also required, that is, the pixels in the training images also need labels. In the disclosed application example, the groundtruth can be generated by first annotating the coronary centerline by segmented naming, so that the label of the pixel in each coronary centerline segment is the annotated name of the corresponding coronary centerline segment.

[0123] Then, the pixels in the training image that belong to the foreground and do not belong to the coronary artery centerline segment are respectively used as the first target pixels, and the label (also annotation) of the pixel on the coronary artery centerline closest to the first target pixel can be used as the label of the first target pixel.

[0124] Through the above process, a training image containing a label can be obtained, and the training image containing a label can be input into the neural network for training, so that a trained neural network can be obtained. In an example, the neural network can use the Res-Vnet neural network as a basic model and use dice loss as a loss function for multi-label segmentation training to obtain the final training result.

[0125] Through the above process, the segmentation and naming task of the coronary centerline can be constructed as an instance segmentation task of the vascular mask; at the same time, the image processing method proposed in the application example of the present disclosure can take the heart coronary mask image and the heart mask image as input. Compared with taking the original image of the heart coronary artery as input, it simplifies the input environment, reduces noise and retains all tree structure information and blood supply position information; in addition, in the naming process, a voting mechanism is adopted for the segmentation results, so that the naming of the coronary centerline segment depends on the segmentation label that appears most frequently in the pixel points of this segment. Even if the segmentation results of some pixels are inaccurate, as long as the correct segmentation results account for the majority, the correct coronary centerline naming results can be obtained; finally, after naming based on the segmentation results, the naming results are further post-processed. Since the segmentation results depend on the training data, the robustness of the final processing results can be maintained even when the training data is small through post-processing.

[0126] It should be noted that the image processing method of the embodiment of the present disclosure is not limited to application in the processing of the above-mentioned coronary artery images, but can be applied to any image processing, and the present disclosure does not impose any limitation on this.

[0127] It can be understood that the above-mentioned various method embodiments mentioned in the present disclosure can be combined with each other to form a combined embodiment without violating the principle logic. Due to space limitations, the present disclosure will not repeat them. It can be understood by those skilled in the art that in the above-mentioned method of the specific implementation method, the specific execution order of each step should be determined according to its function and possible internal logic.

[0128] In addition, the present disclosure also provides an image processing device, an electronic device, a computer-readable storage medium, and a program, all of which can be used to implement any image processing method provided by the present disclosure. The corresponding technical solutions and descriptions are referred to the corresponding records in the method part and will not be repeated here.

[0129] Figure 4 A block diagram of an image processing apparatus according to an embodiment of the present disclosure is shown. The image processing apparatus may be a terminal device, a server, or other processing devices, etc. The terminal device may be a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc.

[0130] In some possible implementations, the image processing apparatus may be implemented by a processor calling computer-readable instructions stored in a memory.

[0131] like Figure 4 As shown, the image processing device 20 may include:

[0132] The target image acquisition module 21 is used to acquire a target image including a target object.

[0133] The segmentation module 22 is used to segment the target image and obtain the name of at least one pixel in the target image as a segmentation result.

[0134] The naming module 23 is used to determine the name of at least one target sub-object in the target object according to the segmentation result.

[0135] In one possible implementation, the segmentation module is used to: input a target image into a neural network; determine, based on an output of the neural network, a name of at least one pixel in the target image as a segmentation result; wherein the neural network is trained using a training image including a target object, and the target object in the training image is labeled using the name of at least one target sub-object.

[0136] In one possible implementation, a neural network is trained using a training image that includes a target object, including: determining labels of at least some pixels in the training image based on a label of the target object in the training image; and training the neural network using a training image that includes labels of at least some pixels.

[0137] In one possible implementation, a neural network is trained using a training image that includes a target object, including: determining a target sub-object that is closest to a first target pixel point within the target object in the training image, wherein the first target pixel point is at least one pixel point other than the target object in the training image; using a label of the determined target sub-object as a label of the first target pixel point; and training the neural network based on the training image that includes the label of the first target pixel point and the label of the target object.

[0138] In one possible implementation, the naming module is used to: determine the name of at least one second target pixel point based on the segmentation result, wherein the second target pixel point is a pixel point contained in the target sub-object; count the names of the second pixel points in the target sub-object to obtain a statistical result, and use the name with the largest number in the statistical result as the name of the target sub-object.

[0139] In one possible implementation, the naming module is further used to: use the segmentation result corresponding to the second target pixel point as the name of the second target pixel point; or determine the name of the second target pixel point based on the segmentation result of at least one pixel point within a preset range of the second target pixel point.

[0140] In a possible implementation manner, the device further includes a processing module, and the processing module is used to: process the target object according to the name of at least one target sub-object to obtain a processing result.

[0141] In a possible implementation, the processing module is further used to: extract target sub-objects having the same name in the target object; and / or, modify the name of at least one target sub-object in the target object according to the names of adjacent target sub-objects.

[0142] In a possible implementation, the target image includes: a heart coronary mask image, or a heart coronary mask image and a heart mask image; the target object includes a coronary centerline.

[0143] The embodiment of the present disclosure also provides a computer-readable storage medium, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the above method is implemented. The computer-readable storage medium may be a non-volatile computer-readable storage medium.

[0144] An embodiment of the present disclosure further proposes an electronic device, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to call the instructions stored in the memory to execute the above method.

[0145] The embodiments of the present disclosure also provide a computer program product, including a computer-readable code. When the computer-readable code is executed on a device, a processor in the device executes instructions for implementing the image processing method provided in any of the above embodiments.

[0146] The embodiment of the present disclosure also provides another computer program product for storing computer-readable instructions, which, when executed, enables a computer to perform the operations of the image processing method provided by any of the above embodiments.

[0147] The electronic device may be provided as a terminal, a server, or a device in other forms.

[0148] Figure 5 A block diagram of an electronic device 800 according to an embodiment of the present disclosure is shown. For example, the electronic device 800 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, or the like.

[0149] Reference Figure 5 , the electronic device 800 may include one or more of the following components: a processing component 802 , a memory 804 , a power component 806 , a multimedia component 808 , an audio component 810 , an input / output (I / O) interface 812 , a sensor component 814 , and a communication component 816 .

[0150] The processing component 802 generally controls the overall operation of the electronic device 800, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0151] The memory 804 is configured to store various types of data to support operations on the electronic device 800. Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0152] The power supply component 806 provides power to the various components of the electronic device 800. The power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 800.

[0153] The multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the electronic device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0154] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC), and when the electronic device 800 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 804 or sent via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

[0155] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: home button, volume button, start button, and lock button.

[0156] The sensor assembly 814 includes one or more sensors for providing various aspects of status assessment for the electronic device 800. For example, the sensor assembly 814 can detect the open / closed state of the electronic device 800, the relative positioning of the components, such as the display and keypad of the electronic device 800, and the sensor assembly 814 can also detect the position change of the electronic device 800 or a component of the electronic device 800, the presence or absence of contact between the user and the electronic device 800, the orientation or acceleration / deceleration of the electronic device 800, and the temperature change of the electronic device 800. The sensor assembly 814 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 814 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0157] The communication component 816 is configured to facilitate wired or wireless communication between the electronic device 800 and other devices. The electronic device 800 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0158] In an exemplary embodiment, the electronic device 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above methods.

[0159] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 804 including computer program instructions, which can be executed by a processor 820 of an electronic device 800 to perform the above method.

[0160] Figure 6 1 is a block diagram of an electronic device 1900 according to an embodiment of the present disclosure. For example, the electronic device 1900 may be provided as a server. Figure 6 , the electronic device 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions executable by the processing component 1922, such as an application. The application stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute instructions to perform the above method.

[0161] The electronic device 1900 may also include a power supply component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input / output (I / O) interface 1958. The electronic device 1900 may operate based on an operating system stored in the memory 1932, such as Windows Server 2000. TM , Mac OS X TM ,UnixTM,Linux TM , FreeBSD TM or similar.

[0162] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions, which can be executed by the processing component 1922 of the electronic device 1900 to perform the above method.

[0163] The present disclosure may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0164] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be interpreted as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.

[0165] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0166] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be personalized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0167] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0168] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0169] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0170] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.

[0171] The computer program product may be implemented in hardware, software or a combination thereof. In one optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (SDK) and the like.

[0172] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. An image processing method, characterized in that: include: acquiring a target image including a target object; Segmenting the target image to obtain a name of at least one pixel in the target image as a segmentation result; Determining, according to the segmentation result, a name of at least one target sub-object in the target object; Determining the name of at least one target sub-object in the target object according to the segmentation result includes: Determine a name of at least one second target pixel point according to the segmentation result, wherein the second target pixel point is a pixel point included in the target sub-object; Counting the names of the second target pixel points in the target sub-object to obtain a statistical result, and taking the name with the largest number and / or the highest appearance ratio in the statistical result as the name of the target sub-object; Determining the name of the second target pixel point according to the segmentation result includes: Using the segmentation result corresponding to the second target pixel as the name of the second target pixel; or, Based on the segmentation result of at least one pixel within a preset range of the second target pixel, a name of the second target pixel is determined.

2. The method according to claim 1, characterized in that The method further comprises: The target object is processed according to the name of at least one of the target sub-objects to obtain a processing result.

3. The method according to claim 2, characterized in that The processing of the target object according to the name of at least one of the target sub-objects includes: Extracting target sub-objects having the same name in the target object; and / or, According to the names of the adjacent target sub-objects, the name of at least one target sub-object in the target object is modified.

4. The method according to any one of claims 1 to 3, characterized in that The target image includes: a heart coronary mask image, or a heart coronary mask image and a heart mask image; The target object includes the coronary artery centerline.

5. The method according to claim 4, characterized in that In a case where the target image includes a heart coronary artery mask image, acquiring the target image including the target object includes: Performing blood vessel segmentation on the original image of the coronary arteries to obtain a mask image of the coronary arteries; Performing centerline extraction on the heart coronary mask image to obtain a coronary centerline image as a target object; The coronary artery centerline is superimposed on the heart coronary artery mask image to obtain a target image containing the target object.

6. The method according to claim 4, characterized in that In a case where the target object includes a coronary artery mask image and a heart mask image, acquiring the target image including the target object includes: Performing blood vessel segmentation on the original image of the coronary arteries to obtain a mask image of the coronary arteries; Extracting the centerline of the heart coronary mask image to obtain the coronary centerline image as the target object; Perform blood vessel segmentation on the original heart image to obtain a heart mask image; The coronary artery centerline, the heart coronary artery mask image and the heart mask image are superimposed to obtain a target image containing the target object.

7. An image processing device, characterized in that: include: A target image acquisition module, used to acquire a target image including a target object; A segmentation module, used to segment the target image and obtain a name of at least one pixel in the target image as a segmentation result; A naming module, used to determine a name of at least one target sub-object in the target object according to the segmentation result; The naming module is used to: determine the name of at least one second target pixel point according to the segmentation result, wherein the second target pixel point is a pixel point included in the target sub-object; count the names of the second target pixel points in the target sub-object to obtain a statistical result, and use the name with the largest number and / or the highest appearance ratio in the statistical result as the name of the target sub-object; The naming module is further used to: use the segmentation result corresponding to the second target pixel point as the name of the second target pixel point; or determine the name of the second target pixel point based on the segmentation result of at least one pixel point within a preset range of the second target pixel point.

8. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images