Image processing method and device

Abstract

The present application provides an image processing method and device, wherein the image processing method comprises: acquiring a sequence of images; acquiring a tubular object to be segmented; performing segmentation processing on tubular object features corresponding to the tubular object to be segmented in the sequence of images; and displaying a tubular object feature image obtained by the segmentation processing.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to an image processing method and device. Background Art

[0002] Vascular diseases have high morbidity and mortality rates worldwide. For patients with vascular diseases, both surgical and drug treatments are very expensive. Diagnosing vascular diseases with the help of CT (Computed Tomography) angiography technology is a non-invasive method that causes less harm to patients. It can also screen vascular diseases and avoid unnecessary invasive treatments.

[0003] Currently, CT angiography is widely used in clinical practice, and with it comes a large number of CTA (Computed Tomography Angiography) images that need to be diagnosed by doctors. Unlike other focal lesions, the diagnosis of vascular diseases is more complicated and tedious. Taking the coronary arteries of the heart as an example, clinical diagnosis generally requires the extraction of coronary arteries, the naming of coronary artery branches, the reconstruction of the coronary artery tree, the detection and diagnosis of stenosis / plaques, and finally the generation of a diagnostic report. The entire diagnostic process takes about 30 minutes to 1 hour, so automatic auxiliary diagnosis of vascular diseases is particularly important. Summary of the invention

[0004] In view of this, the present application provides two image processing methods, two image processing apparatuses, two computing devices, and two computer-readable storage media.

[0005] The present application provides an image processing method, comprising: acquiring a sequence of images; acquiring a tubular object to be segmented; performing segmentation processing on tubular object features corresponding to the tubular object to be segmented in the sequence of images; and displaying a tubular object feature image obtained by the segmentation processing.

[0006] Optionally, the segmentation processing of the tubular features corresponding to the tubular objects to be segmented in the sequence of images includes: determining a sequence of images containing the tubular features in the sequence of images as the tubular image; and segmenting the tubular features corresponding to the tubular objects to be segmented in the tubular images to obtain the tubular feature image.

[0007] Optionally, the segmentation processing of the tubular features corresponding to the tubular object to be segmented in the tubular object image to obtain the tubular feature image includes: extracting tubular feature lines in the tubular object image; performing surface reconstruction on the tubular object image based on the tubular feature lines to obtain a three-dimensional image sequence composed of cross-sectional images; performing tubular feature segmentation processing on the three-dimensional image sequence to obtain a tubular feature image; and performing coordinate conversion processing on the tubular feature image according to a coordinate mapping relationship between the cross-sectional image and the tubular object image to obtain the tubular feature image.

[0008] Optionally, the tubular feature segmentation processing is performed on the three-dimensional image sequence to obtain a tubular feature image, which is implemented using an image segmentation model; wherein the image segmentation model includes a convolutional neural network, a long short-term memory neural network and a fully connected layer; accordingly, the image segmentation model performs tubular feature segmentation processing, including: extracting semantic features of the cross-sectional images contained in the three-dimensional image sequence based on the convolutional neural network to obtain serialized semantic features corresponding to the three-dimensional image sequence; fusing the serialized semantic features based on the long short-term memory neural network to obtain serialized image features; predicting lumen feature data of the tubular lumen features corresponding to the cross-sectional image based on the fully connected layer according to the serialized image features; and generating a lumen binary image corresponding to the cross-sectional image based on the lumen feature data.

[0009] Optionally, the lumen feature data of the tubular lumen feature includes: a direction angle value of the direction angle based on the lumen center of the tubular lumen, and a radius value corresponding to the direction angle; the radius value refers to the distance value from the lumen center to the tubular lumen under the direction angle; wherein the direction angles are equally spaced.

[0010] Optionally, the tubular object to be segmented is input by a user.

[0011] Optionally, the tubular object to be segmented includes at least one of the following: a description text of the tubular object to be segmented, a two-dimensional image of the tubular object to be segmented, and a sequence of three-dimensional images of the tubular object to be segmented.

[0012] The present application provides an image processing device, comprising: an image acquisition module, configured to acquire a sequence of images; a tubular object acquisition module, configured to acquire a tubular object to be segmented; a segmentation processing module, configured to perform segmentation processing on tubular object features corresponding to the tubular object to be segmented in the sequence of images; and a feature image display module, configured to display a tubular feature image obtained by the segmentation processing.

[0013] The present application provides another image processing method, comprising: acquiring at least one tubular object image to be processed; extracting tubular feature lines in the tubular object image; performing surface reconstruction on the tubular object image based on the tubular feature lines to obtain a three-dimensional image sequence composed of cross-sectional images; inputting the three-dimensional image sequence into an image segmentation model to perform tubular feature segmentation processing to obtain a tubular feature image; and performing coordinate transformation processing on the tubular feature image according to a coordinate mapping relationship between the cross-sectional image and the tubular object image to obtain a target image.

[0014] Optionally, the image segmentation model includes a convolutional neural network, a long short-term memory neural network and a fully connected layer; accordingly, the image segmentation model performs tubular feature segmentation processing, including: extracting semantic features of the cross-sectional images contained in the three-dimensional image sequence based on the convolutional neural network, and obtaining serialized semantic features corresponding to the three-dimensional image sequence; fusing the serialized semantic features based on the long short-term memory neural network to obtain serialized image features; predicting lumen feature data of the tubular lumen features corresponding to the cross-sectional image based on the fully connected layer according to the serialized image features; and generating a lumen binary image corresponding to the cross-sectional image based on the lumen feature data.

[0015] Optionally, the lumen feature data of the tubular lumen feature includes: a direction angle value of the direction angle based on the lumen center of the tubular lumen, and a radius value corresponding to the direction angle; the radius value refers to the distance value from the lumen center to the tubular lumen under the direction angle; wherein the direction angles are equally spaced.

[0016] Optionally, the image processing method further includes: segmenting the main tubular features in the tubular object image to obtain a main binary image corresponding to the main tubular features.

[0017] Optionally, the extracting of tubular feature lines in the tubular object image includes: extracting tubular feature lines of branched tubular features in the tubular object image; accordingly, the three-dimensional image sequence composed of cross-sectional images includes a three-dimensional image sequence composed of cross-sectional images corresponding to the branched tubular features; the image segmentation model performs tubular feature segmentation processing, including the image segmentation model performing segmentation processing on the branched tubular features contained in the cross-sectional images to obtain a tubular feature image corresponding to the branched tubular features; the target image obtained after the coordinate transformation processing includes a branched tubular image corresponding to the branched tubular features.

[0018] Optionally, after the step of obtaining a target image by performing coordinate transformation processing on the tubular feature image based on the coordinate mapping relationship between the cross-sectional image and the tubular object image, the step includes: combining the trunk binary image corresponding to the trunk tubular feature and the branch tubular image corresponding to the branch tubular feature to obtain a global tubular image.

[0019] Optionally, the tubular object image includes: a vascular image obtained by angiography; main tubular features in the tubular object image, including aortic features corresponding to the aorta in the vascular image; branch tubular features in the tubular object image, including coronary features corresponding to the coronary arteries in the vascular image.

[0020] Optionally, the coordinate conversion processing is performed on the tubular feature image according to the coordinate mapping relationship between the cross-sectional image and the tubular object image, and before the step of obtaining the target image is executed, the steps include: reading the coordinate information of the cross-sectional image and the coordinate information of the tubular object image; and determining the mapping relationship between the coordinate information of the cross-sectional image and the coordinate information of the tubular object image as the coordinate mapping relationship.

[0021] Optionally, the convolutional neural network includes at least one of the following: a two-dimensional convolutional neural network or a three-dimensional convolutional neural network.

[0022] The present application also provides another image processing device, including: a tubular image acquisition module, configured to acquire at least one tubular image to be processed; a tubular feature line extraction module, configured to extract tubular feature lines in the tubular image; a surface reconstruction module, configured to perform surface reconstruction on the tubular image based on the tubular feature lines to obtain a three-dimensional image sequence composed of cross-sectional images; a segmentation processing module, configured to input the three-dimensional image sequence into an image segmentation model to perform tubular feature segmentation processing to obtain a tubular feature image; and a coordinate transformation processing module, configured to perform coordinate transformation processing on the tubular feature image according to a coordinate mapping relationship between the cross-sectional image and the tubular image to obtain a target image.

[0023] The present application provides a computing device, comprising: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions: acquiring a sequence of images; acquiring a tubular object to be segmented; performing segmentation processing on tubular object features corresponding to the tubular object to be segmented in the sequence of images; and displaying a tubular object feature image obtained by the segmentation processing.

[0024] The present application provides another computing device, comprising: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions: obtaining at least one tubular object image to be processed; extracting tubular feature lines in the tubular object image; performing surface reconstruction on the tubular object image based on the tubular feature lines to obtain a three-dimensional image sequence composed of cross-sectional images; inputting the three-dimensional image sequence into an image segmentation model to perform tubular feature segmentation processing to obtain a tubular feature image; performing coordinate conversion processing on the tubular feature image according to the coordinate mapping relationship between the cross-sectional image and the tubular object image to obtain a target image.

[0025] The present application provides a computer-readable storage medium storing computer instructions, which implement the steps of the image processing method when executed by a processor.

[0026] The present application provides another computer-readable storage medium storing computer instructions, which implement the steps of another image processing method when executed by a processor.

[0027] The image processing method provided in the present application, for the acquired sequence images, starts from the tubular objects to be segmented input by the user for the sequence images, performs segmentation processing on the tubular features corresponding to the tubular objects to be segmented in the sequence images, and displays the tubular feature images obtained by the segmentation processing to the user, thereby improving the flexibility and accuracy of the image segmentation processing.

[0028] The image processing method provided by the present application, in the process of performing image segmentation on the tubular features in the tubular object image, uses the tubular feature lines in the tubular object image to perform surface reconstruction on the tubular object image to form a three-dimensional image sequence of the tubular object, and then inputs the three-dimensional image sequence into an image segmentation model to perform tubular feature segmentation processing, thereby realizing segmentation and extraction of the tubular features in the tubular object image, and restores the tubular feature image obtained by segmentation and extraction to the coordinate system before surface reconstruction through coordinate transformation, thereby obtaining an image corresponding to the tubular object, and realizing refinement and accuracy of tubular feature segmentation. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 is a processing flow chart of an image processing method provided in an embodiment of the present application;

[0030] Figure 2 is a schematic diagram of an image segmentation processing process provided by an embodiment of the present application;

[0031] Figure 3 is a schematic diagram of an image segmentation model provided in an embodiment of the present application;

[0032] Figure 4is a schematic diagram of an image processing device provided in an embodiment of the present application;

[0033] Figure 5 is a processing flow chart of an image processing method provided in an embodiment of the present application;

[0034] Figure 6 is a schematic diagram of an image segmentation processing process provided by an embodiment of the present application;

[0035] Figure 7 is a processing flow chart of an image processing method applied to the coronary artery image segmentation processing process provided by an embodiment of the present application;

[0036] Figure 8 is a schematic diagram of a coronary artery image provided in an embodiment of the present application;

[0037] Fig. 9 is a schematic diagram of an image processing device provided in an embodiment of the present application;

[0038] Fig.10 is a structural block diagram of a computing device provided in an embodiment of the present application;

[0039] Fig.11 It is a structural block diagram of a computing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0040] Many specific details are described in the following description to facilitate a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of this specification, so this specification is not limited to the specific implementation disclosed below.

[0041] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "a", "said" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0042] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0043] An embodiment of the present application provides two image processing methods, two image processing devices, two computing devices, and two computer-readable storage media. The following is a detailed description of each of the embodiments provided in this specification, and each step of the method is described.

[0044] An embodiment of an image processing method provided by the present application is as follows:

[0045] See attached Figure 1 , which shows a processing flow chart of an image processing method provided by this embodiment, see the attached Figure 2 , which shows a schematic diagram of an image segmentation process provided by this embodiment, see the attached Figure 3 , which shows a schematic diagram of an image segmentation model provided by this embodiment.

[0046] Step S102, acquiring a sequence of images.

[0047] The sequence of images described in the embodiments of the present application includes a series of sequence images of human tissues collected by CTA (Computed Tomography Angiography). For example, a series of sequence images of coronary artery tissues in the human body collected by CTA. In addition, the sequence of images can also be a series of image sets collected by tomography technology.

[0048] Step S104, obtaining the tubular object to be segmented.

[0049] In the embodiments of the present application, the tubular object may be vascular tissue in a biological body such as the human body, such as arterial vessels and coronary arteries in the human body. In addition, the tubular object may also be a 3D printed blood vessel applied to the human body or other biological bodies, or an artificial organ applied to the human body or other biological bodies.

[0050] The tubular object to be segmented in the embodiment of the present application may be a description text of a vascular tissue. For example, in the process of image segmentation processing of a sequence of images of coronary tissue, the tubular object to be segmented is the name of the coronary vessel, that is, the description text of "coronary vessel". In addition, in the process of image segmentation processing of a sequence of images of an artificial organ applied to a human body or other organisms, the tubular object to be segmented is a two-dimensional image of the artificial organ, such as an image of the artificial organ taken from one side; or, in the process of image segmentation processing of a sequence of images of 3D-printed blood vessels applied to a human body or other organisms, the tubular object to be segmented is a three-dimensional image sequence composed of a series of images of a 3D model of a 3D-printed blood vessel.

[0051] It should be noted that, in addition to being able to input the tubular object to be segmented in the three ways of description text, two-dimensional image or three-dimensional image sequence, the user can also input the tubular object to be segmented by inputting description text and two-dimensional image. For example, in the process of image segmentation processing of the sequence image of coronary tissue, the user simultaneously inputs the name and image of the coronary vessel, that is, the description text of "coronary vessel" and the image of the coronary vessel. Similarly, the user can also input the tubular object to be segmented by inputting description text and three-dimensional image sequence, or by inputting two-dimensional image and three-dimensional image sequence, or by inputting description text, two-dimensional image and three-dimensional image sequence.

[0052] Here, the tubular object to be segmented refers to the segmentation object for the subsequent segmentation processing of the sequence images. If the acquired tubular object to be segmented is a description text, then in the subsequent segmentation processing of the sequence images, it is necessary to determine the image segmentation target for the subsequent image segmentation processing based on the acquired description text. Similarly, if the acquired tubular object to be segmented is a two-dimensional image of an artificial organ, it is necessary to identify the acquired two-dimensional image of the artificial organ, that is, to identify which organ tissue the acquired two-dimensional image is, and use the identified organ tissue as the image segmentation target for the subsequent image segmentation processing; if the acquired tubular object to be segmented is a three-dimensional image sequence of a 3D-printed blood vessel, it is necessary to identify the acquired three-dimensional image sequence of the 3D-printed blood vessel, that is, to identify which type of blood vessel the acquired three-dimensional image sequence is, whether it is an arterial blood vessel or a venous blood vessel, and use the identified blood vessel as the image segmentation target for the subsequent image segmentation processing.

[0053] The embodiment of the present application takes the vascular tissue in the human body as an example to illustrate the serial image segmentation processing process of the vascular tissue. For the serial image segmentation processing process of 3D printed blood vessels and other artificial tubular objects, refer to the specific implementation of the serial image segmentation processing for vascular tissue provided in this embodiment, which will not be described in detail in this embodiment.

[0054] In this embodiment, the tubular object to be segmented is input by the user, and the purpose of the user inputting the tubular object to be segmented is to perform image segmentation on the basis of the sequence of images acquired above, and to perform segmentation on the sequence of images. For example, after acquiring a series of sequence images of coronary tissue, the user of this method wants to segment the coronary vessels from the sequence images of the coronary tissue, and inputs the segmentation target of the coronary vessels by inputting the description text of "coronary vessels". In the subsequent image segmentation process, the coronary vessels are segmented in the sequence images of the coronary tissue according to the segmentation target of the coronary vessels.

[0055] Step S106, performing segmentation processing on the tubular object features corresponding to the tubular object to be segmented in the sequence of images.

[0056] In an optional implementation manner provided by the embodiment of the present application, segmenting the tubular object features corresponding to the tubular object to be segmented in the sequence of images specifically includes:

[0057] 1) determining a sequence of images containing the tubular object features in the sequence of images as a tubular object image;

[0058] 2) Segmenting the tubular features corresponding to the tubular objects to be segmented in the tubular object image to obtain the tubular object feature image.

[0059] Optionally, segmentation processing is performed on the tubular features corresponding to the tubular objects to be segmented in the tubular object image, which is specifically implemented in the following manner:

[0060] (a) Extracting tubular feature lines from the tubular object image.

[0061] The tubular feature line described in the embodiment of the present application refers to an auxiliary identification line that can identify the area and shape of the tubular object in the tubular object image. In this embodiment, the tubular feature line refers to the center line of the tubular object in the tubular object image. In addition, the tubular feature line can also be the inner cavity contour line of the tubular object in the tubular object image, or other identification lines that can identify the tubular object in the tubular object image. Here, the purpose of extracting the tubular feature line in the tubular object image is to prepare for the subsequent curve reconstruction of the tubular object image.

[0062] (b) performing surface reconstruction on the tubular object image based on the tubular characteristic lines to obtain a three-dimensional image sequence consisting of cross-sectional images.

[0063] Curved Planar Reformation (CPR) presents a series of vascular images at different levels acquired through CTA in a three-dimensional space based on the center line of the tubular image, so that clinical workers can quickly give a diagnostic report based on the internal and external tissue conditions of blood vessels at different levels presented in the three-dimensional space.

[0064] In this embodiment, surface reconstruction is performed based on the center line of the tubular image to obtain vascular image features corresponding to the tubular image in three-dimensional space, so as to present the situation of the tubular object. Specifically, the tubular image features obtained after surface reconstruction exist in the data form of a three-dimensional image sequence composed of cross-sectional images, wherein each cross-sectional image in the three-dimensional image sequence presents the situation of the tubular object at this level.

[0065] (c) performing tubular feature segmentation processing on the three-dimensional image sequence to obtain a tubular feature image.

[0066] In an optional implementation manner provided by this embodiment, the tubular feature segmentation processing performed on the three-dimensional image sequence is implemented by using an image segmentation model;

[0067] The image segmentation model includes a convolutional neural network, a long short-term memory neural network and a fully connected layer. Specifically, the image segmentation model performs tubular feature segmentation processing, including:

[0068] Extracting semantic features of the cross-sectional images contained in the three-dimensional image sequence based on the convolutional neural network to obtain serialized semantic features corresponding to the three-dimensional image sequence;

[0069] Based on the long short-term memory neural network, the serialized semantic features are integrated to obtain serialized image features;

[0070] According to the serialized image features, based on the fully connected layer, predict the lumen feature data of the tubular lumen feature corresponding to the cross-sectional image;

[0071] An inner cavity binary image corresponding to the cross-sectional image is generated based on the inner cavity characteristic data.

[0072] The inner cavity feature data of the tubular inner cavity feature includes:

[0073] The direction angle value of the direction angle based on the inner cavity center of the tubular inner cavity, and the radius value corresponding to the direction angle; the radius value refers to the distance value from the inner cavity center to the tubular inner cavity under the direction angle;

[0074] Wherein, the direction angles are distributed at equal intervals.

[0075] For example, Figure 3 The image segmentation model shown in the figure performs a coronary artery feature segmentation processing process, wherein the input of the image segmentation model is a cross-sectional image in a three-dimensional image sequence obtained after surface reconstruction. After the cross-sectional image of the coronary artery is input into the image segmentation model, the semantic features of the cross-sectional image of the coronary artery are first extracted by a two-dimensional convolutional neural network (2D CNN), and then the extracted serialized semantic features are input into a two-layer long short-term memory neural network (LSTM) for feature fusion. The feature fusion of the long short-term memory neural network enables the intermediate features to consider the semantic features of the previous and next time series. The feature fusion result output by the long short-term memory neural network is input into the fully connected layer of the output layer to predict the inner cavity features of the coronary artery, specifically predicting the radius of the inner cavity of the coronary artery in various directions, where the direction refers to 180 directions with an angle of 2 degrees distributed at equal intervals. The prediction result output by the fully connected layer is the radius values ​​of the 180 directions corresponding to the cross-sectional image of the coronary artery, then according to the radius values ​​of the coronary artery in 180 directions of the cross-sectional image corresponding to the cross-sectional image, the inner cavity feature image corresponding to the cross-sectional image of the coronary artery is generated.

[0076] (d) performing coordinate transformation processing on the tubular feature image according to the coordinate mapping relationship between the cross-sectional image and the tubular object image to obtain the tubular object feature image.

[0077] After the inner cavity characteristic image of the coronary artery in the tubular image is segmented out by using the image segmentation model, the inner cavity characteristic image is subjected to coordinate transformation processing according to the coordinate mapping relationship between the tubular image before the curved surface reconstruction and the cross-sectional image after the curved surface reconstruction, that is, the inner cavity characteristic image is restored to the coordinate system before the curved surface reconstruction to obtain the coronary artery characteristic image of the coronary artery.

[0078] Step S108, displaying the tubular feature image obtained by the segmentation process.

[0079] After the above steps have been performed on the sequence images to segment the tubular features corresponding to the tubular object to be segmented, the tubular feature image obtained after the segmentation process is displayed to the user. For example, after segmenting the coronary vessel feature image of the coronary vessels from the sequence images of the coronary tissue, the coronary vessel feature image obtained after the segmentation process is displayed to the user, as shown in the attached figure. Figure 2 The coronary vessel characteristic image shown in FIG. 1 shows the image segmentation processing result of the segmentation target, the tubular object to be segmented, which is input by the user for the sequence images of the coronary tissue.

[0080] In practical applications, in order to enhance the display effect of the tubular feature image, the tubular feature image can also be enhanced before display, so as to enhance the display effect of the tubular features to be segmented in the tubular feature image. For example, a logarithmic image enhancement algorithm, an exponential image enhancement algorithm, a Laplace operator image enhancement or other image enhancement algorithm can be used to enhance the tubular feature image, and the tubular feature image can be displayed to the user based on the enhanced processing, so as to improve the display effect of the tubular feature image.

[0081] To summarize, the image processing method provided in the present application, for the acquired sequence images, starts from the tubular object to be segmented, performs segmentation processing on the tubular features corresponding to the tubular object to be segmented in the sequence images, and displays the tubular feature image obtained by the segmentation processing to the user, thereby improving the flexibility and accuracy of image segmentation processing.

[0082] The following further describes the image processing method provided by this embodiment by taking the application of the image processing method provided by this embodiment in the C2M (Customer to Manufacturer) scenario as an example. The image processing method applied to the C2M (Customer to Manufacturer) scenario specifically includes the following steps:

[0083] 1) Obtain sequence images.

[0084] In the C2M (Customer to Manufacturer) scenario, users can directly connect to manufacturers or factories. Users can place orders with manufacturers online and entrust manufacturers to manufacture products. During the product processing or after the production is completed, manufacturers can invite users to participate in the supervision or acceptance of product manufacturing, such as inspecting the process of product manufacturing; or, manufacturers inspect products at the request of users, such as users requesting defect detection of products. Specifically, the products that users entrust manufacturers or factories to manufacture can be 3D printed human tissues, organs or other products.

[0085] Whether it is a process inspection or a defect detection of a product, the manufacturer needs to collect images of the product. In this embodiment, image collection for the product refers to collecting a series of images of the product using tomography technology. The collected series of images are called sequence images.

[0086] 2) Obtain the tubular object to be segmented.

[0087] In the process of process inspection or defect detection of the products produced by the manufacturer, the user needs to indicate the specific part of the product to be inspected. For example, if the product package processed by the user entrusts the manufacturer to include a tubular component, and the user needs to perform defect detection on the inner cavity of the tubular component, the tubular component of the product indicated by the user is the tubular object to be segmented. Similarly, the manufacturer can also specify a tubular component of the product for process inspection or defect detection.

[0088] 3) Segmenting the tubular features corresponding to the tubular objects to be segmented in the sequence of images.

[0089] Based on the collected sequence images of the product and the tubular component input by the user, in this step, image segmentation processing is performed on the sequence images of the product, specifically, the image area of ​​the tubular component input by the user is segmented from the sequence images of the product, thereby providing a basis for process inspection or defect detection of the tubular component.

[0090] The following is a specific description of the process of segmenting the tubular features corresponding to the tubular parts in the sequence images of the 3D printed product, taking the inner cavity defect detection of the tubular parts in the 3D printed product as an example:

[0091] (a) determining a sequence of images containing tubular features corresponding to the tubular component among the sequence of images of the 3D printed product as the tubular component image;

[0092] (b) performing segmentation processing on the tubular features corresponding to the tubular component in the tubular component image to obtain a tubular feature image;

[0093] Specifically, the tubular features corresponding to the tubular parts in the tubular part image are segmented. First, the center line of the tubular part in the tubular part image is extracted. Then, the tubular part image is surface reconstructed based on the center line to obtain a three-dimensional image sequence composed of cross-sectional images. Here, the surface reconstruction is performed according to the center line of the tubular part image to obtain the tubular image features corresponding to the tubular part image in the three-dimensional space, so that the situation of the tubular part can be presented. Specifically, the tubular image features obtained after the surface reconstruction exist in the data form of a three-dimensional image sequence composed of cross-sectional images, wherein each cross-sectional image in the three-dimensional image sequence presents the situation of the tubular part at this level. Secondly, the three-dimensional image sequence is segmented to obtain a tubular feature image. Finally, according to the coordinate mapping relationship between the cross-sectional image and the tubular part image, the tubular feature image is coordinate transformed to obtain the tubular feature image.

[0094] 4) Display the tubular feature image obtained by segmentation processing.

[0095] After the above-mentioned segmentation processing of the tubular features corresponding to the tubular parts of the serial images of the product is performed, the tubular feature images obtained after the segmentation processing are displayed here, so as to intuitively show the details of the tubular parts of the product to the user or the manufacturer, so as to facilitate the manufacturer to conduct process inspection on the product, or facilitate the user to conduct defect detection on the product.

[0096] An embodiment of an image processing device provided by the present application is as follows:

[0097] In the above-mentioned embodiment, an image processing method is provided, and correspondingly, an image processing device is also provided, which will be described below with reference to the accompanying drawings.

[0098] See attached Figure 4 , which shows a schematic diagram of an image processing device provided by this embodiment.

[0099] Since the device embodiment corresponds to the method embodiment, the description is relatively simple, and the relevant parts can refer to the corresponding description of the method embodiment provided above. The device embodiment described below is only illustrative.

[0100] The present application provides an image processing device, comprising:

[0101] An image acquisition module 402 is configured to acquire a sequence of images;

[0102] The tubular object to be segmented obtaining module 404 is configured to obtain the tubular object to be segmented;

[0103] The segmentation processing module 406 is configured to perform segmentation processing on the tubular features corresponding to the tubular objects to be segmented in the sequence of images;

[0104] The feature image display module 408 is configured to display the tubular feature image obtained by the segmentation process.

[0105] Optionally, the segmentation processing module 406 includes:

[0106] a tubular object image determination submodule, configured to determine a sequence of images containing the tubular object features among the sequence of images as the tubular object image;

[0107] The segmentation processing submodule is configured to perform segmentation processing on the tubular features corresponding to the tubular objects to be segmented in the tubular object image to obtain the tubular feature image.

[0108] Optionally, the segmentation processing submodule includes:

[0109] a tubular feature line extraction unit, configured to extract the tubular feature line in the tubular object image;

[0110] A curved surface reconstruction unit is configured to perform curved surface reconstruction on the tubular object image based on the tubular characteristic line to obtain a three-dimensional image sequence composed of cross-sectional images;

[0111] a segmentation processing unit, configured to perform tubular feature segmentation processing on the three-dimensional image sequence to obtain a tubular feature image;

[0112] The coordinate conversion processing unit is configured to perform coordinate conversion processing on the tubular characteristic image according to the coordinate mapping relationship between the cross-sectional image and the tubular object image to obtain the tubular object characteristic image.

[0113] Optionally, the segmentation processing unit runs based on an image segmentation model, and the image segmentation model includes a convolutional neural network, a long short-term memory neural network, and a fully connected layer; accordingly, the image segmentation model performs tubular feature segmentation processing, including:

[0114] Extracting semantic features of the cross-sectional images contained in the three-dimensional image sequence based on the convolutional neural network to obtain serialized semantic features corresponding to the three-dimensional image sequence;

[0115] Based on the long short-term memory neural network, the serialized semantic features are integrated to obtain serialized image features;

[0116] According to the serialized image features, based on the fully connected layer, predict the lumen feature data of the tubular lumen feature corresponding to the cross-sectional image;

[0117] An inner cavity binary image corresponding to the cross-sectional image is generated based on the inner cavity characteristic data.

[0118] Optionally, the lumen feature data of the tubular lumen feature includes: a direction angle value of the direction angle based on the lumen center of the tubular lumen, and a radius value corresponding to the direction angle; the radius value refers to the distance value from the lumen center to the tubular lumen under the direction angle; wherein the direction angles are equally spaced.

[0119] Optionally, the tubular object to be segmented includes at least one of the following: a description text of the tubular object to be segmented, a two-dimensional image of the tubular object to be segmented, and a sequence of three-dimensional images of the tubular object to be segmented.

[0120] An embodiment of an image processing method provided by the present application is as follows:

[0121] Refer to the attached Figure 5 , which shows a processing flow chart of an image processing method provided by this embodiment, see the attached Figure 6 , which shows a schematic diagram of an image segmentation process provided by this embodiment, see the attached Figure 3 , which shows a schematic diagram of an image segmentation model provided by this embodiment, see the attached Figure 7 , which shows a processing flow chart of an image processing method provided by this embodiment and applied to the coronary artery image segmentation processing process, see the attached Figure 8 , which shows a schematic diagram of a coronary artery image provided by this embodiment.

[0122] Step S502: acquiring at least one image of a tubular object to be processed.

[0123] In practice, many image processing scenarios require the collection of human vascular image data through CTA (Computed Tomography Angiography). However, the collected vascular image data often contains image features of other human tissues. Therefore, it is necessary to segment the image features of the vascular tissue in the vascular image data from the complex image features of human tissues. However, many blood vessels in the human body are too small. Not only must the vascular image be segmented from the complex human body combination, but the accuracy and precision of the image segmentation must also be guaranteed. Image segmentation is difficult.

[0124] The present application provides an image processing method, in the process of performing image segmentation on tubular features in a tubular object image, using the tubular feature lines in the tubular object image, performing surface reconstruction on the tubular object image to form a three-dimensional image sequence of the tubular object, and then inputting the three-dimensional image sequence into an image segmentation model to perform tubular feature segmentation processing, thereby realizing segmentation and extraction of the tubular features in the tubular object image, and restoring the tubular feature image obtained by segmentation and extraction to the coordinate system before surface reconstruction through coordinate transformation, thereby obtaining an image corresponding to the tubular object, and realizing refinement and accuracy of tubular feature segmentation.

[0125] In the embodiment of the present application, the tubular object may be a vascular tissue in a human body or other organisms, such as an arterial blood vessel and a coronary artery in a human body. In addition, the tubular object may also be a 3D printed blood vessel applied to a human body or other organisms, or an artificial organ applied to a human body or other organisms. The embodiment of the present application takes the vascular tissue in the human body as an example to illustrate the segmentation process of the vascular tissue image. For the image segmentation process of 3D printed blood vessels and other artificial tubular objects, refer to the specific implementation of the segmentation process for the vascular tissue image provided in this embodiment, which will not be described in detail in this embodiment.

[0126] The tubular image described in the embodiment of the present application refers to a blood vessel image of a human body acquired by using CTA (Computed Tomography Angiography), for example, a coronary artery image in a human body acquired by using CTA.

[0127] Step S504: extracting tubular feature lines from the tubular object image.

[0128] The tubular feature line described in the embodiment of the present application refers to an auxiliary identification line that can identify the area and shape of the tubular object in the tubular object image. In this embodiment, the tubular feature line refers to the center line of the tubular object in the tubular object image. In addition, the tubular feature line can also be the inner cavity contour line of the tubular object in the tubular object image, or other identification lines that can identify the tubular object in the tubular object image. Here, the purpose of extracting the tubular feature line in the tubular object image is to prepare for the subsequent curve reconstruction of the tubular object image.

[0129] In practical applications, the coronary images collected by CTA (Computed Tomography Angiography) may contain image features of the coronary arteries (coronary vessels), and may also contain image features of the aorta (aortic vessels) connected to or adjacent to the coronary arteries, and the aorta is often more obvious in the coronary images relative to the coronary features, and is easy to segment and extract. Based on this, in an optional implementation provided by an embodiment of the present application, different segmentation and extraction methods are used to perform corresponding segmentation and extraction for the main tubular features and branch tubular features in the tubular image. Specifically, for the main tubular features in the tubular image, the main tubular features in the tubular image are segmented based on the tubular image to obtain a main binary image corresponding to the main tubular features; and for the branch tubular features in the tubular image, steps S504 to S510 are used to implement the segmentation and extraction of the branch tubular features in the tubular image. For example, in a coronary artery image, the main tubular feature refers to the aortic feature corresponding to the aorta, and the branch tubular feature refers to the coronary feature corresponding to the coronary artery.

[0130] In the process of segmenting and extracting the branched tubular features in the tubular image by using steps S504 to S510, extracting the tubular feature lines in the tubular image here refers to extracting the tubular feature lines of the branched tubular features in the tubular image, such as extracting the center line of the coronary feature in the coronary image.

[0131] Step S506: performing curved surface reconstruction on the tubular object image based on the tubular characteristic line to obtain a three-dimensional image sequence composed of cross-sectional images.

[0132] Curved Planar Reformation (CPR) presents a series of vascular images at different levels acquired through CTA in a three-dimensional space based on the center line of the tubular image, so that the internal and external tissue conditions of blood vessels at different levels can be presented to users more intuitively in three-dimensional space.

[0133] In this embodiment, surface reconstruction is performed based on the center line of the vascular image to obtain vascular image features corresponding to the vascular image in three-dimensional space, so as to present the situation of the vascular cavity tissue. Specifically, the vascular image features obtained after surface reconstruction exist in the data form of a three-dimensional image sequence composed of cross-sectional images, wherein each cross-sectional image in the three-dimensional image sequence presents the situation of the vascular cavity tissue at this level.

[0134] Step S508: input the three-dimensional image sequence into an image segmentation model to perform tubular feature segmentation processing to obtain a tubular feature image.

[0135] In an optional implementation manner provided in an embodiment of the present application, the image segmentation model includes a convolutional neural network, a long short-term memory neural network and a fully connected layer; accordingly, the image segmentation model performs tubular feature segmentation processing, including:

[0136] Extracting semantic features of the cross-sectional images contained in the three-dimensional image sequence based on the convolutional neural network to obtain serialized semantic features corresponding to the three-dimensional image sequence; wherein the convolutional neural network may be a two-dimensional convolutional neural network or a three-dimensional convolutional neural network;

[0137] Based on the long short-term memory neural network, the serialized semantic features are integrated to obtain serialized image features;

[0138] According to the serialized image features, based on the fully connected layer, predict the lumen feature data of the tubular lumen feature corresponding to the cross-sectional image;

[0139] An inner cavity binary image corresponding to the cross-sectional image is generated based on the inner cavity characteristic data.

[0140] Optionally, the lumen feature data of the tubular lumen feature includes: a direction angle value of the direction angle based on the lumen center of the tubular lumen, and a radius value corresponding to the direction angle; the radius value refers to the distance value from the lumen center to the tubular lumen under the direction angle; wherein the direction angles are equally spaced.

[0141] For example, Figure 3The image segmentation model shown in the figure performs tubular feature segmentation processing on coronary artery features, wherein the input of the image segmentation model is a cross-sectional image in a three-dimensional image sequence obtained after surface reconstruction. After the cross-sectional image of the coronary artery is input into the image segmentation model, the semantic features of the cross-sectional image of the coronary artery are first extracted by a two-dimensional convolutional neural network (2D CNN), and then the extracted serialized semantic features are input into a two-layer long short-term memory neural network (LSTM) for feature fusion. The feature fusion of the long short-term memory neural network enables the intermediate features to consider the semantic features of the previous and next time series. The feature fusion result output by the long short-term memory neural network is input into the fully connected layer of the output layer to predict the inner cavity features of the coronary artery, specifically predicting the radius of the inner cavity of the coronary artery in various directions, where the direction refers to 180 directions with an angle of 2 degrees distributed at equal intervals. The prediction result output by the fully connected layer is the radius values ​​of the 180 directions corresponding to the cross-sectional image of the coronary artery, then according to the radius values ​​of the coronary artery in 180 directions of the cross-sectional image corresponding to the cross-sectional image, the inner cavity feature image corresponding to the cross-sectional image of the coronary artery is generated.

[0142] It can be seen that in the process of performing tubular feature segmentation processing on the coronary artery features, this embodiment converts the pixel-by-pixel coronary artery feature segmentation processing problem into a process of performing image segmentation on each coronary artery in the vascular image, thereby reducing the difficulty of image segmentation; and, by combining the two-dimensional convolutional neural network and the long short-term memory neural network, the contextual information of the previous and next time series in the coronary artery image sequence is fully utilized, thereby improving the accuracy of the coronary artery feature segmentation processing.

[0143] Step S510: performing coordinate transformation processing on the tubular feature image according to the coordinate mapping relationship between the cross-sectional image and the tubular object image to obtain a target image.

[0144] After the tubular feature image is obtained in the above step S508, coordinate transformation processing is performed on the tubular feature image according to the coordinate mapping relationship between the tubular object image before the curved surface reconstruction and the cross-sectional image after the curved surface reconstruction in the above step S506, that is, the tubular features contained in the tubular feature image are restored to the coordinate system before the curved surface reconstruction, so as to obtain a target image for characterizing the image features of the tubular object.

[0145] As described above, in the process of respectively segmenting and extracting the main tubular features and the branch tubular features in the tubular object image, the main tubular features in the tubular object image are segmented to obtain a main binary image corresponding to the main tubular features, and steps S504 to S510 are used to implement the segmentation and extraction of the branch tubular features in the tubular object image to obtain branch tubular images corresponding to the branch tubular features. Optionally, the main binary image corresponding to the main tubular features is combined with the branch tubular images corresponding to the branch tubular features to obtain a global tubular image.

[0146] The following combination Figure 7 , taking the application of the image processing method provided by this embodiment in the coronary artery image segmentation process as an example, the image processing method provided by this embodiment is further described. Figure 7 The image processing method used in the coronary artery image segmentation process specifically includes steps S702 to S714.

[0147] Step S702: acquiring a series of coronary artery images to be processed.

[0148] Step S704, segmenting the aorta features in the coronary artery image to obtain an aorta binary image corresponding to the aorta features.

[0149] Step S706, extracting the center line of the coronary artery feature in the coronary artery image.

[0150] Step S708, performing curved reconstruction on the coronary artery image based on the center line of the coronary artery feature to obtain a three-dimensional image sequence composed of cross-sectional images.

[0151] Step S710: Input the three-dimensional image sequence into the image segmentation model to perform coronary artery feature segmentation processing to obtain feature images corresponding to the coronary artery features.

[0152] Among them, the input of the image segmentation model is the cross-sectional image in the three-dimensional image sequence obtained after surface reconstruction. After the cross-sectional image of the coronary artery is input into the image segmentation model, the semantic features of the cross-sectional image of the coronary artery are first extracted by a two-dimensional convolutional neural network, and then the extracted serialized semantic features are input into a two-layer long short-term memory neural network (LSTM, Long Short-Term Memory) for feature fusion. The feature fusion of the long short-term memory neural network enables the intermediate features to consider the semantic features of the previous and next time series. The feature fusion results output by the long short-term memory neural network are input into the fully connected layer of the output layer to predict the inner cavity features of the coronary artery, specifically predicting the radius of the inner cavity of the coronary artery in each direction. The direction here refers to 180 directions with an angle of 2 degrees distributed at equal intervals. The prediction result output by the fully connected layer is the radius value of 180 directions corresponding to the cross-sectional image of the coronary artery. According to the radius value of the coronary artery in 180 directions of the section corresponding to the cross-sectional image, the inner cavity feature image of the coronary artery corresponding to this section is generated.

[0153] Step S712, based on the coordinate mapping relationship between the cross-sectional image and the coronary artery image, coordinate transformation processing is performed on the characteristic image corresponding to the coronary artery feature to obtain the coronary artery characteristic image corresponding to the coronary artery.

[0154] Step S714, combining the aorta binary image corresponding to the aorta feature with the coronary feature image corresponding to the coronary artery to obtain a global coronary artery image, as shown in the attached figure. Figure 8 shown.

[0155] In summary, the image processing method provided by the present application, in the process of image segmentation of tubular features in a tubular object image, uses the tubular feature lines in the tubular object image to perform surface reconstruction on the tubular object image to form a three-dimensional image sequence of the tubular object, and then inputs the three-dimensional image sequence into an image segmentation model to perform tubular feature segmentation processing, thereby realizing the segmentation and extraction of the tubular features in the tubular object image, and restores the tubular feature image obtained by the segmentation and extraction to the coordinate system before the surface reconstruction through coordinate transformation, thereby obtaining an image corresponding to the tubular object, and realizing the refinement and accuracy of the tubular feature segmentation.

[0156] An embodiment of an image processing device provided by the present application is as follows:

[0157] In the above-mentioned embodiment, an image processing method is provided, and correspondingly, an image processing device is also provided, which will be described below with reference to the accompanying drawings.

[0158] See attached Fig. 9 , which shows a schematic diagram of an image processing device provided by this embodiment.

[0159] Since the device embodiment corresponds to the method embodiment, the description is relatively simple, and the relevant parts can refer to the corresponding description of the method embodiment provided above. The device embodiment described below is only illustrative.

[0160] The present application provides an image processing device, comprising:

[0161] The tubular object image acquisition module 902 is configured to acquire at least one tubular object image to be processed;

[0162] A tubular feature line extraction module 904 is configured to extract tubular feature lines in the tubular object image;

[0163] A curved surface reconstruction module 906 is configured to perform curved surface reconstruction on the tubular object image based on the tubular characteristic line to obtain a three-dimensional image sequence composed of cross-sectional images;

[0164] The segmentation processing module 908 is configured to input the three-dimensional image sequence into an image segmentation model to perform tubular feature segmentation processing to obtain a tubular feature image;

[0165] The coordinate transformation processing module 910 is configured to perform coordinate transformation processing on the tubular feature image according to the coordinate mapping relationship between the cross-sectional image and the tubular object image to obtain a target image.

[0166] Optionally, the image segmentation model includes a convolutional neural network, a long short-term memory neural network and a fully connected layer;

[0167] Accordingly, the image segmentation model performs tubular feature segmentation processing, including:

[0168] Extracting semantic features of the cross-sectional images contained in the three-dimensional image sequence based on the convolutional neural network to obtain serialized semantic features corresponding to the three-dimensional image sequence;

[0169] Based on the long short-term memory neural network, the serialized semantic features are integrated to obtain serialized image features;

[0170] According to the serialized image features, based on the fully connected layer, predict the lumen feature data of the tubular lumen feature corresponding to the cross-sectional image;

[0171] An inner cavity binary image corresponding to the cross-sectional image is generated based on the inner cavity characteristic data.

[0172] Optionally, the lumen feature data of the tubular lumen feature includes:

[0173] The direction angle value of the direction angle based on the inner cavity center of the tubular inner cavity, and the radius value corresponding to the direction angle; the radius value refers to the distance value from the inner cavity center to the tubular inner cavity under the direction angle;

[0174] Wherein, the direction angles are distributed at equal intervals.

[0175] Optionally, the image processing device further includes:

[0176] The segmentation module is configured to segment the main tubular features in the tubular object image to obtain a main binary image corresponding to the main tubular features.

[0177] Optionally, the extracting the tubular feature line in the tubular object image includes:

[0178] Extracting tubular feature lines of branched tubular features in the tubular object image;

[0179] Correspondingly, the three-dimensional image sequence composed of the cross-sectional images includes a three-dimensional image sequence composed of the cross-sectional images corresponding to the branched tubular features;

[0180] The image segmentation model performs tubular feature segmentation processing, including the image segmentation model performing segmentation processing on the branch tubular feature contained in the cross-sectional image to obtain a tubular feature image corresponding to the branch tubular feature;

[0181] The target image obtained after the coordinate conversion process includes a branched tubular image corresponding to the branched tubular feature.

[0182] Optionally, the image processing device further includes:

[0183] The combination module is configured to combine the trunk binary image corresponding to the trunk tubular feature with the branch tubular image corresponding to the branch tubular feature to obtain a global tubular image.

[0184] Optionally, the tubular object image includes: a blood vessel image obtained by angiography;

[0185] The main tubular features in the tubular image include aortic features corresponding to the aorta in the blood vessel image;

[0186] The branched tubular features in the tubular object image include coronary artery features corresponding to the coronary arteries in the blood vessel image.

[0187] Optionally, the image processing device further includes:

[0188] a coordinate information reading module, configured to read the coordinate information of the cross-sectional image and the coordinate information of the tubular object image;

[0189] The coordinate mapping relationship determination module is configured to determine a mapping relationship between the coordinate information of the cross-sectional image and the coordinate information of the tubular object image as the coordinate mapping relationship.

[0190] Optionally, the convolutional neural network includes at least one of the following: a two-dimensional convolutional neural network or a three-dimensional convolutional neural network.

[0191] An embodiment of a computing device provided by the present application is as follows:

[0192] Fig.10 1 is a block diagram showing a computing device 1000 according to an embodiment of the present application. The components of the computing device 1000 include but are not limited to a memory 1010 and a processor 1020. The processor 1020 is connected to the memory 1010 via a bus 1030, and a database 1050 is used to store data.

[0193] The computing device 1000 also includes an access device 1040 that enables the computing device 1000 to communicate via one or more networks 1060. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 1040 may include one or more of any type of network interface (e.g., a network interface card (NIC)) that is wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a World Wide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, a near field communication (NFC) interface, and the like.

[0194] In one embodiment of the present application, the above components of the computing device 1000 and Fig.10 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Fig.10 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0195] The computing device 1000 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or PC. The computing device 1000 may also be a mobile or stationary server.

[0196] The present application provides a computing device, including a memory 1010, a processor 1020, and computer instructions stored in the memory and executable on the processor, wherein the processor 1020 is used to execute the following computer executable instructions: acquire a sequence of images; acquire a tubular object to be segmented; perform segmentation processing on tubular object features corresponding to the tubular object to be segmented in the sequence of images; and display a tubular object feature image obtained by the segmentation processing.

[0197] Another computing device embodiment provided by the present application is as follows:

[0198] Fig.11 1 is a block diagram showing a computing device 1100 according to an embodiment of the present application. The components of the computing device 1100 include but are not limited to a memory 1110 and a processor 1120. The processor 1120 is connected to the memory 1110 via a bus 1130, and a database 1150 is used to store data.

[0199] The computing device 1100 also includes an access device 1140 that enables the computing device 1100 to communicate via one or more networks 1160. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 1140 may include one or more of any type of network interface (e.g., a network interface card (NIC)) whether wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a World Wide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, a near field communication (NFC) interface, and the like.

[0200] In one embodiment of the present application, the above components of the computing device 1100 and Fig.11 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Fig.11 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0201] The computing device 1100 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or PC. The computing device 1100 may also be a mobile or stationary server.

[0202] The present application provides a computing device, including a memory 1110, a processor 1120, and computer instructions stored in the memory and executable on the processor, wherein the processor 1120 is used to execute the following computer executable instructions: obtain at least one tubular object image to be processed; extract tubular feature lines in the tubular object image; perform surface reconstruction on the tubular object image based on the tubular feature lines to obtain a three-dimensional image sequence composed of cross-sectional images; input the three-dimensional image sequence into an image segmentation model to perform tubular feature segmentation processing to obtain a tubular feature image; perform coordinate conversion processing on the tubular feature image according to the coordinate mapping relationship between the cross-sectional image and the tubular object image to obtain a target image.

[0203] An embodiment of a computer-readable storage medium provided by the present application is as follows:

[0204] An embodiment of the present application provides a computer-readable storage medium storing computer instructions, which implement the steps of the image processing method when executed by a processor.

[0205] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the above-mentioned image processing method belong to the same concept, and the details not described in detail in the technical scheme of the storage medium can be referred to the description of the technical scheme of the above-mentioned image processing method.

[0206] Another computer-readable storage medium embodiment provided by the present application is as follows:

[0207] An embodiment of the present application provides a computer-readable storage medium storing computer instructions, which implement the steps of another image processing method when executed by a processor.

[0208] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the above-mentioned another image processing method belong to the same concept, and the details not described in detail in the technical scheme of the storage medium can be referred to the description of the technical scheme of the above-mentioned another image processing method.

[0209] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0210] The computer instructions include computer program codes, which may be in source code form, object code form, executable files or some intermediate forms, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0211] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of this specification are not limited by the order of the actions described, because according to the embodiments of this specification, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of this specification.

[0212] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0213] The preferred embodiments of this specification disclosed above are only used to help explain this specification. The optional embodiments do not describe all the details in detail, nor do they limit the invention to only the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of the embodiments of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of this specification, so that technicians in the relevant technical field can well understand and use this specification. This specification is only limited by the claims and their full scope and equivalents.

Claims

1. An image processing method, comprising: Get sequence images; Obtaining a tubular object to be segmented; The tubular features corresponding to the tubular objects to be segmented in the sequence of images are segmented to obtain a tubular feature image, wherein the tubular features corresponding to the tubular objects to be segmented in the sequence of images are segmented to obtain a tubular feature image, and the segmentation process includes: inputting a cross-sectional image of a three-dimensional image sequence generated based on the tubular image into an image segmentation model, wherein the tubular image is a sequence of images containing the tubular features in the sequence of images, and the image segmentation model includes a convolutional neural network, a long short-term memory neural network, and a fully connected layer; extracting semantic features of the cross-sectional images contained in the three-dimensional image sequence based on the convolutional neural network to obtain serialized semantic features corresponding to the three-dimensional image sequence; fusing the serialized semantic features based on the long short-term memory neural network to obtain serialized image features; predicting lumen feature data of the tubular lumen features corresponding to the cross-sectional image based on the fully connected layer according to the serialized image features; generating a tubular feature image corresponding to the cross-sectional image based on the lumen feature data; performing coordinate transformation processing on the tubular feature image to obtain the tubular feature image; The characteristic image of the tubular object obtained by segmentation processing is displayed.

2. The image processing method according to claim 1, wherein the segmentation process of the tubular features corresponding to the tubular objects to be segmented in the sequence of images comprises: Determine a sequence of images containing the tubular object features among the sequence of images as a tubular object image; The tubular object features corresponding to the tubular object to be segmented in the tubular object image are segmented to obtain the tubular object feature image.

3. The image processing method according to claim 2, wherein the step of segmenting the tubular features corresponding to the tubular objects to be segmented in the tubular image to obtain the tubular feature image comprises: Extracting tubular feature lines in the tubular object image; Performing curved surface reconstruction on the tubular object image based on the tubular characteristic line to obtain a three-dimensional image sequence composed of cross-sectional images; Performing tubular feature segmentation processing on the three-dimensional image sequence to obtain a tubular feature image; According to the coordinate mapping relationship between the cross-sectional image and the tubular object image, coordinate transformation processing is performed on the tubular characteristic image to obtain the tubular object characteristic image.

4. The image processing method according to claim 1, wherein the lumen feature data of the tubular lumen feature comprises: The direction angle value of the direction angle based on the inner cavity center of the tubular inner cavity, and the radius value corresponding to the direction angle; The radius value refers to the distance value from the center of the inner cavity to the tubular inner cavity at the direction angle; Wherein, the direction angles are distributed at equal intervals. The image processing method according to claim 1 , wherein the tubular object to be segmented is input by a user.

6. The image processing method according to claim 1, wherein the tubular object to be segmented comprises at least one of the following: Description text of the tubular object to be segmented, two-dimensional image of the tubular object to be segmented, and a sequence of three-dimensional images of the tubular object to be segmented.

7. An image processing device, comprising: An image acquisition module is configured to acquire sequence images; A module for obtaining a tubular object to be segmented, configured to obtain the tubular object to be segmented; a segmentation processing module, configured to perform segmentation processing on the tubular features corresponding to the tubular objects to be segmented in the sequence of images to obtain a tubular feature image; The segmentation processing module is further configured to input the cross-sectional images of the three-dimensional image sequence generated based on the tubular image into the image segmentation model, wherein the tubular image is a sequence of images containing the tubular features in the sequence of images, and the image segmentation model includes a convolutional neural network, a long short-term memory neural network and a fully connected layer; extracting the semantic features of the cross-sectional images contained in the three-dimensional image sequence based on the convolutional neural network to obtain the serialized semantic features corresponding to the three-dimensional image sequence; fusing the serialized semantic features based on the long short-term memory neural network to obtain the serialized image features; predicting the lumen feature data of the tubular lumen features corresponding to the cross-sectional image based on the fully connected layer according to the serialized image features; generating the tubular feature image corresponding to the cross-sectional image based on the lumen feature data; performing coordinate transformation processing on the tubular feature image to obtain the tubular feature image; The feature image display module is configured to display the feature image of the tubular object obtained by segmentation processing.

8. An image processing method, comprising: Acquire at least one image of the tubular object to be processed; Extracting tubular feature lines in the tubular object image; Performing curved surface reconstruction on the tubular object image based on the tubular characteristic line to obtain a three-dimensional image sequence composed of cross-sectional images; Inputting the three-dimensional image sequence into an image segmentation model for tubular feature segmentation processing to obtain a tubular feature image, wherein inputting the three-dimensional image sequence into an image segmentation model for tubular feature segmentation processing to obtain a tubular feature image comprises: inputting a cross-sectional image of a three-dimensional image sequence generated based on a tubular object image into the image segmentation model, wherein the tubular object image is a sequence image containing the tubular object feature in the sequence image, and the image segmentation model comprises a convolutional neural network, a long short-term memory neural network, and a fully connected layer; extracting semantic features of the cross-sectional image contained in the three-dimensional image sequence based on the convolutional neural network to obtain serialized semantic features corresponding to the three-dimensional image sequence; fusing the serialized semantic features based on the long short-term memory neural network to obtain serialized image features; predicting lumen feature data of the tubular lumen feature corresponding to the cross-sectional image based on the fully connected layer according to the serialized image features; and generating a tubular feature image corresponding to the cross-sectional image based on the lumen feature data; According to the coordinate mapping relationship between the cross-sectional image and the tubular object image, coordinate transformation processing is performed on the tubular feature image to obtain a target image.

9. The image processing method according to claim 8, wherein the lumen feature data of the tubular lumen feature comprises: The direction angle value of the direction angle based on the inner cavity center of the tubular inner cavity, and the radius value corresponding to the direction angle; The radius value refers to the distance value from the center of the inner cavity to the tubular inner cavity at the direction angle; Wherein, the direction angles are distributed at equal intervals.

10. The image processing method according to claim 8, further comprising: The main tubular features in the tubular object image are segmented to obtain a main binary image corresponding to the main tubular features.

11. The image processing method according to claim 10, wherein extracting the tubular feature line in the tubular object image comprises: Extracting tubular feature lines of branched tubular features in the tubular object image; Correspondingly, the three-dimensional image sequence composed of the cross-sectional images includes a three-dimensional image sequence composed of the cross-sectional images corresponding to the branched tubular features; The image segmentation model performs tubular feature segmentation processing, including the image segmentation model performing segmentation processing on the branch tubular feature contained in the cross-sectional image to obtain a tubular feature image corresponding to the branch tubular feature; The target image obtained after the coordinate conversion process includes a branched tubular image corresponding to the branched tubular feature.

12. The image processing method according to claim 11, wherein after the step of performing coordinate transformation processing on the tubular feature image according to the coordinate mapping relationship between the cross-sectional image and the tubular object image to obtain the target image is performed, the method further comprises: The trunk binary image corresponding to the trunk tubular feature and the branch tubular image corresponding to the branch tubular feature are combined to obtain a global tubular image.

13. The image processing method according to claim 12, wherein the tubular object image comprises: Vascular images obtained using angiography; The main tubular features in the tubular image include aortic features corresponding to the aorta in the blood vessel image; The branched tubular features in the tubular object image include coronary artery features corresponding to the coronary arteries in the blood vessel image.

14. The image processing method according to claim 8, wherein before the step of performing coordinate transformation processing on the tubular feature image according to the coordinate mapping relationship between the cross-sectional image and the tubular object image to obtain the target image is performed, the method comprises: reading coordinate information of the cross-sectional image and coordinate information of the tubular object image; A mapping relationship between the coordinate information of the cross-sectional image and the coordinate information of the tubular object image is determined as the coordinate mapping relationship.

15. The image processing method according to claim 8, wherein the convolutional neural network comprises at least one of the following: 2D Convolutional Neural Network or 3D Convolutional Neural Network.

16. An image processing device, comprising: A tubular object image acquisition module, configured to acquire at least one tubular object image to be processed; A tubular feature line extraction module, configured to extract tubular feature lines in the tubular object image; A curved surface reconstruction module is configured to perform curved surface reconstruction on the tubular object image based on the tubular characteristic line to obtain a three-dimensional image sequence composed of cross-sectional images; The segmentation processing module is configured to input the three-dimensional image sequence into an image segmentation model for tubular feature segmentation processing to obtain a tubular feature image; the segmentation processing module is further configured to input a cross-sectional image of a three-dimensional image sequence generated based on a tubular object image into the image segmentation model, wherein the tubular object image is a sequence image containing the tubular object feature in the sequence image, and the image segmentation model includes a convolutional neural network, a long short-term memory neural network, and a fully connected layer; based on the convolutional neural network, the semantic features of the cross-sectional image contained in the three-dimensional image sequence are extracted to obtain a serialized semantic feature corresponding to the three-dimensional image sequence; based on the long short-term memory neural network, the serialized semantic features are fused to obtain a serialized image feature; based on the serialized image feature, the lumen feature data of the tubular lumen feature corresponding to the cross-sectional image is predicted based on the fully connected layer; and based on the lumen feature data, a tubular feature image corresponding to the cross-sectional image is generated; The coordinate conversion processing module is configured to perform coordinate conversion processing on the tubular feature image according to the coordinate mapping relationship between the cross-sectional image and the tubular object image to obtain a target image.

17. A computing device comprising: Memory and processor; The memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions: Get sequence images; Obtaining a tubular object to be segmented; The tubular features corresponding to the tubular objects to be segmented in the sequence of images are segmented to obtain a tubular feature image, wherein the tubular features corresponding to the tubular objects to be segmented in the sequence of images are segmented to obtain a tubular feature image, and the segmentation process includes: inputting a cross-sectional image of a three-dimensional image sequence generated based on the tubular image into an image segmentation model, wherein the tubular image is a sequence of images containing the tubular features in the sequence of images, and the image segmentation model includes a convolutional neural network, a long short-term memory neural network, and a fully connected layer; extracting semantic features of the cross-sectional images contained in the three-dimensional image sequence based on the convolutional neural network to obtain serialized semantic features corresponding to the three-dimensional image sequence; fusing the serialized semantic features based on the long short-term memory neural network to obtain serialized image features; predicting lumen feature data of the tubular lumen features corresponding to the cross-sectional image based on the fully connected layer according to the serialized image features; generating a tubular feature image corresponding to the cross-sectional image based on the lumen feature data; performing coordinate transformation processing on the tubular feature image to obtain the tubular feature image; The characteristic image of the tubular object obtained by segmentation processing is displayed.

18. A computing device comprising: Memory and processor; The memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions: Acquire at least one image of the tubular object to be processed; Extracting tubular feature lines in the tubular object image; Performing curved surface reconstruction on the tubular object image based on the tubular characteristic line to obtain a three-dimensional image sequence composed of cross-sectional images; Inputting the three-dimensional image sequence into an image segmentation model for tubular feature segmentation processing to obtain a tubular feature image, wherein inputting the three-dimensional image sequence into an image segmentation model for tubular feature segmentation processing to obtain a tubular feature image comprises: inputting a cross-sectional image of a three-dimensional image sequence generated based on a tubular object image into the image segmentation model, wherein the tubular object image is a sequence image containing the tubular object feature in the sequence image, and the image segmentation model comprises a convolutional neural network, a long short-term memory neural network, and a fully connected layer; extracting semantic features of the cross-sectional image contained in the three-dimensional image sequence based on the convolutional neural network to obtain serialized semantic features corresponding to the three-dimensional image sequence; fusing the serialized semantic features based on the long short-term memory neural network to obtain serialized image features; predicting lumen feature data of the tubular lumen feature corresponding to the cross-sectional image based on the fully connected layer according to the serialized image features; and generating a tubular feature image corresponding to the cross-sectional image based on the lumen feature data; According to the coordinate mapping relationship between the cross-sectional image and the tubular object image, coordinate transformation processing is performed on the tubular feature image to obtain a target image.

19. A computer-readable storage medium storing computer instructions, which, when executed by a processor, implement the steps of the image processing method according to any one of claims 1 to 6.

20. A computer-readable storage medium storing computer instructions, which, when executed by a processor, implement the steps of the image processing method according to any one of claims 8 to 15.

21. A computer program product, characterized in that The method comprises computer instructions, which, when executed by a processor, implement the steps of the image processing method according to any one of claims 1 to 6.

22. A computer program product, characterized in that The method comprises computer instructions which, when executed by a processor, implement the steps of the image processing method according to any one of claims 8 to 15.

Images