Image data processing method and device based on multi-label segmentation model

Abstract

The present application relates to the fields of artificial intelligence and digital medicine, and provides an image data processing method and device based on a multi-label segmentation model, the main purpose of which is to improve the existing technical problems that cannot meet the accuracy and intelligent recognition requirements of lesion area recognition of lung nodules. It includes: acquiring image data of the target area; performing segmentation processing on the image data based on a multi-label segmentation model that has completed model training to obtain a segmentation processing result with multiple labels; in response to output instructions with different labels, obtaining an image segmentation area matching the target label from the segmentation processing result according to the target label carried in the output instruction, and searching for medical project information based on the target label; if matching medical project information is found, the image segmentation area is rendered in combination with the medical project information and the target label for output. It is mainly used for image processing.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence and digital medical technology, and in particular to an image data processing method and device based on a multi-label segmentation model. Background Art

[0002] With the rapid development of digital medical technology, intelligent medical systems have gained a place in the medical field through the support of their own strong artificial intelligence algorithms, especially computed tomography (CT), which has become a common means of screening body lesions. Nowadays, the automatic segmentation algorithm of lung diseases based on deep learning has been widely used in various lesions such as lung nodules and calcifications.

[0003] Currently, chest CT images are segmented to determine lung lesion areas, wherein a lesion area is segmented based on a segmentation processing model. However, since the human chest is composed of multiple organs, multiple lesion areas may be missed when identifying lung lesion areas, or lesion areas of other organs may be used as lesion areas of the lung area. This cannot meet the requirements of accurate and intelligent identification of lesion areas such as lung nodules. Therefore, an image data processing method is urgently needed to solve the above problems. Summary of the invention

[0004] In view of this, the present application provides an image data processing method and device based on a multi-label segmentation model, the main purpose of which is to improve the existing technical problems that cannot meet the requirements of accurate and intelligent identification of lesion areas of lung nodules.

[0005] According to one aspect of the present application, a method for processing image data based on a multi-label segmentation model is provided, comprising:

[0006] Acquire image data of the target area;

[0007] Segmenting the image data based on a multi-label segmentation model for which model training has been completed to obtain a segmentation processing result with multiple labels, wherein the multi-label segmentation model is obtained by model training based on a multi-dimensional one-hot vector as an input parameter and configuring weight parameters according to a preset number of labels;

[0008] In response to output instructions with different labels, obtaining an image segmentation area matching the target label from the segmentation processing result according to the target label carried in the output instruction, and searching for medical project information based on the target label;

[0009] If matching medical project information is found, the image segmentation area is rendered and outputted in combination with the medical project information and the target label.

[0010] Preferably, before acquiring the image data of the target area, the method further comprises:

[0011] Obtain a multi-label multi-region image training sample set;

[0012] Constructing a multi-label neural network including a trunk neural network and at least two branch neural networks, wherein the number of branch neural networks is equal to the number of preset labels;

[0013] Constructing a multi-dimensional one-hot vector based on the sample image data used as input parameters in the multi-label multi-region image training sample set, and configuring weight parameters according to the matching relationship between the number of preset labels and the backbone neural network;

[0014] The multi-label neural network is trained based on the multi-label multi-region image training sample set, the multi-dimensional one-hot vector, and the weight parameters to obtain a multi-label segmentation model.

[0015] Preferably, after constructing a multi-label neural network comprising a trunk neural network and at least two branch neural networks, the method further comprises:

[0016] Normalizing the sample image data in the multi-label multi-region image training sample set, and extracting image blocks of the sample image data after the normalization processing according to a scanning order;

[0017] The image blocks are subjected to image data enhancement processing, and the image blocks subjected to image data enhancement processing are combined to obtain sample image data to be subjected to model training.

[0018] Preferably, after acquiring the image data of the target area, the method further comprises:

[0019] Performing standardization processing on the image data based on a bilinear interpolation method;

[0020] Normalizing the image data after the normalization processing, and extracting image blocks of the image data after the normalization processing according to a scanning order;

[0021] The image blocks are subjected to image data enhancement processing, and the image blocks subjected to image data enhancement processing are combined to obtain image data of the result to be segmented.

[0022] Preferably, searching for medical project information based on the target tag includes:

[0023] Analyzing characteristic information of the target organ corresponding to the target tag;

[0024] The medical project information corresponding to the characteristic information is searched in a preset medical knowledge base, wherein the preset medical knowledge base stores characteristic information of different organs and medical project information matching different characteristic information.

[0025] Preferably, the analyzing the characteristic information of the target organ corresponding to the target tag includes:

[0026] Determine segmentation feature information of the target tag, wherein the segmentation feature information includes shape feature, intensity feature, and texture feature;

[0027] The characteristic information of the target organ is counted according to the segmentation characteristic information, wherein the characteristic information includes the volume ratio of the circumscribed sphere, the standard deviation of the surface-center distance, and the intersection distance of the circumscribed cuboid.

[0028] Preferably, the method further comprises:

[0029] Searching for medical project information, risk information, and symptom information matching the characteristic information based on the intelligent medical lesion database;

[0030] Display labels corresponding to the medical item information, the risk information, and the symptom information are generated, and the display labels are arranged in the image segmentation area to trigger viewing.

[0031] According to another aspect of the present application, there is provided an image data processing device based on a multi-label segmentation model, comprising:

[0032] A first acquisition module, used to acquire image data of a target area;

[0033] A segmentation module, used to perform segmentation processing on the image data based on a multi-label segmentation model for which model training has been completed, to obtain a segmentation processing result with multiple labels, wherein the multi-label segmentation model is obtained by model training based on a multi-dimensional one-hot vector as an input parameter and configuring weight parameters according to a preset number of labels;

[0034] A first search module is used to respond to output instructions with different labels, obtain an image segmentation area matching the target label from the segmentation processing result according to the target label carried in the output instruction, and search for medical project information based on the target label;

[0035] The output module is used to render the image segmentation area in combination with the medical project information and the target label for output if matching medical project information is found.

[0036] Preferably, before the first acquisition module, the device further includes:

[0037] The second acquisition module is used to acquire a multi-label multi-region image training sample set;

[0038] A construction module, used to construct a multi-label neural network including a trunk neural network and at least two branch neural networks, wherein the number of the branch neural networks is equal to the number of preset labels;

[0039] A configuration module, used to construct a multi-dimensional one-hot vector based on the sample image data used as input parameters in the multi-label multi-region image training sample set, and configure weight parameters according to the matching relationship between the number of preset labels and the backbone neural network;

[0040] A training module is used to perform model training on the multi-label neural network based on the multi-label multi-region image training sample set, the multi-dimensional one-hot vector, and the weight parameters to obtain a multi-label segmentation model.

[0041] Preferably, after the building module, the device further comprises:

[0042] A first normalization module, used to perform normalization processing on the sample image data in the multi-label multi-region image training sample set, and extract image blocks of the sample image data after the normalization processing according to a scanning order;

[0043] The first enhancement module is used to perform image data enhancement processing on the image blocks, and combine the image blocks after the image data enhancement processing to obtain sample image data to be used for model training.

[0044] Preferably, after the first acquisition module, the device further includes:

[0045] A standardization module, used for performing standardization processing on the image data based on a bilinear interpolation method;

[0046] A second normalization module, used to perform normalization processing on the image data after the normalization processing, and extract image blocks of the image data after the normalization processing according to a scanning order;

[0047] The second enhancement module is used to perform image data enhancement processing on the image blocks, and combine the image blocks after the image data enhancement processing to obtain image data of the results to be segmented.

[0048] Preferably, the first search module includes:

[0049] A parsing unit, used for parsing characteristic information of the target organ corresponding to the target tag;

[0050] The search unit is used to search for medical project information corresponding to the characteristic information based on a preset medical knowledge base, wherein the preset medical knowledge base stores characteristic information of different organs and medical project information matching different characteristic information.

[0051] Preferably, the parsing unit comprises:

[0052] A determination subunit, used to determine segmentation feature information of the target tag, wherein the segmentation feature information includes shape feature, intensity feature, and texture feature;

[0053] A statistical subunit is used to collect characteristic information of the target organ according to the segmentation characteristic information, wherein the characteristic information includes the volume ratio of the circumscribed sphere, the standard deviation of the surface-center distance, and the intersection distance of the circumscribed cuboid.

[0054] Preferably, the device further comprises:

[0055] A second search module is used to search for medical project information, risk information, and symptom information matching the characteristic information based on the intelligent medical lesion database;

[0056] A generation module is used to generate display labels corresponding to the medical project information, the risk information, and the symptom information, and configure the display labels in the image segmentation area to trigger viewing.

[0057] According to another aspect of the present application, a storage medium is provided, in which at least one executable instruction is stored, and the executable instruction enables a processor to perform operations corresponding to the above-mentioned image data processing method based on the multi-label segmentation model.

[0058] According to another aspect of the present application, there is provided a computer device, comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus;

[0059] The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform operations corresponding to the above-mentioned image data processing method based on the multi-label segmentation model.

[0060] By means of the above technical solution, the technical solution provided by the embodiment of the present application has at least the following advantages:

[0061] The present application provides an image data processing method and device based on a multi-label segmentation model, firstly acquiring image data of a target area; performing segmentation processing on the image data based on a multi-label segmentation model that has completed model training, and obtaining a segmentation processing result with multiple labels, wherein the multi-label segmentation model is based on a multi-dimensional unique hot vector as an input parameter, and the model training is performed by configuring weight parameters according to a preset number of labels; in response to output instructions of different labels, obtaining an image segmentation area matching the target label from the segmentation processing result according to the target label carried in the output instruction, and searching for medical project information based on the target label; if matching medical project information is found, rendering the image segmentation area in combination with the medical project information and the target label, and outputting it. Compared with the prior art, the embodiment of the present application performs segmentation processing on image data through a multi-label segmentation model, which meets the requirements for accurate and intelligent recognition of lesion areas of lung nodules.

[0062] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present application. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0064] Figure 1 A flowchart of an image data processing method based on a multi-label segmentation model provided in an embodiment of the present application is shown;

[0065] Figure 2 A block diagram of an image data processing device based on a multi-label segmentation model provided in an embodiment of the present application is shown;

[0066] Figure 3 A schematic diagram of the structure of a computer device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0067] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0068] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0069] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0070] Based on this, in one embodiment, Figure 1 As shown, a method for processing image data based on a multi-label segmentation model is provided, and the method is applied to a computer device such as a server as an example for explanation, wherein the server can be an independent server, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms, such as intelligent medical systems, digital medical platforms, etc. The above method includes the following steps:

[0071] 101. Obtain image data of a target area.

[0072] In the embodiment of the present application, the user of the terminal can be a patient or a doctor. The target area is the area that the user wants to examine and diagnose, such as the lung area, the liver area, etc. The image data is the image data taken when the user receives treatment and examination, such as CT image data, B-ultrasound image data, etc.

[0073] It should be noted that in the process of acquiring image data, the image data can be acquired by scanning a printed image film, or by photographing and collecting the image through a treatment and examination device (such as a CT machine) connected to the intelligent medical system, and the embodiments of the present application do not make specific limitations.

[0074] For example, chest CT image data of a patient is acquired by taking photos with a CT machine connected to the intelligent medical system.

[0075] 102. Perform segmentation processing on the image data based on the multi-label segmentation model that has completed model training to obtain a segmentation processing result with multiple labels.

[0076] In the embodiment of the present application, the multi-label segmentation model is obtained by training the model based on a multi-dimensional one-hot vector as an input parameter and configuring weight parameters according to the preset number of labels, wherein the one-hot vector is a one-bit effective code. The label is used to mark the result obtained after the image data is segmented.

[0077] It should be noted that when the multi-label segmentation model performs segmentation processing, it uses an N-bit state register to encode N states. Each state has its own independent register bit, and at any time, only one of them is valid. That is, only one bit is 1, and the rest are zero values.

[0078] Exemplarily, in the embodiment of the present application, taking the chest CT image data of the user as an example, the CT image data is spliced ​​into an N-dimensional unique hot vector as the input of the model for prediction processing. Preferably, since the label prediction processing of multiple diseases is performed on one CT image data in the embodiment of the present application, the weight parameters configured in the multi-label segmentation model are determined based on the number of preset labels.

[0079] 103. In response to output instructions with different labels, an image segmentation region matching the target label is obtained from the segmentation processing result according to the target label carried in the output instruction, and medical project information is searched based on the target label.

[0080] In the embodiment of the present application, the medical project information may be characteristic information of the target area obtained by diagnosis based on the target label, or may be further treatment plans or medication categories, etc., which are not specifically limited in the embodiment of the present application. After obtaining the prediction processing results with multiple labels, in order to facilitate the user to observe the conditions of the target areas with different labels, thereby realizing the intelligence of smart medical care, the user can output by clicking on each segmented area that has been marked in the image. In response to the generation instruction of different labels, the matching image segmentation area is obtained according to the target label carried in the output instruction, and further, the corresponding medical project information is searched based on the target label.

[0081] For example, if a doctor wants to identify one of three labels in a CT image, he clicks on the marked area, and the current medical system obtains the segmented image area corresponding to this label in the CT image to determine what type of lesion this area is and further search for medical project information.

[0082] 104. If matching medical project information is found, the image segmentation area is rendered and outputted in combination with the medical project information and the target label.

[0083] In an embodiment of the present application, since the medical project information can assist the user in identifying the image segmentation area, when outputting, each image segmentation area is first rendered in combination with the medical project information and the target label, that is, each segmentation area is rendered in a different color, and the target label is marked in a corresponding position, and the medical project information is configured as matching feature data in the display area. For example, in order to avoid the image of the segmented area being obscured by the medical project information, a link dot can be set at the position of the target label mark in the image, and the medical project information is displayed only when the dot is clicked, so that the user can intuitively observe the information corresponding to different image segmentation areas.

[0084] For further explanation and limitation, in an embodiment of the present application, before obtaining the image data of the target area, the method of this embodiment also includes: obtaining a multi-label multi-region image training sample set; constructing a multi-label neural network including a backbone neural network and at least two branch neural networks, the number of branch neural networks being equal to the number of preset labels; constructing a multi-dimensional unique hot vector based on the sample image data used as input parameters in the multi-label multi-region image training sample set, and configuring weight parameters according to the matching relationship between the preset number of labels and the backbone neural network; performing model training on the multi-label neural network based on the multi-label multi-region image training sample set, the multi-dimensional unique hot vector, and the weight parameters to obtain a multi-label segmentation model.

[0085] The constructed multi-label neural network includes a trunk neural network and multiple branch neural networks, and the number of branch neural networks is at least two, which is specifically determined by the number of target labels. After the branch neural network is connected to the trunk neural network, preferably, the trunk neural network is a UNet network, so as to output the segmentation results of multiple labels based on each branch neural network. The multi-label neural network model is trained using a unique hot vector based on the number of target labels spliced ​​together from the multi-label multi-region image training samples, wherein the network model weight parameters are configured according to the relationship between the number of target labels and the preset trunk branch network.

[0086] Specifically, in the deep learning algorithm training process, before each iteration, training samples are first sampled from the training data sets of different labels, so that the number of training samples of each label sampled is equal, and N is the number of target labels. At the same time, since the input image data for prediction processing in step 102 is preferably image block data Therefore, from each disease data sampled, image blocks with length, width and height of 224×224×28 are extracted in overlapping order according to the scan line order (from top to bottom, from left to right), and the overlap width between image blocks is 10 voxels. Then, the image blocks are randomly scaled, rotated and flipped by any one or more transformations for data enhancement, which is recorded as The transformed image blocks are used to form a mini-batch.

[0087] Among them, the structure of the multi-label neural network in the embodiment of the present application is a backbone network connected to N branch networks, that is, a branched neural network is constructed to uniformly process different label segmentation training data sets. Exemplarily, the neural network uses UNet as the backbone network, denoted as B(x; Θ), x represents the input of the backbone network, and Θ represents the parameter set of the backbone network. The backbone network is followed by N independent three-layer convolutional branch networks to output the segmentation results of each label, respectively denoted as S1(f; Θ1),...,S N (f; θ N ), f is the input of each branch network, that is, the output of the backbone network, Θ i For S i The image block As a training sample, after C convolution layers, the feature matrix is ​​flattened into a 1*N vector (the parameter C is an integer, and its size can be set as needed). In addition, each training sample is concatenated with an N-dimensional one-hot vector Input the neural network for training. The method of constructing a one-hot vector is as follows: If and only if When x comes from the tth lesion dataset, t =1, the rest are 0. This one-hot vector is used to control the error when it is transmitted backwards, and only the t-th branch network is updated. The specific calculation process is as follows: Formula (1). The one-hot vector is set according to the label type. For example, if there are 5 categories (numbered 0-4) of labels, the 0th label sample is input for training. Then the vector is [1, 0, 0, 0, 0]. Use the Adam optimizer to minimize the loss function As the goal, train the backbone network B and branch networks S1,...,S N ,in is the predicted segmentation mask of the tth label, that is, the output of St. Then the loss function Designed for

[0088]

[0089]

[0090]

[0091] It should be noted that after the training is completed, the parameters Θ, Θ1, ..., Θ are determined. N,When predicting, firstly, the predicted image is preprocessed. The predicted image can be scaled using bilinear interpolation according to the median of spacing, normalized using the zscore normalization algorithm, and divided into a series of 224×224×28 image blocks {px according to the order of scan lines with overlap i}, complete the data preprocessing process to avoid missed segmentation or mis-segmentation at the edges of the image blocks. Available Get the image block prediction segmentation mask of the tth label, and then According to the inverse operation of the division, the predicted segmentation mask of the t-th label of the image to be predicted x is obtained. Therefore, when t takes the value [1, N] respectively, the predicted segmentation masks of N labels can be obtained.

[0092] Since there are foreground areas and background areas in the image data, in order to accurately identify each area, in an embodiment of the present application, further, after constructing a multi-label neural network including a backbone neural network and at least two branch neural networks, the method of this embodiment also includes: normalizing the sample image data in the multi-label multi-region image training sample set, and extracting image blocks of the sample image data after the normalization processing in a scanning order; performing image data enhancement processing on the image blocks, and combining the image blocks after the image data enhancement processing to obtain sample image data to be trained on the model.

[0093] Specifically, the mean and standard deviation of the foreground area are first calculated. The specific calculation process is: In the area of The mean and standard deviation of the data are then substituted into the zscore normalization algorithm. Normalize. In addition, since the embodiment of the present application needs to perform segmentation and labeling of multiple diseases from one image based on one model, in order to improve the accuracy of data processing, the image data is overlapped and extracted with a length, width, and height of 224×224×28 in the order of scan lines (from top to bottom, from left to right), and the overlap width between image blocks is 10 voxels. Furthermore, the image blocks are randomly scaled, rotated, and flipped for data enhancement, which is recorded as The changed image blocks are used to form a mini-batch as the data object for the final prediction processing.

[0094] Since image data may contain multiple label information, in order to ensure that the pixel spacing of the image is uniform so as to improve the processing accuracy when processing in the model, the embodiment of the present application is further preferred. After acquiring the image data of the target area, the image data can also be standardized by a bilinear interpolation method, specifically including: standardizing the image data based on the bilinear interpolation method; normalizing the image data after the standardization processing, and extracting image blocks of the image data after the normalization processing in a scanning order; performing image data enhancement processing on the image blocks, and combining the image blocks after the image data enhancement processing to obtain image data to be segmented.

[0095] Specifically, the spacing of all image data in each data set is counted. Spacing is the physical size of each voxel in the image, that is, the pixel spacing, which is the original information carried by the medical imaging data. After determining the median of the spacing, all image data and their corresponding label-annotated image data are scaled. The image is scaled using bilinear interpolation, and the corresponding label-annotated image is scaled using nearest neighbor interpolation. After scaling, the spacing of the image and its corresponding label-annotated image is exactly the median size of the above spacing, so as to achieve the purpose of unifying the spacing of CT images. And further normalization and enhancement processing are performed. The specific processing method has been explained in the above steps and will not be repeated here.

[0096] In an embodiment of the present application, further, after obtaining the image segmentation area matching the target label from the segmentation processing result, the corresponding medical project information can also be searched according to the target label, specifically including: parsing the characteristic information of the target organ corresponding to the target label; searching for medical project information corresponding to the characteristic information based on a preset medical knowledge base.

[0097] Among them, the preset medical knowledge base stores characteristic information of different organs and medical project information matching different characteristic information. The characteristic information is used to characterize the dominant state of the target organ disease. For example, the lungs show a flocculent morphology, indicating that the lungs are inflamed, that is, the characteristic information of the current user's lungs is inflammation. Furthermore, the treatment plan or type of medication corresponding to lung inflammation is searched from the preset medical knowledge base. It can be understood that the medical knowledge base is based on the accumulation and classification of massive historical medical records by medical institutions. It can be stored within the medical institution or in the cloud processor of the intelligent medical system. The embodiments of the present application do not make specific limitations.

[0098] Illustratively, in an embodiment of the present application, taking a lung CT image as an example, parsing the characteristic information of a target organ corresponding to a target label includes: determining segmentation characteristic information of the target label; and calculating characteristic information of the target organ according to the segmentation characteristic information.

[0099] Among them, the segmentation feature information includes shape features, intensity features, and texture features; the feature information includes the circumscribed spherical volume ratio, the surface-center distance standard deviation, and the circumscribed rectangular block intersection line distance. The circumscribed spherical volume ratio is the ratio of each ROI lung lesion extracted from the three-dimensional CT image to its circumscribed spherical volume, which reflects the similarity of the region to the sphere. The surface-center distance standard deviation is the standard deviation of the coordinate distance between each voxel on the surface of the lung nodule and the center of the region, and its value also describes the sphericity of the lung lesion. The circumscribed rectangular block intersection line distance is the distance between the central voxel of the lung lesion and the center of the 12 intersection lines of the six sections of its circumscribed rectangular block, which can indicate the degree of uniform distribution of the voxels in the region in the rectangular body. The specific diagnostic content of the lung lesion can be determined based on the lung feature information such as the circumscribed spherical volume ratio, the surface-center distance standard deviation, and the circumscribed rectangular block intersection line distance. In addition, the property features, intensity features, and texture features can be determined based on image processing technology of the image segmentation area, so that the lung feature information can be statistically analyzed based on the shape features, intensity features, and texture features.

[0100] For the convenience of user use, in the embodiment of the present application, it is further preferred that the method of this embodiment also includes: searching for medical project information, risk information, and symptom information that matches the feature information based on the intelligent medical lesion database; generating display labels corresponding to the medical project information, risk information, and symptom information, respectively, and configuring the display labels in the image segmentation area for triggering viewing.

[0101] The lesion database stores characteristic information of lesions in different organ parts, as well as medical project information, risk information, and symptom information that match the lesion characteristic information. The medical project information includes methods, means, and types of medications for treating lesions; risk information includes the risks to life caused by the lesion, such as survival rate, survival time, etc.; symptom information includes medical symptoms caused by the lesion, such as weight loss, pain, etc. For example, the characteristic information of gastric cancer, as well as surgical plans, stomach pain and weight loss symptoms, and survival rate that match gastric cancer are stored. In order not to affect the doctor's observation angle, when displaying and outputting, a display label can be generated and configured in the corresponding image segmentation area, wherein the display label can be a smaller display component, such as a dot or exclamation mark, so that a text box can be displayed when triggered.

[0102] The present application provides an image data processing method based on a multi-label segmentation model, firstly acquiring image data of a target area; performing segmentation processing on the image data based on a multi-label segmentation model that has completed model training, and obtaining a segmentation processing result with multiple labels, wherein the multi-label segmentation model is based on a multi-dimensional unique hot vector as an input parameter, and the model training is performed by configuring weight parameters according to a preset number of labels; in response to output instructions of different labels, obtaining an image segmentation area matching the target label from the segmentation processing result according to the target label carried in the output instruction, and searching for medical project information based on the target label; if matching medical project information is found, rendering the image segmentation area in combination with the medical project information and the target label, and outputting it. Compared with the prior art, the embodiment of the present application performs segmentation processing on image data through a multi-label segmentation model, which meets the requirements for accurate and intelligent recognition of lesion areas of lung nodules.

[0103] Furthermore, as a response to the above Figure 1 The implementation of the method shown in the embodiment of the present application provides an image data processing device based on a multi-label segmentation model, such as Figure 2 As shown, the device includes: a first acquisition module 21, a segmentation module 22, a first search module 23, and an output module 24.

[0104] A first acquisition module 21, used to acquire image data of a target area;

[0105] A segmentation module 22, used to perform segmentation processing on the image data based on a multi-label segmentation model that has completed model training, to obtain a segmentation processing result with multiple labels, wherein the multi-label segmentation model is obtained by model training based on a multi-dimensional one-hot vector as an input parameter and configuring weight parameters according to a preset number of labels;

[0106] A first search module 23 is used to respond to output instructions with different labels, obtain an image segmentation area matching the target label from the segmentation processing result according to the target label carried in the output instruction, and search for medical project information based on the target label;

[0107] The output module 24 is used to render the image segmentation area in combination with the medical project information and the target label for output if matching medical project information is found.

[0108] In a specific application scenario, before the first acquisition module, the device further includes:

[0109] The second acquisition module is used to acquire a multi-label multi-region image training sample set;

[0110] A construction module, used to construct a multi-label neural network including a trunk neural network and at least two branch neural networks, wherein the number of the branch neural networks is equal to the number of preset labels;

[0111] A configuration module, used to construct a multi-dimensional one-hot vector based on the sample image data used as input parameters in the multi-label multi-region image training sample set, and configure weight parameters according to the matching relationship between the number of preset labels and the backbone neural network;

[0112] A training module is used to perform model training on the multi-label neural network based on the multi-label multi-region image training sample set, the multi-dimensional one-hot vector, and the weight parameters to obtain a multi-label segmentation model.

[0113] In a specific application scenario, after the construction module, the device further includes:

[0114] A first normalization module, used to perform normalization processing on the sample image data in the multi-label multi-region image training sample set, and extract image blocks of the sample image data after the normalization processing according to a scanning order;

[0115] The first enhancement module is used to perform image data enhancement processing on the image blocks, and combine the image blocks after the image data enhancement processing to obtain sample image data to be used for model training.

[0116] In a specific application scenario, after the first acquisition module, the device further includes:

[0117] A standardization module, used for performing standardization processing on the image data based on a bilinear interpolation method;

[0118] A second normalization module, used to perform normalization processing on the image data after the normalization processing, and extract image blocks of the image data after the normalization processing according to a scanning order;

[0119] The second enhancement module is used to perform image data enhancement processing on the image blocks, and combine the image blocks after the image data enhancement processing to obtain image data of the results to be segmented.

[0120] In a specific application scenario, the first search module includes:

[0121] A parsing unit, used for parsing characteristic information of the target organ corresponding to the target tag;

[0122] The search unit is used to search for medical project information corresponding to the characteristic information based on a preset medical knowledge base, wherein the preset medical knowledge base stores characteristic information of different organs and medical project information matching different characteristic information.

[0123] In a specific application scenario, the parsing unit includes:

[0124] A determination subunit, used to determine segmentation feature information of the target tag, wherein the segmentation feature information includes shape feature, intensity feature, and texture feature;

[0125] A statistical subunit is used to collect characteristic information of the target organ according to the segmentation characteristic information, wherein the characteristic information includes the volume ratio of the circumscribed sphere, the standard deviation of the surface-center distance, and the intersection distance of the circumscribed cuboid.

[0126] In a specific application scenario, the device further includes:

[0127] A second search module is used to search for medical project information, risk information, and symptom information matching the characteristic information based on the intelligent medical lesion database;

[0128] A generation module is used to generate display labels corresponding to the medical project information, the risk information, and the symptom information, and configure the display labels in the image segmentation area to trigger viewing.

[0129] The present application provides an image data processing device based on a multi-label segmentation model, which first acquires image data of a target area; performs segmentation processing on the image data based on a multi-label segmentation model that has completed model training, and obtains a segmentation processing result with multiple labels, wherein the multi-label segmentation model is based on a multi-dimensional unique hot vector as an input parameter, and the model training is performed by configuring weight parameters according to a preset number of labels; in response to output instructions of different labels, an image segmentation area matching the target label is obtained from the segmentation processing result according to the target label carried in the output instruction, and medical project information is searched based on the target label; if matching medical project information is found, the image segmentation area is rendered in combination with the medical project information and the target label, and outputted. Compared with the prior art, the embodiment of the present application performs segmentation processing on image data through a multi-label segmentation model, which meets the requirements for accurate and intelligent recognition of lesion areas of lung nodules.

[0130] According to one embodiment of the present application, a storage medium is provided, wherein the storage medium stores at least one executable instruction, and the computer executable instruction can execute the image data processing method based on the multi-label segmentation model in any of the above method embodiments.

[0131] Based on this understanding, the technical solution of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each implementation scenario of the present application.

[0132] Figure 3 A schematic diagram of the structure of a computer device provided according to an embodiment of the present application is shown. The specific embodiment of the present application does not limit the specific implementation of the computer device.

[0133] like Figure 3 As shown, the computer device may include: a processor (processor) 302 , a communication interface (Communications Interface) 304 , a memory (memory) 306 , and a communication bus 308 .

[0134] The processor 302 , the communication interface 304 , and the memory 306 communicate with each other via the communication bus 308 .

[0135] The communication interface 304 is used to communicate with other devices such as clients or other servers.

[0136] The processor 302 is used to execute the program 310, and specifically can execute the relevant steps in the above-mentioned embodiment of the image data processing method based on the multi-label segmentation model.

[0137] Specifically, the program 310 may include program codes, which include computer operation instructions.

[0138] The processor 302 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application. The one or more processors included in the computer device may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0139] The memory 306 is used to store the program 310. The memory 306 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0140] The program 310 may be specifically configured to enable the processor 302 to perform the following operations:

[0141] Acquire image data of the target area;

[0142] Segmenting the image data based on a multi-label segmentation model for which model training has been completed to obtain a segmentation processing result with multiple labels, wherein the multi-label segmentation model is obtained by model training based on a multi-dimensional one-hot vector as an input parameter and configuring weight parameters according to a preset number of labels;

[0143] In response to output instructions with different labels, obtaining an image segmentation area matching the target label from the segmentation processing result according to the target label carried in the output instruction, and searching for medical project information based on the target label;

[0144] If matching medical project information is found, the image segmentation area is rendered and outputted in combination with the medical project information and the target label.

[0145] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages the hardware and software resources of the entity device for business data processing based on the multimodal hybrid model, and supports the operation of information processing programs and other software and / or programs. The network communication module is used to realize communication between components within the storage medium, as well as communication with other hardware and software in the information processing entity device.

[0146] Obviously, those skilled in the art should understand that the above modules or steps of the present application can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, and optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present application is not limited to any specific combination of hardware and software.

[0147] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. An image data processing method based on a multi-label segmentation model, characterized in that: include: Acquire image data of the target area; Segmenting the image data based on a multi-label segmentation model for which model training has been completed to obtain a segmentation processing result with multiple labels, wherein the multi-label segmentation model is obtained by model training based on a multi-dimensional one-hot vector as an input parameter and configuring weight parameters according to a preset number of labels; In response to output instructions with different labels, obtaining an image segmentation area matching the target label from the segmentation processing result according to the target label carried in the output instruction, and searching for medical project information based on the target label; If matching medical project information is found, the image segmentation area is rendered and outputted in combination with the medical project information and the target label; Before acquiring the image data of the target area, the method further includes: Obtain a multi-label multi-region image training sample set; Constructing a multi-label neural network including a trunk neural network and at least two branch neural networks, wherein the number of branch neural networks is equal to the number of preset labels; Constructing a multi-dimensional one-hot vector based on the sample image data used as input parameters in the multi-label multi-region image training sample set, and configuring weight parameters according to the matching relationship between the number of preset labels and the backbone neural network; The multi-label neural network is trained based on the multi-label multi-region image training sample set, the multi-dimensional one-hot vector and the weight parameters to obtain a multi-label segmentation model.

2. The method according to claim 1, characterized in that After constructing the multi-label neural network including a trunk neural network and at least two branch neural networks, the method further includes: Normalizing the sample image data in the multi-label multi-region image training sample set, and extracting image blocks of the sample image data after the normalization processing according to a scanning order; The image blocks are subjected to image data enhancement processing, and the image blocks subjected to image data enhancement processing are combined to obtain sample image data to be subjected to model training.

3. The method according to claim 2, characterized in that After acquiring the image data of the target area, the method further includes: Performing standardization processing on the image data based on a bilinear interpolation method; Normalizing the image data after the normalization processing, and extracting image blocks of the image data after the normalization processing according to a scanning order; The image blocks are subjected to image data enhancement processing, and the image blocks subjected to image data enhancement processing are combined to obtain image data of the result to be segmented.

4. The method according to claim 1, characterized in that The searching for medical project information based on the target tag includes: Analyzing characteristic information of the target organ corresponding to the target tag; The medical project information corresponding to the characteristic information is searched in a preset medical knowledge base, wherein the preset medical knowledge base stores characteristic information of different organs and medical project information matching different characteristic information.

5. The method according to claim 4, characterized in that The analyzing the characteristic information of the target organ corresponding to the target tag includes: Determining segmentation feature information of the target tag, wherein the segmentation feature information includes shape feature, intensity feature and texture feature; The characteristic information of the target organ is counted according to the segmentation characteristic information, wherein the characteristic information includes the volume ratio of the circumscribed sphere, the standard deviation of the surface-center distance, and the intersection distance of the circumscribed cuboid.

6. The method according to claim 4, characterized in that The method further comprises: Searching for medical project information, risk information and disease information matching the characteristic information based on the intelligent medical lesion database; Display labels corresponding to the medical item information, the risk information and the symptom information are generated, and the display labels are arranged in the image segmentation area to trigger viewing.

7. An image data processing device based on a multi-label segmentation model, characterized in that: include: A first acquisition module, used to acquire image data of a target area; A segmentation module, used to perform segmentation processing on the image data based on a multi-label segmentation model for which model training has been completed, to obtain a segmentation processing result with multiple labels, wherein the multi-label segmentation model is obtained by model training based on a multi-dimensional one-hot vector as an input parameter and configuring weight parameters according to a preset number of labels; A first search module is used to respond to output instructions with different labels, obtain an image segmentation area matching the target label from the segmentation processing result according to the target label carried in the output instruction, and search for medical project information based on the target label; An output module, for rendering the image segmentation area in combination with the medical project information and the target label for output if matching medical project information is found; The device also includes: The second acquisition module is used to acquire a multi-label multi-region image training sample set; A construction module, used to construct a multi-label neural network including a trunk neural network and at least two branch neural networks, wherein the number of the branch neural networks is equal to the number of preset labels; A configuration module, used to construct a multi-dimensional one-hot vector based on the sample image data used as input parameters in the multi-label multi-region image training sample set, and configure weight parameters according to the matching relationship between the number of preset labels and the backbone neural network; A training module is used to perform model training on the multi-label neural network based on the multi-label multi-region image training sample set, the multi-dimensional one-hot vector and the weight parameters to obtain a multi-label segmentation model.

8. A storage medium, characterized in that: The storage medium stores at least one executable instruction, and the executable instruction enables the processor to perform operations corresponding to the image data processing method based on a multi-label segmentation model as described in any one of claims 1 to 6.

9. A computer device, characterized in that: include: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform operations corresponding to the image data processing method based on a multi-label segmentation model as described in any one of claims 1-6.

Images