Object recognition method, device, equipment and storage medium

Abstract

The present disclosure provides an object recognition method, apparatus, device, and medium for use in the fields of radiology and image processing. The object recognition method comprises: using a first model to recognize a CT image of a tissue to be tested, obtaining characteristic data of each of multiple target regions in the tissue to be tested; and using a second model to perform object recognition on the multiple target regions based on elemental density data of the tissue to be tested and the characteristic data of each of the multiple target regions, obtaining object recognition results.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing, specifically to the application of image processing in the field of radiological medicine, and more specifically to an object recognition method, apparatus, device, medium, and program product. Background Art

[0002] Automatic delineation of target volumes and organs at risk plays an important role in radiotherapy and is a key step in treatment planning. Automatic delineation refers to the process of automatically identifying and labeling radiotherapy targets and organs at risk from medical images using computer algorithms. For example, automatic delineation of target volumes and organs at risk can be performed on computed tomography (CT) or magnetic resonance imaging (MRI) images.

[0003] However, current automatic outlining technology has low repeatability and high error rate, which increases the complexity of automatic outlining. Summary of the Invention

[0004] In view of the above problems, the present disclosure provides an object recognition method, apparatus, device, medium, and program product.

[0005] According to a first aspect of the present disclosure, an object recognition method is provided, comprising: using a first model to recognize a CT image of a tissue to be tested, and obtaining characteristic data of each of a plurality of target areas in the tissue to be tested; and using a second model to perform object recognition on the plurality of target areas based on element density data of the tissue to be tested and the characteristic data of each of the plurality of target areas, and obtaining an object recognition result.

[0006] A second aspect of the present disclosure provides an object recognition device, comprising: a first recognition module, configured to use a first model to recognize a CT image of a tissue to be tested, and obtain characteristic data of each of a plurality of target areas in the tissue to be tested; and a second recognition module, configured to use a second model to perform object recognition on the plurality of target areas based on element density data of the tissue to be tested and the characteristic data of each of the plurality of target areas, and obtain an object recognition result.

[0007] A third aspect of the present disclosure provides an electronic device, comprising: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the above-mentioned object recognition method.

[0008] A fourth aspect of the present disclosure further provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to execute the above-mentioned object recognition method.

[0009] A fifth aspect of the present disclosure further provides a computer program product, comprising a computer program, which implements the above-mentioned object recognition method when executed by a processor. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The above contents and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0011] Figure 1 Schematically shows a flow chart of an object recognition method according to an embodiment of the present disclosure;

[0012] Figure 2 A schematic diagram of identifying an object according to an embodiment of the present disclosure is schematically shown.

[0013] Figure 3A A schematic diagram schematically illustrates an initial boundary according to an embodiment of the present disclosure;

[0014] Figure 3B A schematic diagram schematically illustrates a target boundary according to an embodiment of the present disclosure;

[0015] Figure 4 Schematically shows a structural block diagram of an object recognition device according to an embodiment of the present disclosure; and

[0016] Figure 5 The block diagram schematically shows an electronic device suitable for implementing the particle object recognition method according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0017] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present disclosure. In the detailed description below, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0018] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise," "include," etc. used herein indicate the presence of features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0019] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0020] When using expressions such as "at least one of A, B, and C," they should generally be interpreted in accordance with the meaning of the expression as commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to systems having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, and C, etc.). In the technical solutions of the present disclosure, the collection, storage, use, processing, transmission, provision, disclosure, and application of the user's personal information involved are in compliance with the provisions of relevant laws and regulations, necessary confidentiality measures have been taken, and they do not violate public order and good morals. In the technical solutions of the present disclosure, the user's authorization or consent is obtained before obtaining or collecting the user's personal information.

[0021] Figure 1 The flowchart of the object recognition method according to the embodiment of the present disclosure is schematically shown.

[0022] like Figure 1 As shown, the object recognition method of this embodiment includes operations S110 to S120.

[0023] In operation S110 , the CT image of the tissue to be tested is recognized using the first model to obtain characteristic data of each of a plurality of target regions in the tissue to be tested.

[0024] In the disclosed embodiments, a CT image is a three-dimensional image obtained by performing a CT scan on a tissue under test using scanning radiation. For example, the tissue under test can be a region of an animal's body under test, and a CT scan of the region under test produces a CT image. The CT image is characterized by the HU (Hounsfield Unit) value of each voxel in the tissue under test. The HU value is a unit of measurement for determining the density of a local tissue or organ and can reflect the degree of X-ray absorption by the tissue or organ. For example, the tissue under test can be considered to be composed of multiple voxels, and the CT image can be represented by a three-dimensional matrix, where each value in the three-dimensional matrix can be the HU value of a voxel.

[0025] In the disclosed embodiments, the multiple target regions may be regions where possible treatment targets or organs at risk are located in the tissue to be tested. By performing object recognition on the CT image using the first model, the possible treatment targets or organs at risk in the CT image can be preliminarily located, thereby obtaining the multiple target regions.

[0026] The characteristic data may include shape information and position information. The shape information is used to characterize the shape and size of the treatment target or organ at risk, and the position information is used to describe the location of the treatment target or organ at risk in the tissue to be measured.

[0027] In the embodiment of the present disclosure, the first model is a pre-trained Unet network. The first model can be trained based on multiple labeled CT images of treatment targets or organs at risk.

[0028] For example, multiple CT images with labeled treatment targets or organs at risk are input into a first initial model to train the first initial model to identify treatment targets or organs at risk based on the CT images. The first initial model can output feature data of the treatment targets or organs at risk. If the accuracy of the output of the first initial model is determined to meet the recognition requirements, the first model is obtained.

[0029] In operation S120 , the second model is used to perform object recognition on the multiple target areas based on the element density data of the tissue to be tested and the characteristic data of each of the multiple target areas to obtain an object recognition result.

[0030] In embodiments of the present disclosure, elemental density data of the tissue under test may include the density distribution of carbon, nitrogen, oxygen, hydrogen, calcium, and phosphorus in the subject under test. For example, the tissue under test may include a plurality of voxels, and the elemental density data may represent the density of carbon, nitrogen, oxygen, hydrogen, calcium, and phosphorus in each voxel. The density distribution of each element may be represented by a three-dimensional matrix, where each value in the three-dimensional matrix may be the elemental density of a voxel.

[0031] The second model may be a nnUnet (no-new-UNet) network. The second model may be trained based on multiple labeled CT images of treatment targets or organs at risk, elemental density data of tissues corresponding to the CT images, and feature data of the CT images output by the first model.

[0032] For example, multiple CT images with labeled treatment targets or organs at risk, elemental density data of the tissue corresponding to the CT images, and feature data of the CT images output by the first model are input into a second initial model to train the second initial model to identify treatment targets or organs at risk based on the feature data and elemental density data. The second initial model can output an object recognition result. The object recognition result is the result of labeling the treatment target or organ at risk in the CT image of the tissue to be tested. When it is determined that the accuracy of the output result of the second initial model meets the recognition requirements, the second model is obtained.

[0033] The elemental density of each voxel can be used to determine the difference in elemental density between multiple voxels in the tissue being examined. When two voxels belong to different treatment targets or organs at risk, the elemental density between the two voxels may differ significantly. Therefore, using elemental density data, treatment targets or organs at risk can be further precisely located within multiple target regions.

[0034] Through the disclosed embodiments, the accuracy of recognition can be improved by performing two identifications on the objects to be identified in the tissue to be tested using a first model and a second model. Using the first model, the objects to be identified that may exist in the tissue to be tested are preliminarily segmented based on the CT image to obtain a preliminary identification result. Using the second model, the output of the first model is further segmented based on the elemental density data of the tissue to be tested to output an accurate identification result of the objects to be identified. Using two models for two-step identification allows the models to effectively extract the HU value information of the CT image of the tissue to be tested and the density information of the elemental density data, respectively, thereby improving recognition accuracy.

[0035] Figure 2 A schematic diagram of identifying an object according to an embodiment of the present disclosure is schematically shown.

[0036] like Figure 2 As shown, the CT images include a first CT image 201 and a second CT image 202 which are input into a first model 203 . The first model 203 performs object recognition based on the first CT image 201 and the second CT image 202 and outputs feature data 204 .

[0037] In the embodiment of the present disclosure, the first model 203 performs operation S110 to identify the CT image of the tissue to be tested and obtain characteristic data of each of the multiple target areas in the tissue to be tested, which may include: acquiring the first CT image 201 and the second CT image 202 included in the CT image; determining multiple initial boundaries formed by the multiple voxels based on the first CT value and the second CT value of each of the multiple voxels; and determining multiple target areas formed by the multiple initial boundaries to obtain characteristic data of each of the multiple target areas.

[0038] In the embodiment of the present disclosure, the first CT image and the second CT image are obtained by scanning rays of different energies of the tissue to be tested. The first CT image includes the first CT value of each of the multiple voxels of the tissue to be tested, and the second CT image includes the second CT value of each of the multiple voxels.

[0039] Conventional CT systems use X-ray sources that emit only a single energy spectrum, known as mono-energy CT. When scanning multiple tissue samples using a single X-ray source, different elemental compositions may result in identical HU values ​​under a single energy spectrum. In this case, further differentiation between the two tissue samples in the CT image is impossible, leading to biased diagnostic results.

[0040] For example, a dual-energy CT system has two energy spectra. These spectra can be generated by different radiation sources, and the energy components included in these spectra can be different. When a CT system scans the same tissue material using different energy spectra, the HU values ​​may be the same or different.

[0041] Therefore, using dual-energy CT, it is possible to compare two CT images of two tissue materials, each scanned using two different energy spectra. When two tissue materials have the same HU value in one energy spectrum, they will often have different HU values ​​in the other energy spectrum, thus further distinguishing the two tissue materials.

[0042] In the disclosed embodiment, the first CT image and the second CT value are used to represent the HU value of a voxel. Multiple target regions can represent regions of multiple objects located in the tissue to be measured. The HU values ​​of voxels in different target regions can vary significantly, so different target regions can be distinguished based on the first CT image 201 and the second CT image 202.

[0043] For example, the tissues to be measured include normal subjects without disease and tumor subjects with disease. The HU values ​​of the voxels that make up the normal subjects differ significantly from the HU values ​​of the voxels that make up the tumor subjects. The first model 203 can distinguish between normal subjects and tumor subjects based on the first CT image 201 and the second CT image 202.

[0044] For another example, the tissue to be measured includes the optic nerve and the eyeball. The HU values ​​of the voxels constituting the optic nerve and the eyeball are significantly different. The first model 203 can distinguish the optic nerve from the eyeball based on the first CT image 201 and the second CT image 202.

[0045] In the disclosed embodiment, based on the differences between the HU values ​​of multiple voxels, voxels belonging to the same target region and voxels belonging to different target regions can be determined. For example, adjacent voxels with a large difference in HU values ​​among the multiple voxels can form an initial boundary. When the HU value difference between the voxels forming the multiple initial boundaries is small, these initial boundaries can form a target region. The first model can determine feature data based on the position information of the multiple voxels within the target region.

[0046] For example, when it is determined that the difference between the two first CT values ​​and the difference between the two second CT values ​​of two adjacent voxels both meet preset conditions, the first model 203 determines the two voxels as boundary voxels, and determines multiple initial boundaries based on the position information of each of the multiple boundary voxels.

[0047] Combine Figure 2 and Figure 3A The initial boundary is schematically illustrated. Figure 3A A schematic diagram schematically illustrates an initial boundary according to an embodiment of the present disclosure.

[0048] Figure 3A The illustrated cell can be considered as a voxel, and voxel 311 is adjacent to voxel 322. When at least one of the difference between the first CT values ​​of voxel 311 and voxel 321 and the difference between the second CT values ​​of voxel 311 and voxel 321 satisfies a preset condition, voxel 311 and voxel 321 are boundary voxels.

[0049] For example, the preset condition may be that the difference in CT values ​​between the two voxels is greater than or equal to 5%. If it is determined that the difference between the first CT values ​​of voxel 311 and voxel 321 is greater than 5% and the difference between the second CT values ​​of voxel 311 and voxel 321 is greater than 5%, voxel 311 and voxel 321 are boundary voxels.

[0050] Because voxels representing different tissue types may have the same HU value in a single-energy CT image, dual-energy CT images can accurately determine whether multiple voxels belong to the same tissue type. For example, if the difference between the first CT values ​​of two voxels does not meet a preset condition, but the difference between the second CT values ​​of the two voxels does, the two voxels are considered boundary voxels. For example, if both the difference between the first CT values ​​of two voxels and the difference between the second CT values ​​of the two voxels do not meet a preset condition, the two voxels belong to the same tissue type.

[0051] In the embodiment of the present disclosure, multiple boundary voxels belonging to the same tissue type may form an initial boundary, and multiple initial boundaries belonging to the same tissue type may form a target area.

[0052] For example, voxel 311, voxel 312, voxel 321, and voxel 322 are all boundary voxels. Voxel 311 and voxel 312 belong to the same tissue type, and voxel 321 and voxel 322 belong to the same tissue type. Therefore, voxel 311 and voxel 312 can form initial boundary 1, and voxel 321 and voxel 322 can form initial boundary 2. Initial boundary 1 and initial boundary 2 belong to different tissue types, so initial boundary 1 and initial boundary 2 cannot form a target region.

[0053] Figure 3A Target region D1 and target region D2 are shown. Target region D1 is represented by cells filled with gray, and target region D2 is represented by cells filled with diagonal lines. The dotted box represents the initial boundary of target region D1, and the solid box represents the initial boundary of target region D2.

[0054] In the embodiment of the present disclosure, the multiple voxels included in the same target area all belong to the same type of tissue, that is, the difference between the first CT values ​​of any two voxels in the multiple voxels included in the same target area does not meet the preset condition and the difference between the second CT values ​​of any two voxels also does not meet the preset condition.

[0055] In the embodiment of the present disclosure, the first model 203 determines multiple target areas formed by multiple initial boundaries in the tissue to be measured, and identifies shape information of each of the multiple target areas and position information of the multiple target areas in the tissue to be measured to obtain feature data 204.

[0056] The CT image can be a three-dimensional image, and a three-dimensional coordinate system can be constructed based on the CT image. The position information of the target area can be described based on the coordinate information of the voxels in the target area, and the shape information of the target area can be described based on the distribution of the voxels in the target area.

[0057] For example, in a three-dimensional matrix representing a CT image, the position information of each HU value in the three-dimensional matrix may represent the position information of the corresponding voxel.

[0058] For example, the dual-energy CT image may be two three-dimensional matrices. For example, a single CT image may be a 512*512*100 matrix, and the dual-energy CT image may be represented by a 512*512*200 matrix obtained by splicing the two three-dimensional matrices.

[0059] The first model 203 can downsample the spliced ​​matrix through the convolution kernel to collect the HU values ​​of multiple voxels and the spatial relationship between the voxels. The output of the first model 203 can be a 512*512*100 matrix. In the output matrix, the voxels belonging to the target area can be marked as "1", and the voxels in other areas can be marked as "0". Therefore, the output matrix is ​​represented by "1" and "0". Based on the distribution of "1" in the output matrix, the position information and shape information of each target area can be determined.

[0060] Return Reference Figure 2 The feature data 204 and the element density data 206 are input to the second model 205 , and the second model 205 performs object recognition on multiple target areas based on the feature data 204 and the element density data 206 , and outputs an object recognition result 207 .

[0061] In the embodiment of the present disclosure, the second model 205 executes operation S120 to perform object recognition on multiple target areas based on the element density data of the tissue to be tested and the characteristic data of each of the multiple target areas to obtain object recognition results, which may include: determining the tissue type of each of the multiple target areas based on the shape information and position information represented by the characteristic data 204; setting weight values ​​for the density data of each of the multiple elements included in the element density data based on the tissue type; determining multiple target boundaries formed by multiple voxels of each of the multiple target areas based on the density data of each of the multiple elements and their respective weight values; and determining the object recognition result 207 based on the target boundaries.

[0062] In the disclosed embodiments, elemental density data of the tissue under test can be determined from a dual-energy CT image of the tissue under test. For example, based on the distribution pattern between dual-energy CT values ​​and elemental density in a dual-energy CT image, the elemental density of each object under test can be determined based on its respective dual-energy CT value. The distribution pattern between dual-energy CT values ​​and elemental density can be derived based on known dual-energy CT values ​​and elemental densities of the sample material.

[0063] For example, a fitting function describing the distribution pattern is obtained by fitting the known dual-energy CT values ​​of a sample material and the elemental density of each voxel in the sample material. The fitting function describes the distribution pattern between the dual-energy CT values ​​and the elemental density. Given the dual-energy CT values ​​and the fitting function of the tissue being tested, the elemental density of the tissue being tested can be calculated.

[0064] For example, the known dual-energy CT values ​​and elemental densities of a sample material are fed into a deep model. The model then learns the data characteristics of the dual-energy CT values, the elemental densities, and the correlation between them, thereby achieving model training. The trained deep model learns the distribution patterns between dual-energy CT values ​​and elemental densities. By feeding the trained deep model with a dual-energy CT image of the tissue being tested, the deep model can output the elemental density of the tissue being tested.

[0065] The element density data represents the element density of each of a plurality of voxels included in the tissue to be measured. The distribution of the element density in the tissue to be measured can be determined based on the element density data.

[0066] In the disclosed embodiments, the target object in the target region is further identified based on the elemental density of multiple voxels within the target region. The distribution characteristics of different elements within the same tissue type may differ. For example, within the same tissue type, oxygen content may be high, but nitrogen content may be low. Therefore, for tissues belonging to this tissue type, the oxygen density of the voxel can accurately determine whether the voxel belongs to that tissue type.

[0067] Based on the positional information and shape information of the tissue under test, the second model 205 can calculate the probability that the tissue under test belongs to various tissue types. For example, different organs in an animal are located at different locations within the body and have different shapes. Based on the positional information of the tissue under test, the multiple organs to which the tissue under test belongs can be determined. Combined with the shape information of the tissue under test, the target organ to which the tissue under test belongs can be determined.

[0068] Set target weights for different elements based on the elemental distribution characteristics of different tissue types. For example, if a tissue type is primarily composed of oxygen and carbon, with low levels of hydrogen and nitrogen, you can set larger target weights for oxygen and carbon, and smaller target weights for hydrogen and nitrogen.

[0069] In the embodiment of the present disclosure, the second model 205 also sets initial weight values ​​for multiple elements for multiple target areas based on the tissue types of each of the multiple target areas, and adjusts the initial weight values ​​based on the tissue types of the adjacent target areas of each of the multiple target areas to obtain target weight values.

[0070] In an embodiment of the present disclosure, when two target regions of a vector have the same distribution characteristics for the same element, the weight value for the element may be adjusted to improve the accuracy of determining the target boundary based on the element density.

[0071] For example, the first type of tissue is mainly composed of oxygen and carbon elements, and the second type of tissue is mainly composed of oxygen and hydrogen elements. For the first type of tissue, larger initial weight values ​​can be set for the oxygen and carbon elements. For the second type of tissue, larger initial weight values ​​can be set for the hydrogen and hydrogen elements. Since the first type of tissue and the second type of tissue are adjacent, the boundary between the first type of tissue and the second type of tissue cannot be accurately identified based on the oxygen element, so the initial weight value of the oxygen element needs to be reduced to avoid the oxygen element misleading the determination of the target boundary. In this case, the weight value of the carbon element of the first type of tissue and the weight value of the hydrogen element of the second type of element can be further increased.

[0072] In the embodiment of the present disclosure, the second model 205 determines the respective distribution information of multiple elements in the multiple target areas based on the respective element density data of the voxels in the multiple target areas, and uses the respective distribution information and respective weight values ​​of the multiple elements to calculate the element distribution difference between two adjacent voxels in the multiple target areas, and then determines multiple target boundaries based on the element distribution difference.

[0073] The element densities of the same element for voxels belonging to different types of tissues vary greatly, so the object to be identified can be identified from the target area based on the element density data of the voxels in each of the multiple target areas.

[0074] For example, there is a significant difference between the oxygen density of voxels constituting a normal subject and the oxygen density of voxels constituting a tumor subject. The second model 205 can distinguish between normal subjects and tumor subjects based on the oxygen density. For another example, there is a significant difference between the carbon density of voxels constituting the optic nerve and the carbon density of voxels constituting the eyeball. The second model 205 can distinguish between the optic nerve and the eyeball based on the carbon density.

[0075] In the disclosed embodiments, based on the differences between the elemental densities of multiple voxels for the same element, voxels within the target region that belong to the object to be identified and voxels that do not belong to the object to be identified can be determined. For example, adjacent voxels within the target region that have a large difference in oxygen density can form a target boundary. When the difference in oxygen density between the voxels forming multiple target boundaries is small, these target boundaries can form the object to be identified.

[0076] For example, when the difference in oxygen density between two adjacent voxels in the target area is greater than 5%, the second model 205 may determine the two voxels as boundary voxels and determine multiple target boundaries based on the position information of each of the multiple boundary voxels.

[0077] In the embodiment of the present disclosure, there may be certain errors in determining the target boundary based on a single element density. Therefore, based on the element density and target weight value of each voxel for multiple elements, the comprehensive element density of the voxel can be calculated, thereby determining whether the element is a boundary voxel based on the comprehensive element density of each voxel.

[0078] Combine Figure 2 、 Figure 3A and Figure 3B The initial boundary is schematically illustrated. Figure 3B A schematic diagram of a target boundary according to an embodiment of the present disclosure is schematically shown.

[0079] Figure 3B The cell shown may be considered as a voxel. For example, based on the output matrix of the first model 203, it is determined that the voxel 313 and the voxel 311 both belong to the target region D1.

[0080] If the difference between the integrated element densities of voxel 311 and voxel 313 is greater than 5%, voxel 311 or voxel 313 belongs to the object to be identified. Further, the integrated element densities of voxel 311 and voxel 313 are compared with the integrated element densities of the remaining voxels in the target region.

[0081] For example, if the number of voxels whose integrated element density differs from that of voxel 313 by less than 5% is greater than the number of voxels whose integrated element density differs from that of voxel 311 by less than 5% among the remaining voxels, voxel 313 may be considered a boundary voxel.

[0082] In the disclosed embodiment, whether a voxel belongs to an object to be identified is determined based on the comprehensive element density of each voxel. Multiple boundary voxels belonging to an object to be identified can form a target boundary, and multiple target boundaries belonging to an object to be identified can form an object to be identified.

[0083] In the target region D1 , voxel 312 and voxel 314 are boundary voxels, and voxel 312 and voxel 314 form target boundary 1. In the target region D2 , voxel 321 and voxel 323 are boundary voxels, and voxel 321 and voxel 323 form target boundary 2.

[0084] The target boundary in the target area D1 forms the object P1 , and the target boundary in the target area D2 forms the object P2 . Figure 3B It is shown that the object P1 is represented by a gray filled cell, and the object P2 is represented by a diagonal filled cell. The dotted line box represents the target boundary of the object P1, and the solid line box represents the target boundary of the object P2.

[0085] In the embodiment of the present disclosure, the element density data may be represented in the form of a three-dimensional matrix. For example, in the three-dimensional matrix representing the element density data, the element density of each voxel is consistent with the position information of the HU value.

[0086] For example, the element density data may include three-dimensional matrices for each of carbon, nitrogen, oxygen, hydrogen, calcium, and phosphorus. For example, if a single element is a 512*512*100 matrix, the element density data may be represented by a 512*512*600 matrix obtained by concatenating six three-dimensional matrices. The output matrix of the first model 203 has a dimension of 512*512*100, so the output matrix representing the feature data 204 is concatenated with the element density data to obtain a 512*512*700 matrix, which is then input into the second model 205.

[0087] The second model 205 can assign weights to the matrices of different elements in the element density data through an attention mechanism and perform object recognition. The output of the second model 205 is also a 512*512*100 matrix. In the output matrix, voxels belonging to the object to be recognized can be marked as "1", and voxels in other areas can be marked as "0". Therefore, the output matrix is ​​represented by "1" and "0". Based on the distribution of "1" in the output matrix, the position information and shape information of the object to be recognized can be determined.

[0088] In the embodiment of the present disclosure, based on the dual-energy CT image, the HU value of the voxel under different energy spectra is combined to perform preliminary identification of the object to be identified in the tested tissue, thereby improving the recognition accuracy based on the CT image. In addition, based on the difference in element distribution between different tissues, the element density distribution is used to correct the initial recognition result to improve the recognition accuracy. For example, when the difference between the dual-energy HU values ​​of two voxels is less than 5%, the two voxels are determined to belong to the same target area or are considered to belong to the object to be identified, and are difficult to distinguish. Combined with the element density zoning of the voxels, when it is determined that the carbon element density of the two voxels differs by 20%, the boundary can be clearly delineated in combination with the carbon element density distribution map.

[0089] Through the disclosed embodiment, a channel attention mechanism is adopted in the second model 205, so that the network model can focus on element density information with greater diagnostic significance, thereby making more accurate delineation predictions based on the element density fusion of CT images, and improving the network's delineation accuracy for target areas and organs at risk.

[0090] In some embodiments, the present disclosure also provides a model training method.

[0091] In the disclosed embodiment, multiple dual-energy CT images are manually outlined and the target areas or organs at risk are annotated, with the dual-energy CT images and the annotated results being used as sample data. For example, in order to ensure the diversity and generalization ability of the training data, the target area or organ at risk data for each disease type may include at least 100 sample data. Data enhancement methods such as rotation, mirroring, and elastic deformation are used to enhance the 100 sample data to expand the amount of original data from 100 to more than 500. In addition, the dual-energy CT images are processed using a specific element decomposition algorithm to extract the density distribution of the six main elements, H, C, N, O, P, and Ca. These element density distributions will assist the network in more accurately identifying and locating target areas and organs at risk during the automatic outlining process.

[0092] The augmented sample data is fed into the first initial model to train it. The input data for the first initial model is a 512*512*200 matrix, and the output data is a 512*512*100 matrix. The output matrix can be used to represent the location and shape information of the target area.

[0093] The element density data representing the element density distribution and the output matrix of the first initial model are input into the second initial model to train the second initial model. The input data of the second output model is a 512*512*700 matrix, and the output data is a 512*512*100 matrix. Based on the output matrix, the position and shape information of the object to be identified can be represented.

[0094] In an embodiment of the present disclosure, the second initial model also sets a weight value for each element density based on the output result of the first initial model, and adjusts the weight value based on the output result of the second initial model until the output result of the second initial model meets the recognition requirements.

[0095] In the embodiment of the present disclosure, the model training method can refer to the object recognition method described above, which will not be repeated for the sake of brevity.

[0096] Based on the above object recognition method, the present disclosure also provides an object recognition device. Figure 4 The device is described in detail.

[0097] Figure 4 The following schematically shows a structural block diagram of an object recognition device according to an embodiment of the present disclosure.

[0098] like Figure 4 As shown, the object recognition device 400 of this embodiment includes a first recognition module 410 and a second recognition module 420 .

[0099] The first recognition module 410 is used to recognize the CT image of the tissue to be tested using the first model to obtain characteristic data of each of multiple target areas in the tissue to be tested. In one embodiment, the first recognition module 410 can be used to perform the operation S110 described above, which will not be repeated here.

[0100] The second recognition module 420 is configured to use the second model to perform object recognition on the multiple target regions based on the element density data of the tissue to be tested and the characteristic data of each of the multiple target regions, thereby obtaining object recognition results. In one embodiment, the second recognition module 420 can be configured to perform operation S120 described above, which will not be further described herein.

[0101] According to an embodiment of the present disclosure, the first identification module 410 is also used to obtain a first CT image and a second CT image included in the CT image, where the first CT image and the second CT image are obtained based on scanning rays of different energies of the tissue to be tested, the first CT image includes a first CT value of each of a plurality of voxels of the tissue to be tested, and the second CT image includes a second CT value of each of a plurality of voxels; based on the first CT value and the second CT value of each of the plurality of voxels, a plurality of initial boundaries formed by the plurality of voxels are determined; and a plurality of target areas formed by the plurality of initial boundaries are determined to obtain characteristic data of each of the plurality of target areas.

[0102] According to an embodiment of the present disclosure, the first identification module 410 is also used to determine multiple target areas formed by multiple initial boundaries in the tissue to be tested; and to identify the shape information of each of the multiple target areas and the position information of the multiple target areas in the tissue to be tested to obtain feature data.

[0103] According to an embodiment of the present disclosure, the first identification module 410 is further configured to determine that two voxels are boundary voxels when at least one of the difference between the two first CT values ​​and the difference between the two second CT values ​​of two adjacent voxels satisfies a preset condition; and to determine multiple initial boundaries based on the position information of each of the multiple boundary voxels.

[0104] According to an embodiment of the present disclosure, the second recognition module 420 is used to determine the tissue type of each of the multiple target areas based on the shape information and position information represented by the feature data; based on the tissue type, set a target weight value for the density data of each of the multiple elements included in the element density data; based on the density data of each of the multiple elements and the respective target weight values, determine a plurality of target boundaries formed by a plurality of voxels of each of the multiple target areas; and determine the object recognition result based on the plurality of target boundaries.

[0105] According to an embodiment of the present disclosure, the second identification module 420 is also used to determine the respective distribution information of multiple elements in multiple target areas based on the respective element density data of the voxels in the multiple target areas; calculate the element distribution difference between two adjacent voxels in the multiple target areas using the respective distribution information of the multiple elements and the respective target weight values; and determine multiple target boundaries based on the element distribution difference.

[0106] According to an embodiment of the present disclosure, the second identification module 420 is also used to set initial weight values ​​for multiple elements for multiple target areas based on the tissue types of each of the multiple target areas; and adjust the initial weight values ​​based on the tissue types of the adjacent target areas of each of the multiple target areas to obtain target weight values.

[0107] According to embodiments of the present disclosure, any multiple modules in the first identification module 410 and the second identification module 420 may be combined into a single module, or any one of them may be split into multiple modules. Alternatively, at least part of the functionality of one or more of these modules may be combined with at least part of the functionality of other modules and implemented in a single module. According to embodiments of the present disclosure, at least one of the first identification module 410 and the second identification module 420 may be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or may be implemented in hardware or firmware through any other reasonable means of circuit integration or packaging, or implemented in any one of the three implementation methods of software, hardware, and firmware, or any appropriate combination thereof. Alternatively, at least one of the first identification module 410 and the second identification module 420 may be at least partially implemented as a computer program module that, when executed, performs the corresponding functionality.

[0108] Figure 5 The block diagram schematically shows an electronic device suitable for implementing the object recognition method according to an embodiment of the present disclosure.

[0109] like Figure 5 As shown, the electronic device 500 according to an embodiment of the present disclosure includes a processor 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage unit 508 into a random access memory (RAM) 503. The processor 501 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 501 may also include onboard memory for caching purposes. The processor 501 may include a single processing unit or multiple processing units for performing different actions of the method flow according to the embodiment of the present disclosure.

[0110] Various programs and data required for the operation of the electronic device 500 are stored in the RAM 503. The processor 501, ROM 502, and RAM 503 are connected to each other via a bus 504. The processor 501 executes the various operations of the method flow according to the embodiment of the present disclosure by executing the programs in the ROM 502 and / or RAM 503. It should be noted that the programs may also be stored in one or more memories other than the ROM 502 and RAM 503. The processor 501 may also execute the various operations of the method flow according to the embodiment of the present disclosure by executing the programs stored in one or more memories.

[0111] According to an embodiment of the present disclosure, electronic device 500 may further include an input / output (I / O) interface 505, which is also connected to bus 504. Electronic device 500 may also include one or more of the following components connected to I / O interface 505: an input section 506 including a keyboard, mouse, etc.; an output section 507 including devices such as a cathode ray tube (CRT), liquid crystal display (LCD), and speakers; a storage section 508 including a hard disk; and a communication section 509 including a network interface card such as a LAN card or modem. Communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to I / O interface 505 as needed. Removable media 511, such as a magnetic disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed in drive 510 as needed, so that computer programs read from the removable media can be installed into storage section 508 as needed.

[0112] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments, or may exist independently and not be incorporated into the device / apparatus / system. The computer-readable storage medium carries one or more programs, and when executed, implements the method according to the embodiments of the present disclosure.

[0113] According to an embodiment of the present disclosure, a computer-readable storage medium may be a non-volatile computer-readable storage medium, and may include, for example, but not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to an embodiment of the present disclosure, a computer-readable storage medium may include the ROM 502 and / or RAM 503 described above, and / or one or more memories other than ROM 502 and RAM 503.

[0114] The embodiments of the present disclosure also include a computer program product, which includes a computer program containing program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the object recognition method provided by the embodiments of the present disclosure.

[0115] The computer program executes the above functions defined in the system / device of the embodiment of the present disclosure when the computer program is executed by the processor 501. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0116] In one embodiment, the computer program may be stored on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may be transmitted and distributed in the form of a signal on a network medium, downloaded and installed via the communication portion 509, and / or installed from a removable medium 511. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to wireless, wired, or any suitable combination thereof.

[0117] In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 509, and / or installed from a removable medium 511. When the computer program is executed by the processor 501, the above-described functions defined in the system of the embodiment of the present disclosure are performed. According to the embodiment of the present disclosure, the systems, devices, means, modules, units, etc. described above can be implemented by computer program modules.

[0118] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computer programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C" or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect via the Internet).

[0119] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the above-mentioned module, program segment, or a part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0120] Those skilled in the art will appreciate that the features described in the various embodiments and / or claims of this disclosure may be combined and / or coupled in various ways, even if such combinations and / or couplings are not explicitly described in this disclosure. In particular, the features described in the various embodiments and / or claims of this disclosure may be combined and / or coupled in various ways without departing from the spirit and teachings of this disclosure. All such combinations and / or couplings are intended to fall within the scope of this disclosure.

[0121] The embodiments of the present disclosure are described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although each embodiment has been described separately above, this does not mean that the measures in each embodiment cannot be used in combination to advantage. The scope of the present disclosure is defined by the appended claims and their equivalents. Without departing from the scope of the present disclosure, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A method for object recognition, comprising: Using the first model to identify the CT image of the tissue to be tested, and obtaining characteristic data of each of a plurality of target areas in the tissue to be tested; as well as Using the second model, performing object recognition on the multiple target areas based on the element density data of the tissue to be tested and the characteristic data of each of the multiple target areas to obtain an object recognition result; The performing object recognition on the multiple target areas based on the element density data of the tissue to be tested and the characteristic data of each of the multiple target areas to obtain the object recognition results includes: determining the tissue type of each of the plurality of target areas based on the shape information and position information represented by the characteristic data; Based on the tissue type, setting a target weight value for density data of each of a plurality of elements included in the element density data; determining a plurality of target boundaries formed by a plurality of voxels of each of the plurality of target regions based on the respective density data and the respective target weight values ​​of the plurality of elements; and An object recognition result is determined based on the plurality of object boundaries.

2. The method according to claim 1, wherein The identification of the CT image of the tissue to be tested to obtain characteristic data of each of the multiple target areas in the tissue to be tested includes: Acquiring a first CT image and a second CT image included in the CT image, wherein the first CT image and the second CT image are obtained by scanning rays of different energies of the tissue to be tested, the first CT image including first CT values ​​of a plurality of voxels of the tissue to be tested, and the second CT image including second CT values ​​of the plurality of voxels; determining a plurality of initial boundaries formed by the plurality of voxels based on respective first CT values ​​and respective second CT values ​​of the plurality of voxels; and The multiple target areas formed by the multiple initial boundaries are determined, and feature data of each of the multiple target areas is obtained.

3. The method according to claim 2, wherein: The determining the multiple target areas formed by the multiple initial boundaries and obtaining feature data of each of the multiple target areas includes: determining a plurality of target areas formed by the plurality of initial boundaries in the tissue to be tested; and The shape information of each of the multiple target areas and the position information of the multiple target areas in the tissue to be measured are identified to obtain the characteristic data.

4. The method according to claim 2, wherein: The determining of a plurality of initial boundaries formed by the plurality of voxels based on the respective first CT values ​​and the respective second CT values ​​of the plurality of voxels comprises: When it is determined that at least one of a difference between two first CT values ​​and a difference between two second CT values ​​of two adjacent voxels satisfies a preset condition, determining the two voxels as boundary voxels; and The plurality of initial boundaries are determined based on the position information of each of the plurality of boundary voxels.

5. The method according to claim 1, wherein The determining, based on the density data and target weight values ​​of the respective elements, a plurality of target boundaries formed by the plurality of voxels of the respective target regions, comprises: determining respective distribution information of a plurality of elements in the plurality of target regions based on the element density data of the respective voxels in the plurality of target regions; Calculating an element distribution difference between two adjacent voxels in the plurality of target areas using the distribution information and target weight values ​​of the respective elements; and Based on the element distribution differences, the plurality of target boundaries are determined.

6. The method according to claim 1, wherein The step of setting a weight value for density data of each of a plurality of elements included in the element density data based on the tissue type includes: setting initial weight values ​​for each of the multiple elements for the multiple target areas based on the respective tissue types of the multiple target areas; The initial weight value is adjusted based on the tissue type of the adjacent target areas of each of the multiple target areas to obtain the target weight value.

7. An object recognition device comprising: a first recognition module, configured to recognize the CT image of the tissue to be tested using the first model, and obtain characteristic data of each of a plurality of target areas in the tissue to be tested; as well as a second recognition module, configured to use a second model to perform object recognition on the multiple target areas based on the element density data of the tissue to be tested and the characteristic data of each of the multiple target areas, to obtain an object recognition result; The performing object recognition on the multiple target areas based on the element density data of the tissue to be tested and the characteristic data of each of the multiple target areas to obtain the object recognition results includes: determining the tissue type of each of the plurality of target areas based on the shape information and position information represented by the characteristic data; Based on the tissue type, setting a target weight value for density data of each of a plurality of elements included in the element density data; determining a plurality of target boundaries formed by a plurality of voxels of each of the plurality of target regions based on the respective density data and the respective target weight values ​​of the plurality of elements; and An object recognition result is determined based on the plurality of object boundaries.

8. An electronic device comprising: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors are enabled to execute the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to perform the method according to any one of claims 1 to 6.

Images