Ultrasonic image processing method, system and equipment, medium and equipment
By acquiring and preprocessing ultrasound images, white light original images, fluorescence original images and depth images, forming a fusion image and performing three-dimensional conversion, the problem of unclear existing ultrasound image data is solved, and clearer tissue edge recognition and better three-dimensional model reconstruction is achieved.
Patent Information
- Application Number
- CN202411972949.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-30
- Publication Date
- 2025-05-16
AI Technical Summary
The existing ultrasound imaging data is unclear, it is impossible to clearly distinguish the edges of the organization, it is difficult to reconstruct three-dimensional model, and it cannot meet the needs of staff.
Preprocessing is performed to obtain the fused image, including depth information, and three-dimensional conversion to form a three-dimensional image by obtaining the target ultrasound image, white light original image, fluorescence original image, and depth image.
It improves the clarity of ultrasound images, can distinguish tissue edges more clearly, enhances the ability to reconstruct three-dimensional model, and meets the needs of staff.
Smart Images
Figure CN120013866A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of image processing, and in particular to a method, system, device, medium, and apparatus for processing ultrasonic images. Background Art
[0002] In recent years, artificial intelligence technology has developed rapidly, and the research on key technologies for medical auxiliary diagnosis is of great significance. Ultrasound imaging is a medical imaging technology that uses high-frequency sound waves to generate real-time images of the internal structure of the human body. It is usually used to examine fetal development, diagnose abdominal organ lesions, and evaluate heart function.
[0003] When ultrasound waves propagate through human tissues, they produce echoes, which are captured by receivers and converted into images, showing the morphology, position, and movement of different tissues. This technology is safe and radiation-free, so it is widely used in many medical fields such as obstetrics and gynecology, cardiology, and hepatology.
[0004] However, the existing ultrasound imaging data still has the disadvantage of being unclear and unable to clearly distinguish tissue edges, so it is difficult to reconstruct three-dimensional models based on two-dimensional images, which makes it difficult to meet the needs of staff. Summary of the invention
[0005] The technical problem to be solved by the present disclosure is to overcome the defect of unclear ultrasound image data in the prior art and to provide a method, system, device, medium and equipment for processing ultrasound image pictures.
[0006] The present invention solves the above technical problems through the following technical solutions:
[0007] According to a first aspect of the present disclosure, a method for processing an ultrasound image is provided, the processing method comprising:
[0008] Acquire targeted ultrasound images of patient tissue;
[0009] Based on the target ultrasound image, a target white light original image, a target fluorescence original image and a target depth image corresponding to the patient tissue are obtained; wherein the viewing angles and parameters of the target white light original image, the target fluorescence original image and the target depth image are consistent;
[0010] Preprocessing the patient tissue based on the target white light original image, the target fluorescence original image and the target depth image to obtain a fused image corresponding to the patient tissue; wherein the fused image includes depth information;
[0011] Based on the fused image, the target ultrasound image is three-dimensionally converted to form a target three-dimensional image of the patient tissue.
[0012] Preferably, the step of acquiring a target white light original image, a target fluorescence original image and a target depth image corresponding to the patient tissue based on the target ultrasound image specifically includes:
[0013] Performing slicing processing on the target ultrasound image to obtain a number of image slice data;
[0014] Determining a patient type corresponding to the patient tissue based on the image slice data;
[0015] Based on a preset shooting tool, the patient type is photographed to obtain a target white light original image, a target fluorescence original image and a target depth image corresponding to the patient tissue.
[0016] Preferably, the step of preprocessing the patient tissue based on the target white light original image, the target fluorescence original image and the target depth image to obtain a fused image corresponding to the patient tissue specifically includes:
[0017] Calculating and fitting the target white light original image and the target depth image to obtain a depth coefficient of any pixel point in the target white light original image;
[0018] Performing normalization calculation processing on the target white light original image and the target fluorescence original image to obtain image normalization processing data;
[0019] A fitting calculation is performed based on the depth coefficient and the normalized processed data to obtain the fused image corresponding to the patient tissue.
[0020] Preferably, the step of performing calculation fitting on the target white light original image and the target depth image to obtain the depth coefficient of any pixel point in the target white light original image specifically includes:
[0021] Adjusting and matching the target white light original image and the target depth image so that pixel information of the target white light original image is consistent with that of the target depth image;
[0022] Based on the target depth image, obtaining the depth value of any pixel in the white light original image;
[0023] A numerical normalization calculation is performed on any of the depth values to obtain a depth coefficient of any pixel point in the target white light original image.
[0024] Preferably, before the step of performing normalization calculation processing on the target white light original image and the target fluorescence original image to obtain image normalization processing data, the processing method further includes:
[0025] Calculating the displacement of the target white light original image and the target fluorescence original image to obtain the displacement between the target white light original image and the target fluorescence original image;
[0026] The target fluorescence original image is processed by calculation to obtain the target region size; the target region size is characterized by the area of the region with a fluorescence intensity exceeding a preset threshold in the target fluorescence original image;
[0027] In response to the displacement amount and the target area size being both greater than a first preset value, stopping the normalization calculation processing on the target white light original image and the target fluorescence original image;
[0028] In response to the displacement amount and / or the target area size being smaller than the first preset value, a normalization calculation process is performed on the target white light original image and the target fluorescence original image.
[0029] Preferably, the step of performing normalization calculation processing on the target white light original image and the target fluorescence original image to obtain image normalization processing data specifically includes:
[0030] Adjusting and matching the target white light original image and the target fluorescence original image so that pixel information of the target white light original image is consistent with that of the target fluorescence original image;
[0031] Binarizing the target fluorescence original image to obtain a fluorescence grayscale image;
[0032] Extracting regions in the fluorescence grayscale image where the grayscale value is greater than a second preset value to obtain a fluorescence region grayscale image;
[0033] The grayscale image of the fluorescent area is superimposed on the original white light image to obtain the image normalization processing data.
[0034] According to a second aspect of the present disclosure, a system for processing ultrasound images is provided, the processing system comprising:
[0035] A first acquisition module, used to acquire a target ultrasound image of a patient's tissue;
[0036] A second acquisition module is used to acquire a target white light original image, a target fluorescence original image and a target depth image corresponding to the patient tissue based on the target ultrasound image; wherein the viewing angles and parameters of the target white light original image, the target fluorescence original image and the target depth image are consistent;
[0037] a processing module, configured to pre-process the patient tissue based on the target white light original image, the target fluorescence original image and the target depth image, so as to obtain a fused image corresponding to the patient tissue; wherein the fused image includes depth information;
[0038] The target image acquisition module is used to perform three-dimensional conversion on the target ultrasound image based on the fused image to form a target three-dimensional image of the patient tissue.
[0039] Preferably, the ultrasonic image processing system further comprises a slicing module, and the slicing module is used to: perform slicing processing on the target ultrasonic image to obtain a number of image slice data;
[0040] Determining a patient type corresponding to the patient tissue based on the image slice data;
[0041] Based on a preset shooting tool, the patient type is photographed to obtain a target white light original image, a target fluorescence original image and a target depth image corresponding to the patient tissue.
[0042] Preferably, the processing module is further used to calculate and fit the target white light original image and the target depth image to obtain the depth coefficient of any pixel point in the target white light original image;
[0043] Performing normalization calculation processing on the target white light original image and the target fluorescence original image to obtain image normalization processing data;
[0044] A fitting calculation is performed based on the depth coefficient and the normalized processed data to obtain the fused image corresponding to the patient tissue.
[0045] Preferably, the processing module is further used to adjust and match the target white light original image and the target depth image so that pixel information of the target white light original image is consistent with that of the target depth image;
[0046] Based on the target depth image, obtaining the depth value of any pixel in the white light original image;
[0047] A numerical normalization calculation is performed on any of the depth values to obtain a depth coefficient of any pixel point in the target white light original image.
[0048] Preferably, the ultrasonic image processing system further comprises a judgment module, wherein the judgment module is used to calculate the displacement of the target white light original image and the target fluorescence original image before performing normalization calculation processing on the target white light original image and the target fluorescence original image to obtain the image normalization processing data, so as to obtain the displacement between the target white light original image and the target fluorescence original image;
[0049] The target fluorescence original image is processed by calculation to obtain the target region size; the target region size is characterized by the area of the region with a fluorescence intensity exceeding a preset threshold in the target fluorescence original image;
[0050] In response to the displacement amount and the target area size being both greater than a first preset value, stopping the normalization calculation processing on the target white light original image and the target fluorescence original image;
[0051] In response to the displacement amount and / or the target area size being smaller than the first preset value, a normalization calculation process is performed on the target white light original image and the target fluorescence original image.
[0052] Preferably, the processing module is further used to adjust and match the target white light original image and the target fluorescence original image so that pixel information of the target white light original image is consistent with that of the target fluorescence original image;
[0053] Binarizing the target fluorescence original image to obtain a fluorescence grayscale image;
[0054] Extracting regions in the fluorescence grayscale image where the grayscale value is greater than a second preset value to obtain a fluorescence region grayscale image;
[0055] The grayscale image of the fluorescent area is superimposed on the original white light image to obtain the image normalization processing data.
[0056] According to a third aspect of the present disclosure, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and used to run on the processor, wherein when the processor executes the computer program, the method for processing ultrasound images described in the first aspect of the present disclosure is implemented.
[0057] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method for processing ultrasound images described in the first aspect of the present disclosure is implemented.
[0058] According to a fifth aspect of the present disclosure, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the method for processing ultrasound images as described in the first aspect of the present disclosure is implemented.
[0059] On the basis of being in accordance with the common sense in the art, the above-mentioned preferred conditions can be arbitrarily combined to obtain the preferred embodiments of the present disclosure.
[0060] The positive and progressive effects of this disclosure are:
[0061] In the processing method of ultrasound images provided in the present disclosure, by comparing through camera shooting, fluorescence camera shooting and ultrasound imaging, when acquiring white light original image, fluorescence original image and depth image, the field of view angle and pixel size are made consistent as much as possible, the depth coefficient is calculated using the white light original image and depth image of the shooting area, and the grayscale value of the pixel points of the grayscale image of the fluorescence area is corrected by the depth coefficient, and the grayscale value is adjusted by the depth change so that the brightness change in the grayscale image can be more consistent with the tissue state of the shooting area, so as to obtain a fused image, and the edge information in the fused image is converted into points, lines or surfaces in three-dimensional space, and the color information of the fused image is mapped to the generated three-dimensional model to display a more realistic state. BRIEF DESCRIPTION OF THE DRAWINGS
[0062] Figure 1 This is a flow chart of the method for processing ultrasound images provided in Embodiment 1 of the present disclosure;
[0063] Figure 2 This is a schematic diagram of a process for obtaining various images of corresponding patient tissues provided in Embodiment 1 of the present disclosure;
[0064] Figure 3 This is a schematic diagram of a process for obtaining a fused image of a corresponding patient tissue provided in Embodiment 1 of the present disclosure;
[0065] Figure 4 This is a schematic diagram of a process for obtaining a depth coefficient provided in Embodiment 1 of the present disclosure;
[0066] Figure 5 This is a schematic diagram of a process for obtaining image normalization processing data provided in Embodiment 1 of the present disclosure;
[0067] Figure 6 This is a schematic diagram of the structure of the ultrasonic image processing system provided in Embodiment 2 of the present disclosure;
[0068] Figure 7 This is a schematic diagram of the structure of an electronic device provided in Embodiment 3 of the present disclosure. DETAILED DESCRIPTION
[0069] The present disclosure is further described below by way of examples, but the present disclosure is not limited to the scope of the examples.
[0070] Prefixes such as "first" and "second" are used in the embodiments of the present disclosure only to distinguish different description objects, and have no limiting effect on the position, order, priority, quantity or content of the described objects. The use of prefixes such as ordinal numbers to distinguish description objects in the embodiments of the present disclosure does not constitute a limitation on the described objects. For the statement of the described objects, please refer to the description in the context of the embodiments, and no unnecessary limitation should be constituted due to the use of such prefixes. In addition, in the description of the present embodiment, unless otherwise specified, the meaning of "plurality" is two or more.
[0071] In the embodiments of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.
[0072] Example 1
[0073] like Figure 1 As shown, this embodiment provides a method for processing an ultrasound image, and the processing method includes:
[0074] S11: Acquire a target ultrasound image of the patient's tissue;
[0075] S12: based on the target ultrasound image, obtaining a target white light original image, a target fluorescence original image, and a target depth image of the corresponding patient tissue;
[0076] Among them, the viewing angles and parameters of the target white light original image, the target fluorescence original image, and the target depth image are consistent;
[0077] S13: preprocessing the patient tissue based on the target white light original image, the target fluorescence original image, and the target depth image to obtain a fused image corresponding to the patient tissue; wherein the fused image includes depth information;
[0078] S14: Based on the fused image, perform three-dimensional conversion on the target ultrasound image to form a target three-dimensional image of the patient's tissue.
[0079] In a specific implementation, the image generally consists of white light original, fluorescent original and depth. White light is a composite light, which is a mixture of lights of multiple wavelengths. The white light original image is an image obtained using white light as the illumination light source, which reflects the natural shape and color of the object being photographed.
[0080] Fluorescence is the cold light emitted by a substance or molecule when irradiated with light of a specific wavelength. A fluorescence raw image is an image obtained by stimulating fluorescence using a fluorescent substance or light of a specific wavelength. It is usually used to detect or analyze the structure, composition or function of a substance.
[0081] A depth image is an image that uses the distance (depth) from the image collector to each point in the scene as the pixel value. It directly reflects the geometric shape of the visible surface of the scene. Unlike ordinary RGB images, each pixel value of the depth image represents the actual distance from the sensor to the object.
[0082] White light raw images and fluorescent raw images are similar in imaging principle, both of which use optical system to image the object on the image sensor. However, their lighting methods are different. White light raw images use white light as the illumination source, while fluorescent raw images use light of a specific wavelength to excite fluorescent substances. The imaging principle of depth image is different from the first two. It mainly obtains image information by measuring the distance from the sensor to each point in the scene.
[0083] White light images provide morphological information of tissues, fluorescence images reveal biological processes, and depth images provide the three-dimensional structure of tissues. The integration of multimodal information helps users identify and evaluate lesions more accurately.
[0084] In this embodiment, by comparing through camera shooting, fluorescence camera shooting and ultrasonic imaging, when acquiring the white light original image, the fluorescence original image and the depth image, the field of view angle and the pixel size are made consistent as much as possible, the depth coefficient is calculated using the white light original image and the depth image of the shooting area, and the grayscale value of the pixel points of the grayscale image of the fluorescence area is corrected by the depth coefficient, and the grayscale value is adjusted by the depth change so that the brightness change in the grayscale image can be more in line with the tissue state of the shooting area, so as to obtain a fused image, and the edge information in the fused image is converted into points, lines or surfaces in three-dimensional space, and the color information of the fused image is mapped to the generated three-dimensional model to display a more realistic state.
[0085] like Figure 2 As shown, based on the target ultrasound image, the steps of obtaining the target white light original image, the target fluorescence original image and the target depth image of the corresponding patient tissue specifically include:
[0086] S21: Slice the target ultrasound image to obtain a number of image slice data;
[0087] S22: Determine the patient type corresponding to the patient tissue based on the image slice data;
[0088] S23: Based on a preset shooting tool, shoot the patient type to obtain a target white light original image, a target fluorescence original image and a target depth image of the corresponding patient tissue.
[0089] In a specific implementation, a preset distance is set for continuous historical ultrasound images for slicing to obtain a number of image slice data. After the image slice data are obtained, the correlation of the image slice nodes is analyzed to obtain the correlated slices.
[0090] Through precise image slice processing and determination of patient type, users can obtain more accurate diagnostic information.
[0091] like Figure 3 As shown, based on the target white light original image, the target fluorescence original image and the target depth image, the steps of preprocessing the patient tissue to obtain the fused image of the corresponding patient tissue specifically include:
[0092] S31: Calculate and fit the target white light original image and the target depth image to obtain the depth coefficient of any pixel point in the target white light original image;
[0093] S32: performing normalization calculation processing on the target white light original image and the target fluorescence original image to obtain image normalization processing data;
[0094] S33: Perform fitting calculation based on the depth coefficient and the normalized processing data to obtain a fused image of the corresponding patient tissue.
[0095] Fusion images combine imaging data from different modalities and can provide more comprehensive and accurate diagnostic information. Furthermore, through image fusion technology, the visualization of the lesion area can be enhanced, making the lesion characteristics more obvious, thereby improving the sensitivity and specificity of diagnosis.
[0096] like Figure 4 In the present embodiment, the step of calculating and fitting the target white light original image and the target depth image to obtain the depth coefficient of any pixel point in the target white light original image specifically includes:
[0097] S41: adjusting and matching the target white light original image and the target depth image so that pixel information of the target white light original image is consistent with that of the target depth image;
[0098] S42: Based on the target depth image, obtain the depth value of any pixel in the original white light image;
[0099] S43: performing numerical normalization calculation on any depth value to obtain a depth coefficient of any pixel point in the target white light original image.
[0100] By performing computational fitting on the target white light original image and the target depth image to obtain the depth coefficient of any pixel point in the target white light original image, the measurement accuracy can be improved, the image quality can be enhanced, the environmental interference can be reduced, and the computational burden can be reduced.
[0101] In this embodiment, before the step of performing normalization calculation processing on the target white light original image and the target fluorescence original image to obtain image normalization processing data, the processing method further includes:
[0102] Calculating the displacement of the target white light original image and the target fluorescence original image to obtain the displacement between the target white light original image and the target fluorescence original image;
[0103] The target fluorescence original image is processed by calculation to obtain the target region size; the target region size is characterized by the area of the region with a fluorescence intensity exceeding a preset threshold in the target fluorescence original image;
[0104] In response to the displacement amount and the target area size being both greater than a first preset value, stopping the normalization calculation processing of the target white light original image and the target fluorescence original image;
[0105] In response to the displacement amount and / or the target area size being smaller than a first preset value, a normalization calculation process is performed on the target white light original image and the target fluorescence original image.
[0106] By calculating the displacement of the target white light original image and the target fluorescence original image, the precise displacement between the two images can be obtained. Image registration can ensure the position consistency of the same structure in the image, thereby improving the accuracy of subsequent analysis.
[0107] like Figure 5 As shown, the steps of performing normalization calculation processing on the target white light original image and the target fluorescence original image to obtain image normalization processing data specifically include:
[0108] S51: adjusting and matching the target white light original image and the target fluorescence original image so that pixel information of the target white light original image and the target fluorescence original image are consistent;
[0109] S52: performing binarization processing on the target fluorescence original image to obtain a fluorescence grayscale image;
[0110] S53: extracting the area in the fluorescence grayscale image whose grayscale value is greater than a second preset value to obtain a fluorescence area grayscale image;
[0111] S54: Overlaying the grayscale image of the fluorescent area with the original white light image to obtain image normalization processing data.
[0112] By performing normalization calculation processing on the target white light original image and the target fluorescence original image, the accuracy and efficiency of data processing can be improved.
[0113] The following is a detailed description of the implementation principle of the ultrasonic image processing method disclosed in the present invention with reference to examples:
[0114] The retrieval module is used to retrieve historical ultrasound images, slice continuous historical ultrasound images, obtain a number of image slice data, analyze the correlation of image slice nodes, obtain slices with correlation, obtain white light original images, fluorescence original images and depth images of the correlated slices, calculate and fit the white light original images and the depth images, and obtain the depth coefficient of each pixel in the white light original image; perform normalization calculation processing on the white light original image and the fluorescence original image to obtain image normalization processing data; perform fitting calculation based on the image normalization processing data and the depth coefficient of each pixel in the white light original image to obtain a fused image with depth information, and perform three-dimensional modeling of the image based on the edge information of the fused image.
[0115] Getting a slice with associativity consists of the following steps:
[0116] Determine the type of slice node, which is divided into continuous type and category type. Continuous slice nodes and category slice nodes are not related;
[0117] For any two continuous slicing nodes 1 and 2, draw a scatter plot with the continuous slicing node 1 as the independent variable and the continuous slicing node 2 as the dependent variable;
[0118] The fitting model is determined according to the scatter plot, and the fitting function is calculated according to the fitting model.
[0119] Obtaining the associated slices specifically includes the following steps:
[0120] The sum of the distances from the points in the scatter plot to the fitting function is calculated as a judgment value. If the judgment value is greater than a first preset value, the continuous type slicing node 1 and the continuous type slicing node 2 are not related. If the judgment value does not exceed the first preset value, the continuous type slicing node 1 and the continuous type slicing node 2 are related.
[0121] Among them, the distance from a point in the scatter plot to the fitting function is the minimum value of the distance from a point in the scatter plot to a point on the fitting function;
[0122] For any two category-type slice nodes one and category-type slice nodes two;
[0123] The data in the category-type slice node 1 is classified according to the categories in the category-type slice node 1, the number of data in each category is counted, and the proportion of data in each category is calculated, and the data is arranged from small to large to obtain a1, a2…, a n ;
[0124] The data in the category-type slice node 2 is classified according to the categories in the category-type slice node 2, the number of data in each category is counted, and the proportion of data in each category is calculated, and arranged from small to large to obtain b1, b2…, b n ;
[0125] calculate If A is greater than the second preset value, there is no correlation between category type slice node one and category type slice node two. If A does not exceed the second preset value, there is a correlation between category type slice node one and category type slice node two.
[0126] It can be understood by those skilled in the art that the purpose of performing slice node correlation analysis is to find out the nodes in the image data that play an analytical role. Since the nodes that play a role in the analysis have certain correlations with each other, nodes that have no correlation with the rest of the nodes in the rest of the data will inevitably have little effect on the analysis and can be discarded and not considered.
[0127] The calculation and fitting of the original white light image and the depth image to obtain the depth coefficient of each pixel in the original white light image includes the following steps:
[0128] Adjust and match the white light original image and the depth image so that the white light original image and the depth image have the same field of view and pixel information;
[0129] Obtain the depth value of each pixel in the original white light image;
[0130] The depth value of each pixel in the original white light image is numerically normalized and calculated to obtain the depth coefficient of each pixel in the original white light image.
[0131] The normalization calculation processing of the white light original image and the fluorescence original image to obtain the image normalization processing data specifically includes the following steps:
[0132] Performing displacement calculation processing on the white light original image and the fluorescence original image to obtain the displacement between the white light original image and the fluorescence original image;
[0133] Processing the original fluorescence image, calculating the size of the region having a fluorescence intensity above a specified threshold, and obtaining the region size;
[0134] Determine whether the ratio of the displacement amount to the area size between the white light original image and the fluorescent original image is greater than a first preset value;
[0135] If the ratio of the displacement amount to the area size between the white light original image and the fluorescence original image is greater than a first preset value, then the normalization calculation of the white light original image and the fluorescence original image is stopped;
[0136] If the ratio of the displacement amount between the white light original image and the fluorescence original image to the size of the region is smaller than a first preset value, a normalization calculation of the white light original image and the fluorescence original image is performed.
[0137] The normalization calculation specifically includes the following steps:
[0138] Adjust and match the white light original image and the fluorescence original image so that the white light original image and the fluorescence original image have the same field of view and pixel information;
[0139] Binarize the original fluorescence image to obtain a fluorescence grayscale image;
[0140] Extracting regions in the fluorescence grayscale image where the grayscale value is greater than a second preset value to obtain a fluorescence region grayscale image;
[0141] The grayscale image of the fluorescence area is superimposed with the original white light image to obtain a normalized fused image.
[0142] A fitting calculation is performed based on the image normalization processing data and the depth coefficient of each pixel in the original white light image to obtain a fused image with depth information, which specifically includes the following steps:
[0143] Extract the depth coefficient of each pixel in the fluorescence area of the normalized fusion image;
[0144] According to the depth coefficient of each pixel in the fluorescent area, the grayscale value of the pixel in the grayscale image of the fluorescent area is corrected according to the grayscale correction formula to obtain a corrected grayscale image of the fluorescent area;
[0145] The corrected grayscale image of the fluorescence area is superimposed with the original white light image to form a fused image with depth information.
[0146] The grayscale correction formula is:
[0147] G 修正 =G 初始 ·(1+D·δ);
[0148] In the formula, G 修正 is the corrected pixel gray value, G 初始 is the initial gray value of the pixel, D is the depth coefficient, and δ is the correction coefficient.
[0149] The correction coefficient range is: 0.01<δ≤0.05.
[0150] Based on the edge information of the fused image, the three-dimensional modeling of the image specifically includes the following steps:
[0151] Convert edge information in the fused image into points, lines or surfaces in three-dimensional space;
[0152] Use point cloud data or voxelized data to reconstruct surfaces and generate smooth three-dimensional surfaces;
[0153] Map the color information of the image to the generated 3D model.
[0154] When acquiring the white light original image, the fluorescence original image and the depth image, the present embodiment tries to make the field of view angle and the pixel size consistent, uses the white light original image and the depth image of the shooting area to calculate the depth coefficient, and uses the depth coefficient to correct the grayscale value of the pixel points of the grayscale image of the fluorescence area. The grayscale value is adjusted by the depth change so that the brightness change in the grayscale image can better fit the tissue state of the shooting area, thereby obtaining a fused image, converting the edge information in the fused image into points, lines or surfaces in three-dimensional space, and mapping the color information of the fused image to the generated three-dimensional model to display a more realistic state.
[0155] Example 2
[0156] Corresponding to the aforementioned embodiment of the method for processing ultrasound images, the present disclosure also provides an embodiment of a system for processing ultrasound images.
[0157] like Figure 6 As shown, the processing system includes:
[0158] A first acquisition module 100 is used to acquire a target ultrasound image of a patient's tissue;
[0159] The second acquisition module 200 is used to acquire a target white light original image, a target fluorescence original image and a target depth image of the corresponding patient tissue based on the target ultrasound image; wherein the viewing angles and parameters of the target white light original image, the target fluorescence original image and the target depth image are consistent;
[0160] The processing module 300 is used to pre-process the patient tissue based on the target white light original image, the target fluorescence original image and the target depth image to obtain a fused image of the corresponding patient tissue; wherein the fused image includes depth information;
[0161] The target image acquisition module 400 is used to perform three-dimensional conversion on the target ultrasound image based on the fused image to form a target three-dimensional image of the patient's tissue.
[0162] The ultrasound image processing system in this embodiment further includes a slicing module 500, which is used to: perform slicing processing on the target ultrasound image to obtain a number of image slice data;
[0163] Determine the patient type corresponding to the patient tissue based on the image slice data;
[0164] Based on the preset shooting tool, the patient type is photographed to obtain the target white light original image, target fluorescence original image and target depth image of the corresponding patient tissue.
[0165] The processing module 300 in this embodiment is also used to calculate and fit the target white light original image and the target depth image to obtain the depth coefficient of any pixel point in the target white light original image;
[0166] Performing normalization calculation processing on the target white light original image and the target fluorescence original image to obtain image normalization processing data;
[0167] A fitting calculation is performed based on the depth coefficient and the normalized processing data to obtain a fused image of the corresponding patient tissue.
[0168] The processing module 300 in this embodiment is also used to adjust and match the target white light original image and the target depth image so that the pixel information of the target white light original image is consistent with the pixel information of the target depth image;
[0169] Based on the target depth image, obtain the depth value of any pixel in the original white light image;
[0170] Perform numerical normalization calculation on any depth value to obtain the depth coefficient of any pixel point in the target white light original image.
[0171] The ultrasonic image processing system in this embodiment further includes a judgment module 600, which is used to calculate the displacement of the target white light original image and the target fluorescence original image before performing normalization calculation processing on the target white light original image and the target fluorescence original image to obtain the image normalization processing data, so as to obtain the displacement between the target white light original image and the target fluorescence original image;
[0172] The target fluorescence original image is processed by calculation to obtain the target region size; the target region size is characterized by the area of the region with a fluorescence intensity exceeding a preset threshold in the target fluorescence original image;
[0173] In response to the displacement amount and the target area size being both greater than a first preset value, stopping the normalization calculation processing of the target white light original image and the target fluorescence original image;
[0174] In response to the displacement amount and / or the target area size being smaller than a first preset value, a normalization calculation process is performed on the target white light original image and the target fluorescence original image.
[0175] The processing module 300 in this embodiment is also used to adjust and match the target white light original image and the target fluorescence original image so that the pixel information of the target white light original image and the target fluorescence original image are consistent;
[0176] Binarize the original target fluorescence image to obtain a fluorescence grayscale image;
[0177] Extracting regions in the fluorescence grayscale image where the grayscale value is greater than a second preset value to obtain a fluorescence region grayscale image;
[0178] The grayscale image of the fluorescent area is superimposed with the original white light image to obtain the image normalization processing data.
[0179] In the ultrasound image processing system provided in the present embodiment, by comparing camera shooting, fluorescence camera shooting and ultrasound imaging, when acquiring the white light original image, the fluorescence original image and the depth image, the field of view angle and the pixel size are made consistent as much as possible, the depth coefficient is calculated using the white light original image and the depth image of the shooting area, and the grayscale value of the pixel points of the grayscale image of the fluorescence area is corrected by the depth coefficient, and the grayscale value is adjusted by the depth change so that the brightness change in the grayscale image can be more consistent with the tissue state of the shooting area, so as to obtain a fused image, and the edge information in the fused image is converted into points, lines or surfaces in three-dimensional space, and the color information of the fused image is mapped to the generated three-dimensional model, so as to display a more realistic state and improve the user experience.
[0180] As for the system embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The system embodiment described above is only illustrative, in which the units described as separate components may or may not be physically separated, and the components as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the disclosed solution.
[0181] Example 3
[0182] Figure 7 The present invention is a schematic diagram of the structure of an electronic device provided in Embodiment 3 of the present disclosure. The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the method in the above embodiment is implemented when the processor executes the program. Figure 7The electronic device 30 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.
[0183] like Figure 7 As shown, the electronic device 30 may be in the form of a general-purpose computing device, for example, it may be a server device. The components of the electronic device 30 may include, but are not limited to: at least one processor 31, at least one memory 32, and a bus 33 connecting different system components (including the memory 32 and the processor 31).
[0184] The bus 33 includes a data bus, an address bus, and a control bus.
[0185] The memory 32 may include a volatile memory, such as a random access memory (RAM) 321 and / or a cache memory 322 , and may further include a read-only memory (ROM) 323 .
[0186] The memory 32 may also include a program / utility 325 having a set (at least one) of program modules 324, such program modules 324 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.
[0187] The processor 31 executes various functional applications and data processing by running the computer programs stored in the memory 32, such as the method in the above-mentioned embodiment of the present disclosure.
[0188] The electronic device 30 may also communicate with one or more external devices 34 (e.g., keyboards, pointing devices, etc.). Such communication may be performed via an input / output (I / O) interface 35. Furthermore, the model generating device 30 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 36. Figure 7 As shown, the network adapter 36 communicates with other modules of the model-generated device 30 via the bus 33. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with the model-generated device 30, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (disk array) systems, tape drives, and data backup storage systems, etc.
[0189] It should be noted that although several units / modules or sub-units / modules of the electronic device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided into multiple units / modules to be embodied.
[0190] Example 4
[0191] The embodiment of the present disclosure also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the method for processing ultrasound images provided in the above-mentioned embodiment 1 is implemented.
[0192] The readable storage medium may include but is not limited to: a portable disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, an optical storage device, a magnetic storage device or any suitable combination of the above.
[0193] Example 5
[0194] The present disclosure also provides a computer program product, including a computer program, which, when executed by a processor, implements the method for processing ultrasound images provided in the above-mentioned embodiment 1.
[0195] Among them, the program code for executing the computer program product of the present disclosure can be written in any combination of one or more programming languages, and the program code can be executed completely on the user device, partially on the user device, as an independent software package, partially on the user device and partially on a remote device, or completely on the remote device.
[0196] Although the specific embodiments of the present disclosure are described above, those skilled in the art should understand that this is only an example, and the protection scope of the present disclosure is defined by the appended claims. Those skilled in the art may make various changes or modifications to these embodiments without departing from the principles and essence of the present disclosure, but these changes and modifications all fall within the protection scope of the present disclosure.
Claims
1. A method for processing ultrasonic images, characterized in that: The processing method comprises: Acquire targeted ultrasound images of patient tissue; Based on the target ultrasound image, a target white light original image, a target fluorescence original image and a target depth image corresponding to the patient tissue are obtained; wherein the viewing angles and parameters of the target white light original image, the target fluorescence original image and the target depth image are consistent; Preprocessing the patient tissue based on the target white light original image, the target fluorescence original image and the target depth image to obtain a fused image corresponding to the patient tissue; wherein the fused image includes depth information; Based on the fused image, the target ultrasound image is three-dimensionally converted to form a target three-dimensional image of the patient tissue.
2. The method for processing ultrasonic images according to claim 1, characterized in that: The step of acquiring a target white light original image, a target fluorescence original image and a target depth image corresponding to the patient tissue based on the target ultrasound image specifically includes: Performing slicing processing on the target ultrasound image to obtain a number of image slice data; Determining a patient type corresponding to the patient tissue based on the image slice data; Based on a preset shooting tool, the patient type is photographed to obtain a target white light original image, a target fluorescence original image and a target depth image corresponding to the patient tissue.
3. The method for processing ultrasonic images according to claim 1, characterized in that: The step of preprocessing the patient tissue based on the target white light original image, the target fluorescence original image and the target depth image to obtain a fused image corresponding to the patient tissue specifically includes: Calculating and fitting the target white light original image and the target depth image to obtain a depth coefficient of any pixel point in the target white light original image; Performing normalization calculation processing on the target white light original image and the target fluorescence original image to obtain image normalization processing data; A fitting calculation is performed based on the depth coefficient and the normalized processed data to obtain the fused image corresponding to the patient tissue.
4. The method for processing ultrasonic images according to claim 3, characterized in that: The step of calculating and fitting the target white light original image and the target depth image to obtain the depth coefficient of any pixel point in the target white light original image specifically includes: Adjusting and matching the target white light original image and the target depth image so that pixel information of the target white light original image is consistent with that of the target depth image; Based on the target depth image, obtaining the depth value of any pixel in the white light original image; A numerical normalization calculation is performed on any of the depth values to obtain a depth coefficient of any pixel point in the target white light original image.
5. The method for processing ultrasonic images according to claim 4, characterized in that: Before the step of performing normalization calculation processing on the target white light original image and the target fluorescence original image to obtain image normalization processing data, the processing method further includes: Calculating the displacement of the target white light original image and the target fluorescence original image to obtain the displacement between the target white light original image and the target fluorescence original image; The target fluorescence original image is processed by calculation to obtain the target region size; the target region size is characterized by the area of the region with a fluorescence intensity exceeding a preset threshold in the target fluorescence original image; In response to the displacement amount and the target area size being both greater than a first preset value, stopping the normalization calculation processing on the target white light original image and the target fluorescence original image; In response to the displacement amount and / or the target area size being smaller than the first preset value, a normalization calculation process is performed on the target white light original image and the target fluorescence original image.
6. The method for processing ultrasonic images according to claim 5, characterized in that: The step of performing normalization calculation processing on the target white light original image and the target fluorescence original image to obtain image normalization processing data specifically includes: Adjusting and matching the target white light original image and the target fluorescence original image so that pixel information of the target white light original image is consistent with that of the target fluorescence original image; Binarizing the target fluorescence original image to obtain a fluorescence grayscale image; Extracting regions in the fluorescence grayscale image where the grayscale value is greater than a second preset value to obtain a fluorescence region grayscale image; The grayscale image of the fluorescent area is superimposed on the original white light image to obtain the image normalization processing data.
7. A system for processing ultrasonic images, characterized in that: The processing system comprises: A first acquisition module, used to acquire a target ultrasound image of a patient's tissue; A second acquisition module is used to acquire a target white light original image, a target fluorescence original image and a target depth image corresponding to the patient tissue based on the target ultrasound image; wherein the viewing angles and parameters of the target white light original image, the target fluorescence original image and the target depth image are consistent; a processing module, configured to pre-process the patient tissue based on the target white light original image, the target fluorescence original image and the target depth image, so as to obtain a fused image corresponding to the patient tissue; wherein the fused image includes depth information; The target image acquisition module is used to perform three-dimensional conversion on the target ultrasound image based on the fused image to form a target three-dimensional image of the patient tissue.
8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and used to run on the processor, characterized in that: When the processor executes the computer program, the method for processing ultrasound images according to any one of claims 1 to 6 is implemented.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for processing ultrasound images according to any one of claims 1 to 6 is implemented.
10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method for processing ultrasound images according to any one of claims 1 to 6 is implemented.