Ultrasound imaging methods, devices, electronic equipment, storage media, and software products

Abstract

This invention provides an ultrasound imaging method, apparatus, electronic device, storage medium, and program product. The method includes: acquiring region of interest (ROI) data and tissue data of a target tissue, wherein the ROI data and tissue data are acquired using different ultrasound imaging methods; rendering the tissue data to obtain a first rendered image; rendering the tissue data and the ROI data to obtain a second rendered image; and fusing the first and second rendered images to obtain a fused image. This fused image better presents the relative spatial position of the ROI within the entire tissue structure. Furthermore, the fused image has a stronger sense of depth, higher smoothness and realism, significantly improving the user experience.

Description

Technical Field

[0001] This invention relates to the field of medical devices, and more specifically, to an ultrasound imaging method, an ultrasound imaging device, an electronic device, a storage medium, and a computer program product. Background Technology

[0002] In the field of medical technology, ultrasound imaging technology has been widely used in medical testing due to its advantages such as high reliability, real-time imaging, and repeatable detection.

[0003] Ultrasound imaging technology is a technique for acquiring, reconstructing, and displaying image data of a target tissue to generate an ultrasound image. Conventional ultrasound imaging techniques, such as B-mode ultrasound, can obtain ultrasound images of the target tissue. However, these ultrasound images can only show the structural information of the target tissue and are difficult to clearly show some local areas of interest within the target tissue, such as blood vessels.

[0004] Contrast-enhanced ultrasound (CEUS) is a technique that uses contrast agents to enhance the contrast of ultrasound images. Specifically, contrast agents are injected into the target tissue to improve the visualization of blood vessels or other tissues, thereby increasing the resolution, sensitivity, and specificity of the tissue in the ultrasound image. For example, three-dimensional contrast-enhanced ultrasound can display the blood perfusion status of tumor tissue in three-dimensional space, showing the origin, number, spatial distribution, and blood perfusion process of the nourishing vessels of the lesion, helping to clarify the tumor boundary and its relationship with surrounding tissues. Furthermore, three-dimensional contrast-enhanced ultrasound is also a preferred imaging method in gynecological examinations. For example, hysterosalpingography (HSG) can be used to determine fallopian tube patency and observe lesions of the uterus, ovaries, and pelvic cavity, providing a comprehensive assessment of the female reproductive system. While contrast-enhanced ultrasound images can clearly show regions of interest within the target tissue, such as blood vessels and fallopian tubes, they only show information about that region of interest and cannot reveal its relative spatial position to the surrounding tissue.

[0005] Some related techniques fuse three-dimensional ultrasound contrast images and ultrasound tissue images to obtain a fused image, thereby determining the relative spatial location of the region of interest within the target tissue. However, the fused images obtained by these techniques have poor detail and realism. Summary of the Invention

[0006] The present invention was proposed in view of the above-mentioned problems.

[0007] According to one aspect of the present invention, an ultrasound imaging method is provided. The ultrasound imaging method includes:

[0008] Acquire local data of interest and tissue data of the target tissue, wherein the local data of interest and tissue data are acquired using different ultrasound imaging methods;

[0009] The organizational data is rendered into an image to obtain a first rendered image;

[0010] Image rendering is performed on the organizational data and the local data of interest to obtain a second rendered image; and

[0011] The first and second rendered images are merged to obtain a merged image.

[0012] For example, acquiring local data of interest and tissue data of a target tissue includes: acquiring local data of interest and tissue data at different times; performing image rendering on the tissue data to obtain a first rendered image, including: performing image rendering on tissue data at some times in different times to obtain at least one first rendered image; performing image rendering on the tissue data and local data of interest to obtain a second rendered image, including: performing image rendering on the tissue data and local data of interest at each time to obtain multiple second rendered images; and fusing the first rendered image and the second rendered image to obtain a fused image, including: fusing the second rendered image at each time with the corresponding first rendered image to obtain a fused image at each time.

[0013] For example, the ultrasound imaging method further includes: obtaining a first weight; rendering the tissue data and the local data of interest to obtain a second rendered image, including: performing mixed rendering of the tissue data and the local data of interest according to the first weight to obtain the second rendered image, wherein the first weight is used to determine the influence of the tissue data and the local data of interest on the second rendered image respectively, the larger the first weight, the greater the influence of the local data of interest on the second rendered image, and the smaller the influence of the tissue data on the second rendered image.

[0014] For example, the ultrasound imaging method further includes: displaying the fused image corresponding to the current first weight; adjusting the first weight and updating the fused image in response to the weight adjustment operation.

[0015] For example, the weight adjustment operation is used to gradually increase or decrease the first weight within the adjustable range of the first weight; the ultrasound imaging method further includes: during the process of updating the fused image, determining the relative spatial depth information of the local area of ​​interest relative to the reference spatial area based on the display change state of the local area of ​​interest corresponding to the local area of ​​interest data; wherein, the display change state includes a state that changes between a clear state and a disappearing state, and the reference spatial area includes at least one of the following: the tissue spatial area corresponding to the rendering light incident point and the tissue data.

[0016] For example, the weight adjustment operation is used to gradually reduce the first weight; the reference spatial region is the organizational spatial region; based on the display change state of the local spatial region corresponding to the local data of interest, the relative spatial depth information of the local spatial region of interest relative to the reference spatial region is determined, including: if the display state of the local spatial region of interest changes from a clear state to a disappearing state, then the relative spatial depth information is determined to be that the spatial depth of the local spatial region of interest is greater than or equal to the spatial depth of the organizational spatial region; if the display state of the local spatial region of interest remains unchanged, then the relative spatial depth information is determined to be that the spatial depth of the local spatial region of interest is less than the spatial depth of the organizational spatial region.

[0017] An exemplary method for blending tissue data and locality of interest (LOI) data to obtain a second rendered image, based on a first weight, includes: for each ray path, obtaining the tissue color value and tissue opacity value of each sampling point on the ray path based on the tissue data, and obtaining the LIO color value and LIO opacity value of each sampling point on the ray path based on the LIO data; determining the blended opacity value of each sampling point based on the first weight, the tissue opacity value, and the LIO opacity value; determining the blended color value of each sampling point based on the first weight, the tissue color value, the tissue opacity value, the LIO color value, and the LIO opacity value; determining the cumulative blended opacity value of each ray path based on the blended opacity value of each sampling point on each ray path; determining the cumulative blended color value of each ray path based on the blended color value and the blended opacity value of each sampling point on each ray path; and determining the color value of each pixel in the second rendered image based on the cumulative blended color value and the cumulative blended opacity value of each ray path to obtain the second rendered image.

[0018] For example, fusing a first rendered image and a second rendered image to obtain a fused image includes: for each ray path, obtaining a cumulative local opacity value on the ray path based on local interest data; and determining the color value of each pixel in the fused image based on the color value of each pixel in the first rendered image, the color value of each pixel in the second rendered image, the cumulative blending opacity value, and the cumulative local opacity value to obtain the fused image.

[0019] For example, the ultrasound imaging method further includes: selecting tissue data to obtain tissue data of a local region corresponding to the target tissue; and rendering the tissue data and the local data of interest to obtain a second rendered image, including: determining the corresponding local data of interest based on the selected tissue data, and rendering the selected tissue data and the corresponding local data of interest.

[0020] For example, selecting tissue data includes: selecting tissue data of multiple local regions corresponding to a target tissue from the tissue data; performing image rendering on the selected tissue data and the corresponding local data of interest includes: for the tissue data of each local region, performing image rendering on the tissue data of the local region and the corresponding local data of interest according to the first weight corresponding to the local region, wherein the first weights corresponding to different local regions are different.

[0021] For example, fusing a first rendered image and a second rendered image to obtain a fused image includes: determining a fusion region in the second rendered image; and fusing the second rendered image within the fusion region into the first rendered image to obtain a fused image.

[0022] For example, in the second rendered image, determining the blending region includes: automatically determining the blending region based on the pixel features of the second rendered image.

[0023] According to another aspect of the present invention, an ultrasound imaging apparatus is also provided. The ultrasound imaging apparatus includes an acquisition module, a first rendering module, a second rendering module, and a fusion module. The acquisition module is used to acquire locality of interest (ROI) data and tissue data of a target tissue, wherein the ROI data and tissue data are acquired using different ultrasound imaging methods; the first rendering module is used to render the tissue data to obtain a first rendered image; the second rendering module is used to render the tissue data and the ROI data to obtain a second rendered image; and the fusion module is used to fuse the first rendered image and the second rendered image to obtain a fused image.

[0024] According to another aspect of the present invention, an electronic device is also provided, comprising: a processor and a memory, wherein the memory stores computer program instructions, which are executed by the processor to perform the above-described ultrasound imaging method.

[0025] According to another aspect of the present invention, a storage medium is also provided, on which program instructions are stored, which, when executed, are used to perform the above-described ultrasound imaging method.

[0026] According to another aspect of the present invention, a computer program product is also provided, including computer program instructions that, when executed, perform the ultrasound imaging method described above.

[0027] The above technical solution obtains a first rendered image by rendering the tissue data separately, a second rendered image by rendering the tissue data and the local data of interest together, and then merges the first and second rendered images to obtain a fused image. This fused image better presents the relative spatial position of the local area of ​​interest of the target tissue within the entire tissue structure. Furthermore, this fused image has a stronger sense of layering, higher smoothness and realism, significantly improving the user experience.

[0028] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0029] The above and other objects, features, and advantages of the present invention will become more apparent from the more detailed description of the embodiments of the invention in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.

[0030] Figure 1 A schematic flowchart of an ultrasound imaging method according to an embodiment of the present invention is shown;

[0031] Figure 2 A schematic diagram of a first rendered image according to an embodiment of the present invention is shown;

[0032] Figure 3A A schematic diagram of a second rendered image according to an embodiment of the present invention is shown;

[0033] Figure 3B A schematic diagram illustrating the cumulative opacity values ​​of a region of interest according to an embodiment of the present invention is shown.

[0034] Figure 4A A schematic diagram of a fused image according to an embodiment of the present invention is shown;

[0035] Figure 4B A schematic diagram of a fused image according to another embodiment of the present invention is shown;

[0036] Figure 4C A schematic diagram of a fused image according to another embodiment of the present invention is shown;

[0037] Figure 5 A schematic block diagram of an ultrasound imaging apparatus according to an embodiment of the present invention is shown;

[0038] Figure 6 A schematic block diagram of an electronic device according to an embodiment of the present invention is shown. Detailed Implementation

[0039] To make the objectives, technical solutions, and advantages of the present invention more apparent, exemplary embodiments according to the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely a subset of embodiments of the present invention.

[0040] To at least partially solve the above-mentioned technical problems, according to one aspect of the present invention, an ultrasound imaging method is provided. Figure 1 A schematic flowchart of an ultrasound imaging method according to an embodiment of the present invention is shown. Figure 1 As shown, the ultrasound imaging method includes steps S110, S120, S130 and S140.

[0041] In step S110, the local data of interest and tissue data of the target tissue are obtained.

[0042] The target tissue can be any body tissue to be examined using ultrasound imaging. Specifically, the target tissue can be a body organ, body part, or diseased tissue. The tissue data of the target tissue can be ultrasound data of the entire target tissue, including its spatial structure information. The region of interest (ROI) data of the target tissue is ultrasound data focused on a specific area within the target tissue, including its spatial structure information. Because the ROI data focuses only on the area of ​​interest and does not include the spatial structure information of other parts of the target tissue, it cannot show the specific spatial location of the ROI within the target tissue.

[0043] The local data of interest (ROI) and tissue data are acquired using different ultrasound imaging methods, thus each type of data possesses its own characteristics as described above. For example, the ultrasound probe can switch operating modes to acquire RIO and tissue data using different ultrasound imaging methods within the same time period. Acquiring both types of data within the same time period ensures spatial alignment, which is beneficial for subsequent image processing. Specifically, the ultrasound imaging methods of the ultrasound probe can be switched at a fixed frequency within the same time period, thereby acquiring RIO and tissue data of the target tissue using different ultrasound imaging methods.

[0044] For example, tissue data can be acquired using conventional ultrasound imaging methods, such as B-mode ultrasound. Data of interest may include blood flow data and / or contrast data. Blood flow data includes blood flow information within blood vessels of the target tissue, which can be obtained using Doppler ultrasound. Contrast data of the target tissue can be obtained using contrast ultrasound techniques. The contrast data may show the spatial distribution of contrast-filled lumens or other chambers in the target tissue, such as blood vessels or fallopian tubes. Specifically, after a contrast agent is injected into the lumen or other chamber of the target tissue, it becomes filled with the contrast agent. An ultrasound probe sends ultrasound waves to the contrast-filled target tissue, and the ultrasound probe receives the ultrasound waves reflected by the contrast agent to obtain contrast data.

[0045] In step S120, the tissue data is rendered to obtain a first rendered image.

[0046] Image rendering refers to the process of converting imaging data into a two-dimensional image. The tissue data obtained in step S110 can be rendered to obtain a first rendered image corresponding to the tissue data. It can be understood that the tissue data can be three-dimensional data, and the first rendered image can be a two-dimensional image. Volume rendering or surface rendering methods can be used to render the tissue data into the first rendered image. Any existing or future surface rendering method can be used to render the tissue data, such as contouring cubes, dual contouring, etc. Alternatively, any existing or future volume rendering method can be used to render the tissue data, such as ray casting, ray tracing, etc. Figure 2 A schematic diagram of a first rendered image according to an embodiment of the present invention is shown. (As shown) Figure 2 As shown, the gray area represents the rendered portion corresponding to the organizational data, while the black area does not correspond to any organizational data. The first rendered image can be a grayscale pseudo-color image, meaning that the RGB values ​​are equal.

[0047] In step S130, the tissue data and the local data of interest are rendered to obtain a second rendered image.

[0048] Tissue data and local data of interest can be combined for image rendering to obtain a second rendered image. Both the tissue data and the local data of interest can be three-dimensional data; however, the local data of interest only reflects the local spatial structure of the target tissue, such as the spatial location of blood vessels inside or around the tumor tissue. In step S130, the tissue data and the local data of interest are mixed and rendered to obtain the second rendered image. The second rendered image is a two-dimensional image. At least some pixels in this second rendered image include information from both the tissue data and the local data of interest. In other words, in the second rendered image, the image portion formed solely based on the tissue data is the background portion, and the image portion formed by the mixed rendering of the tissue data and the local data of interest is the foreground portion.

[0049] Specifically, organizational data and locality of interest (LOI) data can be mixed according to certain rules, such as proportional mixing. During image rendering, the organizational data and LIO data are rendered together proportionally to obtain a second rendered image. The user can set the ratio before image rendering. Alternatively, the ratio of organizational data and LIO data can be preset or calculated based on the organizational data and LIO data. Figure 3A A schematic diagram of a second rendered image according to an embodiment of the present invention is shown. Figure 3A As shown, the gray areas represent the rendered portions corresponding to tissue data and locality of interest (LOOIs), while the black areas do not correspond to any tissue data or LEOIs. The portion of the second rendered image formed solely based on tissue data can be a grayscale pseudo-color image; this portion is gray. The portion of the image formed by rendering a mixture of tissue data and LEOIs is a color image; this portion is colored.

[0050] In step S140, the first rendered image and the second rendered image are merged to obtain a merged image.

[0051] As mentioned earlier, the first rendered image is obtained by rendering the tissue data of the target tissue, and the second rendered image is obtained by blending the tissue data of the target tissue and the local data of interest. Both the first and second rendered images correspond to the same location of the target tissue, and the pixels therein have a spatial correspondence, allowing them to be fused.

[0052] The first and second rendered images can have the same size. Alternatively, they can have different sizes, and their spatial alignment can be used to ensure registration and fusion. The fusion of the first and second rendered images can be achieved by processing each pixel in both images to obtain the fused image. Specifically, corresponding pixels in the first and second images can be processed using methods such as maximum, minimum, and average values ​​to obtain the fused image. Alternatively, frequency fusion, deep learning methods, etc., can also be used to fuse the first and second rendered images.

[0053] The above technical solution obtains a first rendered image by rendering the tissue data separately, a second rendered image by rendering the tissue data and the local data of interest together, and then merges the first and second rendered images to obtain a fused image. This fused image better presents the relative spatial position of the local area of ​​interest of the target tissue within the entire tissue structure. Furthermore, the fused image has a stronger sense of depth, higher smoothness and realism, significantly improving the user experience.

[0054] For example, step S110, acquiring the local area of ​​interest (ROI) data and tissue data of the target tissue, may include: step S110a, acquiring the ROI data and tissue data of the target tissue at different times. Specifically, the ROI data and tissue data can be acquired at different times using an ultrasound probe. The ROI data acquired at one time can be used to obtain a single-frame static two-dimensional image of the ROI through image rendering. Similarly, the tissue data acquired at one time can be used to obtain a single-frame static image of the tissue structure of the target tissue through image rendering. The ROI data acquired at different times can be used to obtain a dynamic video of the ROI by performing image rendering separately. Similarly, the tissue data acquired at different times can be used to obtain a dynamic video of the target tissue by performing image rendering separately.

[0055] Step S120, which involves image rendering of the tissue data to obtain a first rendered image, includes: Step S120a, performing image rendering on tissue data at a subset of different time points to obtain at least one first rendered image. It is understood that the tissue structure of the target tissue does not change significantly over time. For example, the size and shape of tumor tissue do not change in a short period. Therefore, tissue data at different time points within a short period are generally similar. For instance, if the ultrasound probe's detection position remains unchanged at multiple consecutive time points, the target tissue remains essentially unchanged, and thus the tissue data acquired at these consecutive time points are generally the same. An image rendering can be performed on the tissue data acquired at one of these consecutive time points to obtain a first rendered image. This first rendered image can be considered as a rendered image of the tissue data at each of these consecutive time points. After the ultrasound probe's detection position changes, the tissue data changes accordingly, and then image rendering is performed on the changed tissue data to obtain the next first rendered image of the tissue data.

[0056] Step S130 involves rendering the tissue data and the local data of interest to obtain a second rendered image, which includes: step S130a, rendering the tissue data and the local data of interest at each time step to obtain multiple second rendered images.

[0057] The locality of interest (LOOI) data presents the morphology of a locality of interest within the target tissue, which may vary significantly over time. For example, LOCI data may include contrast data and / or blood flow data. It is understood that both contrast data and blood flow data are dynamically changing, differing at each moment. Therefore, the LOCI data and corresponding tissue data at each moment can be combined and rendered to obtain a second rendered image for each moment. These second rendered images can be understood to constitute a dynamic video of the target tissue including the LOCI.

[0058] Step S140, which merges the first rendered image and the second rendered image to obtain a merged image, includes: step S140a, merging the second rendered image at each time step with the corresponding first rendered image to obtain a merged image at each time step.

[0059] As described in step S120a, the tissue data at multiple time points can correspond to a first rendered image. As described in step S130a, each time point corresponds to a second rendered image. Therefore, the second rendered images at multiple time points may correspond to the same first rendered image. The second rendered image at each time point can be fused with the first rendered image corresponding to that time point to obtain a fused image at each time point.

[0060] The above technical solution performs separate image rendering on tissue data acquired at certain time points to obtain corresponding first rendered images. Then, it fuses the first and second rendered images, which have a temporal correspondence, to obtain a fused image for each time point. This solution fully utilizes the characteristic that tissue data at the same location of the target tissue does not change significantly within a short period, rendering images only for tissue data at certain time points while omitting image rendering for tissue data at other time points. This reduces computational load and improves computational speed while supporting video fusion and ensuring the accuracy of each frame of the fused image in the final video.

[0061] Alternatively, step S120, which involves rendering the tissue data to obtain a first rendered image, includes step S120b, rendering the tissue data for each time point at different times to obtain at least one first rendered image. Step S140, which involves fusing the first rendered image and the second rendered image to obtain a fused image, includes step S140b, where, for each time point, the second rendered image and the first rendered image at that time point are fused to obtain the fused image at that time point.

[0062] The above technical solution also supports video fusion. In this solution, image rendering is performed on the tissue data acquired at each moment, ensuring the accuracy of the final fused image.

[0063] For example, the ultrasound imaging method further includes step S150, obtaining a first weight. The adjustable range of the first weight, i.e., the value range, can be 0 to 1. The first weight can be flexibly set according to needs within its adjustable range. The first weight can be set by the user as needed. In some embodiments, the user can set the first weight using input devices such as a mouse, keyboard, trackball, joystick, or touchscreen. Alternatively, the first weight can be automatically set by the ultrasound imaging device; for example, the first weight can have a default value, or the ultrasound imaging device can set the first weight based on past user habits and the target tissue. In some embodiments, different target tissues can correspond to different default first weight values.

[0064] Step S130 performs image rendering on the tissue data and the local data of interest to obtain a second rendered image, which may include step S131: performing mixed rendering on the tissue data and the local data of interest according to a first weight to obtain a second rendered image, wherein the first weight is used to determine the influence of the tissue data and the local data of interest on the second rendered image respectively. The larger the first weight, the greater the influence of the local data of interest on the second rendered image, and the smaller the influence of the tissue data on the second rendered image.

[0065] The first weight influences the proportion of organizational data and locality of interest (LOI) data in the blended rendering. When blending organizational data and LIO data, the proportion of the LIO data can be represented by the first weight, and the proportion of the organizational data can be represented by (1 - the first weight). It can be understood that the larger the proportion of one data point, the greater its influence on the second rendered image. A larger first weight results in a larger proportion of LIO data, and thus a greater influence of LIO data on the second rendered image; conversely, a smaller proportion of organizational data results in a smaller influence of organizational data on the second rendered image. Based on the first weight, the proportions of organizational data and LIO data in the blended rendering can be determined separately, and then the blended rendering can be performed according to these proportions to obtain the second rendered image.

[0066] The above technical solution, based on a first weight, blends and renders the tissue data and the local area of ​​interest (ROI) data in the second rendered image according to the first weight. Therefore, by using an appropriate first weight, different blending effects can be obtained in the second rendered image. This second rendered image can contain both tissue regions and ROI regions, fully representing the spatial relationship between them, thus obtaining a second rendered image that better meets expectations.

[0067] For example, the ultrasound imaging method further includes steps S160 and S170.

[0068] In step S160, the fused image corresponding to the current first weight is displayed.

[0069] A display device can be used to display a fused image, which is the result of fusing the second rendered image corresponding to the current first weight with the first rendered image. Thus, the user can view the fused image through the display device. For example, a first human-computer interaction interface can be provided, in which the fused image can be displayed. Optionally, the first human-computer interaction interface can also display the first weight corresponding to the currently displayed fused image.

[0070] In step S170, in response to the weight adjustment operation, the first weight is adjusted and the fused image is updated.

[0071] The weight adjustment operation can be an adjustment of the first weight performed by the user using a first input device. The first input device can be a mouse, keyboard, trackball, joystick, touch screen, etc. For example, the user can manually adjust the first weight using the first input device. Alternatively, the user can trigger the weight adjustment operation, and the ultrasound imaging device can automatically adjust the first weight, for example, based on the target tissue to which the local data of interest belongs.

[0072] After the user adjusts the first weight, step S131 can be re-executed based on the adjusted first weight. According to the first weight, the organizational data and the local data of interest are mixed and rendered to obtain a second rendered image. The adjusted second rendered image is then merged with the first rendered image to obtain a new merged image, which is then updated and displayed. It can be understood that the organizational data and the local data of interest may remain unchanged; however, due to the change in the first weight, the proportions of the organizational data and the local data of interest during mixed rendering change, thus changing the second rendered image, and consequently, the merged image.

[0073] The above technical solution can display a fused image and support adjustment of the first weight. Therefore, users can adjust the first weight based on the displayed fused image, and then update the fused image according to the adjusted first weight. This allows for flexible adjustment of the first weight based on the presentation effect of local and tissue data of interest in the final fused image, until the most ideal presentation effect is achieved, thus improving the user experience.

[0074] For example, the weight adjustment operation is used to gradually increase or decrease the first weight within the adjustable weight range. As mentioned earlier, the first weight may have a corresponding adjustable range. The magnitude of the first weight should be within the adjustable weight range. For example, a user can sequentially set the first weight to 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, and 0.9 to gradually increase the first weight within the adjustable weight range. In other embodiments, the weight adjustment operation can be performed automatically. For example, a control can be provided that, when triggered by the user, automatically increases or decreases the first weight in steps. The first weight can be gradually increased or decreased based on the current first weight. Alternatively, the first weight can be gradually decreased or increased based on the maximum or minimum value.

[0075] As mentioned earlier, the fused image can be updated progressively in response to weight adjustment operations. Each time the first weight changes, the organizational data and the local data of interest can be mixed and rendered according to the changed first weight to obtain a second rendered image, which is then merged with the first rendered image to obtain the fused image. Each change in the first weight updates the corresponding fused image.

[0076] For example, the ultrasound imaging method further includes step S180: during the updating of the fused image, determining the relative spatial depth information of the local area of ​​interest (LIFO) relative to the reference spatial area based on the display change state of the LIFO corresponding to the LIFO data; wherein, the display change state includes a state changing between a clear state and a disappearing state, and the reference spatial area includes at least one of the following: the tissue spatial area corresponding to the rendering light incident point and the tissue data. The disappearing state can be the state when the LIFO data does not exist, and after rendering, the corresponding area will not be assigned a rendering color, i.e., there is no LIFO region with a color different from the surrounding tissue area; the clear state can be the state when the LIFO data exists and its value is at its maximum, and after rendering, the corresponding area will be assigned a certain rendering color, i.e., there is a LIFO region with a color different from the surrounding tissue area and this region has high brightness and clarity.

[0077] For example, as the first weight is gradually increased or decreased, the rendering proportion of the corresponding Locality of Interest (LOI) data in the second rendered image also gradually increases or decreases. Therefore, the LIO corresponding to the LIO data in the fused image will gradually become clearer or disappear. If the first weight is gradually increased, the proportion of LIO data increases, and the LIO gradually becomes clearer. If the first weight is gradually decreased, the proportion of LIO data decreases, and the LIO gradually disappears. It can be understood that regions closer to the reference region will become clearer earlier and disappear later as the first weight is gradually increased or decreased. Regions farther from the reference region will become clearer later and disappear earlier. The relative spatial depth information of the LIO relative to the reference region can be determined based on the gradual clearing or disappearance of the LIO. It can be understood that as the first weight is gradually increased, the earlier the portion of the LIO becomes clear, the smaller its relative spatial depth relative to the reference region. As the first weight is gradually decreased, the earlier the portion of the LIO disappears, the greater its relative spatial depth relative to the reference region. Therefore, during the weight adjustment operation, the relative spatial depth information can be determined based on the display change status of the local area of ​​interest corresponding to the local data of interest.

[0078] The reference space region can include the tissue space region corresponding to the incident point of the rendered ray and the tissue space region corresponding to the tissue data. When rendering using a ray tracing algorithm, the reference space region can be the first intersection point between each rendered ray and the rendered object, i.e., the incident point. The reference space region can be a region composed of multiple incident points. The reference space region can also be the tissue space region corresponding to the tissue data.

[0079] The relative spatial depth information of a region of interest (ROI) relative to a reference region can represent the positional relationship of the ROI with surrounding tissue regions, such as the boundaries between the ROI and surrounding tissue regions at different depths. This can help physicians assess the nature and extent of lesions in the target tissue, as well as its relationship with surrounding tissues. Therefore, it provides strong evidence for diagnosis, treatment, and prognosis. For example, relative spatial depth information can be used to distinguish between benign and malignant lesions, assess lesion growth patterns, evaluate lesion blood supply characteristics, guide treatment strategies, and determine treatment effectiveness and recurrence probability.

[0080] Specifically, relative spatial depth information can be used to differentiate between benign and malignant lesions. Benign lesions typically exhibit a clear boundary with surrounding tissues, with well-defined and regular borders and a relatively homogeneous enhancement pattern. Malignant lesions, on the other hand, usually show infiltrative growth with surrounding tissues, indistinct and irregular borders, and enhancement patterns that may exhibit rapid enhancement-rapid washing ("fast in, fast out") or heterogeneous enhancement. Therefore, benign and malignant lesions can be identified by the boundary shape of the spatial region of interest corresponding to the local data of interest in the fused image.

[0081] Relative spatial depth information can also be used to assess the growth pattern of lesions. Lesions exhibiting expansive growth are often benign or low-grade malignant, with clear boundaries between the lesion and surrounding tissue. Lesions exhibiting invasive growth are often malignant tumors, potentially invading surrounding tissue, resulting in unclear boundaries. Therefore, benign or malignant lesions can be identified by the boundary shape of the spatial region of interest corresponding to the local data in the fused image.

[0082] Relative spatial depth information can also be used to guide treatment strategies. The positional relationship between the lesion and surrounding important blood vessels, nerves, and other structures directly affects the treatment plan. For localized lesions, surgical resection or local ablation may be considered. For invasive lesions, comprehensive treatment (such as radiotherapy and chemotherapy combined with surgery) may be required. The location of the lesion can also determine the puncture path, reducing damage to important structures. Therefore, the relative positional relationship between the spatial region of interest (ROI) and the surrounding tissue region in the fused image can be used to guide treatment strategies.

[0083] Relative spatial depth information can also be used to assess the blood supply characteristics of lesions. Whether a lesion is close to a blood vessel and whether there is abnormal angiogenesis provides important information about the metabolic activity and nature of the lesion. Highly vascularized lesions, such as hepatocellular carcinoma (HCC), are usually close to blood vessels or have abnormal vascular proliferation. Lowly vascularized lesions, such as metastatic tumors, may not be significantly related to surrounding blood vessels. Therefore, the relative positional relationship between the spatial region of interest (ROI) and the blood vessels in the surrounding tissue region in the fused image can be used to guide treatment strategies.

[0084] Relative spatial depth information can also be used to assess treatment effectiveness and the likelihood of recurrence. If the lesion boundary is close to surrounding important structures and residual lesions may not be easily removed completely, the likelihood of recurrence is high. Ablation treatment effectiveness: By observing enhancement changes in the lesion and surrounding tissues, the thoroughness of treatment can be evaluated.

[0085] Relative spatial depth information can also be used in pathological studies. The location of a lesion within surrounding tissues is related to its histological type. For example, metastatic tumors may be more often located in the peripheral regions of organs, while primary lesions are more commonly located in the central region.

[0086] For example, after obtaining the relative spatial depth information of the local region of interest relative to the reference spatial region, the relative spatial depth information can be displayed. For instance, different parts of the local region of interest can be labeled with different colors based on their different depths in the relative spatial depth information to distinguish spatial depths. Alternatively, the results of the relative spatial depth information can be displayed using text or other methods.

[0087] The above technical solution, during the weight adjustment operation, determines the relative spatial depth information of the local area of ​​interest (HOI) relative to the reference spatial area based on the display change status of the HIO data. In combination with the above, for example, the relative spatial depth information of the HIO of the target tissue of a lesion and the surrounding tissue area can represent the relative positional relationship between them. This not only supports disease diagnosis and classification but also provides important information for developing individualized treatment plans and assessing prognosis.

[0088] For example, the weight adjustment operation is used to gradually reduce the first weight. In other words, the proportion of the local area of ​​interest (HOI) data in the second rendered image gradually decreases. The reference spatial region is the organization spatial region. HOIs located before the organization spatial region are not affected by the change in the first weight. HOIs located after the organization spatial region gradually disappear due to the reduction in the first weight. Therefore, step S180, based on the display change state of the HOIs corresponding to the local area of ​​interest data, determines the relative spatial depth information of the HOIs relative to the reference spatial region, and may include steps S181 and S182.

[0089] In step S181, if the display state of the local spatial region of interest changes from a clear state to a disappearing state, the relative spatial depth information is determined to be that the spatial depth of the local spatial region of interest is greater than or equal to the spatial depth of the tissue spatial region.

[0090] For regions within a local area of ​​interest whose display state changes from a clear state to a vanishing state, the earlier a region vanishes, the greater its spatial depth, and the later a region vanishes, the smaller its spatial depth. Furthermore, regions within a local area of ​​interest that gradually vanish as the first weight decreases have a relative spatial depth greater than or equal to the spatial depth of the tissue region.

[0091] In step S182, if the display status of the local spatial region of interest remains unchanged, the relative spatial depth information is determined to be that the spatial depth of the local spatial region of interest is less than the spatial depth of the tissue spatial region.

[0092] As the first weight gradually decreases, for regions of interest whose display state remains unchanged, their relative spatial depth is less than the depth of the tissue spatial region. Therefore, the relative spatial depth information of regions of interest whose display state remains unchanged can be determined as having a spatial depth less than the spatial depth of the tissue spatial region.

[0093] The above technical solution uses the tissue space region as the reference space region and determines the relative spatial depth of the local spatial region of interest based on a progressively decreasing first weight. This allows for a better determination of the spatial depth of the local region of interest relative to the tissue space region, and a more accurate determination of the positional relationship of the local region of interest relative to the surrounding tissue.

[0094] For example, the ultrasound imaging method described above further includes: step S125, selecting tissue data to obtain tissue data of a local area corresponding to the target tissue.

[0095] Step S125 can be performed before step S130. In some embodiments, a portion of the tissue data of the target tissue can be selected for subsequent data processing, for example, including tissue data of a local region of interest. It is understood that the embodiments of this application aim to provide the relative positional relationship of a local region within the target tissue as reflected by the local data of interest; therefore, only tissue data including that local region can be selected for subsequent data processing. The local region can be the area where the lesion is located, the area where the organ of interest is located, etc., and can be determined according to user needs. It is understood that if the tissue data of the selected local region of the corresponding target tissue is used for image rendering, the resulting rendered image is a local region of the first rendered image.

[0096] The selection of organizational data can be automatic, based on the organizational data and / or local data of interest. In one example, the organizational data is segmented based on factors such as gradients between adjacent data points. Then, the minimum bounding cube of the segmented organizational data, including the local regions, is determined to obtain the organizational data for the corresponding local region of the target organization. In another example, the portion of the organizational data corresponding to the location of the local data of interest can be determined based on the local data of interest; that is, the organizational data for the corresponding local region of the target organization. Alternatively, the organizational data for the corresponding local region of the target organization can be manually selected from the organizational data according to user needs.

[0097] Step S130 involves image rendering of the tissue data and the local area of ​​interest (LOI) to obtain a second rendered image, including: step S132, determining the corresponding LIO based on the selected tissue data, and performing image rendering on the selected tissue data and the corresponding LIO to obtain a second rendered image. In some embodiments, the LIO corresponding to a local region of the target tissue corresponding to the selected tissue data can be determined. Image rendering is then performed on the selected tissue data and the corresponding LIO to obtain a second rendered image. In other words, the second rendered image includes tissue data and LIO of a local region of the target tissue.

[0098] Therefore, in the subsequent step S140, only the corresponding parts of the second rendered image and the first rendered image need to be merged.

[0099] The above-described technical solution allows for the selection of tissue data to participate in subsequent hybrid rendering and image fusion. This can reduce the computational load of ultrasound imaging methods, accelerate ultrasound imaging speed, and reduce ultrasound imaging latency.

[0100] For example, step S125 selects tissue data, including step S125a selecting tissue data of multiple local regions corresponding to the target tissue from the tissue data.

[0101] In some embodiments, the organizational data can be segmented based on gradients or other parameters between adjacent data. Then, the smallest bounding cube of the segmented organizational data, including local regions, is determined to obtain the organizational data of the corresponding local region of the target organization. There can be multiple local regions, and organizational data for multiple local regions can be obtained. In other embodiments, based on the local data of interest, the portion of the organizational data corresponding to its location can be determined, i.e., the organizational data of the corresponding local region of the target organization. It is understood that the local data of interest can correspond to multiple regions. Organizational data for multiple local regions of the target organization corresponding to the local data of interest can be determined. Alternatively, organizational data for multiple local regions of the target organization can be manually selected from the organizational data according to user needs.

[0102] Image rendering of the selected tissue data and the corresponding local data of interest may include step S1321, whereby for the tissue data of each local region, image rendering of the tissue data of the local region and the corresponding local data of interest is performed according to the first weight corresponding to the local region, wherein the first weights corresponding to different local regions are different.

[0103] Each local region can correspond to a first weight, and different local regions can correspond to different first weights. Based on the tissue data and the local data of interest corresponding to each local region, the tissue data and the local data of interest can be rendered according to the first weight of the local region to obtain the rendered image corresponding to each local region, which together serve as the second rendered image.

[0104] The above technical solution supports selecting multiple local regions of a target organization, and the primary weight of different local regions can be different. This allows for more flexible rendering of organizational data and local data of interest, providing richer and more diverse information and improving the user experience.

[0105] For example, step S140 merges the first rendered image and the second rendered image to obtain a merged image, including steps S141 and S142.

[0106] In step S141, the fusion region is determined in the second rendered image.

[0107] This step can be performed manually or semi-automatically. In one specific example, the user can manually outline the blending region in the second rendered image using a line drawing tool. In another specific example, the blending region can be determined by marking keypoint locations on the second rendered image. This determination of the blending region based on keypoint locations can be achieved using methods such as interactive segmentation, thresholding segmentation, and automatic growth.

[0108] This step can also be implemented fully automatically by the user. For example, step S141, determining the fusion region in the second rendered image, may include: step S1411, automatically determining the fusion region based on the pixel features of the second rendered image. Specifically, step S1411 can be implemented using a trained artificial intelligence model, automatic segmentation algorithm, etc.

[0109] For example, in hysterosalpingography (HSG) imaging, the fusion region can include both fallopian tubes and the uterine cavity. Based on the characteristic that the pixels corresponding to the fallopian tubes and uterine cavity in the second rendered image have larger pixel values, the second rendered image can be automatically segmented to determine the fusion region.

[0110] The above technical solution automatically determines the fusion region based on the second rendered image. No user intervention is required, avoiding the influence of personal user experience on the determination of the fusion region, and improving the quality and speed of ultrasound imaging.

[0111] In a specific example, the blending region is first determined in the first rendered image. Then, based on the position of the blending region in the first rendered image, its corresponding position in the second rendered image can be determined. Thus, the blending region is determined in the second rendered image.

[0112] In step S142, the second rendered image within the fusion region is fused into the first rendered image to obtain a fused image.

[0113] In some embodiments, based on the fusion region determined in step S141, only the second rendered image within the fusion region is fused into the first rendered image. In other words, the second rendered image outside the fusion region is not fused into the first rendered image.

[0114] The above technical solution determines a fusion region in the second rendered image and merges the second rendered image within the fusion region into the first rendered image. Users can flexibly select the fusion region to meet their personalized needs. Simultaneously, it reduces the computational load of fusion, improves the computational speed of the imaging method, and reduces ultrasound imaging latency.

[0115] For example, step S131 performs mixed rendering of organization data and local data of interest according to a first weight to obtain a second rendered image, including steps S1311, S1312, S1313, S1314, S1315 and S1316.

[0116] In step S1311, for each ray path, based on the tissue data, the tissue color value and tissue opacity value of each sampling point on the ray path are obtained, and based on the local interest data, the local interest color value and local interest opacity value of each sampling point on the ray path are obtained.

[0117] Tissue data can include tissue color values ​​and tissue opacity values. Similarly, locality of interest data can include locality of interest color values ​​and locality of interest opacity values. The color value represents the color information of the corresponding location of the target tissue, and the opacity value represents the visibility of the corresponding location of the target tissue.

[0118] Specifically, the size and resolution of the second rendered image to be rendered can be determined first. The size and resolution of the second rendered image can be the same as those of the first rendered image. Based on the size and resolution of the second rendered image, the number and position of pixels in the second rendered image can be determined. In this embodiment, ray casting can be used to perform volume rendering on tissue data and locality of interest (LOI) data to obtain the second rendered image. In ray casting, a ray is emitted from each pixel of the second rendered image, and multiple sampling points can be set along the ray's path. The tissue color value and tissue opacity value of each sampling point on the ray path can be determined based on the tissue data. Similarly, the LIO color value and LIO opacity value of each sampling point on the ray path can be determined based on the LIO data. It is understood that at some sampling points on the ray path, tissue color value, tissue opacity value, LIO color value, and LIO opacity value can be obtained. At some sampling points on the ray path, only tissue color value and tissue opacity value may be obtained, without LIO color value and LIO opacity value. There may be no LIO data at the sampling point location.

[0119] In step S1312, the mixed opacity value of each sampling point is determined based on the first weight, the organization opacity value, and the opacity value of the local area of ​​interest.

[0120] Specifically, the mixed opacity value of each sample point can be obtained by weighted summation of the tissue opacity value and the opacity value of the area of ​​interest, based on the first weight. In a specific example, the mixed opacity value of each sample point can be calculated according to the following formula (1):

[0121] alpha mix =alpha gray ·(1-mixscale)+alpha contrast • mixscale (1);

[0122] Where, alpha mix This represents the blending opacity value at that sampling point; alpha gray This represents the tissue opacity value at that sampling point; alpha contrast This represents the local opacity value of the sampling point; mixscale represents the first weight.

[0123] For example, the first weight can be 0.7, the tissue opacity value of the sampling point can be 0.3, the contrast opacity value can be 0.5, and the mixed value of different transparency of the sampling point can be 0.44 calculated by the above formula.

[0124] In step S1313, the mixed color value of each sampling point is determined based on the first weight, the tissue color value, the tissue opacity value, the color value of the local area of ​​interest, and the opacity value of the local area of ​​interest.

[0125] In a specific example, the mixed color value can be calculated according to the following formula (2):

[0126] color mix =color grag ·alpha gray (1-mixscale)+color contrast ·alpha cintrast • mixscale (2)

[0127] Among them, color mix This represents the mixed color value of the sample point; color gray This represents the tissue color value at that sampling point; alpha gray This indicates the tissue opacity value at that sampling point; color contrast This represents the local color value of interest at that sampling point; alpha contrast This represents the local opacity value of the sampling point; mixscale represents the first weight.

[0128] In step S1314, the cumulative mixed opacity value on each light path is determined based on the mixed opacity value of each sampling point on each light path.

[0129] Each ray path has multiple sampling points. All sampling points can be uniformly distributed along the ray path with a preset step size, or they can be set according to parameters such as the data density of tissue data and locality of interest (LOO). The blend opacity of each sampling point along each ray path can be accumulated, for example, by summing, to obtain the cumulative blend opacity value for that ray path. In some embodiments, the cumulative blend opacity of each ray path can be set with a maximum value threshold. When the cumulative blend opacity of the preceding sampling points on the ray path reaches the maximum value threshold, subsequent sampling points no longer participate in the image rendering calculation of tissue data and LEO. The penetrating power of light is limited; when the cumulative blend opacity value reaches the maximum value threshold, the light cannot reach subsequent sampling points. Therefore, it is not necessary to determine the tissue color value, tissue opacity value, LEO color value, and LEO opacity value of subsequent sampling points.

[0130] In step S1315, the cumulative mixed color value on each light path is obtained based on the mixed color value of each sampling point on each light path and the mixed opacity value of each sampling point.

[0131] The blended color value of each sampling point along the ray path can be accumulated, for example, by summing, to obtain the cumulative blended color value along the ray path. In some embodiments, since the cumulative blended opacity has a maximum threshold, when the cumulative blended opacity reaches the maximum threshold, the tissue color value and the local color value of interest at subsequent sampling points will no longer be acquired. Therefore, the calculation of the cumulative blended color value along the ray path will no longer refer to the blended color value and blended opacity value of subsequent sampling points.

[0132] In step S1316, the color value of each pixel in the second rendered image is determined based on the cumulative blend color value and cumulative blend opacity value on each ray path, so as to obtain the second rendered image.

[0133] The color value of each pixel in the second rendered image is related to the cumulative blend color value and cumulative blend opacity value along the ray path of the light emitted by that pixel.

[0134] In a specific example, the color value of a pixel in the second rendered image can be calculated according to the following formula (3):

[0135] color pixel =coloracc·alphaacc (3)

[0136] Among them, color pixe1This represents the color value of a pixel in the second rendered image; coloracc represents the cumulative blended color value along the ray path corresponding to that pixel; alphaacc represents the cumulative blended opacity value along the ray path corresponding to that pixel.

[0137] The above technical solution performs hybrid rendering of organizational data and local data of interest based on a first weight. The proportion of information from each type of organizational data and local data of interest in the second rendered image can be controlled. This technical solution integrates information from organizational data and local data of interest with relatively low computational cost, ensuring the realism of the obtained second rendered image and thus guaranteeing the smoothness of the fused image.

[0138] It is understood that the specific example above demonstrates the calculation of the blend opacity value and blend color value of each sampling point using formulas (1) and (2) respectively. This scheme has low computational cost, fast calculation speed, and the resulting second rendered image has a more realistic imaging effect. However, this scheme is only an example of this application and does not constitute a limitation of this application.

[0139] Alternatively, the first weight can be applied only to the opacity value of the area of ​​interest, thereby affecting its influence in the second rendered image.

[0140] In the examples above, which calculate the blend opacity and blend color values ​​of each sampling point based on formulas (1) and (2) respectively, the larger the first weight, the more the local data of interest in the deeper areas can participate in image rendering, and the more prominent the local data of interest will be in the final fused image; otherwise, the first weight decreases, the proportion of local data of interest decreases, and the nearby tissue data can quickly make the cumulative blend opacity value reach a certain value. In this case, the local data of interest in the deeper areas does not participate in image rendering, and the presentation effect of the local data of interest will no longer appear in the fused image.

[0141] Figure 4A A schematic diagram of a fused image according to an embodiment of the present invention is shown. Figure 4B A schematic diagram of a fused image according to another embodiment of the present invention is shown. Figure 4C A schematic diagram of a fused image according to another embodiment of the present invention is shown. Figure 4A , Figure 4B and Figure 4C Image rendering is performed on identical tissue data and localities of interest (LOIs) with different first weights to obtain a second rendered image. This second rendered image is then fused with the first rendered image to obtain a fused image. The LIOs are imaging data; specifically... Figure 4A The first weight corresponding to the fused image shown is 0.9; Figure 4B The first weight corresponding to the fused image shown is 0.7; Figure 4CThe first weight corresponding to the fused image shown is 0.4. (As shown...) Figure 4A , Figure 4B and Figure 4C The fused images generated from contrast data and tissue data at different proportions have different display effects.

[0142] Alternatively, a first weight can be omitted. In a specific example, the blended opacity value of each sampling point can be determined solely based on the tissue opacity value and the opacity value of the region of interest. The blended color value of each sampling point is determined solely based on the tissue color value, tissue opacity value, region of interest color value, and region of interest opacity value. This specific example also achieves the goal of integrating tissue data and region of interest data, successfully showing the specific location of the region of interest within the target tissue in the fused image.

[0143] For example, step S140 merges the first rendered image and the second rendered image to obtain a merged image, including steps S143 and S144.

[0144] In step S143, for each ray path, the cumulative opacity value of the local area of ​​interest on that ray path is obtained based on the local area of ​​interest data.

[0145] As described above, each sampling point along each ray path can obtain its corresponding local color value and local opacity value based on the locality of interest (LOI) data. Therefore, the LIO values ​​of each sampling point along the ray path can be accumulated, for example, by summing, to obtain a cumulative LIO value. In some embodiments, the cumulative LIO value also has a maximum threshold; once the sum of the LIO values ​​of preceding sampling points reaches the maximum threshold, the LIO values ​​of subsequent sampling points are not included in the calculation. In other words, since the penetrating power of light is limited, it cannot reach the positions of subsequent sampling points, and thus the LIO values ​​of those subsequent sampling points cannot be obtained. Figure 3B A schematic diagram illustrating the cumulative opacity values ​​of a region of interest according to an embodiment of the present invention is shown. Figure 3B As shown, the gray area represents the cumulative local opacity value; the higher the brightness, the greater the opacity value. The black area does not correspond to any cumulative local opacity value.

[0146] In step S144, the color value of each pixel in the fused image is determined based on the color value of each pixel in the first rendered image, the color value of each pixel in the second rendered image, the cumulative blending opacity value, and the cumulative local opacity value of interest, so as to obtain the fused image.

[0147] The color value of each pixel in the blended image can be calculated based on the color value of the corresponding pixel in the first rendered image, the color value of the corresponding pixel in the second rendered image, the cumulative blending opacity value, and the cumulative local opacity value. In a specific example, it can be calculated according to the following formula (4):

[0148] colorpixel out =colorpixel mix ·alphaacc contrast +colorpixel grag ·(1-alphaacc mix ·alphaacc contrast (4)

[0149] Among them, colorpixel out Indicates a specific pixel in the merged image; colorpixel mix This represents the color value of a pixel in the corresponding second rendered image; alphaacc contrast This represents the cumulative local opacity value of the pixel in the corresponding second rendered image along the ray path; colorpixel gray This represents the color value of a pixel in the corresponding first rendered image; alphaacc mix This represents the cumulative blending opacity value along the ray path corresponding to the pixel in the second rendered image.

[0150] In some embodiments, local interest data can be obtained only at certain locations of the target tissue. In other words, local interest data can be data from a portion of the target tissue. Tissue data can be data from all locations of the target tissue. It can be understood that some regions in the second rendered image do not have local interest data involved in the calculation, only tissue data. Therefore, the cumulative local interest opacity value on the ray path corresponding to the pixel in these regions can be 0, i.e., completely transparent. The second rendered image includes regions where both local interest data and tissue data are involved in the calculation, and the cumulative local interest opacity value on the ray path corresponding to the pixel in these regions is not 0. The cumulative local interest opacity value alphaacc in the above formula (4) contrastThis allows the fused image to display only the color values ​​of the pixels in the first rendered image corresponding to the organizational data in regions of the second rendered image that only contain organizational data and no local interest data (LOGs) in the calculation. In other words, in these regions of the fused image, only the corresponding regions of the first rendered image generated solely from the organizational data are displayed. Regions containing both organizational data and LIGs are merged from the first and second rendered images. It can be understood that at the boundary between regions containing only organizational data and regions containing both organizational data and LIGs, the cumulative LIG opacity value is relatively small. This results in a more natural blending transition between the second and first rendered images at the boundary.

[0151] The above technical solution fuses the first and second rendered images based on the accumulated opacity values ​​of the local area of ​​interest to obtain a fused image. This controls the fusion range of the first and second rendered images, reduces computational complexity, and ensures the quality of the fused image.

[0152] According to another aspect of the present invention, an ultrasound imaging apparatus is also provided. Figure 5 A schematic block diagram of an ultrasound imaging apparatus 500 according to an embodiment of the present invention is shown. Figure 5 As shown, the ultrasound imaging device 500 includes an acquisition module 510, a first rendering module 520, a second rendering module 530, and a fusion module 540. The acquisition module 510 acquires region of interest (ROI) data and tissue data of the target tissue. The ROI data and the tissue data are acquired using different ultrasound imaging methods. The first rendering module 520 renders the tissue data to obtain a first rendered image. The second rendering module 530 renders the tissue data and the ROI data to obtain a second rendered image. The fusion module 540 fuses the first rendered image and the second rendered image to obtain a fused image.

[0153] For example, the acquisition module 510 includes a first acquisition submodule, the first rendering module 520 includes a first rendering submodule, the second rendering module 530 includes a second rendering submodule, and the fusion module 540 includes a first fusion submodule. The first acquisition submodule is used to acquire the local area of ​​interest (HOI) data and the tissue data at different times. The first rendering submodule is used to perform image rendering on the tissue data at some of the different times to obtain at least one first rendered image. The second rendering submodule is used to perform image rendering on the tissue data and the HIO data at each time point to obtain multiple second rendered images. The first fusion submodule is used to fuse the second rendered image at each time point with the corresponding first rendered image to obtain a fused image at each time point.

[0154] For example, the ultrasound imaging device 500 further includes a second acquisition module. The second rendering module 530 includes a third rendering submodule. The second acquisition module is used to acquire a first weight. The third rendering submodule is used to perform mixed rendering of the tissue data and the region of interest data according to the first weight to obtain a second rendered image, wherein the first weight is used to determine the influence of the tissue data and the region of interest data on the second rendered image, and the larger the first weight, the greater the influence of the region of interest data on the second rendered image, and the smaller the influence of the tissue data on the second rendered image.

[0155] For example, the ultrasound imaging device 500 further includes a display module and a first adjustment module. The display module is used to display the fused image corresponding to the current first weight. The first adjustment module is used to adjust the first weight and update the fused image in response to a weight adjustment operation.

[0156] For example, the weight adjustment operation is used to gradually increase or decrease the first weight within the weight adjustment range. The ultrasound imaging device 500 also includes a first determining module. The first determining module is used to determine the relative spatial depth information of the local area of ​​interest (HOI) relative to a reference spatial region based on the display change state of the HIOI corresponding to the HIOI data during the updating of the fused image; wherein the display change state includes a state changing between a clear state and a disappearing state, and the reference spatial region includes at least one of the following: the rendering ray incident point and the tissue spatial region corresponding to the tissue data.

[0157] For example, the weight adjustment operation is used to gradually reduce the first weight; the reference spatial region is the tissue spatial region. The first determining module includes a first determining submodule and a second determining submodule. The first determining submodule is used to determine the relative spatial depth information as follows: if the display state of the local spatial region of interest changes from a clear state to a disappearing state, the spatial depth information is determined to be greater than or equal to the spatial depth of the tissue spatial region. The second determining submodule is used to determine the relative spatial depth information as follows: if the display state of the local spatial region of interest remains unchanged, the spatial depth information is determined to be less than the spatial depth of the tissue spatial region.

[0158] For example, the third rendering submodule includes a second acquisition unit, a first determination unit, a second determination unit, a third determination unit, a fourth determination unit, and a fifth determination unit. The second acquisition unit is used to, for each ray path, acquire the tissue color value and tissue opacity value of each sampling point on the ray path based on the tissue data, and acquire the local color value and local opacity value of each sampling point on the ray path based on the locality of interest data. The first determination unit is used to determine the blended opacity value of each sampling point based on a first weight, the tissue opacity value, and the local opacity value. The second determination unit is used to determine the blended color value of each sampling point based on the first weight, the tissue color value, the tissue opacity value, the local color value, and the local opacity value. The third determination unit is used to determine the cumulative blended opacity value of each ray path based on the blended opacity value of each sampling point on each ray path. The fourth determination submodule is used to determine the cumulative blended color value of each ray path based on the blended color value and the blended opacity value of each sampling point on each ray path. The fifth determining unit is used to determine the color value of each pixel in the second rendered image based on the cumulative blended color value and the cumulative blended opacity value on each ray path, so as to obtain the second rendered image.

[0159] For example, the blending module 540 includes a second blending submodule and a third blending submodule. The second blending submodule is used to obtain, for each ray path, a cumulative locality of interest (LOI) opacity value on that ray path based on the LIO data. The third blending submodule is used to determine the color value of each pixel in the blended image based on the color value of each pixel in the first rendered image, the color value of each pixel in the second rendered image, the cumulative blending opacity value, and the cumulative LIO opacity value, to obtain the blended image.

[0160] For example, the ultrasound imaging device further includes a selection module. The second rendering module 530 includes a fourth rendering submodule. The selection module is used to select the tissue data to obtain tissue data corresponding to a local region of the target tissue. The fourth rendering submodule is used to determine the corresponding local area of ​​interest based on the selected tissue data, and to perform image rendering on the selected tissue data and the corresponding local area of ​​interest.

[0161] For example, the selection module includes a first selection submodule. The fourth rendering submodule includes a fourth rendering unit. The first selection submodule is used to select organization data of multiple local regions corresponding to the target organization from the organization data. The fourth rendering unit is used to perform image rendering on the organization data of each local region and the corresponding local data of interest according to a first weight corresponding to that local region, wherein different local regions have different first weights.

[0162] For example, the fusion module 540 includes a sixth determining submodule and a fourth fusion submodule. The sixth determining submodule is used to determine a fusion region in the second rendered image. The fourth fusion submodule is used to fuse the second rendered image within the fusion region into the first rendered image to obtain the fused image.

[0163] For example, the sixth determining submodule includes a sixth determining unit. The sixth determining unit is used to automatically determine the fusion region based on the pixel features of the second rendered image.

[0164] According to another aspect of the present invention, an electronic device is also provided. Figure 6 A schematic block diagram of an electronic device 600 according to an embodiment of the present invention is shown. Figure 6 As shown, the electronic device 600 includes a processor 610 and a memory 620. The memory 620 stores computer program instructions, which are executed by the processor 610 to perform the ultrasound imaging method described above.

[0165] According to another aspect of the present invention, a storage medium is also provided, on which program instructions are stored, which, when executed, are used to perform the ultrasound imaging method of the above embodiments. The storage medium may, for example, include an erasable programmable read-only memory (EPROM), a portable read-only memory (CD-ROM), a USB memory, or any combination of the above storage media. The storage medium may be any combination of one or more computer-readable storage media.

[0166] According to another aspect of the present invention, a computer program product is also provided, including computer program instructions that, when executed, perform the ultrasound imaging method described in the above embodiments.

[0167] Those skilled in the art can understand the specific implementation schemes and beneficial effects of the above-described ultrasound imaging methods, including the ultrasound imaging device, electronic device, storage medium, and computer program product, by reading the relevant descriptions. For the sake of brevity, these will not be elaborated further here.

[0168] Although exemplary embodiments have been described herein with reference to the accompanying drawings, it should be understood that the above exemplary embodiments are merely illustrative and are not intended to limit the scope of the invention. Various changes and modifications can be made therein by those skilled in the art without departing from the scope and spirit of the invention. All such changes and modifications are intended to be included within the scope of the invention as claimed in the appended claims.

[0169] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0170] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0171] Similarly, it should be understood that, in order to streamline the invention and aid in understanding one or more of the various aspects of the invention, features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof in the description of exemplary embodiments of the invention. However, this approach should not be construed as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as reflected in the corresponding claims, its inventive point lies in solving the corresponding technical problem with fewer features than all of those in a single disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of the invention.

[0172] Those skilled in the art will understand that, apart from the mutual exclusion of features, all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or apparatus so disclosed can be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0173] Furthermore, those skilled in the art will understand that although some embodiments described herein include certain features but not others included in other embodiments, combinations of features from different embodiments are intended to be within the scope of the invention and form different embodiments. For example, in the claims, any of the claimed embodiments can be used in any combination.

[0174] The various component embodiments of the present invention can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement the functions in the electronic devices according to embodiments of the present invention. The present invention can also be implemented as an apparatus program (e.g., a computer program and computer program product) for performing some or all of the methods described herein. Such programs implementing the present invention can be stored on a computer-readable medium or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0175] It should be noted that the above embodiments are illustrative of the invention and not restrictive, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

[0176] The above description is merely a specific embodiment of the present invention or an explanation of that embodiment. The scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. The scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. An ultrasound imaging method, characterized in that, include: Acquire local data of interest and tissue data of the target tissue, wherein the local data of interest and the tissue data are acquired using different ultrasound imaging methods; The tissue data is rendered to obtain a first rendered image; The tissue data and the region of interest are rendered to obtain a second rendered image; and The first rendered image and the second rendered image are merged to obtain a merged image.

2. The ultrasound imaging method according to claim 1, characterized in that, The acquisition of the target organization's local data of interest and organizational data includes: At different times, acquire the local data of interest and the tissue data; The step of rendering the tissue data to obtain a first rendered image includes: Image rendering is performed on the organizational data of a portion of the different time points to obtain at least one first rendered image; The step of rendering the tissue data and the local data of interest to obtain a second rendered image includes: The tissue data and the local data of interest at each time point are rendered into images to obtain multiple second rendered images; The step of fusing the first rendered image and the second rendered image to obtain a merged image includes: The second rendered image at each time step is fused with the corresponding first rendered image to obtain the fused image at each time step.

3. The ultrasound imaging method according to claim 1 or 2, characterized in that, The ultrasound imaging method further includes: Obtain the first weight; The step of rendering the tissue data and the local data of interest to obtain a second rendered image includes: The tissue data and the local data of interest are mixed and rendered according to the first weight to obtain the second rendered image. The first weight is used to determine the influence of the tissue data and the local data of interest on the second rendered image respectively. The larger the first weight, the greater the influence of the local data of interest on the second rendered image, and the smaller the influence of the tissue data on the second rendered image.

4. The ultrasound imaging method according to claim 3, characterized in that, The ultrasound imaging method further includes: Displays the fused image corresponding to the current first weight; In response to the weight adjustment operation, the first weight is adjusted and the fused image is updated.

5. The ultrasound imaging method according to claim 4, characterized in that, The weight adjustment operation is used to gradually increase or decrease the first weight within the adjustable range of the weight. The ultrasound imaging method further includes: During the updating of the fused image, the relative spatial depth information of the local area of ​​interest relative to the reference spatial area is determined based on the display change state of the local area of ​​interest corresponding to the local area of ​​interest data; wherein, the display change state includes a state that changes between a clear state and a disappearing state, and the reference spatial area includes at least one of the following: the rendering ray incident point and the tissue spatial area corresponding to the tissue data.

6. The ultrasound imaging method according to claim 5, characterized in that, The weight adjustment operation is used to gradually reduce the first weight; the reference space region is the tissue space region; The step of determining the relative spatial depth information of the local area of ​​interest relative to the reference spatial area based on the display change state of the local area of ​​interest corresponding to the local area of ​​interest data includes: If the display state of the local area of ​​interest changes from a clear state to a disappearing state, then the relative spatial depth information is determined to be that the spatial depth of the local area of ​​interest is greater than or equal to the spatial depth of the tissue space region. If the display status of the local area of ​​interest remains unchanged, then the relative spatial depth information is determined to be that the spatial depth of the local area of ​​interest is less than the spatial depth of the tissue space region.

7. The ultrasound imaging method according to claim 3, characterized in that, The step of performing mixed rendering of the tissue data and the local data of interest according to the first weight to obtain the second rendered image includes: For each ray path, based on the tissue data, the tissue color value and tissue opacity value of each sampling point on the ray path are obtained, and based on the region of interest data, the region of interest color value and region of interest opacity value of each sampling point on the ray path are obtained. The mixed opacity value of each sampling point is determined based on the first weight, the tissue opacity value, and the opacity value of the region of interest. The mixed color value of each sampling point is determined based on the first weight, the tissue color value, the tissue opacity value, the color value of the region of interest, and the opacity value of the region of interest. The cumulative blending opacity value on each ray path is determined based on the blending opacity value at each sampling point on each ray path; The cumulative mixed color value along each light path is determined based on the mixed color value and the mixed opacity value of each sampling point along each light path. The color value of each pixel in the second rendered image is determined based on the cumulative blend color value and the cumulative blend opacity value on each ray path, so as to obtain the second rendered image.

8. The ultrasound imaging method according to claim 7, characterized in that, The step of fusing the first rendered image and the second rendered image to obtain a merged image includes: For each ray path, the cumulative opacity value of the local area of ​​interest on that ray path is obtained based on the local area of ​​interest data; The color value of each pixel in the fused image is determined based on the color value of each pixel in the first rendered image, the color value of each pixel in the second rendered image, the cumulative blending opacity value, and the cumulative locality of interest opacity value, so as to obtain the fused image.

9. The ultrasound imaging method according to claim 1 or 2, characterized in that, The ultrasound imaging method further includes: The tissue data is selected to obtain tissue data for a local region corresponding to the target tissue; The step of rendering the tissue data and the local data of interest to obtain a second rendered image includes: Based on the selected tissue data, the corresponding local data of interest is determined, and image rendering is performed on the selected tissue data and the corresponding local data of interest.

10. The ultrasound imaging method according to claim 9, characterized in that, The selection of the tissue data includes: Select tissue data from multiple local regions corresponding to the target tissue in the tissue data; The image rendering of the selected tissue data and the corresponding local data of interest includes: For each local region's organizational data, image rendering is performed on the local region's organizational data and the corresponding local data of interest based on the first weight corresponding to that local region. The first weight is different for different local regions.

11. The ultrasound imaging method according to claim 1 or 2, characterized in that, The step of fusing the first rendered image and the second rendered image to obtain a merged image includes: In the second rendered image, the fusion region is determined; The second rendered image within the fusion region is fused into the first rendered image to obtain the fused image.

12. The ultrasound imaging method according to claim 11, characterized in that, Determining the blending region in the second rendered image includes: The fusion region is automatically determined based on the pixel features of the second rendered image.

13. An ultrasonic imaging device, characterized in that, include: The acquisition module is used to acquire local data of interest and tissue data of the target tissue, wherein the local data of interest and the tissue data are acquired using different ultrasound imaging methods; The first rendering module is used to perform image rendering on the tissue data to obtain a first rendered image; The second rendering module is used to perform image rendering on the tissue data and the region of interest data to obtain a second rendered image; and The fusion module is used to fuse the first rendered image and the second rendered image to obtain a fused image.

14. An electronic device comprising: Processor and memory, characterized in that, The memory stores computer program instructions, which, when executed by the processor, are used to perform the ultrasound imaging method as described in any one of claims 1 to 12.

15. A storage medium on which program instructions are stored, characterized in that, The program instructions, when executed, are used to perform the ultrasound imaging method as described in any one of claims 1 to 12.

16. A computer program product comprising computer program instructions, characterized in that, The computer program instructions, when executed, are used to perform the ultrasound imaging method as described in any one of claims 1 to 12.

Images