Image rendering method and device of interested volume and display method and device of interested volume

Abstract

The invention provides a volume-of-interest image rendering method and device and a volume-of-interest display method and device. The image rendering method comprises the steps of obtaining an interested volume; determining grids on each surface of a target area, wherein the target area is an area including an area where the volume of interest is located; generating a forward depth texture map and a backward depth texture map of the target area based on the position information of the determined grid; determining a starting point and an ending point of each light in the at least one light based on the forward depth texture map and the backward depth texture map; acquiring color data of the interested volume corresponding to each light at least based on the starting point and the ending point corresponding to each light; and rendering the volume of interest based on the color data of the volume of interest corresponding to each light to obtain a rendered target image. According to the scheme, the image rendering efficiency of the interested volume can be improved.

Description

Technical Field

[0001] This application belongs to the technical field of ultrasonic imaging, and particularly relates to an image rendering method for a volume of interest, an image display method for a volume of interest, an image rendering device for a volume of interest, an image display device for a volume of interest, an electronic device, and a storage medium. Background Art

[0002] Three-dimensional ultrasound examination is of great significance in clinical diagnosis. During the three-dimensional ultrasound examination, a two-dimensional image of the volume of interest can be obtained by performing volume rendering on the volume of interest in the three-dimensional ultrasound image. Since this two-dimensional image retains the detailed information of the volume of interest, the overall and complete view of the volume of interest can be observed based on this two-dimensional image.

[0003] In the related art, generally, the ray casting algorithm is used to generate the two-dimensional image of the volume of interest. The ray casting algorithm is a direct volume rendering algorithm based on an image sequence. From each pixel of the image, a ray is emitted along a fixed direction (usually the line-of-sight direction). The ray passes through the entire image sequence, and during this process, color information is sampled from the image sequence, and at the same time, the color values are accumulated according to the ray absorption model until the ray passes through the entire image sequence. The finally obtained color value is the color of the two-dimensional image to be rendered. The ray casting algorithm can determine the ray casting distance of each ray in the volume of interest by finding the intersection points of the rays and the volume of interest. Since the observation surface of the volume of interest is usually a curved surface with a complex structure, the process of finding the intersection of the rays and the curved surface of the volume of interest is very time-consuming, which affects the rendering efficiency of the two-dimensional image. Summary of the Invention

[0004] In view of the above problems, this application is proposed. This application provides an image rendering method for a volume of interest, an image display method for a volume of interest, an image rendering device for a volume of interest, an image display device for a volume of interest, an electronic device, and a storage medium.

[0005] According to one aspect of the present application, an image rendering method for a volume of interest is provided, including: obtaining a volume of interest, which is determined based on a three-dimensional ultrasound image; determining a mesh on each surface of a target region, where the target region is a region including the region where the volume of interest is located; generating a forward depth texture map and a backward depth texture map of the target region based on the position information of the determined mesh; determining the starting point and the ending point of each ray among at least one ray; where at least one ray is used to collect color data for image rendering, and for each ray, the starting point of the ray is the intersection point of the ray and the forward depth texture map, and the ending point of the ray is the intersection point of the ray and the backward depth texture map; obtaining the color data corresponding to each ray of the volume of interest at least based on the starting point and the ending point corresponding to each ray respectively; rendering the volume of interest based on the color data corresponding to each ray of the volume of interest to obtain a rendered target image.

[0006] Exemplarily, the volume of interest includes at least one curved surface; the target region is the region where the volume of interest is located, and determining a mesh on each surface of the target region includes: for each curved surface among the at least one curved surface, dividing the curved surface into a preset number of meshes; determining an expression function for representing the curved surface based on the type of the curved surface and the position information of a plurality of feature points on the curved surface; determining the position information of each mesh on the curved surface based on the expression function and the positions of the preset number of meshes on the curved surface.

[0007] Exemplarily, determining a mesh on each surface of the target region further includes: for each plane in the volume of interest, determining the plane as a mesh, and determining the position information of the mesh corresponding to the plane based on the position information of the plane.

[0008] Exemplarily, determining an expression function for representing the curved surface based on the type of the curved surface and the position information of a plurality of feature points on the curved surface includes: determining a function for representing a feature curve of the curved surface based on the type of the curved surface and the position information of a plurality of feature points on the curved surface, and the expression function is the function of the feature curve; where the feature curve includes curves obtained by projecting the curved surface onto a first plane and a second plane respectively, and the first plane and the second plane are two mutually perpendicular shear planes on the volume of interest.

[0009] Exemplarily, dividing the curved surface into a preset number of meshes includes: projecting the curved surface onto a third plane, where the third plane is a shear plane on the volume of interest that is perpendicular to the first plane and the second plane respectively; evenly dividing the projected curved surface by the preset number to obtain evenly divided projected meshes; determining the meshes on the curved surface that correspond one-to-one to the evenly divided projected meshes as the preset number of meshes divided.

[0010] Exemplarily, the target region is the bounding box region of the volume of interest, and grids are determined on each surface of the target region, including: determining grids on each surface of the bounding box region.

[0011] Exemplarily, the volume of interest includes at least one curved surface; before determining grids on each surface of the target region, the method further includes: for each curved surface in the at least one curved surface, determining an expression function for representing the curved surface based on the type of the curved surface and the position information of a plurality of feature points on the curved surface; determining the position information of the vertices of the curved surface based on the expression function; and determining the bounding box region based on the position information of the vertices of each surface in the volume of interest.

[0012] Exemplarily, determining grids on each surface of the bounding box region includes: for each plane in the bounding box region, determining the plane as a grid and determining the position information of the grid corresponding to the plane based on the position information of the plane.

[0013] Exemplarily, the plurality of feature points includes at least two preset feature points and at least one adjustable feature point, and the position information of the at least two preset feature points is known. Before determining an expression function for representing the curved surface based on the type of the curved surface and the position information of the plurality of feature points on the curved surface, the method further includes: in response to a position setting operation of the user, determining the position information of the at least one adjustable feature point.

[0014] Exemplarily, acquiring color data of the volume of interest corresponding to each ray based at least on the starting point and the ending point corresponding to each ray includes: for each ray, performing resampling along the corresponding ray projection direction to collect color data corresponding to a plurality of resampled points within the bounding box region; rendering the volume of interest based on the color data of the volume of interest corresponding to each ray includes: for each resampled point among the plurality of resampled points corresponding to each ray, determining whether the resampled point is located on the volume of interest; when the resampled point is not located on the volume of interest, resetting the color data of the resampled point to 0; accumulating the color data of each resampled point corresponding to each ray; and rendering the volume of interest based on the accumulated color data.

[0015] Exemplarily, determining whether the resampled point is located on the volume of interest includes: projecting the volume of interest onto a first plane and a second plane respectively to obtain a first projection image and a second projection image; the first plane and the second plane are two mutually perpendicular shear planes on the volume of interest; projecting the resampled point onto the first plane and the second plane respectively; when the first projection point is within the projection area obtained by projecting the volume of interest onto the first plane and the second projection point is within the projection area obtained by projecting the volume of interest onto the second plane, determining that the resampled point is located on the volume of interest; wherein, the first projection point is the projection point of the resampled point on the first plane; the second projection point is the projection point of the resampled point on the second plane.

[0016] Exemplarily, obtaining color data of the volume of interest corresponding to each ray based at least on the starting point and the ending point corresponding to each ray includes: for each ray, starting resampling when the ray intersects with the starting point corresponding to the ray; ending resampling when the ray meets any one of the requirements in the target requirements; wherein, the target requirements include at least one of the following requirements: a first target requirement, a second target requirement, and a third target requirement; the first target requirement is that the ray intersects with the ending point corresponding to the ray; the second target requirement is that the cumulative opacity value of the resampled points passed by the ray is greater than the transparency threshold; the third target requirement includes that the ray projection distance is greater than or equal to the ray projection distance threshold; the ray projection distance is the distance between the current resampled point passed by the ray and the starting point corresponding to the ray; the ray projection distance threshold is the distance between the ending point corresponding to the ray and the starting point corresponding to the ray.

[0017] According to another aspect of the present application, there is also provided an image display method for a volume of interest, including: rendering the volume of interest by using the above-mentioned rendering method to obtain a rendered target image; displaying the target image.

[0018] According to another aspect of the present application, there is also provided an image rendering device for a volume of interest, including: a first acquisition module for acquiring a volume of interest, which is determined based on a three-dimensional ultrasound image; a first determination module for determining a grid on each surface of a target region, where the target region is a region including the region where the volume of interest is located; a generation module for generating a forward depth texture map and a backward depth texture map of the target region based on the position information of the determined grid; a second determination module for determining the starting point and ending point of each ray among at least one ray based on the forward depth texture map and the backward depth texture map, where at least one ray is used to collect color data for image rendering, and for each ray, the starting point of the ray is the intersection point of the ray and the forward depth texture map, and the ending point of the ray is the intersection point of the ray and the backward depth texture map; a second acquisition module for acquiring color data corresponding to each ray of the volume of interest at least based on the starting point and ending point corresponding to each ray; a first rendering module for rendering the volume of interest based on the color data corresponding to each ray of the volume of interest to obtain a rendered target image.

[0019] According to another aspect of the present application, there is also provided an image display device for a volume of interest, including: a second rendering module for rendering the volume of interest by using the above-mentioned rendering method to obtain a rendered target image; a display module for displaying the target image.

[0020] According to another aspect of the present application, there is also provided an electronic device, including a processor and a memory, where computer program instructions are stored in the memory, and when the computer program instructions are run by the processor, they are used to execute the above-mentioned rendering method or the above-mentioned display method.

[0021] According to another aspect of the present application, there is also provided a storage medium, on which program instructions are stored, and when the program instructions are run, they are used to execute the above-mentioned rendering method or the above-mentioned display method.

[0022] According to the above technical solutions of the embodiments of the present application, by using the forward depth texture map and the backward depth texture map generated based on the grid on each surface of the target region, the starting point and ending point of each ray can be quickly determined, which helps to quickly obtain the color data of the volume of interest. This solution is simple to implement and does not require a complex intersection algorithm to calculate the intersection point of the ray and the region of interest, which helps to improve the image rendering efficiency of the volume of interest.

[0023] The above description is only an overview of the technical solutions of the present application. In order to be able to understand the technical means of the present application more clearly, it can be implemented according to the content of the specification. And in order to make the above and other purposes, features and advantages of the present application more obvious and understandable, the specific embodiments of the present application are specifically exemplified below. Description of the Drawings

[0024] The following drawings of the present application are hereby incorporated as part of the present application for understanding the present application. The embodiments of the present application shown in the drawings and their descriptions are used to explain the principles of the present application. In the drawings,

[0025] Figure 1 A schematic flowchart showing a method for image rendering of a volume of interest according to an embodiment of the present application;

[0026] Figure 2 A schematic diagram showing a volume of interest according to an embodiment of the present application;

[0027] Figure 3 A schematic diagram showing a volume of interest according to another embodiment of the present application;

[0028] Figure 4 A schematic diagram showing a single Bezier surface according to an embodiment of the present application;

[0029] Figure 5 A schematic diagram showing a projection image of a volume of interest according to an embodiment of the present application;

[0030] Figure 6 A schematic flowchart showing a method for rendering a volume of interest according to a specific embodiment of the present application;

[0031] Figure 7 A schematic diagram showing a projected surface obtained by projecting a surface onto a third plane according to an embodiment of the present application;

[0032] Figure 8 A schematic diagram showing a bounding box region of a volume of interest according to an embodiment of the present application;

[0033] Figure 9 A schematic diagram showing projection images obtained by projecting a bounding box region onto two planes respectively according to an embodiment of the present application;

[0034] Figure 10 A schematic flowchart showing a method for rendering a volume of interest according to another specific embodiment of the present application;

[0035] Figure 11 A schematic flowchart showing a method for image display of a volume of interest according to an embodiment of the present application;

[0036] Figure 12 A schematic block diagram showing an image rendering apparatus for a volume of interest according to an embodiment of the present application;

[0037] Figure 13Schematic block diagram of an image display device for a volume of interest according to an embodiment of the present application; and

[0038] Figure 14 Schematic block diagram of an electronic device according to an embodiment of the present application. Detailed implementation manners

[0039] In order to make the objectives, technical solutions, and advantages of the present application more obvious, exemplary embodiments according to the present application will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. It should be understood that the present application is not limited by the exemplary embodiments described herein. Based on the embodiments described in the present application, all other embodiments obtained by those skilled in the art without creative efforts shall fall within the protection scope of the present application.

[0040] According to one aspect of the present application, a method for rendering an image of a volume of interest is provided. Figure 1 Schematic flowchart of a method for rendering an image of a volume of interest according to an embodiment of the present application. As Figure 1 shown, the method 100 may include the following steps S110, step S120, step S130, step S140, step S150, and step S160.

[0041] In step S110, a volume of interest is acquired, and the volume of interest is determined based on a three-dimensional ultrasound image.

[0042] It can be understood that when performing three-dimensional ultrasound image acquisition, the entire spatial region including the imaging target (i.e., the volume of interest) can be scanned, ultrasonic waves are emitted and ultrasonic echoes are received to obtain volume data within the spatial region. This spatial region can be referred to as an ultrasound imaging space. The volume of interest (VOI) can also be referred to as the region of interest (ROI), which refers to a geometric shape defined in the ultrasound imaging space. When performing three-dimensional ultrasound image acquisition, generally, tissues larger than the volume of interest are acquired for imaging to ensure that the volume data in the region where the volume of interest is located is fully acquired.

[0043] Optionally, determining the volume of interest in a three-dimensional ultrasound image can be performed when the three-dimensional ultrasound image is not displayed. Alternatively, determining the volume of interest in a three-dimensional ultrasound image can be performed when the three-dimensional ultrasound image is displayed. In this embodiment, the volume of interest can be determined based on the displayed three-dimensional ultrasound image, facilitating viewing of whether the volume of interest determined in the three-dimensional ultrasound image already contains the target object for which three-dimensional imaging is desired. Therefore, the solution of this embodiment helps to more accurately determine the volume of interest in the three-dimensional ultrasound image.

[0044] The overall shape of the volume of interest can be set as needed. Optionally, the volume of interest can be a hexahedron or a polyhedron with a different number of surfaces. The shape of each surface of the volume of interest can also be set as needed. For example, each surface of the volume of interest can be a plane. Alternatively, at least one of the surfaces of the volume of interest can include a curved surface. The number of curved surfaces can be set as needed, such as one, two, three, or more. In a specific embodiment, the starting observation surface of the volume of interest can be a curved surface, and the other five surfaces can be planes.

[0045] Optionally, the size of the volume of interest can be adjusted according to user needs. For example, the dimensions in the three directions of length, width, and height of the volume of interest can be preset according to the size of the target object so that the volume of interest can enclose the target object. Alternatively, the shearing line can be adjusted by adjusting the control points on the shearing line of the volume of interest on any slice, and a shearing plane can be generated based on the two shearing lines on two perpendicular slices. By changing the positions of the edges on this shearing plane, the shape of the volume of interest can be changed. In one embodiment, the volume of interest can be projected onto the A, B, and C planes of the three-dimensional imaging system, and the shape of the volume of interest can be changed by adjusting the positions of the edges of each projection in the A, B, and C planes. Exemplarily, the A plane can be a plane perpendicular to a predefined horizontal direction in the three-dimensional ultrasound imaging space, and is typically used to observe the projection of the volume of interest on this plane. The B plane can be a plane perpendicular to the horizontal direction and perpendicular to the A plane in the three-dimensional ultrasound imaging space, and is typically used to observe the projection of the volume of interest on this plane. The C plane can be a plane parallel to the horizontal direction in the three-dimensional ultrasound imaging space, and is typically used to observe the projection of the volume of interest on the horizontal plane.

[0046] Figure 2 A schematic diagram showing the volume of interest according to an embodiment of the present application. In this embodiment, the starting observation surface of the volume of interest is a smooth curved surface, and the other five surfaces are planes. As Figure 2As shown, projections of the volume of interest on three planes A, B, and C can be obtained. The projections of the volume of interest on the three planes A, B, and C are all polygons. The user can change the shape of the starting observation plane by adjusting one or several sides of the polygon. Exemplarily, for the ultrasonic imaging space where the volume of interest is located, a three-dimensional coordinate system can be established, such as the XYZ coordinate system. The above-mentioned plane A can be the XY plane, that is, the plane where the X-axis (the axis corresponding to fROIX) and the Y-axis (the axis corresponding to fROIY) are located. The above-mentioned plane B can be the YZ plane, that is, the plane where the Y-axis and the Z-axis (the axis corresponding to fROIZ) are located. The above-mentioned plane C can be the XZ plane, that is, the plane where the X-axis and the Z-axis are located.

[0047] In step S120, grids are determined on each surface of the target region, and the target region is the region including the region where the volume of interest is located.

[0048] Optionally, the target region can be the region where the volume of interest is located. Alternatively, the target region can be larger than the region where the volume of interest is located. For example, the target region can be a bounding box containing the volume of interest.

[0049] Optionally, the number of grids on each surface can be determined according to the shape of each surface. For example, for any surface of the target region, when the surface is a plane, the surface can be directly determined as one grid. When the shape of the surface is a curved surface, the surface can be divided into multiple grids. The above embodiment of directly determining the surface as one grid when the surface is a plane is only an example and not a limitation of the present application. When the surface is a plane, it can also be optionally divided into multiple grids.

[0050] Optionally, the shape of the grid can be set as needed. For example, the shape of the grid can be any polygon such as a quadrilateral or a triangle.

[0051] Optionally, the grid shapes on each surface of the target region can be the same. For example, the grid shapes on each surface of the target region can all be quadrilaterals. Alternatively, the grid shapes on each surface of the target region can also be different. For example, when there are curved surfaces among the surfaces of the target region, the grid shape corresponding to the plane among the surfaces of the target region can be a quadrilateral, and the grid shape corresponding to the curved surface among the surfaces of the target region can be a triangle.

[0052] In step S130, based on the position information of the determined grids, a forward depth texture map and a backward depth texture map of the target region are generated.

[0053] Optionally, the position information of the grid can be represented by the vertex information of each vertex of the grid. In this article, the vertex of the grid is each corner point of the grid. Figure 3A schematic diagram showing a volume of interest according to another embodiment of the present application. In this embodiment, the volume of interest includes five planes and one curved surface. For any one of the five planes, it can be directly determined that the plane is a grid. As Figure 3 shown, the volume of interest is a hexahedron. The 8 vertices of the volume of interest can be numbered. The vertex information can be the numbers corresponding to the respective vertices. In this embodiment, the numbers of the vertices corresponding to the respective surfaces of the volume of interest can be recorded in a clockwise or counterclockwise manner. For example, the numbers of the vertices corresponding to the respective surfaces can be recorded in a counterclockwise order. Among them, the grids corresponding to the five planes of the volume of interest can be respectively represented as {6, 2, 3, 7}, {5, 1, 0, 4}, {7, 3, 1, 5}, {4, 0, 2, 6}, {2, 0, 1, 3}. For the curved surface in the volume of interest, the grid can be divided on the curved surface in any manner, and the vertices of each grid can be numbered in a manner similar to the grids on the plane, and the grid can be marked based on the numbers of the vertices of the grid in a manner similar to the grids on the plane. Of course, the above solution of using the vertex information of the respective vertices of the grid to represent the position information of the grid is only an example and not a limitation to the present application. The present application can use other suitable ways to represent the position information of the grid. For example, the position information of the grid can be directly represented by the coordinates of two or more vertices of the grid in the XYZ coordinate system, or alternatively, the position information of the grid can be represented by the coordinates of one or more vertices of the grid or the coordinates of the center point of the grid in the XYZ coordinate system, in combination with the length and / or width of the grid.

[0054] In some embodiments, off-screen rendering can be performed on the target area through any one of the existing or future-developed open image libraries for rendering images, and depth culling can be enabled during the rendering to cull the back faces of the target area, thereby generating a forward depth texture map of the target area. In this step, the vertex information of the grids corresponding to the respective surfaces of the target area can be stored in a vertex table (VertTable) in advance, and it is specified that the surface with the vertex order in the counterclockwise direction is the front face. Each time, 4 vertices can be taken out from the vertex table to determine the face corresponding to a grid, and it is confirmed whether to perform the depth culling operation until all the vertices in the vertex table are used up. Thus, the generation of the forward depth texture map is completed.

[0055] In some embodiments, off-screen rendering of the volume of interest can be performed using any existing or future-developed open image library for rendering images, and depth culling can be enabled during the rendering to cull the front-facing surfaces of the volume of interest, thereby generating a back-facing depth texture map of the volume of interest. In this step, the vertex information of the meshes corresponding to the surfaces of the target region can be stored in a vertex table (VertTable) in advance, and the surfaces with a clockwise vertex order are defined as the back surfaces. Each time, 4 vertices can be taken from the vertex table to determine the face corresponding to a mesh, and it is confirmed whether depth culling is to be performed until all the vertices in the vertex table are used up. Thus, the generation of the back-facing depth texture map is completed.

[0056] In step S140, based on the forward depth texture map and the back-facing depth texture map, determine the starting point and ending point of each ray in at least one ray; wherein, at least one ray is used to collect color data for image rendering, and for each ray, the starting point of the ray is the intersection point of the ray and the forward depth texture map, and the ending point of the ray is the intersection point of the ray and the back-facing depth texture map.

[0057] It can be understood that each point on the forward depth texture map represents the point on the volume of interest that is closest to the view point along the ray projection direction, and each point on the back-facing depth texture map represents the point on the volume of interest that is farthest from the view point along the ray projection direction. In step S140, by determining the intersection points of each ray with the forward depth texture map and the back-facing depth texture map, the starting point and ending point of each ray can be determined, thereby providing a relatively accurate basis for subsequent steps (such as step S150). Those skilled in the art can understand the specific steps of determining the intersection points of each ray with the forward depth texture map and the back-facing depth texture map, which will not be elaborated here.

[0058] In step S150, obtain the color data corresponding to each ray of the volume of interest based at least on the starting point and ending point corresponding to each ray.

[0059] In some embodiments, the color data of the target region can be resampled at equal intervals based at least on the starting point and ending point corresponding to each ray. Each sampling point on the ray can be called a resampled point. Optionally, obtaining the color data of the volume of interest corresponding to each ray can include the following steps: for any ray, accumulate the color data of each resampled point on the ray to obtain the color data of the ray.

[0060] In step S160, render the volume of interest based on the color data corresponding to each ray of the volume of interest to obtain the rendered target image.

[0061] The step of rendering the volume of interest based on color data can be implemented in any rendering manner, and the present application does not limit this.

[0062] In the above technical solution, by using the forward depth texture map and the backward depth texture map generated based on the meshes of the surfaces of the target region, the starting point and the ending point of each ray can be quickly determined, which helps to quickly obtain the color data of the volume of interest. This solution is simple to implement and does not require a complex intersection algorithm to calculate the intersection points of the rays and the region of interest, which helps to improve the image rendering efficiency of the volume of interest.

[0063] Exemplarily, the volume of interest includes at least one curved surface. The target region is the region where the volume of interest is located. Determining the meshes on the surfaces of the target region may include the following steps: For each curved surface in the at least one curved surface, divide the curved surface into a preset number of meshes. Based on the type of the curved surface and the position information of a plurality of feature points on the curved surface, determine an expression function for representing the curved surface. Based on the expression function and the positions of the preset number of meshes on the curved surface, determine the position information of each mesh on the curved surface.

[0064] Optionally, the plurality of feature points may be at least two. In some embodiments, the expression function of the curved surface can be determined by the position information of two feature points and the type of the corresponding curved surface. In other embodiments, the expression function of the curved surface can be determined by the position information of three feature points and the type of the corresponding curved surface. It can be understood that the above number of feature points is only an example, and the number of feature points can also be four, five or more.

[0065] Optionally, the preset number can be set as needed. For example, the preset number can be determined according to the area of the current curved surface. The larger the area of the current curved surface, the more the preset number can be; the smaller the area of the current curved surface, the fewer the preset number can be. In one embodiment, the preset number can be 128. When dividing the curved surface into a preset number of meshes, the division method can be arbitrary, and an equal division method or a non-equal division method can be adopted.

[0066] Optionally, the surface of the volume of interest can be a single surface or composed of multiple spliced surfaces. The types of any two surfaces among the multiple surfaces can be the same or different. Exemplarily, when the surface of the volume of interest is a single surface, the surface can be any one of a Bezier surface, a B-spline surface, a non-uniform rational B-spline (NURBS) surface, a paraboloid, etc. Exemplarily, when the surface of the volume of interest is composed of multiple spliced surfaces, each surface for splicing can be any one of a Bezier surface, a B-spline surface, a NURBS surface, a paraboloid, etc. In a specific embodiment, the surface of the volume of interest can be composed of 4 quadratic Bezier surfaces spliced together. Figure 4 A schematic diagram showing a single Bezier surface according to an embodiment of the present application. Exemplarily, the surface of the volume of interest can be composed of 4 single Bezier surfaces as Figure 4 shown spliced together.

[0067] Optionally, the expression function of the surface can be determined by any existing or future-developed method for determining the expression function of the surface. For example, the expression function of the surface can be represented by the functions of at least two characteristic curves corresponding to the surface. For another example, the functions of at least two characteristic curves corresponding to the surface can be determined first, and then, based on the functions of the at least two characteristic curves, the expression function of the surface can be calculated. Those skilled in the art can understand the method of calculating the expression function of the surface based on the functions of at least two characteristic curves, which will not be elaborated here.

[0068] After obtaining the expression function of the surface, the position information of each grid can be determined according to the positions of a preset number of grids on the surface. For example, the coordinates of each vertex of each grid can be determined according to the expression function of the surface and the positions of the grids on the surface. The position information of each grid can be represented by the coordinates of the vertices corresponding to the grid.

[0069] According to the above technical solution, by determining the expression function of the surface and dividing the surface into a preset number of grids, the position information of each grid can be determined more accurately, and thus a more reliable basis can be provided for subsequent steps (such as step S130).

[0070] Exemplarily, determining grids on each surface of the target area may further include the following steps: For each plane in the volume of interest, the plane is determined as a grid, and the position information of the grid corresponding to the plane is determined based on the position information of the plane.

[0071] In this solution, a plane is directly determined as a grid without further dividing the plane. This solution helps to reduce the number of vertices, improve the generation speed of the forward depth texture map and the backward depth texture map, and reduce the calculation amount.

[0072] Exemplarily, based on the type of the surface and the position information of a plurality of feature points on the surface, determining an expression function for representing the surface may include the following steps: Based on the type of the surface and the position information of a plurality of feature points on the surface, determining a function of a feature curve for representing the surface. The expression function is a function of the feature curve. Wherein, the feature curve includes curves obtained by projecting the surface onto a first plane and a second plane respectively, and the first plane and the second plane are two mutually perpendicular shear planes on the volume of interest.

[0073] Take Figure 2 as an example to illustrate the feature curve of the surface. As Figure 2 shown, the first plane may be plane A, and the second plane may be plane B. The feature curve of any surface may be the curve obtained by projecting the surface onto plane A and the curve obtained by projecting the surface onto plane B.

[0074] Optionally, the type of the surface is a quadratic Bezier surface. It can be understood that the calculation of the quadratic Bezier surface can be obtained by finding a quadratic Bezier curve in one direction and then finding a quadratic Bezier curve in another direction. The Bezier curves in different directions can be calculated from the shear lines of the volume of interest on the three-dimensional virtual cutting plane (i.e., the shear plane).

[0075] In one embodiment, the function of the feature curve (i.e., the quadratic Bezier curve) on the quadratic Bezier surface may be represented by the following formula:

[0076] B(t) = (1 - t) 2 P + 2t(1 - t)B + t 2 Q (1)

[0077] Wherein, P, B, and Q are control points (i.e., feature points) on the quadratic Bezier curve. t is a parameter of the quadratic Bezier curve, and t is in the range of [0, 1]. Any one or more of P, B, and Q may be preset feature points, and any one or more of P, B, and Q may also be adjustable feature points adjusted by the user. Figure 5 A schematic diagram showing a projection image of a volume of interest according to an embodiment of the present application. Figure 5 In the shown embodiment, the plane for projection is plane A. As Figure 5 shown, the feature curve obtained by projecting the volume of interest onto plane A can be divided into two Bezier curves, P 1 , B 1 and Q 1 are feature points on Bezier curve 1, P 2 , B 2and Q 2 are characteristic points on Bezier curve 2, where Q 1 and P 2 are the same point. In this embodiment, the function of Bezier curve 1 can be determined according to the coordinates of P 1 , B 1 and Q 1 . Similarly, the function of Bezier curve 2 can be determined according to the coordinates of P 2 , B 2 and Q 2 .

[0078] In Figure 5 the embodiment shown, P 1 and Q 2 are control points on the volume of interest, and their spatial coordinates change as the user adjusts the size of the volume of interest. Q 1 (i.e., the P 2 point) is a control point whose coordinates can be directly adjusted by the user. The coordinates of B 1 can be calculated according to the coordinates of P 1 , Q 1 and Q 2 . The coordinates of B 2 can be calculated according to the coordinates of P 1 , P 2 and Q 2 . To ensure that Bezier curve 1 passes through P 1 and Q 1 , and Bezier curve 2 can pass through P 2 and Q 2 point, the calculation methods of the coordinates of B 1 and B 2 points are as follows:

[0079]

[0080]

[0081] As described above, the three control points of the first - order quadratic Bezier curve are P 1 , B 1 and Q 1 , and the three control points of the second - order quadratic Bezier curve are P 2 , B 2 and Q 2 . Substituting the control points on the Bezier curve into formula (1), the value of B(t) corresponding to any t ∈ [0, 1] can be calculated.

[0082] Similarly, the control points of two Bézier curves can be obtained by the same method on plane B. In this way, 4 Bézier planes can be formed by the four Bézier curves on the two planes. The 4 Bézier planes form the surface of the volume of interest. Therefore, the expression function of the surface of the volume of interest can be represented by the functions corresponding to the curves obtained by projecting the surface onto two mutually perpendicular planes.

[0083] The above embodiment illustrates the determination of the surface expression function with the type of the surface being a quadratic Bézier surface. It can be understood that the above embodiment is only an example, and the type of the surface can also be a B-spline surface, a NURBS surface, a paraboloid, etc., which are not limited herein. When using different types of surfaces, the expression function of the surface can be determined by using the function of the characteristic curve corresponding to the surface. Those skilled in the art can understand the method for determining the expression functions of different surfaces, which will not be elaborated herein.

[0084] Figure 6 A schematic flowchart showing a method for rendering a volume of interest according to a specific embodiment of the present application is shown. As Figure 6 shown, first, step S610 is executed to determine the expression function of the surface on the volume of interest. Then, step S620 is executed to determine the meshes of the respective surfaces of the volume of interest to obtain the position information of each mesh. After obtaining the position information of the meshes of the respective surfaces of the volume of interest, step S630 is executed to generate a forward depth texture map and a backward depth texture map. Next, step S640 is executed to calculate the starting point, ending point, and ray casting distance of each ray in the ray casting direction by using the forward depth texture map and the backward depth texture map. In this step, the corresponding starting point and ending point can be obtained by calculating each ray with the forward depth texture map and the backward depth texture map. The ray casting distance of each ray is the distance between the corresponding starting point and ending point. After obtaining the starting point, ending point, and ray casting distance of each ray, step S650 is executed to perform volume rendering by using a ray casting algorithm. In this step, based on the starting point, ending point, and ray casting distance of each ray, the color data of the volume of interest corresponding to each ray can be obtained, and based on the color data of the volume of interest corresponding to each ray, the volume of interest is rendered to obtain the rendered target image.

[0085] In the above technical solution, the function of the characteristic curve of the surface is directly used as the expression function of the surface. This solution helps to improve the calculation efficiency, and thus helps to further improve the image rendering efficiency.

[0086] Exemplarily, dividing the surface into a preset number of meshes may include the following steps: projecting the surface onto a third plane. The third plane is a shear plane perpendicular to the first plane and the second plane respectively on the volume of interest. Dividing the projected surface evenly according to the preset number to obtain evenly divided projected meshes. Determining the meshes on the surface that correspond one-to-one to the evenly divided projected meshes as the preset number of meshes to be divided.

[0087] Still taking Figure 2 as an example to illustrate the third plane. As Figure 2 shown, the third plane may be plane C.

[0088] Figure 7 FIG. shows a schematic diagram of the projected surface obtained by projecting the surface onto the third plane according to an embodiment of the present application. As Figure 7 shown, the projection of the surface onto the third plane is a quadrilateral, and the quadrilateral can be evenly divided into N*M quadrilaterals, and each quadrilateral is a projected mesh. N*M is the preset number, and its value can be set according to needs, for example, equal to 128. Since the meshes on the surface of the volume of interest correspond one-to-one to the evenly divided projected meshes, therefore, the surface can also be approximately pieced together by N*M quadrilaterals. In this embodiment, the number of vertices of the quadrilaterals on the surface of the volume of interest is (N + 1)*(M + 1). Since the vertices of each quadrilateral on the surface satisfy the Bezier curve characteristics, the coordinates of the vertices of each quadrilateral can be calculated according to the expression function of the surface. At the same time, the vertex numbers of the quadrilaterals can be sorted in the counterclockwise direction for each quadrilateral corresponding vertex to record the vertex numbers of the quadrilaterals as the position information of each mesh on the surface. In a specific embodiment, the vertex numbers of the quadrilaterals on the surface can be stored in VertTable for use in the generation of the forward depth texture map and the backward depth texture map in subsequent steps.

[0089] According to the above technical solution, by using the projection of the surface onto the third plane to divide the surface into a preset number of meshes, it helps to improve the accuracy of the forward depth texture map and the backward depth texture map generated based on the position information of the meshes, thereby helping to ensure the image rendering effect of the volume of interest.

[0090] Exemplarily, the target area is the bounding box area of the volume of interest. Determining meshes on each surface of the target area may include the following steps: determining meshes on each surface of the bounding box area.

[0091] The bounding box region can be the circumscribed bounding box region of the volume of interest or a region larger than the circumscribed bounding box region of the volume of interest. Optionally, the bounding box region of the volume of interest can be generated by any existing or future-developed bounding box algorithm. For example, the algorithm can be an Axis-Aligned Bounding Box (AABB) algorithm, an Oriented bounding box (OBB) algorithm, etc. The solution of determining a grid on the surface of the bounding box region with the bounding box region as the target region can be executed when the volume of interest has any shape. Preferably, it is executed when at least one surface of the volume of interest is a curved surface. When at least one surface of the volume of interest is a curved surface, especially when two or more surfaces are curved surfaces, the shape of the volume of interest is relatively irregular and it is not easy to determine the grid on its surface. In this case, it will be relatively easier to determine the grid by using the bounding box region as the target region.

[0092] It can be understood that the bounding box is a geometric body with a simple shape, and each surface of the bounding box is a plane. In some embodiments, each surface of the bounding box can be determined as a grid. Figure 8 A schematic diagram showing the bounding box region of the volume of interest according to an embodiment of the present application. As Figure 8 shown, the bounding box region is a hexahedron, and each surface of the bounding box region is a plane. In this embodiment, the 8 vertices of the bounding box region can be labeled first, and then the labels (i.e., vertex information) of the vertices corresponding to each surface of the volume of interest can be recorded in a clockwise or counterclockwise manner. For example, the labels of the vertices corresponding to each surface can be recorded in a counterclockwise order. The grids corresponding to the flat surfaces of the volume of interest can be respectively represented as {6, 2, 3, 7}, {5, 1, 0, 4}, {7, 3, 1, 5}, {4, 0, 2, 6}, {2, 0, 1, 3}, {7, 5, 4, 6}.

[0093] In the above technical solution, the bounding box region of the volume of interest is used as the target region, and grids are determined on the surfaces of the bounding box region. This solution does not need to divide the grids on the curved surfaces that may exist on the volume of interest, which helps to improve the calculation speed and thus helps to improve the rendering efficiency of the volume of interest.

[0094] Exemplarily, the volume of interest includes at least one curved surface. Before determining the grid for each surface of the target region, method 100 may further include the following steps: For each curved surface in the at least one curved surface, based on the type of the curved surface and the position information of a plurality of feature points on the curved surface, determine an expression function for representing the curved surface. Based on the expression function, determine the position information of the vertices of the curved surface. Based on the position information of the vertices of each surface in the volume of interest, determine the bounding box region. The specific method for determining the expression function for representing the curved surface has been described in detail above. For the sake of brevity, it will not be repeated here.

[0095] Taking Figure 8 as an example, the determination of the bounding box region of the volume of interest will be described. As Figure 8 shown, first, based on the expression function of the top curved surface of the volume of interest, determine the position information of vertex D of the curved surface. Then, based on the position information of the vertices of each surface in the volume of interest, determine the bounding box region. It can be understood that for a plane, any point on the plane can be used as a vertex of the plane. Therefore, based on point D and any vertex on each of the other five planes, the bounding box region can be determined. The surfaces of the bounding box region can correspond one by one to the surfaces of the volume of interest. The plane of the bounding box region corresponding to the curved surface of the volume of interest can pass through the vertices of the curved surface, and the plane of the bounding box region corresponding to the plane of the volume of interest can be the plane of the volume of interest itself.

[0096] According to the above technical solution, by determining the position information of the vertices of the curved surface, the bounding box region of the volume of interest can be determined more accurately, thereby being able to provide a more accurate basis for subsequent steps (such as step S120), which helps to improve the image rendering effect of the volume of interest.

[0097] Exemplarily, determining the grid for each surface of the bounding box region may include the following steps: For each plane in the bounding box region, determine the plane as a grid, and based on the position information of the plane, determine the position information of the grid corresponding to the plane.

[0098] It can be understood that each surface of the bounding box region is a plane. Therefore, each surface of the bounding box region can be directly determined as a grid. For example, when the bounding box region is a hexahedron, the six faces of the bounding box region can be directly determined as six grids.

[0099] The above technical solution directly determines a plane as a grid without further dividing the plane. This solution helps to reduce the number of vertices, improve the generation speed of the forward depth texture map and the backward depth texture map, and reduce the amount of calculation. Of course, the above embodiment of determining each plane as a grid is only an example, and each of any one or more planes in the bounding box area can be further divided into multiple grids by an equal or unequal division method.

[0100] Exemplarily, the multiple feature points include at least two preset feature points and at least one adjustable feature point, and the position information of the at least two preset feature points is known. Before determining the expression function for representing the surface based on the type of the surface and the position information of the multiple feature points on the surface, the method may further include the following steps: in response to a position setting operation of the user, determining the position information of the at least one adjustable feature point.

[0101] In an embodiment where the target area is the area where the volume of interest is located, when determining the expression function of any surface in the volume of interest, the multiple feature points used may include the preset feature points and adjustable feature points in this embodiment. In an embodiment where the target area is the bounding box area, when determining the bounding box area, the vertices of the surface may be determined first. And when determining the vertices of the surface for the purpose of determining the expression function of any surface, the multiple feature points used may also include the preset feature points and adjustable feature points in this embodiment. The position information of the adjustable feature points can be set by the user.

[0102] Optionally, the user can set the position information of any adjustable feature point by inputting text information. The operation of the user inputting text information is the position setting operation. For example, the user can directly input the coordinates of any adjustable feature point in a text box through an operation device such as a mouse or a keyboard. Alternatively, the user can set the position information of the adjustable feature point by directly controlling the movement of any adjustable feature point. The operation of the user controlling the movement of the adjustable feature point is the position setting operation. Exemplarily, in an embodiment where the user controls the movement of the adjustable feature point, the position information of the moved adjustable feature point can be determined according to the relative movement distance of the adjustable feature point and the initial coordinates of the adjustable feature point. The initial coordinates of the adjustable feature point can be set as needed. Taking Figure 5 as an example to illustrate the adjustable feature point. As Figure 5 shown, the length and height of the volume of interest are ROIX and ROIY respectively. When defining the volume of interest, its length, width, and height will be defined in advance, for example, ROIX, ROIY, and ROIZ respectively (which can be seen in Figure 2As shown). The volume of interest is preferably symmetrically distributed relative to the center point of the entire display screen, and this center point can correspond to the origin (0, 0, 0) in the XYZ coordinate system. It can be understood that in the projected image, the z - coordinates of all points on the volume of interest are normalized to 0. In this embodiment, the projected image can be symmetrically distributed relative to the center point (0, 0, 0) of the XYZ coordinate system. P 1 and Q 2 are preset feature points. P 1 has the coordinates (-ROIX / 2, ROIY / 2, 0), and Q 2 has the coordinates (ROIX / 2, ROIY / 2, 0). Q 1 (i.e., the P 2 point) is an adjustable feature point, and the initial coordinates can be, for example, (0, 0.5, 0). The user can adjust the coordinates of the Q 1 point. When moving, the coordinates of the Q 1 point can be calculated according to the relative moving distance of the Q 1 point. For example, if the height of the current projected image is PlaneH, and the moving distances of the Q 1 point on the plane are Xoffset and Yoffset, then the coordinates of the Q 1 point are (Xoffset / PlaneH, 0.5 + Yoffset / PlaneH, 0).

[0103] It can be understood that when determining the volume of interest, by adjusting the position of the adjustable feature point, the shape and size of the surface of the volume of interest can be changed. In this solution, the user is allowed to set the position information of at least one adjustable feature point according to the needs, so that the user can customize the shape and size of the volume of interest. This solution can meet the personalized needs of users and the user experience is relatively good.

[0104] Exemplarily, step S150, obtaining the color data of the volume of interest corresponding to each ray, at least based on the starting point and the ending point corresponding to each ray, may include the following steps: For each ray, along the corresponding ray projection direction, resampling is performed to collect the color data corresponding to multiple resampling points within the bounding box area.

[0105] Step S160, rendering the volume of interest based on the color data of the volume of interest corresponding to each ray, may include the following steps: For each resampling point among the multiple resampling points corresponding to each ray, determine whether the resampling point is located on the volume of interest. When the resampling point is not located on the volume of interest, reset the color data of the resampling point to 0. Accumulate the color data of each resampling point corresponding to each ray. Render the volume of interest based on the accumulated color data. Those skilled in the art can understand the method of accumulating the color data of each resampling point on each ray, which will not be elaborated here.

[0106] Optionally, for any resampling point, it can be determined whether the resampling point is located on the volume of interest according to the coordinates of the resampling point in the coordinate system where the volume of interest is located (such as the above XYZ coordinate system). Alternatively, for any resampling point, the resampling point can be projected onto the projection plane of the volume of interest, and the position of the projection point of the resampling point can be used to determine whether the resampling point is located on the volume of interest. This method will be described in detail below.

[0107] It can be understood that when the volume of interest includes at least one curved surface, the size of the bounding box region is larger than the region where the volume of interest is located. Therefore, when collecting color data using rays, there is color data in the obtained color data that does not belong to the volume of interest. In this solution, after obtaining the multiple resampling points corresponding to each ray, further determining whether each resampling point is located within the volume of interest helps to eliminate the color data that does not belong to the volume of interest in the obtained color data, thereby helping to ensure the rendering effect of the volume of interest image.

[0108] Exemplarily, determining whether the resampling point is located on the volume of interest may include the following steps: Project the volume of interest onto a first plane and a second plane respectively. The first plane and the second plane are two mutually perpendicular shear planes on the volume of interest. Project the resampling point onto the first plane and the second plane respectively. When the first projection point is located within the projection region obtained by projecting the volume of interest onto the first plane and the second projection point is located within the projection region obtained by projecting onto the second plane, it is determined that the resampling point is located on the volume of interest. Wherein, the first projection point is the projection point of the resampling point on the first plane; the second projection point is the projection point of the resampling point on the second plane.

[0109] Figure 9 A schematic diagram showing the projection images obtained by projecting the bounding box region according to an embodiment of the present application onto two planes respectively. In this embodiment, the first plane is plane A and the second plane is plane B. As Figure 9As shown, first, project the volume of interest onto plane A and plane B respectively to obtain a first projection image and a second projection image. The projection area corresponding to the volume of interest in the first projection image can be called the first projection area, and the projection area corresponding to the volume of interest in the second projection image can be called the second projection area. Optionally, in this step, the pixel values belonging to their respective projection areas in the first projection image and the second projection image can be filled with 255 (i.e., the pixel values of the areas indicated by the slashes in plane A and plane B in the figure are filled with 255), and the pixel values of the parts outside the first projection area in the first projection image and the parts outside the second projection area in the second projection image are filled with 0. Then project the current resampling point onto the first plane and the second plane respectively to obtain a first projection point and a second projection point, and determine the pixel values corresponding to the first projection point and the second projection point respectively. When the pixel values of the first projection point and the second projection point are both 255, it can be determined that the current resampling point is located on the volume of interest. When any one of the pixel values of the first projection point and the second projection point is 0, it can be determined that the current resampling point is not located on the volume of interest. At this time, the color data of the current resampling point can be reset to 0.

[0110] Figure 10 FIG. shows a schematic flowchart of a rendering method for a volume of interest according to another specific embodiment of the present application. As Figure 10As shown in the figure, first, step S1010 is executed to calculate the bounding box of the region where the volume of interest is located, that is, to determine the bounding box region. Then, step S1020 is executed to determine the meshes of each surface of the bounding box region to obtain the position information of each mesh. After obtaining the position information of the meshes of each surface of the bounding box region, step S1030 is executed to generate a forward depth texture map and a backward depth texture map. Next, step S1040 is executed to calculate the starting point, ending point, and ray casting distance of each ray in the ray casting direction by using the forward depth texture map and the backward depth texture map. In this step, the corresponding starting point and ending point can be obtained by calculating each ray with the forward depth texture map and the backward depth texture map. The ray casting distance of each ray is the distance between the corresponding starting point and ending point. After obtaining the starting point, ending point, and ray casting distance of each ray, step S1050 is executed to trim the rendering data (i.e., color data) by applying two plane clipping templates. Among them, the two plane clipping templates are the projection of the volume of interest on the first plane and the projection of the volume of interest on the second plane. In this step, the position of the projection point of the resampling point in the first plane and the second plane can be used to determine whether the resampling point is located on the volume of interest. If the resampling point is not located on the volume of interest, the color data of the resampling point is reset to 0. After judging each resampling point by using the above steps, the color data of each resampling point corresponding to each ray can be accumulated, and based on the accumulated color data, the volume of interest is rendered to obtain the rendered target image.

[0111] In the above technical solution, by projecting the volume of interest and the resampling points onto the first plane and the second plane, and using the relationship between the projected points and the projection regions after projection, it is possible to more accurately determine whether the resampling point is located on the volume of interest, which helps to ensure the accuracy of the color data of the volume of interest determined based on each resampling point, and further helps to improve the image rendering effect of the volume of interest.

[0112] Exemplarily, in step S150, obtaining color data of the volume of interest corresponding to each ray based at least on the starting point and the ending point corresponding to each ray may include the following steps: For each ray, start resampling when the ray intersects with the starting point corresponding to the ray; end resampling when the ray meets any one of the target requirements. The target requirements include at least one of the following requirements: a first target requirement, a second target requirement, and a third target requirement. The first target requirement is that the ray intersects with the ending point corresponding to the ray. The second target requirement is that the cumulative opacity value of the resampling points passed by the ray is greater than the transparency threshold. The third target requirement includes that the ray projection distance is greater than or equal to the ray projection distance threshold. The ray projection distance is the distance between the current resampling point passed by the ray and the starting point corresponding to the ray. The ray projection distance threshold is the distance between the ending point corresponding to the ray and the starting point corresponding to the ray.

[0113] Optionally, the target requirements may include the first target requirement. In this embodiment, for any ray, when the ray intersects with the ending point corresponding to the ray, stop resampling.

[0114] Optionally, the target requirements may include the second target requirement. The color data of each resampling point includes the color value of the resampling point and the opacity of the resampling point. In this embodiment, for any ray, the opacities of the respective resampling points on the ray can be cumulatively obtained in real time to obtain the cumulative opacity value corresponding to the ray. When the cumulative opacity value is greater than or equal to the transparency threshold, stop resampling. The transparency threshold can be set as needed. For example, the transparency threshold can be in the range of [0.9, 1]. In a specific embodiment, the transparency threshold can be 0.95.

[0115] Optionally, the target requirements may include the third target requirement. Before step S150 of obtaining color data of the volume of interest corresponding to each ray based at least on the starting point and the ending point corresponding to each ray, method 100 may further include the following steps: determining the ray projection distance threshold corresponding to each ray based on the starting point and the ending point corresponding to each ray. In this embodiment, the ray projection distance n after each ray starts resampling can be obtained in real time, and the size of n and the ray projection distance threshold m can be compared. When n ≥ m, stop resampling.

[0116] Optionally, the target requirements may include multiple requirements among the first target requirement, the second target requirement, and the third target requirement. In this embodiment, when the ray meets any one of the target requirements, resampling can be ended. The judgment methods of the respective target requirements have been described in detail above and will not be elaborated here.

[0117] When any of the requirements of the above technical solution is met by the light, the resampling ends. Thereby, it is possible to avoid the color values of resampling points with little rendering value from participating in the rendering. Therefore, this solution helps to improve the resampling speed of the color data of the volume of interest, and thus helps to improve the image rendering speed of the volume of interest.

[0118] According to another aspect of the present application, there is provided a method for displaying an image of a volume of interest. Figure 11 FIG. shows a schematic flowchart of a method for displaying an image of a volume of interest according to an embodiment of the present application. As Figure 11 shown, the method 1100 may include the following steps S1110 and step S1120.

[0119] In step S1110, the rendering method 100 is used to render the volume of interest to obtain a rendered target image.

[0120] In step S1120, the target image is displayed.

[0121] According to still another aspect of the present application, there is provided an image rendering apparatus for a volume of interest. Figure 12 FIG. shows a schematic block diagram of an image rendering apparatus for a volume of interest according to an embodiment of the present application. As Figure 12 shown, the apparatus 1200 includes a first acquisition module 1210, a first determination module 1220, a generation module 1230, a second determination module 1240, a second acquisition module 1250, and a first rendering module 1260.

[0122] The first acquisition module 1210: is configured to acquire a volume of interest, and the volume of interest is determined based on a three-dimensional ultrasound image.

[0123] The first determination module 1220: is configured to determine a grid on each surface of the target area, and the target area is an area including the area where the volume of interest is located.

[0124] The generation module 1230: generates a forward depth texture map and a backward depth texture map of the target area based on the position information of the determined grid.

[0125] The second determination module 1240: is configured to determine the starting point and the ending point of each ray in at least one ray; wherein, at least one ray is used to collect color data for image rendering, and for each ray, the starting point of the ray is the intersection point of the ray and the forward depth texture map, and the ending point of the ray is the intersection point of the ray and the backward depth texture map.

[0126] The second acquisition module 1250: is configured to acquire color data of the volume of interest corresponding to each ray at least based on the starting point and the ending point corresponding to each ray.

[0127] The first rendering module 1260: Render the volume of interest based on the color data corresponding to each ray in the volume of interest to obtain a rendered target image.

[0128] Exemplarily, the volume of interest includes at least one curved surface; the target area is the area where the volume of interest is located. The first determination module 1220 includes: a first division sub-module for dividing each curved surface in the at least one curved surface into a preset number of grids; a first determination sub-module for, for each curved surface in the at least one curved surface, determining an expression function for representing the curved surface based on the type of the curved surface and the position information of a plurality of feature points on the curved surface; a second determination sub-module for, for each curved surface in the at least one curved surface, determining the position information of each grid on the curved surface based on the expression function and the positions of the preset number of grids on the curved surface.

[0129] Exemplarily, the first determination module 1220 further includes: a third determination sub-module for, for each plane in the volume of interest, determining the plane as a grid and determining the position information of the grid corresponding to the plane based on the position information of the plane.

[0130] Exemplarily, the first determination sub-module includes: a first determination unit for determining a function of a characteristic curve for representing the curved surface based on the type of the curved surface and the position information of a plurality of feature points on the curved surface, and the expression function is the function of the characteristic curve; wherein, the characteristic curve includes curves obtained by projecting the curved surface onto a first plane and a second plane respectively, and the first plane and the second plane are two mutually perpendicular shear planes on the volume of interest.

[0131] Exemplarily, the first division sub-module includes: a first projection unit for projecting the curved surface onto a third plane, where the third plane is a shear plane on the volume of interest that is perpendicular to the first plane and the second plane respectively; an equal division unit for equally dividing the projected curved surface by a preset number to obtain equally divided projected grids; a second determination unit for determining the grids on the curved surface that correspond one-to-one to the equally divided projected grids as the preset number of grids to be divided.

[0132] Exemplarily, the target area is the bounding box area of the volume of interest. The first determination module 1220 includes: a fourth determination sub-module for determining grids on each surface of the bounding box area.

[0133] Exemplarily, the volume of interest includes at least one curved surface. The apparatus 1200 further includes: a third determination module configured to, for each of the at least one curved surfaces, determine an expression function for representing the curved surface based on the type of the curved surface and the position information of a plurality of feature points on the curved surface; a fourth determination module configured to, for each of the at least one curved surfaces, determine the position information of the vertices of the curved surface based on the expression function; a fifth determination module configured to, for each of the at least one curved surfaces, determine a bounding box region based on the position information of the vertices of each surface in the volume of interest.

[0134] Exemplarily, the fourth determination sub-module includes: a third determination unit configured to, for each plane in the bounding box region, determine the plane as a grid and determine the position information of the grid corresponding to the plane based on the position information of the plane.

[0135] Exemplarily, the plurality of feature points includes at least two preset feature points and at least one adjustable feature point. The apparatus 1200 further includes: a sixth determination module configured to determine the position information of the at least one adjustable feature point in response to a position setting operation of a user.

[0136] Exemplarily, the second acquisition module 1250 includes: a resampling sub-module configured to, for each ray, perform resampling along the corresponding ray projection direction to acquire the color data corresponding to a plurality of resampled points within the bounding box region; the first rendering module 1260 includes: a determination sub-module configured to, for each resampled point among the plurality of resampled points corresponding to each ray, determine whether the resampled point is located on the volume of interest; a reset sub-module configured to, for each resampled point among the plurality of resampled points corresponding to each ray, reset the color data of the resampled point to 0 when the resampled point is not located on the volume of interest; an accumulation sub-module configured to accumulate the color data of each resampled point corresponding to each ray; a rendering sub-module configured to render the volume of interest based on the accumulated color data.

[0137] Exemplarily, the determination sub-module includes: a second projection unit configured to project the volume of interest onto a first plane and a second plane respectively to obtain a first projection image and a second projection image; the first plane and the second plane are two mutually perpendicular shear planes on the volume of interest; a third projection unit configured to project the resampled point onto the first plane and the second plane respectively; a fourth determination unit configured to determine that the resampled point is located on the volume of interest when the first projection point is located within the projection region obtained by projecting the volume of interest onto the first plane and the second projection point is located within the projection region obtained by projecting the volume of interest onto the second plane; wherein, the first projection point is the projection point of the resampled point on the first plane; the second projection point is the projection point of the resampled point on the second plane.

[0138] Exemplarily, the second acquisition module 1250 includes: a start sub-module, configured to start resampling for each ray when the ray intersects with the starting point corresponding to the ray; an end sub-module, configured to end resampling for each ray when the ray meets any one of the target requirements; wherein the target requirements include at least one of the following requirements: a first target requirement, a second target requirement, and a third target requirement; the first target requirement is that the ray intersects with the ending point corresponding to the ray; the second target requirement is that the cumulative opacity value of the resampling points passed by the ray is greater than the transparency threshold; the third target requirement includes that the ray projection distance is greater than or equal to the ray projection distance threshold; the ray projection distance is the distance between the current resampling point passed by the ray and the starting point corresponding to the ray; the ray projection distance threshold is the distance between the ending point corresponding to the ray and the starting point corresponding to the ray.

[0139] According to another aspect of the present application, an image display device for an interested volume is provided. Figure 13 The schematic block diagram of an image display device for an interested volume according to an embodiment of the present application is shown. As Figure 13 shown, the device 1300 includes: a second rendering module 1310: configured to render the interested volume by using the above-mentioned rendering method to obtain a rendered target image; a display module 1320: configured to display the target image.

[0140] According to another aspect of the present application, an electronic device is further provided. Figure 14 The schematic block diagram of an electronic device 1400 according to an embodiment of the present application is shown. As Figure 14 shown, the electronic device 1400 includes a processor 1410 and a memory 1420. Wherein, computer program instructions are stored in the memory 1420, and when the computer program instructions are run by the processor 1410, the following operations are performed: acquiring an interested volume, the interested volume being determined based on a three-dimensional ultrasound image; determining a grid on each surface of a target region, the target region being a region including the region where the interested volume is located; generating a forward depth texture map and a backward depth texture map of the target region based on the position information of the determined grid; determining the starting point and the ending point of each ray in at least one ray; wherein, at least one ray is used to collect color data for image rendering, and for each ray, the starting point of the ray is the intersection point of the ray and the forward depth texture map, and the ending point of the ray is the intersection point of the ray and the backward depth texture map; acquiring color data corresponding to each ray of the interested volume at least based on the starting point and the ending point corresponding to each ray respectively; rendering the interested volume based on the color data corresponding to each ray of the interested volume to obtain a rendered target image.

[0141] The electronic device may be, for example, any one of a desktop ultrasonic diagnostic device, a portable ultrasonic diagnostic device (such as a notebook or tablet ultrasonic diagnostic device), and an ultrasonic imaging workstation.

[0142] Exemplarily, the volume of interest includes at least one curved surface; the target region is the region where the volume of interest is located, and the steps of determining a grid on each surface of the target region, which are performed when the computer program instructions are run by the processor 1410, may include the following steps: for each curved surface in the at least one curved surface, dividing the curved surface into a preset number of grids; based on the type of the curved surface and the position information of a plurality of feature points on the curved surface, determining an expression function for representing the curved surface; and based on the expression function and the positions of the preset number of grids on the curved surface, determining the position information of each grid on the curved surface.

[0143] Exemplarily, the steps of determining a grid on each surface of the target region, which are performed when the computer program instructions are run by the processor 1410, may further include the following steps: for each plane in the volume of interest, determining the plane as a grid, and based on the position information of the plane, determining the position information of the grid corresponding to the plane.

[0144] Exemplarily, the steps of determining an expression function for representing the curved surface, which are performed when the computer program instructions are run by the processor 1410, based on the type of the curved surface and the position information of a plurality of feature points on the curved surface, may include the following steps: based on the type of the curved surface and the position information of a plurality of feature points on the curved surface, determining a function for representing a feature curve of the curved surface, and the expression function is the function of the feature curve; wherein, the feature curve includes curves obtained by projecting the curved surface onto a first plane and a second plane respectively, and the first plane and the second plane are two mutually perpendicular shear planes on the volume of interest.

[0145] Exemplarily, the steps of dividing the curved surface into a preset number of grids, which are performed when the computer program instructions are run by the processor 1410, may include the following steps: projecting the curved surface onto a third plane, where the third plane is a shear plane on the volume of interest that is perpendicular to the first plane and the second plane respectively; evenly dividing the projected curved surface by the preset number to obtain evenly divided projected grids; and determining the grids on the curved surface that correspond one-to-one to the evenly divided projected grids as the preset number of grids to be divided.

[0146] Exemplarily, the target region is the bounding box region of the volume of interest, and the steps of determining a grid on each surface of the target region, which are performed when the computer program instructions are run by the processor 1410, may include the following steps: determining a grid on each surface of the bounding box region.

[0147] Exemplarily, the volume of interest includes at least one curved surface; before the step of determining a grid on each surface of the target region when the computer program instructions are run by the processor 1410, it is further used to perform the following steps: for each curved surface in the at least one curved surface, based on the type of the curved surface and the position information of a plurality of feature points on the curved surface, determine an expression function for representing the curved surface; based on the expression function, determine the position information of the vertices of the curved surface; based on the position information of the vertices of each surface in the volume of interest, determine the bounding box region.

[0148] Exemplarily, the step of determining a grid on each surface of the bounding box region that the computer program instructions are used to perform when run by the processor 1410 may include the following steps: for each plane in the bounding box region, determine the plane as a grid, and based on the position information of the plane, determine the position information of the grid corresponding to the plane.

[0149] Exemplarily, the plurality of feature points include at least two preset feature points and at least one adjustable feature point, and the position information of the at least two preset feature points is known. Before the step of determining an expression function for representing the curved surface based on the type of the curved surface and the position information of the plurality of feature points on the curved surface when the computer program instructions are run by the processor 1410, it is further used to perform the following steps: in response to a position setting operation of the user, determine the position information of the at least one adjustable feature point.

[0150] Exemplarily, the step of obtaining color data of the volume of interest corresponding to each ray based at least on the starting point and the ending point corresponding to each ray that the computer program instructions are used to perform when run by the processor 1410 may include the following steps: for each ray, along the corresponding ray projection direction, perform resampling to collect the color data corresponding to a plurality of resampled points within the bounding box region; the step of rendering the volume of interest based on the color data of the volume of interest corresponding to each ray that the computer program instructions are used to perform when run by the processor 1410 may include the following steps: for each resampled point among the plurality of resampled points corresponding to each ray, determine whether the resampled point is located on the volume of interest; when the resampled point is not located on the volume of interest, reset the color data of the resampled point to 0; accumulate the color data of each resampled point corresponding to each ray; based on the accumulated color data, render the volume of interest.

[0151] Exemplarily, the steps for determining whether the resampled point is located on the volume of interest, which are performed when the computer program instructions are run by the processor 1410, may include the following steps: projecting the volume of interest onto a first plane and a second plane respectively to obtain a first projection image and a second projection image; the first plane and the second plane are two mutually perpendicular shear planes on the volume of interest; projecting the resampled point onto the first plane and the second plane respectively; when the first projection point is located within the projection area obtained by projecting the volume of interest onto the first plane and the second projection point is located within the projection area obtained by projecting the volume of interest onto the second plane, determining that the resampled point is located on the volume of interest; wherein, the first projection point is the projection point of the resampled point on the first plane; the second projection point is the projection point of the resampled point on the second plane.

[0152] Exemplarily, the steps for obtaining color data of the volume of interest corresponding to each ray at least based on the starting point and the ending point corresponding to each ray, which are performed when the computer program instructions are run by the processor 1410, may include the following steps: for each ray, starting resampling when the ray intersects with the starting point corresponding to the ray; ending resampling when the ray meets any one of the requirements in the target requirements; wherein, the target requirements include at least one of the following requirements: a first target requirement, a second target requirement, and a third target requirement; the first target requirement is that the ray intersects with the ending point corresponding to the ray; the second target requirement is that the cumulative opacity value of the resampled points passed by the ray is greater than the transparency threshold; the third target requirement includes that the ray projection distance is greater than or equal to the ray projection distance threshold; the ray projection distance is the distance between the current resampled point passed by the ray and the starting point corresponding to the ray; the ray projection distance threshold is the distance between the ending point corresponding to the ray and the starting point corresponding to the ray.

[0153] According to another aspect of the present application, an electronic device is further provided. The electronic device includes a processor and a memory. Among them, computer program instructions are stored in the memory, and when the computer program instructions are run by the processor, they are used to perform the following operations: rendering the volume of interest by using the above rendering method to obtain a rendered target image; displaying the target image.

[0154] The electronic device may be, for example, any one of a desktop ultrasonic diagnostic device, a portable ultrasonic diagnostic device (such as a notebook or tablet ultrasonic diagnostic device), and an ultrasonic imaging workstation.

[0155] According to another aspect of the present application, a storage medium is further provided. Program instructions are stored on the storage medium, and when the program instructions are running, they are used to perform the following operations: obtaining a volume of interest, which is determined based on a three-dimensional ultrasound image; determining a grid on each surface of a target region, where the target region is a region including the region where the volume of interest is located; generating a forward depth texture map and a backward depth texture map of the target region based on the position information of the determined grid; determining the starting point and the ending point of each ray in at least one ray; where at least one ray is used to collect color data for image rendering, for each ray, the starting point of the ray is the intersection point of the ray and the forward depth texture map, and the ending point of the ray is the intersection point of the ray and the backward depth texture map; obtaining the color data corresponding to each ray of the volume of interest at least based on the starting point and the ending point corresponding to each ray respectively; rendering the volume of interest based on the color data corresponding to each ray of the volume of interest to obtain a rendered target image.

[0156] The storage medium may for example include a storage component of a tablet computer, a hard disk of a personal computer, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disc read-only memory (CD-ROM), a USB memory, or any combination of the above storage media. The computer-readable storage medium may be any combination of one or more computer-readable storage media.

[0157] Exemplarily, the volume of interest includes at least one curved surface; the target region is the region where the volume of interest is located, and the step of determining a grid on each surface of the target region that the program instructions are used to perform when running may include the following steps: for each curved surface in the at least one curved surface, dividing the curved surface into a preset number of grids; determining an expression function for representing the curved surface based on the type of the curved surface and the position information of a plurality of feature points on the curved surface; determining the position information of each grid on the curved surface based on the expression function and the positions of the preset number of grids on the curved surface.

[0158] Exemplarily, the step of determining a grid on each surface of the target region that the program instructions are used to perform when running may further include the following steps: for each plane in the volume of interest, determining the plane as a grid, and determining the position information of the grid corresponding to the plane based on the position information of the plane.

[0159] Exemplarily, the steps for determining an expression function for representing the surface based on the type of the surface and the position information of a plurality of feature points on the surface, which the program instructions are used to execute during runtime, may include the following steps: determining a function for representing a feature curve of the surface based on the type of the surface and the position information of the plurality of feature points on the surface, where the expression function is the function of the feature curve; wherein, the feature curve includes curves obtained by projecting the surface onto a first plane and a second plane respectively, and the first plane and the second plane are two mutually perpendicular shear planes on the volume of interest.

[0160] Exemplarily, the steps for dividing the surface into a preset number of grids, which the program instructions are used to execute during runtime, may include the following steps: projecting the surface onto a third plane, where the third plane is a shear plane on the volume of interest that is perpendicular to the first plane and the second plane respectively; evenly dividing the projected surface into the preset number to obtain evenly divided projected grids; determining the grids on the surface that correspond one-to-one with the evenly divided projected grids as the preset number of grids to be divided.

[0161] Exemplarily, the target region is the bounding box region of the volume of interest, and the steps for determining grids on each surface of the target region, which the program instructions are used to execute during runtime, may include the following steps: determining grids on each surface of the bounding box region.

[0162] Exemplarily, the volume of interest includes at least one surface; before the program instructions execute the steps for determining grids on each surface of the target region during runtime, they are further used to execute the following steps: for each surface in the at least one surface, determining an expression function for representing the surface based on the type of the surface and the position information of a plurality of feature points on the surface; determining the position information of the vertices of the surface based on the expression function; determining the bounding box region based on the position information of the vertices of each surface in the volume of interest.

[0163] Exemplarily, the steps for determining grids on each surface of the bounding box region, which the program instructions are used to execute during runtime, may include the following steps: for each plane in the bounding box region, determining the plane as a grid, and determining the position information of the grid corresponding to the plane based on the position information of the plane.

[0164] Exemplarily, the plurality of feature points includes at least two preset feature points and at least one adjustable feature point, and the position information of the at least two preset feature points is known. Before the program instructions execute the steps for determining an expression function for representing the surface based on the type of the surface and the position information of the plurality of feature points on the surface during runtime, they are further used to execute the following steps: in response to a position setting operation of the user, determining the position information of the at least one adjustable feature point.

[0165] Exemplarily, the step of obtaining color data of the volume of interest corresponding to each ray based at least on the starting point and the ending point corresponding to each ray when the program instruction runs may include the following steps: for each ray, along the corresponding ray projection direction, perform resampling to collect the color data corresponding to each of the multiple resampled points within the bounding box area; the step of rendering the volume of interest based on the color data of the volume of interest corresponding to each ray when the program instruction runs may include the following steps: for each resampled point among the multiple resampled points corresponding to each ray, determine whether the resampled point is located on the volume of interest; when the resampled point is not located on the volume of interest, reset the color data of the resampled point to 0; accumulate the color data of each resampled point corresponding to each ray; based on the accumulated color data, render the volume of interest.

[0166] Exemplarily, the step of determining whether the resampled point is located on the volume of interest when the program instruction runs may include the following steps: project the volume of interest onto a first plane and a second plane respectively to obtain a first projection image and a second projection image; the first plane and the second plane are two mutually perpendicular shear planes on the volume of interest; project the resampled point onto the first plane and the second plane respectively; when the first projection point is located within the projection area obtained by projecting the volume of interest onto the first plane and the second projection point is located within the projection area obtained by projecting the volume of interest onto the second plane, determine that the resampled point is located on the volume of interest; wherein, the first projection point is the projection point of the resampled point on the first plane; the second projection point is the projection point of the resampled point on the second plane.

[0167] Exemplarily, the step of obtaining color data of the volume of interest corresponding to each ray based at least on the starting point and the ending point corresponding to each ray when the program instruction runs may include the following steps: for each ray, start resampling when the ray intersects with the starting point corresponding to the ray; end resampling when the ray meets any one of the target requirements; wherein, the target requirements include at least one of the following requirements: a first target requirement, a second target requirement, and a third target requirement; the first target requirement is that the ray intersects with the ending point corresponding to the ray; the second target requirement is that the cumulative opacity value of the resampled points passed by the ray is greater than the transparency threshold; the third target requirement includes that the ray projection distance is greater than or equal to the ray projection distance threshold; the ray projection distance is the distance between the current resampled point passed by the ray and the starting point corresponding to the ray; the ray projection distance threshold is the distance between the ending point corresponding to the ray and the starting point corresponding to the ray.

[0168] According to another aspect of the present application, a storage medium is also provided. Program instructions are stored on the storage medium, and when the program instructions are running, they are used to perform the following operations: rendering the volume of interest by using the above-mentioned rendering method to obtain a rendered target image; and displaying the target image.

[0169] The storage medium may include, for example, a storage component of a tablet computer, a hard disk of a personal computer, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disc read-only memory (CD-ROM), a USB memory, or any combination of the above storage media. The computer-readable storage medium may be any combination of one or more computer-readable storage media.

[0170] Those of ordinary skill in the art can understand the specific implementation solutions of the above-mentioned image display method for the volume of interest, the image rendering device for the volume of interest, the image display device for the volume of interest, the electronic device, and the storage medium by reading the relevant descriptions of the above-mentioned image rendering method for the volume of interest. For the sake of brevity, they will not be elaborated here.

[0171] Although example embodiments have been described herein with reference to the accompanying drawings, it should be understood that the above example embodiments are merely exemplary and are not intended to limit the scope of the present application. Those of ordinary skill in the art can make various changes and modifications therein without departing from the scope and spirit of the present application. All such changes and modifications are intended to be included within the scope of the present application as claimed by the appended claims.

[0172] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, or by a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. A professional technician can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0173] In several embodiments provided by the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed.

[0174] In the specification provided herein, a number of specific details are set forth. It will be appreciated, however, that embodiments of the present application may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail so as not to obscure an understanding of the present specification.

[0175] Similarly, it should be understood that, in order to streamline the present application and assist in understanding one or more of the various aspects of the application, in the description of the exemplary embodiments of the present application, the various features of the present application are sometimes grouped together in a single embodiment, figure, or description thereof. However, the methods of the present application should not be construed as reflecting an intention that the claimed present application requires more features than are expressly recited in each claim. Rather, as reflected by the corresponding claims, the point of the application is that the corresponding technical problem can be solved with features less than all the features of a single disclosed embodiment. Accordingly, the claims following the detailed description are hereby expressly incorporated into the detailed description, with each claim standing on its own as a separate embodiment of the present application.

[0176] Those skilled in the art will appreciate that, except where features are mutually exclusive, any combination may be employed of all the features disclosed in this specification (including the accompanying claims, abstract and drawings), as well as of all the processes or units of any method or apparatus so disclosed. Each feature disclosed in this specification (including the accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.

[0177] In addition, those skilled in the art will appreciate that, although some embodiments herein include certain features included in other embodiments but not others, combinations of features of different embodiments mean within the scope of the present application and form different embodiments. For example, in the claims, any one of the claimed embodiments may be used in any combination.

[0178] Each component embodiment of the present application can be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art should understand that a microprocessor or a digital signal processor (DSP) can be used in practice to implement some or all of the functions of some modules in the image rendering device of the volume of interest, the image display device of the volume of interest, and the electronic device according to the embodiments of the present application. The present application can also be implemented as a device program (for example, a computer program and a computer program product) for executing part or all of the methods described herein. Such a program implementing the present application can be stored on a computer-readable medium, or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0179] It should be noted that the above embodiments illustrate the present application rather than limit the present application, and those skilled in the art can design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in the claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present application can be implemented by means of hardware including several different elements and by means of a suitably programmed computer. In the unit claims listing several devices, several of these devices can be embodied by the same item of hardware. The use of the words first, second, and third, etc. does not denote any order. These words can be interpreted as names.

[0180] The above is only the specific implementation manner of the present application or the description of the specific implementation manner, and the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present application, and all of them should be covered by the protection scope of the present application. The protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A method for rendering an image of a volume of interest, characterized in that, it includes: Obtain a volume of interest, where the volume of interest is determined based on a three-dimensional ultrasound image; Determine a grid on each surface of a target area, where the target area is an area including the area where the volume of interest is located; Generate a forward depth texture map and a backward depth texture map of the target area based on the position information of the determined grid; Determine the starting point and ending point of each ray in at least one ray; where, the at least one ray is used to collect color data for image rendering, and for each ray, the starting point of the ray is the intersection point of the ray and the forward depth texture map, and the ending point of the ray is the intersection point of the ray and the backward depth texture map; Obtain the color data corresponding to each ray of the volume of interest at least based on the starting point and ending point corresponding to each ray; Render the volume of interest based on the color data corresponding to each ray of the volume of interest to obtain a rendered target image.

2. The rendering method according to claim 1, characterized in that, the volume of interest includes at least one curved surface; the target area is the area where the volume of interest is located, and determining a grid on each surface of the target area includes: For each curved surface in the at least one curved surface, Divide the curved surface into a preset number of grids; Determine an expression function for representing the curved surface based on the type of the curved surface and the position information of multiple feature points on the curved surface; Determine the position information of each grid on the curved surface based on the expression function and the positions of the preset number of grids on the curved surface.

3. The rendering method according to claim 2, characterized in that, determining a grid on each surface of the target area further includes: For each plane in the volume of interest, determine the plane as a grid, and determine the position information of the grid corresponding to the plane based on the position information of the plane.

4. The rendering method according to claim 2, characterized in that, the determining an expression function for representing the curved surface based on the type of the curved surface and the position information of multiple feature points on the curved surface includes: Determine a function for representing a feature curve of the curved surface based on the type of the curved surface and the position information of multiple feature points on the curved surface, and the expression function is the function of the feature curve; wherein, the feature curve includes curves obtained by projecting the curved surface onto a first plane and a second plane respectively, and the first plane and the second plane are two mutually perpendicular shear planes on the volume of interest.

5. The rendering method according to claim 4, characterized in that, the dividing the curved surface into a preset number of grids includes: Project the curved surface onto a third plane, where the third plane is a shear plane on the volume of interest that is perpendicular to the first plane and the second plane respectively; Divide the projected curved surface evenly according to the preset number to obtain evenly divided projected grids; Determine the grids on the curved surface that correspond one-to-one with the evenly divided projected grids as the preset number of grids divided.

6. The rendering method according to claim 1, wherein, the target area is the bounding box area of the volume of interest, and determining a grid on each surface of the target area includes: determining a grid on each surface of the bounding box area.

7. The rendering method according to claim 6, wherein, the volume of interest includes at least one curved surface; before determining a grid on each surface of the target area, the method further includes: for each of the at least one curved surface, determining an expression function for representing the curved surface based on the type of the curved surface and the position information of a plurality of feature points on the curved surface; determining the position information of the vertices of the curved surface based on the expression function; determining the bounding box area based on the position information of the vertices of each surface in the volume of interest.

8. The rendering method according to claim 6, wherein, determining a grid on each surface of the bounding box area includes: for each plane in the bounding box area, determining the plane as a grid and determining the position information of the grid corresponding to the plane based on the position information of the plane.

9. The rendering method according to claim 2, wherein, the plurality of feature points include at least two preset feature points and at least one adjustable feature point, and the position information of the at least two preset feature points is known, before determining an expression function for representing the curved surface based on the type of the curved surface and the position information of a plurality of feature points on the curved surface, the method further includes: responding to a position setting operation of the user to determine the position information of the at least one adjustable feature point.

10. The rendering method according to claim 6, wherein, obtaining color data of the volume of interest corresponding to each ray based on at least the starting point and the ending point corresponding to each ray respectively includes: for each ray, performing resampling along the corresponding ray projection direction to collect color data corresponding to a plurality of resampled points within the bounding box area; rendering the volume of interest based on the color data of the volume of interest corresponding to each ray respectively includes: for each resampled point among the plurality of resampled points corresponding to each ray, judging whether the resampled point is located on the volume of interest; when the resampled point is not located on the volume of interest, resetting the color data of the resampled point to 0; accumulating the color data of each resampled point corresponding to each ray; rendering the volume of interest based on the accumulated color data.

11. The rendering method according to claim 10, wherein, judging whether the resampled point is located on the volume of interest includes: projecting the volume of interest onto a first plane and a second plane respectively; the first plane and the second plane are two mutually perpendicular shear planes on the volume of interest; projecting the resampled point onto the first plane and the second plane respectively; When the first projection point is within the projection area obtained by projecting the volume of interest onto the first plane and the second projection point is within the projection area obtained by projecting the volume of interest onto the second plane, it is determined that the resampled point is on the volume of interest; wherein, the first projection point is the projection point of the resampled point on the first plane; the second projection point is the projection point of the resampled point on the second plane.

12. The rendering method according to any one of claims 1-11, characterized in that, the obtaining the color data of the volume of interest corresponding to each ray based at least on the starting point and the ending point corresponding to each ray respectively includes: for each ray, starting resampling when the ray intersects with the starting point corresponding to the ray; ending resampling when the ray meets any one of the requirements in the target requirements; wherein, the target requirements include at least one of the following requirements: a first target requirement, a second target requirement, and a third target requirement; the first target requirement is that the ray intersects with the ending point corresponding to the ray; the second target requirement is that the cumulative opacity value of the resampled points passed by the ray is greater than the transparency threshold; the third target requirement includes that the ray projection distance is greater than or equal to the ray projection distance threshold; the ray projection distance is the distance between the current resampled point passed by the ray and the starting point corresponding to the ray; the ray projection distance threshold is the distance between the ending point corresponding to the ray and the starting point corresponding to the ray.

13. An image display method for a volume of interest, characterized in that, including: rendering the volume of interest by using the rendering method according to any one of claims 1-12 to obtain a rendered target image; displaying the target image.

14. An image rendering device for a volume of interest, characterized in that, including: a first obtaining module: for obtaining a volume of interest, the volume of interest being determined based on a three-dimensional ultrasound image; a first determining module: for determining a mesh on each surface of a target area, the target area being an area including the area where the volume of interest is located; a generating module: generating a forward depth texture map and a backward depth texture map of the target area based on the position information of the determined mesh; a second determining module: for determining the starting point and the ending point of each ray in at least one ray based on the forward depth texture map and the backward depth texture map; wherein, the at least one ray is used for collecting color data used for image rendering, and for each ray, the starting point of the ray is the intersection point of the ray and the forward depth texture map, and the ending point of the ray is the intersection point of the ray and the backward depth texture map; a second obtaining module: for obtaining the color data of the volume of interest corresponding to each ray based at least on the starting point and the ending point corresponding to each ray respectively; a first rendering module: rendering the volume of interest based on the color data of the volume of interest corresponding to each ray to obtain a rendered target image.

15. An image display device for a volume of interest, characterized in that, including: Second rendering module: configured to render the volume of interest by using the rendering method according to any one of claims 1-12 to obtain a rendered target image; Display module: configured to display the target image.

16. An electronic device, characterized in that, it includes a processor and a memory, and computer program instructions are stored in the memory, and when the computer program instructions are run by the processor, they are used to execute the rendering method according to any one of claims 1-12 or the display method according to claim 13.

17. A storage medium, characterized in that, program instructions are stored on the storage medium, and when the program instructions are run, they are used to execute the rendering method according to any one of claims 1-12 or the display method according to claim 13.

Images