Transfer function generation method and device, image rendering method and device and computer equipment

Abstract

The invention relates to a transfer function generation method and device, an image rendering method and device and computer equipment. The method comprises the following steps: acquiring a 2D reference rendering image of a visual image data target part comprising each pixel point, acquiring an organization label corresponding to each pixel point in the 2D reference rendering image, and generating a transfer function of the 2D reference rendering image through machine learning according to the visual image data and the organization label of each pixel point. According to the method, the user operation is simplified, the user does not need to manually configure pixel points one by one, the 2D reference rendering graph is directly adopted as the reference sample, the transfer function between the volume data and the visual image data is autonomously generated, the subjective influence of the user is reduced, the accuracy of the generated transfer function is improved, and the user experience is improved. The matching degree between the rendered image rendered based on the transfer function and the rendering effect expected by the user is correspondingly improved, so that the actual rendering effect is improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a transfer function generation method, an image rendering method, an apparatus and a computer device. Background Art

[0002] In the medical field, image rendering refers to the process of converting three-dimensional medical volume data (such as CT values) into visual image data (such as color and transparency) using transfer functions, thereby rendering the three-dimensional medical volume data onto a two-dimensional screen. The transfer function is used to characterize the conversion relationship between volume data and image data.

[0003] In the related art, a user may manually configure a transfer function in a rendering configuration interface to implement image rendering using the transfer function manually configured by the user.

[0004] However, affected by the user's subjective operation, the image rendered in the related art is quite different from the expectation, thus reducing the actual rendering effect. Summary of the invention

[0005] Based on this, it is necessary to provide a transfer function generation method, image rendering method, device, computer equipment, storage medium and program product to address the above technical problems.

[0006] In a first aspect, the present application provides a transfer function generation method, comprising:

[0007] Obtain a 2D reference rendering of the target part; the 2D reference rendering includes visual image data of each pixel point;

[0008] Get the tissue label corresponding to each pixel in the 2D reference rendering;

[0009] Based on the visualized image data and organizational labels of each pixel, a transfer function of a 2D reference rendering is generated through machine learning.

[0010] In one of the embodiments, the tissue label corresponding to each pixel in the 2D reference rendering is obtained through machine learning.

[0011] In one embodiment, the method further comprises:

[0012] receiving an adjustment operation for a transfer function;

[0013] In response to the adjustment operation, the transfer function is adjusted to obtain an adjusted transfer function.

[0014] In a second aspect, the present application also provides an image rendering method, comprising:

[0015] Obtain a 2D reference rendering of the target part; the 2D reference rendering includes visual image data of each pixel point;

[0016] Get the tissue label corresponding to each pixel in the 2D reference rendering;

[0017] Generate a transfer function of a 2D reference rendering using machine learning based on the visualized image data and tissue labels of each pixel.

[0018] The to-be-rendered volume data of the target part is rendered at least according to the transfer function to obtain a target rendering image.

[0019] In a third aspect, the present application also provides a transfer function generating device, comprising:

[0020] A reference rendering acquisition module is used to acquire a 2D reference rendering of a target part; the 2D reference rendering includes visual image data of each pixel point;

[0021] A label acquisition module, used to obtain the tissue label corresponding to each pixel in the 2D reference rendering;

[0022] The function determination module is used to generate a transfer function of a 2D reference rendering image through machine learning based on the visualized image data and organizational labels of each pixel.

[0023] In a fourth aspect, the present application further provides an image rendering device, comprising:

[0024] A reference rendering acquisition module is used to acquire a 2D reference rendering of a target part; visualization image data of each pixel point in the 2D reference rendering;

[0025] A label acquisition module, used to obtain the tissue label corresponding to each pixel in the 2D reference rendering;

[0026] The function determination module generates the transfer function of the 2D reference rendering image through machine learning based on the visualized image data and tissue labels of each pixel;

[0027] The rendering processing module is used to perform rendering processing on the to-be-rendered volume data of the target part at least according to the transfer function to obtain a target rendering image.

[0028] In a fifth aspect, the present application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any of the above methods when executing the computer program.

[0029] In a sixth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of any of the above methods when the computer program is executed by a processor.

[0030] In a seventh aspect, the present application also provides a computer program product, comprising a computer program, which implements the steps of any of the above methods when executed by a processor.

[0031] In the above transfer function generation method, image rendering method, device, computer equipment, storage medium and program product, a 2D reference rendering image of the target part including the visualized image data of each pixel is obtained, and the tissue label corresponding to each pixel in the 2D reference rendering image is obtained. According to the visualized image data and tissue label of each pixel, a transfer function of the 2D reference rendering image is generated through machine learning. In the above method, user operation is simplified, and the user does not need to manually configure each pixel. The 2D reference rendering image is directly used as a reference sample to autonomously generate a transfer function between the volume data and the visualized image data, thereby reducing the subjective influence of the user, improving the accuracy of the generated transfer function, and correspondingly improving the matching degree between the rendering image rendered based on the transfer function and the rendering effect expected by the user, thereby improving the actual rendering effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] FIG1 is a diagram showing the internal structure of a computer device in one embodiment;

[0033] FIG2 is a schematic flow chart of a transfer function generation method in one embodiment;

[0034] FIG3 is a schematic diagram showing a change from a reference effect diagram to a reference label diagram in one embodiment;

[0035] FIG4 is a schematic diagram of a standard truth value diagram in one embodiment;

[0036] FIG5 is a schematic flow chart of a transfer function generating method in another embodiment;

[0037] FIG6 is a schematic diagram of a flow chart of an image rendering method in one embodiment;

[0038] FIG7 is a schematic diagram of a flow chart of an image rendering method in another embodiment;

[0039] FIG8 is a diagrammatic illustration of an image rendering process in one embodiment;

[0040] FIG9 is a block diagram of a transfer function generating device in one embodiment;

[0041] FIG. 10 is a structural block diagram of an image rendering device in one embodiment. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0043] The transfer function generation method / image rendering method provided in the embodiment of the present application can be applied to the computer device shown in Figure 1. The computer device can be a terminal, including a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a transfer function generation method / image rendering method is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a key, trackball or touchpad set on the housing of the computer device, or an external keyboard, touchpad or mouse.

[0044] Exemplarily, the computer device may be an imaging device / system that can perform imaging based on volume data, such as a computed tomography (CT) system or a magnetic resonance (MR) system, or may be a processing device / system that can communicate with the above-mentioned imaging device / system.

[0045] Those skilled in the art will understand that the structure shown in FIG. 1 is merely a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0046] In one embodiment, as shown in FIG. 2 , a transfer function generation method is provided, which is described by taking the method applied to the computer device in FIG. 1 as an example, and includes the following steps:

[0047] S210, obtaining a 2D reference rendering of the target part; the 2D reference rendering includes visualized image data of each pixel point.

[0048] Among them, the 2D reference rendering is a 2D image whose rendering visual effect meets the user's needs. The visual image data is used to characterize the visual characteristics of the pixel points. Exemplarily, the visual image data may include at least one of color, grayscale value, and transparency. In the case where the 2D reference rendering is an RGB image, the color of the pixel point can be represented by the red (Red, R) component value, the green component value (Green, G) and the blue component value (Blue, B). In the case where the 2D reference rendering is an HSV image, the color of the pixel point can be represented by the hue (Hue, H), saturation (Saturation, S) and hue (Value, V).

[0049] Optionally, the computer device may obtain a 2D image imported by the user in response to an import operation of the user, and use the 2D image as a 2D reference rendering, or may receive a 2D image sent by an external device as a 2D reference rendering.

[0050] S220 , obtaining a tissue label corresponding to each pixel in the 2D reference rendering.

[0051] The tissue label of a pixel is used to characterize the type of tissue region to which the pixel belongs.

[0052] The 2D reference rendering may be a body tissue image drawn by art software, or a body tissue image rendered based on a body tissue image in DICOM format. Accordingly, the computer device may directly determine the tissue label corresponding to each pixel based on the 2D reference rendering, or may determine the tissue label corresponding to each pixel based on the original image forming the 2D reference rendering. As mentioned above, the original image forming the 2D reference rendering may be a 2D / 3D medical image, such as a CT image, or a nuclear magnetic resonance MR image.

[0053] For example, the computer device can receive a user's manual division operation for different types of tissue regions in the reference rendering, or a rule for tissue regions can be pre-defined, and the computer device can automatically divide the tissue regions based on the rule. A large number of image processing software already available in the industry can realize the automated segmentation operation of the tissue region.

[0054] Alternatively, the computer device may also perform tissue segmentation on the reference rendering through machine learning, and add corresponding tissue labels to the pixels in each type of tissue area to obtain the tissue labels corresponding to each pixel in the 2D reference rendering. The machine learning may be deep learning. Typically, the tissue labels of each pixel in the same type of tissue area are the same.

[0055] S230: Generate a transfer function of a 2D reference rendering image through machine learning based on the visualized image data and tissue labels of each pixel.

[0056] After the computer device obtains the visualized image data and tissue labels of each pixel in the 2D reference rendering, it can obtain the transfer function of the above 2D reference rendering through machine learning, such as deep learning, where the transfer function expresses the relationship between the volume data of different pixel points and the visualized image data.

[0057] Optionally, after obtaining the transfer function of the 2D reference rendering, a save file of the transfer function may be generated for subsequent rendering calls. Exemplarily, a save file of the transfer function in xml or json format may be generated.

[0058] In the embodiment of the present application, a 2D reference rendering of the target part including the visualized image data of each pixel is obtained, and the tissue label corresponding to each pixel in the 2D reference rendering is obtained. According to the visualized image data and tissue label of each pixel, a transfer function of the 2D reference rendering is generated through machine learning. In the above method, the user operation is simplified, and the user does not need to manually configure each pixel. The 2D reference rendering is directly used as a reference sample to autonomously generate a transfer function between the volume data and the visualized image data, which reduces the subjective influence of the user, improves the accuracy of the generated transfer function, and correspondingly improves the matching degree between the rendering obtained by rendering based on the transfer function and the rendering effect expected by the user, thereby improving the actual rendering effect.

[0059] To improve the efficiency and accuracy of tissue segmentation, in one embodiment, the tissue label corresponding to each pixel point in the 2D reference rendering image is obtained in the above S220 through machine learning.

[0060] The computer device is equipped with a tissue segmentation model pre-trained based on machine learning. The computer device can input the acquired 2D reference rendering image into the tissue segmentation model to obtain the tissue label corresponding to each pixel point in the 2D reference rendering image.

[0061] The tissue segmentation model is a pre-trained segmentation network model that can be used to identify the type and location of each tissue region in an image. For example, the tissue segmentation model can be trained using a U-net, a dense convolutional network (DenseNet), etc. as a backbone network structure.

[0062] For example, the tissue label corresponding to each pixel in the 2D reference rendering can be embodied in the form of a reference label map formed by adding tissue labels to each pixel in the 2D reference rendering. As shown in FIG3 , the target part is the chest and abdomen of the human body. The left figure shows the input segmentation network model N seg 2D reference rendering of I ref, the right figure shows the segmentation network model N seg Output reference label image I label I ref It can be a common image format such as jpeg or png. ref The data structure is a tensor . , R represents array / matrix, N is the number of images, C is the number of channels, usually C=4 (color RGB+transparency A), H and W are the height and width of the image respectively. label The numerical value in represents the label value, where the label value 1 represents the heart, 2 represents the kidney, 3 represents the spleen, and 0 represents no tissue at that location. label The data structure is a tensor . , C L =C+1, the additional channel 1 is the organization label.

[0063] In an optional embodiment, the computer device may also input the acquired 2D reference rendering image into a preset network model for tissue segmentation, obtain the position and type of each tissue area in the 2D reference rendering image, form a tissue segmentation map of the 2D reference rendering image, and then determine the tissue label of each tissue area in the tissue segmentation map based on the correspondence between the pre-stored type and the tissue label, and add the corresponding type of tissue label to the pixel points of each tissue area in the tissue segmentation map to obtain the tissue label corresponding to each pixel point in the 2D reference rendering map.

[0064] Exemplarily, for example, the tissue segmentation map includes type A and type B, and the correspondence between the type and the tissue label includes type A corresponding to tissue label 1, and type B corresponding to tissue label 2. The computer device adds tissue label 1 to the pixel points in the tissue area corresponding to type A in the tissue segmentation map, and adds tissue label 2 to the pixel points in the tissue area corresponding to type B in the tissue segmentation map, thereby forming a reference label map corresponding to the 2D reference rendering map.

[0065] Next, we briefly introduce the tissue segmentation model N seg The training process:

[0066] The rendered images I1 of multiple frames of the target part and the standard truth images I1 corresponding to each frame I1 can be prepared in advance. n As training samples. Standard truth image I n is a priori image for correctly labeling the rendered image I1 with tissue labels. For example, FIG4 shows a standard truth image of the target part being the chest and abdomen of a human body.

[0067] The computer device can input I1 into the initial segmentation model N s , by N s Output label image I1', and based on I1' and In Loss function adjustment between N s The model parameters in, and then input the new I1 to get the new I1', and adjust N based on the new loss function s The model parameters in this cycle are repeated until the loss function reaches the minimum, and the tissue segmentation model N is obtained. seg .

[0068] Exemplarily, the cross entropy L is used as the training tissue segmentation model N seg The loss function of L is calculated as follows:

[0069]

[0070] Among them, M is the total number of all pixels in the image, K is the category of the tissue area corresponding to the pixel (including no tissue), that is, the tissue label. represents the sign function (0 or 1), which takes 1 if the correct tissue label of pixel i is equal to l, otherwise it takes 0. The probability that pixel i belongs to category k can be calculated by the following softmax function.

[0071]

[0072] It should be noted that, in all label maps, there may be no tissue label in the non-tissue area, and a preset tissue label representation, such as 0, may be used.

[0073] In the embodiment of the present application, the tissue labels corresponding to each pixel in the 2D reference rendering are obtained by machine learning. In the above method, different types of tissue areas in the reference rendering can be accurately identified and located based on machine learning, thereby improving the efficiency and accuracy of obtaining tissue labels.

[0074] The transfer function can be obtained based on a function prediction model obtained through machine learning training. Based on this, in one embodiment, the above S230, based on the visualized image data and tissue labels of each pixel point, generates a transfer function of the 2D reference rendering through machine learning, including:

[0075] The visualized image data and tissue labels of each pixel are input into the function prediction model to obtain the transfer function output by the function prediction model and used as the transfer function of the 2D reference rendering.

[0076] Among them, the function prediction model can be used to infer the transfer function representing the conversion relationship between volume data and visual image data through implicit expression layer by layer based on the visual image data and tissue labels of each pixel in the image. Exemplarily, the function prediction model can be trained using a U-net, an autoencoder, or a deep residual network (ResNet) as the backbone network structure.

[0077] Optionally, after adding tissue labels to the 2D reference rendering, a reference label map including the visualization image data of each pixel and the tissue label can be obtained. The computer device can input the reference label map into its own function prediction model to obtain the transfer function of the 2D reference rendering output by the function prediction model.

[0078] For example, the function prediction model N tf Input That is, the tissue segmentation model N seg Output (i.e., the organizational label of each pixel) and a 2D reference rendering of the visualization image data of each pixel. ref ,Right now Function prediction model N tf Output is the transfer function. , Represents the visualization image data corresponding to each volume data. The data structure of the transfer function is a one-dimensional or two-dimensional texture, which can be a tensor of (V, C) or (V, G, C), where V represents the numerical intensity range, v∈V, C represents the number of channels, and G represents the data gradient.

[0079] The transfer function, i.e., the tensor (V, C), can be represented by a correspondence table between different volume data and visual image data. Taking the volume data CT value data intensity range V[-1024, 2764] (which can be mapped to [1, 3789] in the figure), and the visual image data including R, G, B and transparency (A) as an example, the transfer function can be specifically represented by the following table:

[0080]

[0081] After obtaining the above transfer function, when performing image rendering, the computer device can use the transfer function to determine the visualization image data (such as R, G, B, A) corresponding to each volume data (such as CT value), thereby forming a rendered image.

[0082] In one of the embodiments, sub-transfer functions corresponding to different tissue labels may also be obtained based on machine learning.

[0083] Optionally, the computer device inputs the visualized image data corresponding to the same tissue label in the 2D reference rendered image into a function prediction model obtained based on machine learning, and the function prediction model outputs a sub-transfer function corresponding to the same tissue label.

[0084] For example, the reference label diagram 1 shown in FIG. 3 label , the computer device can obtain I label Visual image data of all pixels corresponding to label value 0 in I label The visualized image data of all pixels corresponding to the label value 1 in I, ..., and the obtained I label The visualized image data of all pixels corresponding to the label value 5 are input into the function prediction model to obtain the sub-transfer functions corresponding to the label values ​​0 / 1 / 2 / 3 / 4 / 5 respectively.

[0085] In the embodiment of the present application, sub-transfer functions corresponding to different tissue labels are obtained based on machine learning. In the above method, corresponding sub-transfer functions are determined for different types of tissue regions in the whole image, and different types of tissue regions can be rendered independently during subsequent rendering without affecting each other, so that different types of tissue regions in the whole image can present different visualization effects, thereby improving rendering diversity.

[0086] In one embodiment, as shown in FIG5 , the method further includes:

[0087] S510: Receive an adjustment operation on a transfer function.

[0088] The adjustment operation is used to further adjust the obtained transfer function to change the visualization image data corresponding to the partial volume data.

[0089] Optionally, after obtaining the transfer function of the 2D reference rendering image, the computer device may display the transfer function in a graphical form, and receive an adjustment operation of the transfer function by the user on a display interface of the transfer function.

[0090] S520: In response to the adjustment operation, adjust the transfer function to obtain an adjusted transfer function.

[0091] Optionally, after receiving the user's adjustment operation on the transfer function, the computer device receives the corresponding relationship between the adjusted volume data and the visualization image data input by the user in response to the adjustment operation, and obtains the adjusted transfer function.

[0092] In the embodiment of the present application, an adjustment operation for the transfer function is received, and in response to the adjustment operation, the transfer function is adjusted to obtain the adjusted transfer function. In the above method, after the transfer function is obtained, the transfer function can also be adjusted based on the actual needs of the user, thereby improving the flexibility and convenience of the transfer function adjustment.

[0093] Based on the above transfer function generation method, in one embodiment, as shown in FIG6 , an image rendering method is also provided, comprising the following steps:

[0094] S610, obtaining a 2D reference rendering of the target part; the 2D reference rendering includes visualized image data of each pixel point.

[0095] S620, obtaining a tissue label corresponding to each pixel in the 2D reference rendering;

[0096] S630: Generate a transfer function of a 2D reference rendering image through machine learning based on the visualized image data and the tissue label of each pixel.

[0097] The process of generating the transfer function of the reference rendering image is the same as the embodiment of the transfer function generation method described above. The specific process can be referred to the embodiment described above and will not be described again here.

[0098] S640: Render the to-be-rendered volume data of the target part at least according to the transfer function to obtain a to-be-rendered target image.

[0099] The volume data to be rendered of the target part is a physical quantity obtained by scanning the target part with radiation, for example, a CT value obtained by scanning the target part with X-rays, or an MRI signal intensity value obtained by performing MRI on the target part.

[0100] Optionally, the computer device may scan the target part to obtain the volume data to be rendered, or directly obtain the volume data to be rendered of the target part obtained by scanning the scanning system. After obtaining the volume data to be rendered of the target part, the computer device converts the volume data to be rendered based on a transfer function obtained based on a 2D reference rendering image of the target part, thereby obtaining a target rendering image.

[0101] Next, we will briefly introduce the training process of the function prediction model Ntf:

[0102] Prepare training samples, including a 2D reference rendering and a standard transfer function corresponding to the 2D reference rendering. The standard transfer function includes information in five dimensions, of which four dimensions are visual image data from the 2D reference rendering, such as R, G, B, and A, and one dimension is volume data of a rendered 3D image corresponding to the 2D reference rendering, such as CT value and MR image grayscale information. The rendered 3D image is a 3D image that has been reconstructed and rendered manually or in an existing manner.

[0103] The computer device can input the 2D reference rendering image I2 with tissue labels into the initial prediction model N t , by N t Output transfer function T', adjust N based on the loss function of transfer function T' and standard transfer function t The model parameters in the above equation are input, and then the new I2 is input to obtain the new T'. Then, N is adjusted based on the loss function between the new transfer function T' and the standard transfer function. t The model parameters in this cycle are repeated until the loss function reaches the minimum, and the function prediction model N is obtained. tf , and N tf The final transfer function T is determined.

[0104] For example, the transfer function may be represented by texture, and specifically may be a correspondence table between different volume data and visual image data. For example:

[0105]

[0106] For example, the tensor distance may be As a training function, the prediction model N tf The loss function is The calculation formula is as follows:

[0107]

[0108] in, Indicates standard rendering I m The corresponding standard transfer function is the true value texture, The transfer function that represents the output during model training is the output texture.

[0109] In an embodiment of the present application, a reference rendering of a target part including 2D visualization image data of each pixel is obtained, and a tissue label corresponding to each pixel in the 2D reference rendering is obtained. Based on the visualization image data and tissue label of each pixel, a transfer function of the 2D reference rendering is generated through machine learning, so as to render the to-be-rendered volume data of the target part at least according to the transfer function to obtain a target rendering. In the above method, user operations are simplified, and the user does not need to manually configure each pixel. The 2D reference rendering is directly used as a reference sample to autonomously generate a transfer function between the volume data and the visualization image data, thereby reducing the subjective influence of the user, improving the accuracy of the generated transfer function, and correspondingly improving the matching degree between the rendering obtained by rendering based on the transfer function and the rendering effect expected by the user, thereby improving the actual rendering effect.

[0110] To facilitate understanding by those skilled in the art, the image rendering method provided by the present application is described in detail below. As shown in FIG7 , the method may include:

[0111] S701, obtaining a 2D reference rendering of the target part;

[0112] S702, inputting the 2D reference rendering image into a tissue segmentation model to obtain a tissue label corresponding to each pixel in the 2D reference rendering image; the tissue segmentation model is obtained based on machine learning;

[0113] S703, inputting the visualized image data and tissue label of each pixel into a function prediction model to obtain a transfer function of a 2D reference rendering image; the function prediction model is obtained based on machine learning;

[0114] S704 , performing rendering processing on the to-be-rendered volume data of the target part at least according to the transfer function to obtain a target rendering image.

[0115] Combined with Figure 8, the specific process of the image rendering method is as follows:

[0116] Reference Figure 1 of the target part where the computer device receives the user input ref (i.e. 2D reference rendering), I ref Input medical segmentation network N seg (i.e. tissue segmentation model), through N seg Output I ref Then the tissue label of each pixel and the I including the visualization image data of each pixel are combined. ref Input transfer function prediction network N tf(i.e., function prediction model), obtain the transfer function, and save the transfer function as a transfer function file in XML format. The computer device can call the transfer function file to apply the obtained transfer function to the to-be-rendered volume data of the target part to obtain the application effect image I res (i.e., the target rendering). On this basis, the computer device can also receive the user's fine-tuning operation on the transfer function, and obtain the final rendering I through the fine-tuned transfer function. final .

[0117] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0118] Based on the same inventive concept, the embodiment of the present application also provides a transfer function generating device for implementing the transfer function generating method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more transfer function generating device embodiments provided below can refer to the limitations of the transfer function generating method above, and will not be repeated here.

[0119] In one embodiment, as shown in FIG. 9 , a transfer function generating device is provided, including: a reference rendering image acquisition module 901, a label acquisition module 902 and a function determination module 903, wherein:

[0120] The reference rendering acquisition module 901 is used to acquire a 2D reference rendering of the target part; the 2D reference rendering includes visual image data of each pixel point;

[0121] The label acquisition module 902 is used to acquire the tissue label corresponding to each pixel in the 2D reference rendering;

[0122] The function determination module 903 is used to generate a transfer function of a 2D reference rendering image through machine learning according to the visualized image data and the organizational label of each pixel.

[0123] Each module in the transfer function generating device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each module.

[0124] Based on the same inventive concept, the embodiment of the present application also provides an image rendering device for implementing the above-mentioned image rendering method. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above-mentioned method, so the specific limitations in the one or more transfer function generation device embodiments provided below can refer to the limitations on the image rendering method above, and will not be repeated here.

[0125] In one embodiment, as shown in FIG. 10 , an image rendering device is provided, including: a reference rendering image acquisition module 1001, a label acquisition module 1002, a function determination module 1003, and a rendering processing module 1004, wherein:

[0126] The reference rendering acquisition module 1001 is used to acquire a 2D reference rendering of a target part; and visualized image data of each pixel point in the 2D reference rendering.

[0127] The label acquisition module 1002 is used to acquire the tissue label corresponding to each pixel in the 2D reference rendering;

[0128] The function determination module 1003 generates a transfer function of a 2D reference rendering image according to the visualized image data and tissue labels of each pixel point;

[0129] The rendering processing module 1004 is used to perform rendering processing on the to-be-rendered volume data of the target part at least according to the transfer function to obtain a target rendering image.

[0130] Each module in the above image rendering device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0131] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps of any of the above methods when executing the computer program.

[0132] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above methods are implemented.

[0133] In one embodiment, a computer program product is provided, comprising a computer program, which implements the steps of any of the above methods when executed by a processor.

[0134] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0135] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0136] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be construed as limiting the scope of the present application. It should be noted that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A transfer function generation method, characterized in that: The method comprises: Obtaining a 2D reference rendering of the target part; the 2D reference rendering includes visual image data of each pixel point; Obtaining a tissue label corresponding to each pixel in the 2D reference rendering; A transfer function of the 2D reference rendering image is generated through machine learning according to the visualized image data of each pixel and the tissue label.

2. The method according to claim 1, characterized in that The obtaining of the tissue label corresponding to each pixel point in the 2D reference rendering image is obtained through machine learning.

3. The method according to claim 1 or 2, characterized in that: The method further comprises: receiving an adjustment operation for the transfer function; In response to the adjustment operation, the transfer function is adjusted to obtain an adjusted transfer function.

4. An image rendering method, characterized in that: The method comprises: Obtaining a 2D reference rendering of the target part; the 2D reference rendering includes visual image data of each pixel point; Obtaining a tissue label corresponding to each pixel in the 2D reference rendering; Generate a transfer function of the 2D reference rendering image through machine learning based on the visualized image data of each pixel and the tissue label; The to-be-rendered volume data of the target part is rendered at least according to the transfer function to obtain a target rendering image.

5. A transfer function generating device, characterized in that: The device comprises: A reference rendering acquisition module is used to acquire a 2D reference rendering of the target part; the 2D reference rendering includes visual image data of each pixel point; A label acquisition module, used to acquire a tissue label corresponding to each pixel in the 2D reference rendering; The function determination module is used to generate a transfer function of the 2D reference rendering image through machine learning based on the visualized image data of each pixel and the tissue label.

6. An image rendering device, characterized in that: The device comprises: A reference rendering acquisition module, used to acquire a 2D reference rendering of a target part; visualization image data of each pixel point in the 2D reference rendering; A label acquisition module, used to acquire a tissue label corresponding to each pixel in the 2D reference rendering; A function determination module, which generates a transfer function of the 2D reference rendering image through machine learning based on the visualized image data of each pixel and the tissue label; The rendering processing module is used to perform rendering processing on the to-be-rendered volume data of the target part at least according to the transfer function to obtain a target rendering image.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 are implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

9. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images