An image pickup method, apparatus, device, and storage medium

Abstract

The application discloses a picking method and device of an image, equipment and a storage medium. In response to an operation of selecting a target region on a screen, a first texture map corresponding to a first image is obtained. In the first texture map, the target region is in a non-transparent state, and other regions are in a transparent state. According to parameters of a screen camera, a first perspective matrix of an off-screen camera of the first image is obtained. The off-screen camera is matched with the first image. The first perspective matrix is converted into coordinates on the first texture map, and a first picking result of the first image is obtained. The first picking result includes a part corresponding to the target region in the first image. It can be seen that, by the method, efficient and accurate data picking of an image in a volume rendering model is realized based on simple mapping technology, so that cutting, filling and other operations of the volume rendering model can be quickly and accurately performed.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to an image picking method and device, equipment and storage medium. BACKGROUND

[0002] Volume rendering is a technique for generating a two-dimensional image on a screen directly from a three-dimensional data field. The main idea is to calculate the contribution of all sampling points in the three-dimensional data field to the screen pixels in the projection process. The advantage is that it can explore the internal structure of the object. Therefore, it is a common display method for medical images.

[0003] In order to more clearly display the accurate position of the object, it is usually necessary to crop or fill the volume rendering model. The cropping or filling of the volume rendering model requires picking up data of each layer image (such as a computed tomography (CT) image) in the volume rendering model. However, the current picking method of data in the image has the problem that the picked data is not accurate enough.

[0004] Therefore, there is an urgent need to provide a technical solution capable of accurately picking up data in an image to provide a reliable data basis for the cropping or filling of the volume rendering model. SUMMARY

[0005] To solve the above technical problems, the embodiments of the present application provide an image picking method, device, equipment and storage medium, which realizes data picking of the image by means of mapping, and improves the accuracy of data picking in the image.

[0006] In a first aspect, the embodiments of the present application provide an image picking method, comprising:

[0007] In response to an operation of selecting a target region on a screen, a first texture map corresponding to a first image is obtained, the target region in the first texture map is in a non-transparent state, and other regions in the first texture map except the target region are in a transparent state;

[0008] According to parameters of a screen camera, a first perspective matrix of an off-screen camera of the first image is obtained, the off-screen camera is matched with the first image;

[0009] The first perspective matrix is converted into coordinates on the first texture map to obtain a first picking result of the first image, and the first picking result includes a part corresponding to the target region in the first image.

[0010] Optionally, the obtaining of the first perspective matrix of the off-screen camera of the first image according to the parameters of the screen camera comprises:

[0011] mapping the first image to a target plane of the off-screen camera based on the parameters of the screen camera to obtain the first perspective matrix.

[0012] Optionally, the mapping the first image to a target plane of the off-screen camera based on the parameters of the screen camera comprises:

[0013] obtaining a local coordinate matrix of the off-screen camera according to the voxel coordinates corresponding to the points of the target plane;

[0014] obtaining a world coordinate matrix of the off-screen camera according to the local coordinate matrix of the off-screen camera and the rotation matrix of the screen camera;

[0015] obtaining an inverse matrix of the world coordinate matrix of the off-screen camera according to the inverse matrix of the world coordinate matrix of the screen camera and the world coordinate matrix of the off-screen camera;

[0016] obtaining the first perspective matrix according to the inverse matrix of the world coordinate matrix of the off-screen camera and the projection matrix of the screen camera.

[0017] Optionally, the converting the first perspective matrix into coordinates on the first texture map to obtain the first picking result of the first image comprises:

[0018] comparing the first perspective matrix with the coordinates of the first texture map on the target plane to obtain the first picking result.

[0019] Optionally, the method further comprises:

[0020] obtaining a second texture map corresponding to a second image, wherein the target region in the second texture map is in a non-transparent state, and other regions in the second texture map except the target region are in a transparent state;

[0021] obtaining a second perspective matrix of an off-screen camera of the second image according to the parameters of the screen camera, wherein the off-screen camera matches the second image;

[0022] converting the second perspective matrix into coordinates on the second texture map to obtain a second picking result of the second image, wherein the second picking result comprises a part corresponding to the target region in the second image.

[0023] Optionally, the first image and the second image belong to a target object, and the method further comprises:

[0024] clipping or filling the target region of a volume rendering model of the target object according to the first picking result and the second picking result.

[0025] Secondly, embodiments of this application also provide an image picking device, comprising:

[0026] The first obtaining unit is configured to obtain a first texture map corresponding to the first image in response to an operation of selecting a target area on the screen, wherein the target area in the first texture map is in a non-transparent state and other areas in the first texture map other than the target area are in a transparent state.

[0027] The second obtaining unit is used to obtain the first perspective matrix of the off-screen camera of the first image based on the parameters of the screen camera, wherein the off-screen camera is matched with the first image.

[0028] The third obtaining unit is used to convert the first perspective matrix into coordinates on the first texture map to obtain a first picking result of the first image, wherein the first picking result includes the part corresponding to the target region in the first image.

[0029] Optionally, the second obtaining unit is specifically used for:

[0030] Based on the parameters of the screen camera, the first image is pasted onto the target plane of the off-screen camera to obtain the first perspective matrix.

[0031] Optionally, the second obtaining unit is specifically used for:

[0032] The local coordinate matrix of the off-screen camera is obtained based on the voxel coordinates corresponding to the points on the target plane.

[0033] The world coordinate matrix of the off-screen camera is obtained based on the local coordinate matrix of the off-screen camera and the rotation matrix of the on-screen camera.

[0034] The inverse matrix of the world coordinate matrix of the off-screen camera is obtained based on the inverse matrix of the world coordinate matrix of the on-screen camera and the world coordinate matrix of the off-screen camera.

[0035] The first perspective matrix is ​​obtained by using the inverse of the world coordinate matrix of the off-screen camera and the projection matrix of the on-screen camera.

[0036] Optionally, the third obtaining unit is specifically used for:

[0037] On the target plane, the coordinates of the first perspective matrix and the first texture map are compared to obtain the first picking result.

[0038] Optionally, the device further includes:

[0039] The fourth obtaining unit is used to obtain a second texture map corresponding to the second image, wherein the target area in the second texture map is in a non-transparent state, and other areas in the second texture map other than the target area are in a transparent state;

[0040] The fifth obtaining unit is used to obtain the second perspective matrix of the off-screen camera of the second image based on the parameters of the screen camera, wherein the off-screen camera is matched with the second image;

[0041] The sixth obtaining unit is used to convert the second perspective matrix into coordinates on the second texture map to obtain a second picking result of the second image, the second picking result including the part corresponding to the target region in the second image.

[0042] Optionally, the first image and the second image belong to the target object, and the device further includes:

[0043] The processing unit is configured to crop or fill the target region of the volume drawing model of the target object based on the first picking result and the second picking result.

[0044] Thirdly, embodiments of this application also provide an image acquisition device, the device including a processor and a memory:

[0045] The memory is used to store program code and transmit the program code to the processor;

[0046] The processor is configured to execute the method provided in the first aspect above according to the instructions in the program code.

[0047] Fourthly, a storage medium is also provided for storing program code for executing the method provided in the first aspect.

[0048] Compared with the prior art, the embodiments of this application have at least the following advantages:

[0049] In this embodiment, firstly, the user can trigger a target area selection operation on the screen by drawing a target area or other means according to the data acquisition needs. In response to the target area selection operation, the image acquisition device can obtain a first texture map corresponding to the first image. The target area in the first texture map is in a non-transparent state, and other areas in the first texture map other than the target area are in a transparent state. Next, the image acquisition device obtains a first perspective matrix of the off-screen camera of the first image according to the parameters of the screen camera. The off-screen camera matches the first image. Thus, the image acquisition device converts the first perspective matrix into coordinates on the first texture map to obtain a first acquisition result of the first image. The first acquisition result includes the part corresponding to the target area in the first image. As can be seen, the method provided in this application transforms the voxel mapping of the volume rendering model into image cropping and the image data picking into GPU texture mapping by rendering textures layer by layer on the layered image (i.e., the first image, such as a CT image) of the volume rendering model using a graphics processing unit (GPU). This greatly reduces the computational complexity. Moreover, by using multi-scene and off-screen rendering techniques, the picking results are more accurate. Thus, this method enables the picking of data in the image to be fast and accurate, thereby making efficient and accurate cropping or filling of the volume rendering model possible. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings.

[0051] Figure 1 A schematic flowchart illustrating an image picking method provided in an embodiment of this application;

[0052] Figure 2 This is a schematic diagram of the structure of the integrated drawing model provided in the embodiments of this application;

[0053] Figure 3 A schematic diagram of a target region for drawing, provided in an embodiment of this application;

[0054] Figure 4 A schematic diagram of an example of a first texture map provided in an embodiment of this application;

[0055] Figure 5 A schematic diagram of a four-channel fragment shader provided in an embodiment of this application;

[0056] Figure 6A schematic diagram illustrating an example of a first picking result provided in an embodiment of this application;

[0057] Figure 7 A schematic diagram of the structure of an image picking device provided in an embodiment of this application;

[0058] Figure 8 This is a schematic diagram of the structure of an image picking device provided in an embodiment of this application. Detailed Implementation

[0059] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0060] Volume rendering is a common method of display in medical imaging. To achieve a sense of three-dimensionality, volume rendering often employs a perspective view (i.e., objects appear larger when closer and smaller when farther away). Due to the perspective effect, a cross-section drawn on a plane will appear as a truncated cone in volume rendering due to perspective relationships. The extent of the cross-section will differ in each layer of the volume rendering model, therefore, a good data acquisition method is needed as the foundation for its implementation.

[0061] To obtain the precise location desired by the user, the structure of a portion of the volumetric rendering model is obtained through clipping or filling, while unnecessary structures are removed, making the volumetric rendering model more focused and efficient. Taking the clipping of the volumetric rendering model as an example, the current methods include: Method 1, affine transformation, which projects the area drawn on the screen onto each layer of the volumetric rendering model and then performs projection calculations with each layer image. This projection calculation is complex and prone to errors. Method 2, perspective, which uses rays to penetrate the selected surface. Since the selected area is a view frustum, many voxels behind the selected surface of the volumetric rendering model cannot be selected, leading to data selection errors for some voxels. Method 3, screen transformation, which maps each voxel in the volumetric rendering model to a two-dimensional position, projects the two-dimensional position onto the screen, and then compares it with the screen drawing area to obtain the clipping area. This method requires traversing all voxels of the volumetric rendering model. A typical voxel consists of tens of millions of points, requiring tens of millions of calculations through this method. The computational load is huge, the speed is very slow, and such a large amount of computation is very prone to errors.

[0062] Based on this, the method provided in this application embodiment can quickly, conveniently, and accurately pick up data of each layer of the image in the volume drawing model according to data picking requirements. The image picking method provided in this application embodiment may include, for example, the following: First, the user can trigger a target area selection operation on the screen by drawing a target area, etc., according to the data picking requirements. In response to the target area selection operation, a first texture map corresponding to the first image is obtained. The target area in the first texture map is non-transparent, and other areas outside the target area in the first texture map are transparent. Next, based on the parameters of the screen camera, a first perspective matrix of the off-screen camera of the first image is obtained. The off-screen camera matches the first image. Then, the first perspective matrix is ​​converted into coordinates on the first texture map to obtain a first picking result of the first image. The first picking result includes the portion corresponding to the target area in the first image.

[0063] As can be seen, this method transforms the voxel mapping of the volume rendering model into image cropping by rendering textures layer by layer on the layered image of the volume rendering model using the GPU, and transforms image data picking into GPU texture mapping, which greatly reduces the computational complexity. Moreover, by using multi-scene and off-screen rendering technology, the picking results are more accurate. Thus, this method enables the picking of data in the image to be fast and accurate, thereby making efficient and accurate cropping or filling of the volume rendering model possible.

[0064] It should be noted that the main body implementing the image picking method can be the image picking device provided in the embodiments of this application, which can be carried in an electronic device or a functional module of an electronic device. The electronic device in the embodiments of this application can be any device capable of implementing the image picking method in the embodiments of this application, such as an Internet of Things (IoT) device.

[0065] The various non-limiting embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0066] Figure 1 This is a schematic flowchart illustrating an image picking method provided in an embodiment of this application. See also... Figure 1 In this embodiment, the method may specifically include the following steps 101 to 103:

[0067] S101, in response to the operation of selecting a target area on the screen, a first texture map corresponding to the first image is obtained, wherein the target area in the first texture map is in a non-transparent state, and other areas in the first texture map other than the target area are in a transparent state.

[0068] It is understandable that the volume rendering model is at least one image rendered by the GPU, with a length of w and a height of h. The image with length w and height h can be understood as the size of the screen or the size of the window displaying the volume rendering model on the screen. After projection rendering, the volume rendering model has a length of vx, a width of vy, and a height of vz. For the screen camera, the actual 3D volume rendering model has a length of xDim, a width of yDim, and a layer count of zDim.

[0069] The desired region (i.e., the target region) is drawn on the volume rendering model corresponding to the first image on the screen, obtaining a cross-sectional fence of the target region. This cross-section is the projection of the truncated cone onto the screen. The cross-sectional fence obtained through the target region forms a first texture map. This first texture map is a transparent image with a width of w and a height of h. In the first texture map, the target region is in a non-transparent state, while other regions are in a transparent state.

[0070] The first image is any image obtained by cutting the three-dimensional voxels (hereinafter referred to as voxels) of the volume rendering model into images along the z-axis. In this embodiment, each image is used as the first image, and a separate operation is performed in S101 to S103. In this way, the number of calculations required to pick up the data of the target area in the volume rendering model is equal to the number of layers, which greatly reduces the computational complexity.

[0071] For example, suppose the volume rendering model is as follows: Figure 2 As shown, the cross-sectional fence drawn on the first image of the volumetric model is as follows: Figure 3 As shown, the obtained first texture map can be found in [reference needed]. Figure 4 As shown.

[0072] It is understood that the method provided in this application embodiment is for obtaining which point in the first image is selected, thereby obtaining the selected data of the volume rendering model based on the selected points in each layer of the volume rendering model. The output of the off-screen camera is a transparent image, the length and width of which are the same size as the first image, and the opaque positions on this image are the selected target areas in the image.

[0073] S102, based on the parameters of the screen camera, obtain the first perspective matrix of the off-screen camera of the first image, wherein the off-screen camera is matched with the first image.

[0074] In this embodiment, the purpose of using the screen camera parameters is to use the GPU to simulate the current position, viewing angle, and other information of the screen camera, so that the picked-up data is more accurate.

[0075] It is understood that the off-screen camera can load a first off-screen rendered image onto the GPU. This image is the size of one layer of the volume rendering model, i.e., an image with length x and width y. The GPU changes the z-axis parameter (i.e., the number of layers in the volume rendering model) for each rendering iteration. This allows for the capture of one layer of voxels as a single image, enabling the execution of the method provided in this application's embodiments. The obtained images are sequentially imported into the GPU for subsequent texturing operations. The image can be adjusted using the parameters of the on-screen camera to ensure it fills the off-screen display area.

[0076] In some possible implementations, S102 may include, for example, pasting the first image onto the target plane of the off-screen camera based on the parameters of the screen camera to obtain the first perspective matrix.

[0077] As an example, in S102, based on the parameters of the screen camera, the first image is pasted onto the target plane of the off-screen camera to obtain the first perspective matrix. This may include, for example, obtaining the local coordinate matrix of the off-screen camera according to the voxel coordinates corresponding to the points on the target plane; obtaining the world coordinate matrix of the off-screen camera according to the local coordinate matrix of the off-screen camera and the rotation matrix of the screen camera; obtaining the inverse matrix of the world coordinate matrix of the off-screen camera according to the inverse matrix of the world coordinate matrix of the screen camera and the world coordinate matrix of the off-screen camera; and obtaining the first perspective matrix according to the inverse matrix of the world coordinate matrix of the off-screen camera and the projection matrix of the screen camera.

[0078] Taking the first image, a layer in the volume rendering model, as an example, we will introduce the off-screen rendering process.

[0079] It's understandable that off-screen cameras use GPU computation, but the results aren't displayed on the screen; off-screen rendering can be understood as GPU computation. The display area of ​​the off-screen camera has the same length (x) and width (y) as the first image. In the off-screen rendering camera's scene, there's a plane S that fills the display area, with length x and height y. In the screen camera's scene, the display area size is length w and height h. Assume the center of the first image is at the center of a cuboid with sides of length vx, width vy, and height vz, which actually correspond to the width xDim, height yDim, and layer number zDim of the entity (e.g., CT) in the first image. Assume the screen camera's rotation matrix is ​​R1, projection matrix is ​​P1, and the inverse of the world coordinate matrix is ​​W1. -1 Therefore, the parameters of the on-screen camera are input to the off-screen camera, and then passed to the GPU. The GPU applies the first image texture to plane S of the scene. The specific calculation process is as follows:

[0080] The first step is to take the coordinates of a point on plane S, whose index in the voxel is (x, y, z). x and y are the coordinate values ​​on plane S. z is the layer number of the volume rendering model in which the first image is located. The (x, y, z) of this point is converted into the local coordinate matrix (LocalPosition.x, LocalPosition.y, LocalPosition.z) of the off-screen camera, for example, by using the following formula (1):

[0081]

[0082] The second step is to calculate the world coordinate matrix W2 of the off-screen camera based on the local coordinate matrix of the off-screen camera and the rotation matrix R1 of the screen camera. The local coordinate matrix of the off-screen camera can be represented as LocalPosition, which consists of the local coordinate matrices (LocalPosition.x, LocalPosition.y, LocalPosition.z) corresponding to each point on the S-plane. For example, the world coordinate matrix W2 of the off-screen camera can be calculated using the following formula (2):

[0083] W2=LocalPosition*R1......Formula (2)

[0084] The third step involves using the inverse matrix W1 of the world coordinate matrix of the off-screen camera and the world coordinate matrix of the on-screen camera. -1 Calculate the inverse world coordinate matrix W2 of the off-screen camera. -1 For example, the inverse world coordinate matrix W2 of the off-screen camera can be calculated using the following formula (3). -1 :

[0085] W2 -1 =W2*W1 -1 ...Formula (3)

[0086] The fourth step is to use the inverse world coordinate matrix W2 of the off-screen camera. -1 Given the projection matrix P1 of the screen camera, calculate the perspective matrix P2 of the off-screen camera. For example, the perspective matrix P2 of the off-screen camera can be calculated using the following formula (4):

[0087] P2 = W2 -1 *P1...Formula (4)

[0088] Thus, through S102, the first perspective matrix of the off-screen camera for the first image is obtained using the parameters of the screen camera, and the GPU is used to simulate the current position, viewing angle and other information of the screen camera, thus preparing for the accurate acquisition of data in the first image.

[0089] S103, convert the first perspective matrix into coordinates on the first texture map to obtain a first picking result of the first image, the first picking result including the part corresponding to the target region in the first image.

[0090] S103 can be understood as comparing the coordinates of plane S in the off-screen camera scene with the first texture map and outputting a transparent image, i.e., the first picking result.

[0091] In some possible implementations, S103 may include, for example, comparing the coordinates of the first perspective matrix and the first texture map on the target plane to obtain the first picking result.

[0092] For example, based on the example in S102, the first texture map can be normalized to the range of [0,1], and the perspective matrix P2 of the off-screen camera can be transformed into the coordinate system after the first texture map is normalized. For example, the transformation can be performed using the following formula (5):

[0093]

[0094] Among them, the coordinate values ​​of (Screen.x, Screen.y) also belong to the range of [0,1].

[0095] The coordinate matrix is ​​transformed into a standard aspect ratio (Standard Variable) to form an image. This image uses the actual load of the four-channel fragment shader as its aspect ratio. The image value of the first texture map is retrieved using the proportional coordinates of the four-channel fragment shader. When a value is available, the position of the voxel of the corresponding volume rendering model in that layer of the image (i.e., the first image) is the valid value of that image. In other words, the first picking result includes the position of that voxel in the first image. Specifically, the color values ​​on the first texture map are picked and output to a transparent image. The output image is a four-channel color RGBA image with a width and height of xy. When this image overlaps with the original first image, the non-transparent areas represent the first picking result.

[0096] For example, using Figure 5 The four-channel fragment shader shown Figure 4 The first texture map shown is converted to obtain the first picking result, which can be found in [reference]. Figure 6 As shown.

[0097] In this way, data picking of any layer of image in the volume rendering model is achieved.

[0098] In some possible implementations, the method may further include: obtaining a second texture map corresponding to the second image, wherein the target region in the second texture map is in a non-transparent state and other regions in the second texture map other than the target region are in a transparent state; obtaining a second perspective matrix of the off-screen camera of the second image according to the parameters of the on-screen camera, wherein the off-screen camera matches the second image; converting the second perspective matrix into coordinates on the second texture map to obtain a second picking result of the second image, wherein the second picking result includes the portion corresponding to the target region in the second image.

[0099] If the first image and the second image belong to the volume rendering model of the target object, then after applying the method provided in this embodiment to all layer images of the target object, the method may further include: cropping the volume rendering model of the target object based on the picking results of each layer image to obtain the structure of the target region of the target object; or, the method may further include: filling the volume rendering model of the target object based on the picking results of each layer image to obtain the structure of the target region of the target object. For example, the method may further include: cropping or filling the target region of the volume rendering model of the target object according to the first picking result and the second picking result.

[0100] As can be seen, this method uses GPU mapping and data picking on the layered images of the volume rendering model, transforms the picking relationship from voxel mapping to image cropping, and transforms the value acquisition into texture mapping. By using multi-scene and off-screen rendering, the picking operation becomes fast, accurate, and effective, greatly reducing the computational complexity and thus improving the speed and effect of subsequent operations such as cropping and filling.

[0101] Accordingly, embodiments of this application also provide an image picking device 700, such as... Figure 7 As shown, the device 700 may specifically include:

[0102] The first obtaining unit 701 is configured to obtain a first texture map corresponding to the first image in response to an operation of selecting a target area on the screen, wherein the target area in the first texture map is in a non-transparent state and other areas in the first texture map other than the target area are in a transparent state.

[0103] The second obtaining unit 702 is used to obtain the first perspective matrix of the off-screen camera of the first image based on the parameters of the screen camera, wherein the off-screen camera is matched with the first image.

[0104] The third obtaining unit 703 is used to convert the first perspective matrix into coordinates on the first texture map to obtain a first picking result of the first image, wherein the first picking result includes the part corresponding to the target region in the first image.

[0105] Optionally, the second obtaining unit 702 is specifically used for:

[0106] Based on the parameters of the screen camera, the first image is pasted onto the target plane of the off-screen camera to obtain the first perspective matrix.

[0107] Optionally, the second obtaining unit 702 is specifically used for:

[0108] The local coordinate matrix of the off-screen camera is obtained based on the voxel coordinates corresponding to the points on the target plane.

[0109] The world coordinate matrix of the off-screen camera is obtained based on the local coordinate matrix of the off-screen camera and the rotation matrix of the on-screen camera.

[0110] The inverse matrix of the world coordinate matrix of the off-screen camera is obtained based on the inverse matrix of the world coordinate matrix of the on-screen camera and the world coordinate matrix of the off-screen camera.

[0111] The first perspective matrix is ​​obtained by using the inverse of the world coordinate matrix of the off-screen camera and the projection matrix of the on-screen camera.

[0112] Optionally, the third obtaining unit 703 is specifically used for:

[0113] On the target plane, the coordinates of the first perspective matrix and the first texture map are compared to obtain the first picking result.

[0114] Optionally, the device 700 further includes:

[0115] The fourth obtaining unit is used to obtain a second texture map corresponding to the second image, wherein the target area in the second texture map is in a non-transparent state, and other areas in the second texture map other than the target area are in a transparent state;

[0116] The fifth obtaining unit is used to obtain the second perspective matrix of the off-screen camera of the second image based on the parameters of the screen camera, wherein the off-screen camera is matched with the second image;

[0117] The sixth obtaining unit is used to convert the second perspective matrix into coordinates on the second texture map to obtain a second picking result of the second image, the second picking result including the part corresponding to the target region in the second image.

[0118] Optionally, the first image and the second image belong to the target object, and the device 700 further includes:

[0119] The processing unit is configured to crop or fill the target region of the volume drawing model of the target object based on the first picking result and the second picking result.

[0120] The above description is a description of the image picking device 700. For the specific implementation method and the achieved effect, please refer to the description of the above image picking method embodiment, which will not be repeated here.

[0121] Furthermore, embodiments of this application also provide an image picking device 800, such as... Figure 8 As shown, the device 800 includes a processor 801 and a memory 802:

[0122] The memory 802 is used to store program code and transmit the program code to the processor 801;

[0123] The processor 801 is used to execute the above-described memory resource allocation method according to the instructions in the program code.

[0124] The specific implementation method and the effect achieved by the image picking device can be found in the description of the above-mentioned image picking method embodiment, and will not be repeated here.

[0125] Furthermore, this application embodiment also provides a storage medium for storing program code for executing the above-described image picking method.

[0126] In the embodiments of this application, the "first" in names such as "first image" and "first picking result" is only used for name identification and does not represent the first in order. The same rule applies to "second" and so on.

[0127] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that all or part of the steps in the methods of the above embodiments can be implemented by means of software plus a general-purpose hardware platform. Based on this understanding, the technical solution of this application can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as a read-only memory (ROM) / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, a server, or a network communication device such as a router) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0128] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device and apparatus embodiments are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions in the method embodiments. The device and apparatus embodiments described above are merely illustrative. Modules described as separate components may or may not be physically separate, and components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0129] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. It should be noted that those skilled in the art can make various improvements and modifications without departing from this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. An image picking method, characterized in that, include: In response to the operation of selecting a target area on the screen, a first texture map corresponding to a first image is obtained, wherein the target area in the first texture map is in a non-transparent state, and other areas outside the target area in the first texture map are in a transparent state; wherein, the first image is any image obtained by cutting a three-dimensional voxel of a volume rendering model into images along the z-axis direction; Based on the parameters of the screen camera, the first perspective matrix of the off-screen camera for the first image is obtained, and the off-screen camera is matched with the first image; The first perspective matrix is ​​converted into coordinates on the first texture map to obtain a first picking result of the first image, the first picking result including the part corresponding to the target region in the first image; The method further includes: A second texture map corresponding to the second image is obtained, wherein the target area in the second texture map is in a non-transparent state, and other areas outside the target area in the second texture map are in a transparent state; Based on the parameters of the on-screen camera, a second perspective matrix of the off-screen camera for the second image is obtained, wherein the off-screen camera is matched with the second image; The second perspective matrix is ​​converted into coordinates on the second texture map to obtain a second picking result of the second image, the second picking result including the part corresponding to the target region in the second image; The first image and the second image belong to the target object, and the method further includes: Based on the first picking result and the second picking result, the target region of the volume drawing model of the target object is cropped or filled.

2. The method according to claim 1, characterized in that, The step of obtaining the first perspective matrix of the off-screen camera for the first image based on the parameters of the on-screen camera includes: Based on the parameters of the screen camera, the first image is pasted onto the target plane of the off-screen camera to obtain the first perspective matrix.

3. The method according to claim 2, characterized in that, The step of pasting the first image onto the target plane of the off-screen camera based on the parameters of the screen camera includes: The local coordinate matrix of the off-screen camera is obtained based on the voxel coordinates corresponding to the points on the target plane. The world coordinate matrix of the off-screen camera is obtained based on the local coordinate matrix of the off-screen camera and the rotation matrix of the on-screen camera. The inverse matrix of the world coordinate matrix of the off-screen camera is obtained based on the inverse matrix of the world coordinate matrix of the on-screen camera and the world coordinate matrix of the off-screen camera. The first perspective matrix is ​​obtained by using the inverse of the world coordinate matrix of the off-screen camera and the projection matrix of the on-screen camera.

4. The method according to claim 2 or 3, characterized in that, The step of converting the first perspective matrix into coordinates on the first texture map to obtain the first picking result of the first image includes: On the target plane, the coordinates of the first perspective matrix and the first texture map are compared to obtain the first picking result.

5. An image acquisition device, characterized in that, include: The first obtaining unit is configured to obtain a first texture map corresponding to the first image in response to an operation of selecting a target area on the screen. The target area in the first texture map is in a non-transparent state, and other areas in the first texture map other than the target area are in a transparent state. The first image is any image obtained by cutting a three-dimensional voxel of a volume rendering model into images along the z-axis. The second obtaining unit is used to obtain the first perspective matrix of the off-screen camera of the first image based on the parameters of the screen camera, wherein the off-screen camera is matched with the first image. The third obtaining unit is used to convert the first perspective matrix into coordinates on the first texture map to obtain a first picking result of the first image, wherein the first picking result includes the part corresponding to the target region in the first image; The device further includes: The fourth obtaining unit is used to obtain a second texture map corresponding to the second image, wherein the target area in the second texture map is in a non-transparent state, and other areas in the second texture map other than the target area are in a transparent state; The fifth obtaining unit is used to obtain the second perspective matrix of the off-screen camera of the second image based on the parameters of the screen camera, wherein the off-screen camera is matched with the second image; The sixth obtaining unit is used to convert the second perspective matrix into coordinates on the second texture map to obtain a second picking result of the second image, wherein the second picking result includes the portion corresponding to the target region in the second image; The first image and the second image belong to the target object, and the device further includes: The processing unit is configured to crop or fill the target region of the volume drawing model of the target object based on the first picking result and the second picking result.

6. The apparatus according to claim 5, characterized in that, The second obtaining unit is specifically used for: Based on the parameters of the screen camera, the first image is pasted onto the target plane of the off-screen camera to obtain the first perspective matrix.

7. An image acquisition device, characterized in that, The device includes a processor and a memory: The memory is used to store program code and transmit the program code to the processor; The processor is configured to execute the method according to any one of claims 1 to 4 according to the instructions in the program code.

8. A storage medium, characterized in that, The storage medium is used to store program code, which is used to execute the method according to any one of claims 1 to 4.

Images