A method and system for imaging and processing target organs in laparoscopic surgery

Abstract

The present invention proposes an imaging and processing method and system for target organs in laparoscopic surgery, which relates to the field of medical image processing. By acquiring a scanned image of the target organ, the scanned image includes a preliminary location of the lesion; then performing significant area detection on the scanned image; then segmenting the scanned image along the significant area to obtain a significant area image and a non-significant area image; then performing three-dimensional reconstruction of the significant area image using a ray casting algorithm to generate a first three-dimensional image; then performing three-dimensional reconstruction of the non-significant area image using a moving cube algorithm to generate a second three-dimensional image; finally, combining the first three-dimensional image and the second three-dimensional image to obtain a three-dimensional image of the target organ. The reconstruction calculation amount of the entire target organ is reduced, and the reconstruction speed is increased. The internal structural information and details of the object can be displayed, thereby improving the accuracy of the lesion location and helping doctors to locate the lesion.

Description

Technical Field

[0001] The present invention relates to the field of medical image processing, and in particular to an imaging and processing method and system for target organs in laparoscopic surgery. Background Art

[0002] 3D reconstruction technology refers to the use of image data output by medical imaging equipment such as CT and MRI, and the selection of appropriate 3D reconstruction algorithms as needed to obtain 3D projection images that can be observed from any perspective, so that diagnostic doctors can easily observe and diagnose the structure of internal tissues or organs of the human body. After targeted processing of medical images, 3D reconstruction technology is used to construct a 3D model of tissues or organs, and then the 3D model is displayed on the display screen. For organs of interest to doctors, qualitative or quantitative information such as size, shape and spatial position can also be extracted for easy analysis. The use of 3D reconstruction technology enables medical personnel to observe the 3D structure of internal organs of the human body more intuitively and quantitatively, and can also enhance certain original details in the image according to the needs of different disease diagnosis, thereby helping doctors to make correct disease diagnoses more easily.

[0003] There are two main types of image 3D reconstruction: surface rendering and volume rendering. Existing 3D reconstruction generally uses volume rendering, which processes each voxel in the volume data field. Therefore, the amount of calculation increases and the rendering speed is slow. Summary of the invention

[0004] The purpose of the present invention is to provide a method and system for imaging and processing target organs in laparoscopic surgery, so as to improve the problems of increased computational complexity and slow rendering speed in the prior art using volume rendering surfaces.

[0005] In a first aspect, an embodiment of the present application provides an imaging and processing method for a target organ in laparoscopic surgery, comprising the following steps:

[0006] Obtaining a scanned image of the target organ, wherein the scanned image includes a preliminary location of the lesion;

[0007] Performing salient region detection on the scanned image to obtain salient regions, wherein the salient regions include the preliminary location of the lesion;

[0008] Segment the scanned image along the salient region to obtain a salient region image and a non-salient region image;

[0009] Performing three-dimensional reconstruction on the salient region image by using a ray casting algorithm to generate a first three-dimensional image;

[0010] The non-significant area image is three-dimensionally reconstructed using a marching cube algorithm to generate a second three-dimensional image;

[0011] The first three-dimensional image and the second three-dimensional image are combined to obtain a three-dimensional image of the target organ.

[0012] In the above implementation process, a scanned image of the target organ is obtained, and the scanned image includes the preliminary location of the lesion, so that the scanned image can display the preliminary location of the lesion; then, a significant area detection is performed on the scanned image to obtain a significant area, and the significant area includes the preliminary location of the lesion; then, the scanned image is segmented along the significant area to obtain a significant area image and a non-significant area image. By performing a significant detection on the scanned image, the scanned image can be segmented into an image including the preliminary location of the lesion and an image not including the image; then, the significant area image is three-dimensionally reconstructed using a ray casting algorithm to generate a first three-dimensional image. The significant area image is the part that is focused on in laparoscopic surgery, so a more accurate three-dimensional reconstruction method is used for reconstruction, which can display the internal structure information and details of the object; then, the non-significant area image is three-dimensionally reconstructed using a marching cube algorithm to generate a second three-dimensional image. The non-significant area is the part that does not need to be focused on in laparoscopic surgery, so a three-dimensional reconstruction algorithm that cannot display the internal feature information of the object is used for reconstruction; finally, the first three-dimensional image and the second three-dimensional image are combined to obtain a three-dimensional image of the target organ. Since the marching cubes algorithm belongs to surface rendering, which processes part of the data in the volume data field, it has the advantages of small amount of calculation and fast speed, thereby reducing the amount of calculation for the reconstruction of the entire target organ and improving the reconstruction speed. At the same time, the image of the preliminary location of the lesion is reconstructed using a ray casting algorithm, which can display the internal structure information and details of the object, thereby improving the accuracy of the lesion location and helping doctors to locate the lesion.

[0013] Based on the first aspect, in some embodiments of the present invention, obtaining a scanned image of a target organ, wherein the step of preliminarily locating a lesion in the scanned image comprises the following steps:

[0014] Obtain preliminary information on the location of target organs;

[0015] The initially located target organ is scanned according to the initially located target organ information to obtain a scanned image of the target organ, wherein the scanned image includes the initially located lesion position.

[0016] Based on the first aspect, in some embodiments of the present invention, the following steps are further included:

[0017] A1: Calculate and compare the area of ​​the significant region with the preset region area threshold to obtain a comparison result;

[0018] A2: Determine whether the salient region is valid based on the comparison result. If it is valid, segment the scanned image along the salient region to obtain a salient region image and a non-salient region image. If it is invalid, execute A3.

[0019] A3: Expand the salient region according to the preset expansion rules to obtain a new salient region;

[0020] A4: Segment the scanned image along the salient regions to obtain salient region images and non-salient region images.

[0021] Based on the first aspect, in some embodiments of the present invention, the step of combining the first three-dimensional image and the second three-dimensional image to obtain a three-dimensional image of the target organ includes the following steps:

[0022] The first three-dimensional image and the second three-dimensional image are combined along the edge of the salient region to obtain a three-dimensional image of the target organ.

[0023] Based on the first aspect, in some embodiments of the present invention, the following steps are further included:

[0024] Mark the edges of the salient regions for easy identification.

[0025] Based on the first aspect, in some embodiments of the present invention, the step of performing three-dimensional reconstruction on the non-significant region image using a marching cubes algorithm to generate a second three-dimensional image includes the following steps:

[0026] Extract multiple data points connected to any two layers in the non-significant area image to form cubes of equal size;

[0027] Compare each corner point of each cube with a preset threshold value to obtain a state value of each corner point in each cube;

[0028] Find the cube with equal value points according to the state value;

[0029] Extract and calculate the grayscale values ​​of equal-value points in the cube;

[0030] Each isovalue point is connected into a triangular face according to a preset topological form to form an isovalue surface and generate a second three-dimensional image.

[0031] Based on the first aspect, in some embodiments of the present invention, the step of performing three-dimensional reconstruction on the salient region image using a ray casting algorithm to generate a first three-dimensional image includes the following steps:

[0032] A ray casting algorithm is used to perform spatial scanning on the salient region image to generate a first three-dimensional image.

[0033] In a second aspect, an embodiment of the present application provides an imaging and processing system for a target organ in laparoscopic surgery, comprising:

[0034] A scanning image acquisition module is used to acquire a scanning image of the target organ, wherein the scanning image includes a preliminary location of the lesion;

[0035] A salient region detection module is used to perform salient region detection on the scanned image to obtain salient regions, wherein the salient regions include the preliminary location of the lesion;

[0036] Region segmentation is used to segment the scanned image along the salient region to obtain a salient region image and a non-salient region image;

[0037] A first three-dimensional reconstruction module, used to perform three-dimensional reconstruction on the salient region image by using a ray casting algorithm to generate a first three-dimensional image;

[0038] A second three-dimensional reconstruction module is used to perform three-dimensional reconstruction on the non-significant area image by using a marching cube algorithm to generate a second three-dimensional image;

[0039] The three-dimensional image combination module is used to combine the first three-dimensional image and the second three-dimensional image to obtain a three-dimensional image of the target organ.

[0040] In the above implementation process, a scanning image of the target organ is acquired by a scanning image acquisition module, and the scanning image includes a preliminary location of the lesion, so that the preliminary location of the lesion can be displayed in the scanning image; then the significant region detection module performs significant region detection on the scanning image to obtain a significant region, and the significant region includes the preliminary location of the lesion; then the region segmentation segments the scanning image along the significant region to obtain a significant region image and a non-significant region image, and by performing a significant detection on the scanning image, the scanning image can be segmented into an image including the preliminary location of the lesion and an image not including the lesion; then the first three-dimensional reconstruction module segments the significant region image For example, the ray casting algorithm is used for three-dimensional reconstruction to generate the first three-dimensional image. For the significant area image, it is the focus of laparoscopic surgery. Therefore, a more accurate three-dimensional reconstruction method is used for reconstruction, which can show the internal structure information and details of the object. The second three-dimensional reconstruction module uses the moving cube algorithm to reconstruct the non-significant area image in three dimensions to generate the second three-dimensional image. For the non-significant area, it is the part that does not need to be focused on in laparoscopic surgery. Therefore, a three-dimensional reconstruction algorithm that cannot show the internal feature information of the object is used for reconstruction. The three-dimensional image combination module combines the first three-dimensional image and the second three-dimensional image to obtain the three-dimensional image of the target organ. Since the moving cube algorithm belongs to surface drawing, which processes part of the data in the volume data field, it has the advantages of small amount of calculation and fast speed, thereby reducing the amount of calculation for reconstruction of the entire target organ and improving the reconstruction speed. At the same time, the ray casting algorithm is used to reconstruct the image for the preliminary positioning of the lesion position, which can show the internal structure information and details of the object, thereby improving the accuracy of the lesion position and helping doctors to locate the lesion.

[0041] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory for storing one or more programs and a processor. When the one or more programs are executed by the processor, any method in the first aspect is implemented.

[0042] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a method as described in any one of the first aspects above.

[0043] The embodiments of the present invention have at least the following advantages or beneficial effects:

[0044] The embodiment of the present invention provides an imaging and processing method and system for a target organ in laparoscopic surgery. The method comprises acquiring a scanned image of the target organ, wherein the scanned image includes a preliminary localization lesion position, so that the scanned image can display the preliminary localization lesion position. Then, the scanned image is subjected to a significant region detection to obtain a significant region, wherein the significant region includes the preliminary localization lesion position. Then, the scanned image is segmented along the significant region to obtain a significant region image and an insignificant region image. By performing a significant region detection on the scanned image, the scanned image can be segmented into an image including the preliminary localization lesion position image and an image not including the image. Then, the significant region image is subjected to a three-dimensional reconstruction using a ray casting algorithm to generate a first three-dimensional image. Since the significant region image is a portion that is focused on in laparoscopic surgery, a more accurate three-dimensional reconstruction method is used for reconstruction to display the internal structural information and details of the object. Then, the insignificant region image is subjected to a three-dimensional reconstruction using a marching cube algorithm to generate a second three-dimensional image. Since the insignificant region is a portion that does not need to be focused on in laparoscopic surgery, a three-dimensional reconstruction algorithm that cannot display the internal characteristic information of the object is used for reconstruction. Finally, the first three-dimensional image and the second three-dimensional image are combined to obtain a three-dimensional image of the target organ. Since the marching cubes algorithm belongs to surface rendering, which processes part of the data in the volume data field, it has the advantages of small amount of calculation and fast speed, thereby reducing the amount of calculation for the reconstruction of the entire target organ and improving the reconstruction speed. At the same time, the image of the preliminary location of the lesion is reconstructed using a ray casting algorithm, which can display the internal structure information and details of the object, thereby improving the accuracy of the lesion location and helping doctors to locate the lesion. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0046] Figure 1 A flow chart of an imaging and processing method for a target organ in laparoscopic surgery provided by an embodiment of the present invention;

[0047] Figure 2 A detailed flowchart of step S110 provided in an embodiment of the present invention;

[0048] Figure 3 A flow chart of the scanning image segmentation steps provided by an embodiment of the present invention;

[0049] Figure 4A structural block diagram of an imaging and processing system for target organs in laparoscopic surgery provided by an embodiment of the present invention;

[0050] Figure 5 A structural block diagram of an electronic device provided by an embodiment of the present invention.

[0051] Icons: 110 - scanning image acquisition module; 120 - salient region detection module; 130 - region segmentation; 140 - first three-dimensional reconstruction module; 150 - second three-dimensional reconstruction module; 160 - three-dimensional image combination module; 101 - memory; 102 - processor; 103 - communication interface. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0053] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0054] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0055] Example

[0056] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0057] Please see Figure 1-3 , Figure 1 A flow chart of an imaging and processing method for a target organ in laparoscopic surgery provided by an embodiment of the present invention, Figure 2 A detailed flowchart of step S110 provided in an embodiment of the present invention, Figure 3 A flow chart of the scanning image segmentation steps provided in an embodiment of the present invention. The imaging and processing method for the target organ in laparoscopic surgery comprises the following steps:

[0058] Step S110: Obtain a scanned image of the target organ, which includes a preliminary location of the lesion; the scanned image of the target organ may be image data output by medical imaging equipment such as CT and MRI, and the target organ refers to an organ that requires laparoscopic surgery, such as the stomach, intestines, etc. In this embodiment, a preliminary detection of the target organ may be performed before scanning, and a lesion location mark may be set, which may be accomplished by the following steps:

[0059] First, obtain preliminary information on the target organ; when the lesion of the target organ is detected through endoscopy, clamp the positioning hook with identification function at the lesion to obtain preliminary information on the target organ. For example, in a gastroscopy operation, the positioning hook is clamped at the lesion inside the stomach wall. The positioning hook can be mounted through an endoscope and clamped inside the stomach wall after endoscopic surgery. It has the characteristics of small size, light weight, safety and non-toxicity. A positioning tag with positioning and identification function is equipped at the grip of the hook. There are two ways to implement it: the first is a micro RFID tag, and identification requires the installation of a positioning antenna on the positioning laparoscope pole. The second is a small permanent magnet, and a magnetic sensor can be used on the laparoscope pole for positioning and identification. It is also possible to use both technologies at the same time for dual protection.

[0060] Then, the target organ for preliminary positioning is scanned according to the information of the target organ for preliminary positioning, and a scanned image of the target organ is obtained, and the scanned image includes the position of the preliminary positioned lesion. Since the metal components in the hook will be highlighted in the scanned image such as the CT image, the highlighted position in the image is the position of the preliminary positioned lesion. By obtaining a scanned image of the target organ with the position of the preliminary positioned lesion, the preliminary positioning of the lesion position can be achieved.

[0061] Step S120: Perform salient area detection on the scanned image to obtain salient areas, which include the preliminary location of the lesion. The above salient area detection refers to the machine simulating humans to quickly and accurately identify salient areas in the visual field, simulating human visual characteristics through intelligent algorithms, and extracting salient areas in the image (i.e., areas of interest to humans). Since there will be highlighted images in the scanned image, this part of the image will be displayed in the salient area. The above salient area detection can be achieved using existing salient detection technology, which will not be repeated here. The preliminary location of the lesion can be detected through saliency detection, which is convenient for later image segmentation.

[0062] Step S130: Segment the scanned image along the significant area to obtain a significant area image and a non-significant area image; the significant area image obtained above includes the preliminary location of the lesion, and the non-significant area image includes the normal position of the target organ. In laparoscopic surgery, the location of the lesion that is initially located needs to be paid special attention to, and the preliminary location of the lesion needs to be further confirmed. To ensure the effectiveness of the significant area, the area judgment can be performed before segmentation, which specifically includes the following steps:

[0063] Step 1: Calculate and compare the area of ​​the significant region with the preset region area threshold to obtain a comparison result; the preset region area threshold can be set based on experience. After obtaining the significant region, first calculate the area, and compare the obtained area value with the preset region area threshold. The obtained comparison results include less than the preset region area threshold and not less than the preset region area threshold. For example: the area of ​​the significant region is calculated to be 5 square centimeters, and the region area threshold is 4 square centimeters. After comparison, the comparison result is: less than the preset region area threshold.

[0064] Step 2: Determine whether the salient region is valid based on the comparison result. If it is valid, segment the scanned image along the salient region to obtain the salient region image and the non-salient region image. If it is invalid, execute the third step. If the comparison result is less than the preset area threshold, it is considered invalid. Otherwise, it is considered valid. If it is invalid, it is considered that the salient region is too small and needs to be further expanded, so execute the third step.

[0065] Step 3: Expand the salient area according to the preset expansion rules to obtain a new salient area; the expansion refers to expanding the salient area circumferentially toward the non-salient area, and the preset expansion rules refer to pre-set rules, which can be set based on experience, such as: expansion of 5 cm. The expansion makes the salient area larger, thereby making the new salient area effective.

[0066] Step 4: Segment the scanned image along the salient region to obtain a salient region image and a non-salient region image. The new salient region obtained is valid and meets the requirements, and can then be segmented. The above segmentation is performed along the edge of the salient region.

[0067] Step S140: The salient region image is three-dimensionally reconstructed using a ray casting algorithm to generate a first three-dimensional image; the above three-dimensional reconstruction can be drawn using a visualization toolkit (VTK). The three-dimensional image reconstruction based on VTK mainly includes two categories: surface drawing and volume drawing. The ray casting algorithm belongs to volume drawing. Volume drawing is oriented to the entire volume data, and processes each voxel in the volume data field to make the reconstruction effect more accurate. The above volume drawing is to use a ray casting algorithm to perform spatial scanning on the salient region image, and the high-definition image generated is the first three-dimensional image. Specifically, sampling points are set on a ray that passes through the three-dimensional volume data field emitted by each pixel on the screen, and the distances between the sampling points are equal; the eight data points closest to a certain sampling point are formed into a voxel, and the color value and opacity of the voxel are respectively calculated by cubic linear interpolation; thereby, the color values ​​and opacity of all sampling points are obtained and they are synthesized respectively, and the synthesis method can be from front to back or from back to front. The volume rendering process is generally divided into four steps: projection, blanking, rendering and synthesis. After all the data is calculated by the ray casting algorithm, the data is processed by window drawing, display, interaction, etc. to complete the volume rendering. In volume rendering, different materials are assigned corresponding transparency, which can compare the shape characteristics and hierarchical relationships between the materials. Therefore, the introduction of transparency improves the overall display effect of the data.

[0068] Step S150: The non-significant area image is three-dimensionally reconstructed using the marching cube algorithm to generate a second three-dimensional image; the three-dimensional reconstruction can be drawn using the Visualization Toolkit (VTK). The three-dimensional image reconstruction based on VTK mainly includes two types: surface drawing and volume drawing. The marching cube algorithm belongs to surface drawing. Surface drawing is to extract the surface contour information of the part to be reconstructed in the image for three-dimensional drawing. Only the surface of the object is reconstructed. Since surface drawing processes part of the data in the volume data field, it has the advantages of small amount of calculation and fast speed. The surface drawing based on VTK mainly includes the following steps:

[0069] First, extract multiple data points connected in any two layers in the non-significant region image to form cubes of equal size; the above scanned image can be a scanned image including multiple angles, and the salient region image obtained after performing salient region detection can also include salient region images of multiple angles and multiple non-significant region images. These salient region images are regarded as a three-dimensional volume data field, and 8 adjacent data points in two consecutive layers are taken to form cubes of equal size, which are called voxels, and the 8 vertices of the voxels are called corner points.

[0070] Then, each corner point of each cube is compared with the preset threshold to obtain the state value of each corner point in each cube; set the threshold T, traverse the data values ​​of the 8 corner points u (i, j, k) in each voxel, and compare them with the preset threshold T to obtain the state value of the corner point in each voxel. The value less than the preset threshold is marked as 0, and the value greater than the preset threshold is marked as 1.

[0071] Then, find the cube with equal value points according to the state value; each point in the voxel can be calculated by linear interpolation method, and the formula is:

[0072] p(x, y, z)=k0+k1x+k2y+k3z+k4xy+k5xz+k6yz+k7xyz, where k i (i=0, 1, ..., 7) is a constant uniquely determined by the corner point.

[0073] Then, the grayscale values ​​of the equal-value points in the cube are extracted and calculated;

[0074] Finally, each isovalue point is connected into triangular facets according to a preset topological form to form an isovalue surface and generate a second three-dimensional image. The above triangular facets are spliced ​​to form an isovalue surface, that is:

[0075] The correct connection of the isosurface {(x, y, z)|p(x, y, z) = T} requires the calculation of the coordinates of the isovalue points and the normal vector by linear interpolation method. The calculation formula is:

[0076] Among them: C is the coordinate of the equivalent point, C1 and C2 are the coordinates of the two end points of the equivalent point; N is the normal vector, N1 and N2 are the normal vectors of the two end points; T1 and T2 are the gray values ​​of the two end points.

[0077] Step S160: Combine the first three-dimensional image and the second three-dimensional image to obtain a three-dimensional image of the target organ. The above combination is to combine the first three-dimensional image and the second three-dimensional image along the edge of the significant area to obtain the three-dimensional image of the target organ. In order to facilitate rapid identification of the combined surface during combination, the edge of the significant area can be marked for easy identification. The above mark can be a color mark or a highlight mark. During combination, the surfaces with marked edges are combined. The final three-dimensional image of the target organ can be obtained by combination.

[0078] In the above implementation process, a scanned image of the target organ is obtained, and the scanned image includes the preliminary location of the lesion, so that the scanned image can display the preliminary location of the lesion; then, a significant area detection is performed on the scanned image to obtain a significant area, and the significant area includes the preliminary location of the lesion; then, the scanned image is segmented along the significant area to obtain a significant area image and a non-significant area image. By performing a significant detection on the scanned image, the scanned image can be segmented into an image including the preliminary location of the lesion and an image not including the image; then, the significant area image is three-dimensionally reconstructed using a ray casting algorithm to generate a first three-dimensional image. The significant area image is the part that is focused on in laparoscopic surgery, so a more accurate three-dimensional reconstruction method is used for reconstruction, which can display the internal structure information and details of the object; then, the non-significant area image is three-dimensionally reconstructed using a marching cube algorithm to generate a second three-dimensional image. The non-significant area is the part that does not need to be focused on in laparoscopic surgery, so a three-dimensional reconstruction algorithm that cannot display the internal feature information of the object is used for reconstruction; finally, the first three-dimensional image and the second three-dimensional image are combined to obtain a three-dimensional image of the target organ. Since the marching cubes algorithm belongs to surface rendering, which processes part of the data in the volume data field, it has the advantages of small amount of calculation and fast speed, thereby reducing the amount of calculation for the reconstruction of the entire target organ and improving the reconstruction speed. At the same time, the image of the preliminary location of the lesion is reconstructed using a ray casting algorithm, which can display the internal structure information and details of the object, thereby improving the accuracy of the lesion location and helping doctors to locate the lesion.

[0079] Based on the same inventive concept, the present invention also proposes an imaging and processing system for target organs in laparoscopic surgery, see Figure 4 , Figure 4 A structural block diagram of an imaging and processing system for a target organ in laparoscopic surgery provided by an embodiment of the present invention, the imaging and processing system for a target organ in laparoscopic surgery comprises:

[0080] A scanning image acquisition module 110 is used to acquire a scanning image of a target organ, wherein the scanning image includes a preliminary location of a lesion;

[0081] A significant region detection module 120 is used to perform significant region detection on the scanned image to obtain a significant region, wherein the significant region includes a preliminary location of the lesion;

[0082] Region segmentation 130, for segmenting the scanned image along the salient regions to obtain salient region images and non-salient region images;

[0083] A first 3D reconstruction module 140 is used to perform 3D reconstruction on the salient region image by using a ray casting algorithm to generate a first 3D image;

[0084] A second 3D reconstruction module 150 is used to perform 3D reconstruction on the non-significant region image by using a marching cube algorithm to generate a second 3D image;

[0085] The three-dimensional image combination module 160 is used to combine the first three-dimensional image and the second three-dimensional image to obtain a three-dimensional image of the target organ.

[0086] In the above implementation process, a scanning image of the target organ is acquired by the scanning image acquisition module 110, and the scanning image includes the preliminary location of the lesion, so that the preliminary location of the lesion can be displayed in the scanning image; then the significant region detection module 120 performs significant region detection on the scanning image to obtain a significant region, and the significant region includes the preliminary location of the lesion; then the region segmentation 130 segments the scanning image along the significant region to obtain a significant region image and a non-significant region image. By performing a significant detection on the scanning image, the scanning image can be segmented into an image including the preliminary location of the lesion and an image not including the lesion; then the first three-dimensional reconstruction module 140 segments the significant region into a non-significant region image. The regional image is three-dimensionally reconstructed using a ray casting algorithm to generate a first three-dimensional image. For the significant regional image, it is the part that is focused on in laparoscopic surgery, so a more accurate three-dimensional reconstruction method is used for reconstruction, which can display the internal structural information and details of the object; the second three-dimensional reconstruction module 150 uses a moving cube algorithm to three-dimensionally reconstruct the non-significant regional image to generate a second three-dimensional image. For the non-significant region, it is the part that does not need to be focused on in laparoscopic surgery, so a three-dimensional reconstruction algorithm that cannot display the internal characteristic information of the object is used for reconstruction; the three-dimensional image combination module 160 combines the first three-dimensional image and the second three-dimensional image to obtain a three-dimensional image of the target organ. Since the moving cube algorithm belongs to surface rendering, and surface rendering processes part of the data in the volume data field, it has the advantages of small amount of calculation and fast speed, thereby reducing the amount of reconstruction calculation of the entire target organ and improving the reconstruction speed. At the same time, the image for the preliminary positioning of the lesion position is reconstructed using a ray casting algorithm, which can display the internal structural information and details of the object, thereby improving the accuracy of the lesion position and helping doctors to locate the lesion.

[0087] See also Figure 5 , Figure 5A schematic structural block diagram of an electronic device provided in an embodiment of the present application. The electronic device includes a memory 101, a processor 102 and a communication interface 103, and the memory 101, the processor 102 and the communication interface 103 are electrically connected to each other directly or indirectly to realize data transmission or interaction. For example, these elements can be electrically connected to each other through one or more communication buses or signal lines. The memory 101 can be used to store software programs and modules, such as a program instruction / module corresponding to an imaging and processing system for a target organ in laparoscopic surgery provided in an embodiment of the present application. The processor 102 executes various functional applications and data processing by executing the software programs and modules stored in the memory 101. The communication interface 103 can be used to communicate signaling or data with other node devices.

[0088] Among them, the memory 101 can be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable read-only memory (EEPROM), etc.

[0089] The processor 102 may be an integrated circuit chip with signal processing capability. The processor 102 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0090] Understandably, Figure 5 The structure shown is for illustration only. The electronic device may also include Figure 5 More or fewer components as shown, or with Figure 5 Different configurations are shown. Figure 5 Each component shown in the figure can be implemented by hardware, software or a combination thereof.

[0091] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to the multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or the flowchart, and the combination of boxes in the block diagram and / or the flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0092] In addition, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0093] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0094] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

[0095] It will be apparent to those skilled in the art that the present application is not limited to the details of the exemplary embodiments described above, and that the present application can be implemented in other specific forms without departing from the spirit or essential features of the present application. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the present application is defined by the appended claims rather than the above description, and it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims be included in the present application. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

Claims

1. A method for imaging and processing a target organ in laparoscopic surgery, characterized in that: The following steps are involved: Acquiring a scanned image of the target organ, wherein the scanned image includes a preliminary location of the lesion; Performing salient region detection on the scanned image to obtain a salient region, wherein the salient region includes a preliminary localized lesion position; The scanned image is segmented along the salient region to obtain a salient region image and a non-salient region image, including: A1: calculating and comparing the area of ​​the salient region with a preset region area threshold to obtain a comparison result; A2: judging whether the salient region is valid according to the comparison result, if valid, the scanned image is segmented along the salient region to obtain a salient region image and a non-salient region image; if invalid, executing A3; A3: expanding the salient region according to a preset expansion rule to obtain a new salient region; A4: segmenting the scanned image along the salient region to obtain a salient region image and a non-salient region image; Performing three-dimensional reconstruction on the salient region image by using a ray casting algorithm to generate a first three-dimensional image; The non-significant region image is subjected to three-dimensional reconstruction using a marching cube algorithm to generate a second three-dimensional image, including: extracting a plurality of data points connected in any two layers in the non-significant region image to form cubes of equal size; comparing each corner point of each cube with a preset threshold value to obtain a state value of each corner point in each cube; finding a cube with equal-value points according to the state value; extracting and calculating the grayscale values ​​of the equal-value points in the cube; connecting each equal-value point into a triangular facet according to a preset topological form to form an isosurface to generate a second three-dimensional image; The first three-dimensional image and the second three-dimensional image are combined to obtain a three-dimensional image of the target organ.

2. The imaging and processing method for target organs in laparoscopic surgery according to claim 1, characterized in that: The step of obtaining a scanned image of the target organ, wherein the scanned image includes a step of preliminarily locating the position of the lesion, comprises the following steps: Obtain preliminary information on the location of target organs; The initially located target organ is scanned according to the initially located target organ information to obtain a scanned image of the target organ, wherein the scanned image includes the initially located lesion position.

3. The imaging and processing method for target organs in laparoscopic surgery according to claim 1, characterized in that: The step of combining the first three-dimensional image and the second three-dimensional image to obtain a three-dimensional image of the target organ comprises the following steps: The first three-dimensional image and the second three-dimensional image are combined along the edge of the salient region to obtain a three-dimensional image of the target organ.

4. The imaging and processing method for target organs in laparoscopic surgery according to claim 3, characterized in that: The following steps are also included: Mark the edges of the salient regions for easy identification.

5. The imaging and processing method for target organs in laparoscopic surgery according to claim 1, characterized in that: The step of performing three-dimensional reconstruction of the salient region image by using a ray casting algorithm to generate a first three-dimensional image comprises the following steps: A ray casting algorithm is used to perform spatial scanning on the salient region image to generate a first three-dimensional image.

6. An imaging and processing system for target organs in laparoscopic surgery, characterized in that: include: A scanning image acquisition module, used to acquire a scanning image of the target organ, wherein the scanning image includes a preliminary location of the lesion; A salient region detection module is used to perform salient region detection on the scanned image to obtain a salient region, wherein the salient region includes a preliminary localized lesion position; Region segmentation, used for segmenting the scanned image along the significant region to obtain the significant region image and the non-significant region image, including: A1: calculating and comparing the area of ​​the significant region with the preset region area threshold to obtain the comparison result; A2: judging whether the significant region is valid according to the comparison result, if valid, segmenting the scanned image along the significant region to obtain the significant region image and the non-significant region image; if invalid, executing A3; A3: expanding the significant region according to the preset expansion rule to obtain a new significant region; A4: segmenting the scanned image along the significant region to obtain the significant region image and the non-significant region image; A first three-dimensional reconstruction module, used to perform three-dimensional reconstruction on the salient region image by using a ray casting algorithm to generate a first three-dimensional image; The second three-dimensional reconstruction module is used to perform three-dimensional reconstruction on the non-significant area image by using a moving cube algorithm to generate a second three-dimensional image, including: extracting multiple data points connected in any two layers in the non-significant area image to form cubes of equal size; comparing each corner point of each cube with a preset threshold value to obtain a state value of each corner point in each cube; finding a cube with equal-value points according to the state value; extracting and calculating the grayscale values ​​of the equal-value points in the cube; connecting each equal-value point into a triangular facet according to a preset topological form to form an isosurface to generate the second three-dimensional image; The three-dimensional image combination module is used to combine the first three-dimensional image and the second three-dimensional image to obtain a three-dimensional image of the target organ.

7. An electronic device, characterized in that: include: A memory for storing one or more programs; processor; When the one or more programs are executed by the processor, the method according to any one of claims 1 to 5 is implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images