A laparoscopic ultrasound imaging method and ultrasound imaging device

Abstract

The laparoscopic ultrasonic imaging method and ultrasonic imaging device provided by the present invention identify the vascular structures in the ultrasonic images of multiple target areas and obtain sub-three-dimensional vascular images of each target area; determine the vascular type of the vascular structure in the ultrasonic image of each target area; obtain the position change information of the ultrasonic probe when performing ultrasonic scanning on each target area, and determine the position information of the vascular structure in the ultrasonic image of each target area based on the position change information; and then, based on the determined position information of the vascular structure, splice the sub-three-dimensional vascular images of each target area to obtain a three-dimensional vascular image and display it. The vascular type can also be displayed differently in the three-dimensional vascular image, so that the doctor can more comprehensively observe the lesions and the three-dimensional spatial status of the blood vessels in the surgical-related area, and better plan and implement puncture or resection surgery.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of medical devices, in particular to a laparoscopic ultrasound imaging method and an ultrasound imaging device. BACKGROUND

[0002] With the development of minimally invasive surgery technology, especially laparoscopic surgery technology, the treatment of many abdominal organ diseases has increasingly adopted laparoscopic surgery technology. In the process of laparoscopic surgery for abdominal organ diseases including liver cancer, laparoscopic ultrasound (LUS) is often used for diagnosis and evaluation and auxiliary guidance. Since LUS can help doctors see the structure below the surface and perform palpation, it is a good auxiliary guidance image paradigm in laparoscopic surgery.

[0003] Currently, laparoscopic ultrasound imaging can provide two-dimensional ultrasound section images. During the surgery, the doctor needs to rely on imagination to form a three-dimensional cognition according to the two-dimensional ultrasound section images for good auxiliary guidance, which requires higher experience and ability of the doctor.

[0004] In view of the above, there is a certain demand for three-dimensional reconstruction imaging of laparoscopic ultrasound data. SUMMARY

[0005] In view of the above, the present application provides a laparoscopic ultrasound imaging method and an ultrasound imaging device, which will be described in detail below.

[0006] According to a first aspect, in an embodiment, a laparoscopic ultrasound imaging method is provided, comprising:

[0007] obtaining an ultrasound image of each target region of a plurality of target regions of a target tissue obtained by ultrasound scanning of the plurality of target regions;

[0008] identifying a blood vessel structure in the ultrasound image of each target region to obtain a sub-three-dimensional blood vessel image of each target region;

[0009] obtaining a color Doppler image or a vector blood flow image of each target region;

[0010] obtaining a blood vessel type of the blood vessel structure in the ultrasound image of each target region according to the color Doppler image or the vector blood flow image of each target region, wherein the blood vessel type at least includes an arterial blood vessel type and a venous blood vessel type;

[0011] obtaining position change information of an ultrasound probe during ultrasound scanning of the plurality of target regions, and determining position information of the blood vessel structure in the ultrasound image of each target region based on the position change information;

[0012] splicing the sub-three-dimensional blood vessel images of the target regions to obtain one three-dimensional blood vessel image of the multiple target regions based on the position information of the blood vessel structure in the ultrasound images of the target regions, and displaying the three-dimensional blood vessel image, and differentiating the blood vessel types in the three-dimensional blood vessel image;

[0013] The differentiating the blood vessel types in the three-dimensional blood vessel image includes:

[0014] The differentiating the blood vessel types in the three-dimensional blood vessel image includes:

[0015] The three-dimensional reconstruction includes:

[0016] According to a second aspect, an embodiment provides a laparoscopic ultrasound imaging method, including:

[0017] obtaining ultrasound images of each target region in the multiple target regions of the target tissue obtained by ultrasound scanning of the multiple target regions of the target tissue;

[0018] identifying blood vessel structures in the ultrasound images of the target regions to obtain sub-three-dimensional blood vessel images of the target regions;

[0019] determining blood vessel types of the blood vessel structures in the ultrasound images of the target regions, wherein the blood vessel types at least include arterial blood vessel types and venous blood vessel types;

[0020] obtaining position change information of an ultrasound probe during ultrasound scanning of the multiple target regions, and determining position information of the blood vessel structures in the ultrasound images of the target regions based on the position change information;

[0021] splicing the sub-three-dimensional blood vessel images of the target regions to obtain one three-dimensional blood vessel image of the multiple target regions based on the position information of the blood vessel structure in the ultrasound images of the target regions, and displaying the three-dimensional blood vessel image, and differentiating the blood vessel types in the three-dimensional blood vessel image.

[0022] According to a third aspect, an embodiment provides an ultrasound imaging device, including:

[0023] an ultrasound probe configured with a position sensor for obtaining real-time position information of the ultrasound probe;

[0024] transmitting and receiving circuitry for controlling the ultrasound probe to transmit ultrasound waves to a target tissue, and receive ultrasound echo signals;

[0025] a processor for performing the method of any of the above.

[0026] According to a fourth aspect, in an embodiment there is provided a computer program product comprising computer programs and / or instructions which, when executed by a processor, implement the method of any of the above.

[0027] According to the laparoscopic ultrasound imaging method and the ultrasound imaging device of the above embodiments, the blood vessel structures in the ultrasound images of the multiple target regions are identified to obtain sub-three-dimensional blood vessel images of the target regions; the blood vessel types of the blood vessel structures in the ultrasound images of the target regions are determined; the position change information of the ultrasound probe when performing ultrasound scanning on the target regions is obtained, and the position information of the blood vessel structures in the ultrasound images of the target regions is determined based on the position change information; then, based on the determined position information of the blood vessel structures, a three-dimensional blood vessel image is obtained by splicing the sub-three-dimensional blood vessel images of the target regions and displayed, and the blood vessel types can be differentially displayed in the three-dimensional blood vessel image, so that the doctor can more comprehensively observe the three-dimensional spatial state of the lesions and blood vessels in the operation related region, and better perform puncture or resection surgery planning and implementation. BRIEF DESCRIPTION OF DRAWINGS

[0028] Figure 1 a structural block diagram of an embodiment of the ultrasound imaging device provided by the present application;

[0029] Figure 2 a flowchart of an embodiment of the laparoscopic ultrasound imaging method provided by the present application;

[0030] Figure 3 a schematic diagram of splicing sub-three-dimensional blood vessel images of multiple target regions to obtain a three-dimensional blood vessel image;

[0031] Figure 4 a schematic diagram of differentially displaying blood vessel types in a three-dimensional blood vessel image;

[0032] Figure 5 a flowchart of another embodiment of the laparoscopic ultrasound imaging method provided by the present application;

[0033] Figure 6 a flowchart of still another embodiment of the laparoscopic ultrasound imaging method provided by the present application. DETAILED DESCRIPTION

[0034] The present invention will be further described in detail below by means of specific embodiments in conjunction with the accompanying drawings. Similar elements in different embodiments are numbered with associated similar elements. In the following embodiments, many detailed descriptions are provided to enable the present application to be better understood. However, those skilled in the art will readily appreciate that some of the features may be omitted in different circumstances, or may be replaced by other elements, materials, or methods. In some cases, some operations related to the present application are not shown or described in the specification. This is to avoid the core portion of the present application being overwhelmed by excessive descriptions, and for those skilled in the art, it is not necessary to describe these related operations in detail. They will fully understand the related operations based on the description in the specification and the general technical knowledge in the art.

[0035] In addition, the features, operations, or characteristics described in the specification may be combined in any appropriate manner to form various embodiments. Furthermore, the steps or actions in the method description may be reordered or adjusted in a manner readily apparent to those skilled in the art. Therefore, the various sequences in the specification and drawings are provided solely for the purpose of clearly describing a particular embodiment and are not intended to be mandatory, unless otherwise specified.

[0036] The serial numbers assigned to components herein, such as "first," "second," etc., are used solely to distinguish the objects being described and do not convey any sequential or technical meaning. References to "connection" and "coupling" herein, unless otherwise specified, include both direct and indirect connections (couplings).

[0037] The present invention obtains ultrasonic data of multiple target areas (such as laparoscopic ultrasonic data) by performing multiple ultrasonic scans on the target tissue, and performs three-dimensional reconstruction of lesions and blood vessels on these ultrasonic data. Based on the navigation positioning information during the ultrasonic scan, three-dimensional splicing is performed, and colored display is performed in combination with blood flow information. This technical solution can provide doctors with a large-scale three-dimensional display of vascular trees and lesions during laparoscopic surgery, and provide blood flow direction information, so that doctors no longer need to rely on spatial imagination. When performing puncture or resection, they can more clearly understand the spatial position relationship between liver blood vessels and lesions in the intraoperative state, and understand the branch properties of the vascular tree, making it more convenient to plan puncture or resection and perform surgery. This is explained in detail below through some embodiments.

[0038] like Figure 1 As shown, the ultrasonic imaging device includes an ultrasonic probe 10 , a transmitting and receiving circuit 20 , an echo processing module 30 and a processor 40 .

[0039] The ultrasound probe 10 is used to transmit ultrasound waves to the target tissue A and receive corresponding ultrasound echo signals to obtain ultrasound data, such as two-dimensional ultrasound data or three-dimensional ultrasound data. In some embodiments, the ultrasound probe 10 includes a plurality of array elements for converting electrical signals and ultrasound waves to each other, so as to transmit ultrasound waves to the target tissue A and receive corresponding ultrasound echo signals. The array elements can transmit ultrasound waves according to excitation electrical signals, or convert received ultrasound waves into electrical signals. Therefore, each array element can be used to transmit ultrasound waves to the target tissue A, and can also be used to receive ultrasound echo waves returned by the tissue. When performing ultrasound detection, it can be controlled by a transmission sequence and a reception sequence which array elements are used to transmit ultrasound waves and which array elements are used to receive ultrasound waves, or it can be controlled by time slots in which array elements are used to transmit ultrasound waves or receive ultrasound echo waves. All array elements participating in ultrasound wave transmission can be excited by electrical signals at the same time, so as to transmit ultrasound waves at the same time; or the array elements participating in ultrasound wave transmission can also be excited by a plurality of electrical signals with a time interval, so as to continuously transmit ultrasound waves with a time interval.

[0040] The transmission and reception circuit 20 is used to control the ultrasound probe 10 to perform transmission of ultrasound waves and reception of ultrasound echo signals. For example, the transmission and reception circuit 20 is used to control the ultrasound probe 10 to transmit ultrasound waves to the target tissue A, and is also used to control the ultrasound probe 10 to receive ultrasound echo signals reflected by the target tissue A. In some embodiments, the transmission and reception circuit 20 is used to generate a transmission sequence and a reception sequence, and outputs the transmission sequence and the reception sequence to the ultrasound probe 10. The transmission sequence is used to control part or all of the plurality of array elements in the ultrasound probe 10 to transmit ultrasound waves to the target tissue A, and parameters of the transmission sequence include the number of array elements used for transmission and ultrasound wave transmission parameters (such as amplitude, frequency domain, transmission times, transmission interval, transmission angle, wave type and / or focus position, etc.). The reception sequence is used to control part or all of the plurality of array elements to receive echoes of ultrasound waves after passing through the target tissue, and parameters of the reception sequence include the number of array elements used for reception and echo reception parameters (such as reception angle, depth, etc.). The ultrasound wave parameters in the transmission sequence and the echo parameters in the reception sequence are different according to different uses of the ultrasound echo or different images generated according to the ultrasound echo.

[0041] The echo processing module 30 is used to process the ultrasonic echo signals received by the ultrasonic probe 10, for example, by filtering, amplifying, and beamforming the ultrasonic echo signals to obtain ultrasonic image data. In a specific embodiment, the echo processing module 30 can output the ultrasonic image data to the processor 40, or it can first store the ultrasonic image data in a memory. When a calculation based on the ultrasonic image data is required, the processor 40 reads the ultrasonic image data from the memory. Those skilled in the art will understand that in some embodiments, when filtering, amplifying, beamforming, and other processing of the ultrasonic echo signals are not required, the echo processing module 30 can be omitted. In some embodiments, some or even all of the functions of the echo processing module 30 can also be performed by the processor 40, that is, the echo processing module 30 can be part of the processor 40.

[0042] The processor 40 is used to acquire ultrasound image data and use relevant algorithms to obtain the required parameters or images. In some embodiments of the present invention, the processor 40 includes, but is not limited to, a central processing unit (CPU), a microcontroller unit (MCU), a field-programmable gate array (FPGA), and a digital signal processing unit (DSP), etc., which are devices used to interpret computer instructions and process data in computer software. In some embodiments, the processor 40 is used to execute various computer applications stored in a computer-readable storage medium, thereby realizing the various functions of the ultrasound imaging device.

[0043] In some embodiments, the ultrasound imaging device may further include a human-computer interaction device 50. The human-computer interaction device 50 is used for human-computer interaction, namely, receiving user input and outputting visual information. The user input may be received using a keyboard, operating buttons, a mouse, a trackball, or a touch screen integrated with a display. The visual information may be output using a display. The display may be used to display information, such as parameters and images calculated by the processor 40. Those skilled in the art will appreciate that in some embodiments, the ultrasound imaging device may not have an integrated display, but may instead be connected to an external display to display information.

[0044] It should be noted that Figure 1 The structure is only for illustration and may also include Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown. Figure 1 Each component shown in the figure may be implemented using hardware and / or software.

[0045] The ultrasound probe 10 of the embodiment is configured with a position sensor. The position sensor is used to acquire real-time position information of the ultrasound probe 10.

[0046] The processor 40 can control the ultrasound imaging device to perform laparoscopic ultrasound imaging, and the specific process is as follows Figure 2 As shown in the figure, the method comprises the following steps:

[0047] Step 1: The processor 40 acquires the ultrasound images of each of the plurality of target regions obtained by performing ultrasound scanning on the plurality of target regions of the target tissue. In the embodiment, the ultrasound probe 10 is configured with a position sensor, which is used to detect the position of the ultrasound probe 10 to obtain the position change information of the ultrasound probe 10. The position sensor can be a magnetic positioning device, such as a magnetic navigation system and a magnetic positioning sensor. However, the position sensor can also not be limited to a magnetic positioning device, and other positioning devices with the same or similar positioning capabilities can also be used, such as inertial navigation and inertial measurement positioning sensors, or optical navigation and positioning sensors, etc. The preferred installation method of the position sensor is to be built-in in the laparoscopic ultrasound probe 10, that is, the ultrasound probe 10 is built-in with the position sensor. Of course, in other embodiments, the position sensor can also be installed on the shell of the ultrasound probe 10 using an external fixing device.

[0048] The target tissue can be various tissues of a living body, such as the uterus, heart, thyroid, ovary, kidney, liver, etc. In the embodiment, the target tissue is taken as the liver tissue for illustration. The medical staff can use the laparoscopic ultrasound probe with a built-in electromagnetic positioning device to perform liver scanning to obtain the ultrasound images of the plurality of target regions of the target tissue, so as to confirm the target tumor and the blood vessel information of the related region based on the ultrasound images. Among them, the ultrasound images of the target regions can be three-dimensional ultrasound images or multiple frames of two-dimensional ultrasound images, which can be obtained by the medical staff in B mode, C mode, contrast mode, or elasticity mode, which will be described below.

[0049] The medical staff can start ultrasound data acquisition through the keys or touch screen operation on the ultrasound probe 10, and at the same time, the position sensor is also started, such as performing ultrasound data acquisition on the related liver region. When performing ultrasound data acquisition, the ultrasound probe 10 can be controlled to move from the head side to the foot side of the patient in the plurality of target regions Figure 3The two target regions D1 and D2 are scanned, collected, and the lesions (as shown in the green region in the figure), the blood vessel contours (as shown in the red region in the figure), and the positioning information are obtained, and the color Doppler blood flow signals (C mode) are collected to obtain the blood flow direction and blood flow. The ultrasound data collection can be 3D ultrasound data collection, automatic reconstruction to generate 3D data (three-dimensional ultrasound image), such as Freehand 3D scanning reconstruction based on a positioning navigation system, 3D ultrasound collection using a volume probe. Of course, two-dimensional ultrasound data collection can also be performed. The collection scanning method of the medical staff can be to control the ultrasound probe 10 to scan from the patient's head side to the foot side, and the purpose is to obtain the scanning direction as a reference for blood vessel type (artery / vein) identification. However, the scanning and collection method to achieve this purpose is not limited to this, and can also include scanning from the patient's foot side to the head side, from the patient's left side (right side) to the right side (left side), or free scanning combined with automatic judgment of positioning information.

[0050] It can be seen that the present application can utilize the ultrasound image obtained by the doctor during the conventional laparoscopic ultrasound examination for subsequent processing, and ultimately obtain a relatively large three-dimensional blood vessel image without the doctor performing additional scanning operations.

[0051] Step 2, the processor 40 identifies the blood vessel structure in the ultrasound image of each target region to obtain a sub-three-dimensional blood vessel image of each target region. The first choice can be to directly perform 3D detection and segmentation on the three-dimensional ultrasound image (3D ultrasound data) to obtain a blood vessel model, and of course, it can also be to first perform tracking detection frame by frame based on a single slice, and then perform 3D reconstruction according to the positioning information. That is, the ultrasound image of the target region can be a three-dimensional ultrasound image (3D ultrasound data), or a plurality of two-dimensional ultrasound images, so the specific process of obtaining the sub-three-dimensional blood vessel image of the target region is different, which will be illustrated below.

[0052] In one embodiment, the ultrasound image of each target region is a three-dimensional ultrasound image. For this case, the processor 40 can directly detect and segment the blood vessel structure in the three-dimensional ultrasound image to obtain the sub-three-dimensional blood vessel image of each target region. That is, the three-dimensional ultrasound image of each target region is detected and segmented to obtain the blood vessel image, thereby obtaining the sub-three-dimensional blood vessel image of each target region. The processor 40 automatically detects and segments the blood vessel structure in the three-dimensional ultrasound image, which can have various implementation manners, three of which will be illustrated below.

[0053] The first implementation is to detect and segment the blood vessel structure in the three-dimensional ultrasound image based on a traditional image processing segmentation algorithm. For example, the processor 40 uses algorithms such as filtering, edge detection, and / or threshold segmentation to perform threshold division based on image pixels, grayscale, and other information in the three-dimensional ultrasound image, thereby segmenting the blood vessel structure in the three-dimensional ultrasound image. The filtering, edge detection, threshold segmentation, and other algorithms can specifically use Roberts algorithm, Prewitt algorithm, Otsu method (OTSU), and the like.

[0054] The second implementation is to detect and segment the blood vessel structure in the three-dimensional ultrasound image based on a traditional machine learning semantic segmentation algorithm. For example, the idea of patching and then classifying the image can be used. This implementation mainly includes two steps: the processor 40 first extracts features from the image blocks of the three-dimensional ultrasound image. Specifically, the PCA (Principal Component Analysis) algorithm, LDA (Linear Discriminant Analysis) algorithm, or deep neural network can be used to extract features from the image blocks of the three-dimensional ultrasound image to establish a feature vector, and then the feature vector is classified. Specifically, the k-NN algorithm (k-Nearest Neighbor algorithm), Random Forest, or SVM (Support Vector Machine) algorithm can be used to classify the feature vector. The processor 40 then splices the classified image blocks back into the entire image to obtain the segmentation result (blood vessel structure).

[0055] The third implementation is to detect and segment the blood vessel structure in the three-dimensional ultrasound image based on a deep learning semantic segmentation algorithm. For example, a supervised learning strategy can be used in advance. A deep neural network is established by stacking convolutional layers, pooling layers, up-sampling layers, deconvolutional layers, and fully connected layers, and a real segmentation annotation is made into a mask with the same size as the input image (three-dimensional ultrasound image) as supervision information, so that the neural network model learns and outputs the region to be segmented. The two-dimensional segmentation network used can use common Mask R-CNN, U-Net, Solo, and the like. In this way, the processor 40 only needs to input the three-dimensional ultrasound image into the trained neural network model to obtain the region to be segmented (i.e., the blood vessel structure) output by the neural network model.

[0056] After the processor 40 detects and segments the blood vessel structure in the three-dimensional ultrasound image by any of the above implementations, the three-dimensional reconstruction of the target region can be performed to obtain a sub-three-dimensional blood vessel image of the target region.

[0057] In another embodiment, the ultrasound images of each target region are multiple two-dimensional ultrasound images (e.g., ultrasound B-mode images). In this embodiment, there are also multiple ways to obtain the sub-three-dimensional blood vessel images of each target region, which are also illustrated by taking three examples.

[0058] In a first way, the processor 40 reconstructs the multiple two-dimensional ultrasound images into a three-dimensional ultrasound image based on the position change information, which is known, i.e., the orientation and movement trajectory of the ultrasound probe 10 are known, and the relative position relationship between the ultrasound probe 10 and each pixel point of the two-dimensional ultrasound images scanned by the ultrasound probe 10 is also known, so that the three-dimensional ultrasound image can be reconstructed according to the two. This is also a common three-dimensional imaging method. Then, the processor 40 detects and segments the blood vessel structure in the three-dimensional ultrasound image to obtain the sub-three-dimensional blood vessel images of each target region. After the multiple two-dimensional ultrasound images are reconstructed into a three-dimensional ultrasound image, it is the same as the previous embodiment, so the specific implementation ways of obtaining the sub-three-dimensional blood vessel images of each target region mentioned in the previous embodiment can be used. That is, the processor 40 can use the first, second, or third implementation way mentioned above to detect and segment the blood vessel structure in the three-dimensional ultrasound image, so as to obtain the sub-three-dimensional blood vessel images of each target region, which will not be repeated here.

[0059] In a second way, the processor 40 displays the multiple two-dimensional ultrasound images through the display of the human-computer interaction device 50, obtains the blood vessel structure in the multiple two-dimensional ultrasound images in response to the marking operation of the user on the displayed blood vessel structure in the multiple two-dimensional ultrasound images, and obtains the sub-three-dimensional blood vessel images of each target region based on the blood vessel structure in the multiple two-dimensional ultrasound images. That is, the display will display the multiple two-dimensional ultrasound images of the target region, and the user will mark the blood vessels on the two-dimensional ultrasound images, where the marking operation can be manual tracing or using a regular shape (such as a circle, an ellipse, a rectangle, etc.) to mark, so that the processor 40 knows the blood vessel structure and its position on each two-dimensional ultrasound image. Based on each blood vessel structure and its position, the sub-three-dimensional blood vessel images can be generated, such as based on the position of each blood vessel structure in the two-dimensional ultrasound image, the image, and the position relationship between the multiple two-dimensional ultrasound images, three-dimensional reconstruction is performed to obtain the sub-three-dimensional blood vessel images. This way is equivalent to semi-automatically obtaining the sub-three-dimensional blood vessel images.

[0060] The third way is that the processor 40 detects and segments the blood vessel structure of each two-dimensional ultrasound image in the plurality of two-dimensional ultrasound images to obtain the blood vessel structure in the plurality of two-dimensional ultrasound images. Compared with the second way, this way does not need user participation, and the blood vessel structure is automatically detected and segmented, so that the processor 40 knows the blood vessel structure and its position, and the sub-three-dimensional blood vessel image can be generated based on each blood vessel structure and its position. This way is equivalent to automatically obtaining the sub-three-dimensional blood vessel image.

[0061] In this way, the blood vessel structure in the two-dimensional ultrasound image is detected and segmented, and the specific implementation manner can be the same as that of detecting and segmenting the blood vessel structure in the three-dimensional ultrasound image, such as the first, second or third implementation manner. That is, the processor 40 can use filtering, edge detection and / or threshold segmentation algorithm to perform threshold division based on the image pixel, gray scale and other information in the two-dimensional ultrasound image, so as to segment the blood vessel structure in the two-dimensional ultrasound image. The processor 40 can also first extract features of the image block of the two-dimensional ultrasound image, establish a feature vector, then classify the feature vector, and then splice the classified image block back to the whole image to obtain the segmentation result (blood vessel structure). The processor 40 can also input the two-dimensional ultrasound image into the trained neural network model to obtain the region (i.e. blood vessel structure) segmented by the neural network model.

[0062] In the second way and the third way, the processor 40 obtains the sub-three-dimensional blood vessel image of each target region based on the blood vessel structure in the plurality of two-dimensional ultrasound images of the target region. Specifically, the sub-three-dimensional blood vessel image of each target region can be obtained by three-dimensional reconstruction based on the blood vessel structure in the plurality of two-dimensional ultrasound images of the target region and the position information of the blood vessel structure in the plurality of two-dimensional ultrasound images of the target region. The position information of the blood vessel structure in the two-dimensional ultrasound image of the target region can be determined based on the position change information, which will be described in detail in step 4.

[0063] Each target region has blood vessels, and this step is equivalent to extracting the three-dimensional image of the blood vessels of each target region through the ultrasound image. This facilitates subsequent splicing.

[0064] Step 3, the processor 40 determines the blood vessel type of the blood vessel structure in the ultrasound image of each target region, wherein the blood vessel type at least includes an arterial blood vessel type and a venous blood vessel type. That is, the processor 40 determines which blood vessels in each target region are arterial blood vessels and which blood vessels are venous blood vessels.

[0065] The ultrasound scan performed by the user in step 1 can be performed in B+C mode, that is, the ultrasound image obtained in step 1 is a color Doppler image (including an ultrasound B image obtained by B mode and a blood flow image obtained by C mode), of course, the ultrasound scan performed by the user in step 1 can also be performed in vector blood flow mode, that is, the ultrasound image obtained in step 1 is a vector blood flow image. Regardless of the manner, the ultrasound B image and the blood flow information can be obtained. Therefore, the processor 40 can obtain the color Doppler image or the vector blood flow image of each target region; according to the color Doppler image or the vector blood flow image of each target region, the blood vessel type of the blood vessel structure in the ultrasound image of each target region is obtained. Specifically, the processor 40 can obtain the blood flow information of the blood vessel structure in the ultrasound image of each target region according to the color Doppler image or the vector blood flow image of each target region, the blood flow information includes: blood flow intensity and / or blood flow velocity; in some embodiments, the blood flow information can also include blood flow direction and the like. Further, the processor 40 determines the blood vessel type of the blood vessel structure in the ultrasound image of each target region according to the blood flow information of the blood vessel structure in the ultrasound image of each target region. In some embodiments, the processor 40 can also obtain the scanning direction of the ultrasound probe as a reference for determining the blood vessel type. There is a relatively large difference between the blood flow intensity and the flow velocity in the arteries and the veins, and according to these information, it can be determined which blood vessels in the image are arteries and which blood vessels are veins.

[0066] The blood vessel type of the blood vessel structure can be determined by a predetermined model / algorithms. Considering that the more information input into the model / algorithms, the more accurate the blood vessel type result given, the ultrasound image, the sub-three-dimensional blood vessel image of the target region and the like can also be input into the model / algorithms. For example, if the ultrasound image of each target region is a three-dimensional ultrasound image, the processor 40 can input the ultrasound image (such as three-dimensional image point cloud data) of the target region, the sub-three-dimensional blood vessel image (such as three-dimensional image point cloud data) of the target region, and the blood flow information (at least one of signal intensity, direction, and flow velocity) of the blood vessel structure in the ultrasound image of the target region into the pre-trained multi-modal three-dimensional blood vessel type recognition model or into the predetermined multi-modal three-dimensional blood vessel type recognition algorithm, to obtain the blood vessel type of the blood vessel structure in the ultrasound image of the target region output by the multi-modal three-dimensional blood vessel type recognition model or the multi-modal three-dimensional blood vessel type recognition algorithm.

[0067] Among them, the multi-modal three-dimensional blood vessel type recognition model / algorithms can adopt one of the following two:

[0068] One kind of multi-modal three-dimensional blood vessel type recognition model / algorithm is a traditional machine learning-based classification model / algorithm. The classification model / algorithm establishes a feature vector based on the image pixels, grayscale, etc. of the ultrasound image of the target region and the sub-three-dimensional blood vessel image of the target region, for example, using PCA or LDA algorithm to process the image pixels, grayscale, etc. of the ultrasound image of the target region and the sub-three-dimensional blood vessel image of the target region, and establishes a feature vector. At the same time, the text data (such as blood flow information of the blood vessel structure) is also processed into a feature vector, and then the two feature vectors are fused. Then, the fused feature vector is classified, which can be classified using algorithms such as k-NN, Random Forest, or SVM, so as to output the classification result, that is, the blood vessel structure is classified to obtain the type (artery or vein) of each blood vessel structure.

[0069] Another multi-modal three-dimensional blood vessel type recognition model / algorithm is a deep learning-based classification algorithm. The classification model / algorithm usually adopts a supervised learning strategy, establishes a deep neural network through stacking convolutional layers, pooling layers, up-sampling layers, deconvolutional layers, and fully connected layers, etc. modules, and uses artificially labeled true classification results as supervision information. The neural network takes image or point cloud data as input, that is, the ultrasound image of the target region and the sub-three-dimensional blood vessel image of the target region are input into the model / algorithm, and the graphical feature vector is obtained through model operation. The text feature vector obtained based on the blood flow information of the blood vessel structure is input into the model / algorithm for learning and outputting the classification result, that is, the blood vessel structure is classified to obtain the type (artery or vein) of each blood vessel structure. Of course, the text feature vector can also be processed into an array data with the same dimension as the image or point cloud data, and the image or point cloud data can be spliced and input into the neural network for learning and outputting the classification result.

[0070] For example, if the ultrasound image of each target region is a multi-frame two-dimensional ultrasound image, the processor 40 can input the multi-frame two-dimensional ultrasound image of the target region, the blood vessel structure (such as a segmentation mask) segmented in the two-dimensional ultrasound image, and the blood flow information (at least one of signal intensity, direction, and flow rate) of the blood vessel structure in the ultrasound image of the target region into a pre-trained multi-modal two-dimensional blood vessel type recognition model or a preset multi-modal two-dimensional blood vessel type recognition algorithm to obtain the blood vessel type of the blood vessel structure in the ultrasound image of the target region output by the multi-modal two-dimensional blood vessel type recognition model or the multi-modal two-dimensional blood vessel type recognition algorithm.

[0071] Similarly, the multi-modal two-dimensional blood vessel type recognition model / algorithm can adopt one of the following two:

[0072] One kind of multi-modal two-dimensional blood vessel type recognition model / algorithm is a traditional machine learning-based classification model / algorithm. The classification model / algorithm is based on the two-dimensional ultrasound image of the target region and the image pixels, grayscale, etc. information of the blood vessel structure in the two-dimensional ultrasound image, establishes a feature vector, for example, the two-dimensional ultrasound image of the target region and the image pixels, grayscale, etc. information of the blood vessel structure in the two-dimensional ultrasound image are processed using PCA or LDA algorithm, a feature vector is established, and the text data (such as blood flow information of the blood vessel structure) is also processed into a feature vector, and then the two feature vectors are fused, and then the fused feature vectors are classified, which can be classified using algorithms such as k-NN, Random Forest, or SVM, etc. to output the classification result.

[0073] Another multi-modal two-dimensional blood vessel type recognition model / algorithm is a deep learning-based classification algorithm. The classification model / algorithm usually adopts a supervised learning strategy, establishes a deep neural network by stacking convolutional layers, pooling layers, up-sampling layers, deconvolutional layers, and fully connected layers, etc. modules, uses artificially labeled true classification results as supervision information, and lets the neural network take image or point cloud data as input, that is, inputs the two-dimensional ultrasound image of the target region and the blood vessel structure in the two-dimensional ultrasound image into the model / algorithm, obtains the graph feature vector through model operation, inputs the text feature vector based on the blood flow information of the blood vessel structure into the model / algorithm, learns and outputs the classification result. Of course, the text feature vector can also be processed into an array data with the same dimension as the two-dimensional ultrasound image, spliced with the image, input into the neural network for learning, and output the classification result.

[0074] Step 4, the processor 40 acquires the position change information of the ultrasound probe 10 when the ultrasound probe 10 is scanning the plurality of target regions, for example, the processor 40 acquires the position change information of the ultrasound probe 10 by using the position sensor arranged on the ultrasound probe 10 when the ultrasound probe 10 is scanning the plurality of target regions. The position change information of the ultrasound probe 10 when scanning the target region is known, such as the orientation and movement trajectory of the ultrasound probe 10 are known, for the same spatial coordinate system, the position of the ultrasound probe 10 is known, and the plane of the ultrasound image obtained by each position scanning is also known. Therefore, the processor 40 can determine the position information of the blood vessel structure in the ultrasound image of each target region based on the position change information, that is, the position of the ultrasound probe 10 when scanning the ultrasound image is known, and the relative position relationship between the blood vessel structure in the ultrasound image and the ultrasound probe is known, so the position information of the blood vessel structure in the ultrasound image can be determined according to the two information. These position relationships can be based on the same spatial coordinate system, such as the spatial coordinate system based on the position sensor, which facilitates the calculation of the position of the blood vessel structure in the ultrasound image of each target region (such as determining the coordinates of the blood vessel structure in the spatial coordinate system).

[0075] Step 5, the processor 40 stitches the sub-three-dimensional vascular images of the target regions to obtain a three-dimensional vascular image of the target regions based on the determined position information of the vascular structure in the ultrasound images of the target regions and displays the three-dimensional vascular image. It can be seen that the main method of stitching the sub-three-dimensional vascular images is automatic stitching based on the position information. Specifically, the processor 40 stitches the sub-three-dimensional vascular images of the target regions in the target regions to obtain a three-dimensional vascular image based on the sub-three-dimensional vascular images of the target regions and the determined position information of the vascular structure in the ultrasound images of the target regions, Figure 3 An example of stitching the sub-three-dimensional vascular images in the target regions D1 and D2 to obtain a three-dimensional vascular image of a larger region D3 is given, and then the three-dimensional vascular image is displayed (as shown in the right three-dimensional vascular image). Figure 3

[0076] After the three-dimensional vascular image is obtained, pose correction can be performed on the three-dimensional vascular image to make the three-dimensional vascular image as little as possible with traces of stitching and improve the accuracy of the position of the blood vessels in the image. The processor 40 can perform pose correction on the three-dimensional vascular image based on the vascular tree structure. For example, the processor 40 can specifically perform pose correction on the three-dimensional vascular image according to the vascular centerline to align the vascular tree structure. For another example, the processor 40 can specifically align each sub-three-dimensional vascular image in the three-dimensional vascular image based on the vascular profile of the stitching surface. For yet another example, after the three-dimensional vascular image is displayed, the position of each sub-three-dimensional vascular image in the displayed three-dimensional vascular image can be edited, that is, medical personnel are allowed to manually stitch and adjust the position of the multiple sub-three-dimensional vascular images according to observation and understanding. In this way, the smoothness and accuracy of the three-dimensional vascular image can be further improved.

[0077] Since a laparoscope is used in laparoscopic ultrasound examination and ultrasound is also used, the processor 40 can display the three-dimensional vascular image in the display interface of the ultrasound imaging device, in the display interface of the laparoscope device, or in both display interfaces. The display interface of the laparoscope device usually displays optical images taken by the laparoscope, such as real-time videos, pictures, etc. The three-dimensional vascular image displayed in the display interface of the laparoscope device can be displayed on the same screen as the optical image or can be superimposed, such as being superimposed on the optical image displayed in the display interface of the laparoscope device. In this way, the user can see both the real-time video inside the target tissue and the three-dimensional image of the blood vessels hidden by the tissue, which provides great convenience for doctors whether they are doing examination or surgery.

[0078] For example, the processor 40 can display the three-dimensional vascular image in the display interface of the ultrasound imaging device, in the display interface of the laparoscope device, or in both display interfaces. Figure 4 ​As shown, processor 40 can also differentiate vessel types in a 3D vascular image, for example, highlighting arteries to remind surgeons to pay attention to them during surgery. This allows users to see which vessels are arteries and which are veins in the 3D vascular image. Processor 40 can differentiate vessel types in a 3D vascular image in various ways, two of which are exemplified below.

[0079] In the first method, the processor 40 can perform differentiated display for the three-dimensional vascular image based on the vascular type of the vascular structure in the ultrasound image of each target area, that is, directly perform differentiated display on the vascular structure in the three-dimensional vascular image, for example, the vascular structure of the arterial vascular type and the vascular structure of the venous vascular type have different colors in the three-dimensional vascular image.

[0080] In the second method, if Figure 4 As shown, the processor 40 can 3D reconstruct a sub-3D vessel type image for each target region based on the color Doppler image of each target region and the positional information of the vascular structures in the ultrasound image of each target region, and overlay the sub-3D vessel type image on the 3D vessel image for display. In the sub-3D vessel type image, vascular structures of different vessel types have different colors. By overlaying the sub-3D vessel type images on the 3D vessel image, the user can see arteries and veins represented by different colors. Specifically, if the ultrasound image is a 2D ultrasound image, after identifying the vascular structures (vascular segmentation mask) in the 2D ultrasound image in step 2, the processor 40 can colorize the vascular structures in the 2D ultrasound image. Then, based on the positional information of the vascular structures in the ultrasound image of each target region, 3D reconstruct a colored sub-3D vessel type image for each target region. Of course, the processor 40 can also directly extract images of the vascular structures from the C-mode blood flow image in the color Doppler image, and reconstruct these extracted images of the vascular structures based on the positional information of the vascular structures in the ultrasound image of each target region to obtain colored sub-3D vessel type images for the target region.

[0081] In some embodiments, in step 1, the doctor triggers an ultrasound scan, and an ultrasound image of a target area can be collected within a preset time (for example, about 3 seconds); the doctor triggers an ultrasound scan again to obtain an ultrasound image of another target area. The doctor can perform as many ultrasound scans as he wants. After multiple scans, ultrasound images of multiple target areas can be obtained, and subsequent steps can be performed to obtain three-dimensional vascular images of the entire blood vessels in these target areas.

[0082] In another embodiment, in step 1, the doctor triggers an ultrasound scan, and an ultrasound image of a target area can be collected within a preset time (for example, about 3 seconds); the doctor triggers an ultrasound scan again, and an ultrasound image of another target area can be obtained. The processor 40 can execute the ultrasound images of the target area obtained by the two scans. Figure 2 The processing shown is performed to obtain and display a three-dimensional blood vessel image, that is, steps 1-5 are completed. The doctor may then continue to scan ultrasound images of other target areas. Specifically, the processor 40 may obtain an ultrasound image of the currently scanned target area, identify the vascular structure in the ultrasound image of the current target area to obtain a sub-3D vascular image of the current target area (the specific process is the same as step 2, except that the number of target areas is different and will not be described in detail here). The processor 40 may determine the vascular type of the vascular structure in the ultrasound image of the current target area (the specific process is the same as step 3, except that the number of target areas is different and will not be described in detail here). The processor 40 may obtain position change information of the ultrasound probe when performing ultrasound scans on the previous target area and the current target area, and determine the position information of the vascular structure in the ultrasound image of the current target area based on the position change information (the specific process is the same as step 4, except that the two target areas are limited and will not be described in detail here). Based on the position information of the vascular structure in the current target area, the sub-3D vascular image of the current target area may be spliced ​​into the currently displayed 3D vascular image, and the 3D vascular image on the display interface may be updated. That is, if the display interface currently displays a 3D vascular image, after obtaining the sub-3D vascular image through the aforementioned steps, the sub-3D vascular image may be spliced ​​into the existing 3D vascular image, thereby updating the 3D vascular image. This is equivalent to scanning multiple target areas to obtain and display three-dimensional vascular images at the beginning. Every time the doctor scans the ultrasound image of a target area, a sub-three-dimensional vascular image of the target area is added to the three-dimensional vascular image displayed on the display interface. The doctor can see the results of his real-time scanning and real-time stitching, and get feedback, which is conducive to quickly scanning the area he wants.

[0083] Taking into account the limited field of view during laparoscopic ultrasound imaging, the ultrasound images obtained from each scan can only show some local blood vessels, and are two-dimensional, which is not conducive to the doctor's grasp of the location of lesions and blood vessels from a global perspective, resulting in time-consuming and labor-intensive surgery. One scan can obtain an ultrasound image of a target area. The present invention processes the ultrasound images of multiple target areas obtained by multiple scans through the above method, thereby splicing together a three-dimensional image of blood vessels in a relatively large area, which greatly facilitates the doctor's inspection and surgery. In addition to three-dimensional splicing of blood vessels, of course, lesions (such as kidney tumors, lung tumors, uterine tumors, adnexal tumors, etc.) can also be three-dimensionally spliced, such as Figure 5 As shown, the processor 40 may specifically perform three-dimensional stitching of the lesions including the following steps:

[0084] Step 2', the processor 40 identifies the lesion structure in the ultrasound image of each target region (such as the green region in Figure 3 and Figure 4 ) to obtain a sub-three-dimensional lesion image of each target region. The specific method of identifying the lesion structure to obtain the sub-three-dimensional lesion image is the same as that of identifying the blood vessel structure to obtain the sub-three-dimensional blood vessel image in step 2 of the foregoing embodiment, that is, replace "blood vessel structure" with "lesion structure", "sub-three-dimensional blood vessel image" with "sub-three-dimensional lesion image" in the content of step 2 of the foregoing embodiment, and no further description is made here. Step 2' and step 2 can be performed simultaneously, that is, the processor 40 can identify the blood vessel structure in the ultrasound image of each target region to obtain a sub-three-dimensional blood vessel image of each target region, and identify the lesion structure in the ultrasound image of each target region to obtain a sub-three-dimensional lesion image of each target region.

[0085] Step 4', the processor 40 determines the position information of the lesion structure in the ultrasound image of each target region based on the position change information of the ultrasound probe when performing ultrasound scanning on the plurality of target regions. Similarly, the method of determining the position information of the lesion structure is the same as that of determining the position information of the blood vessel structure in step 4 of the foregoing embodiment, that is, replace "blood vessel structure" with "lesion structure" in the content of step 4 of the foregoing embodiment, and no further description is made here. Step 4' and step 4 can be performed simultaneously, that is, the processor 40 can determine the position information of the blood vessel structure and the position information of the lesion structure in the ultrasound image of each target region based on the position change information.

[0086] Step 5', the processor 40 splices the sub-three-dimensional lesion images of the plurality of target regions to obtain a three-dimensional lesion image of the plurality of target regions based on the determined position information of the lesion structure in the ultrasound image of each target region. Similarly, the method of splicing the sub-three-dimensional lesion images of the plurality of target regions to obtain a three-dimensional lesion image is the same as that of splicing the sub-three-dimensional blood vessel images of the plurality of target regions to obtain a three-dimensional blood vessel image in step 5 of the foregoing embodiment, that is, replace "blood vessel structure" with "lesion structure", "sub-three-dimensional blood vessel image" with "sub-three-dimensional lesion image", and "three-dimensional blood vessel image" with "three-dimensional lesion image" in the content of step 5 of the foregoing embodiment, and no further description is made here. Step 5' and step 5 can be performed simultaneously, that is, the processor 40 can splice the sub-three-dimensional lesion images of the plurality of target regions to obtain a three-dimensional lesion image of the plurality of target regions based on the determined position information of the lesion structure in the ultrasound image of each target region, and splice the sub-three-dimensional blood vessel images of the plurality of target regions to obtain a three-dimensional blood vessel image based on the determined position information of the blood vessel structure in the ultrasound image of each target region.

[0087] After obtaining the three-dimensional lesion image, the processor 40 can display the three-dimensional lesion image, for example, display the three-dimensional lesion image in the display interface of the ultrasound imaging device, display the three-dimensional lesion image in the display interface of the laparoscope device, etc. In this embodiment, the three-dimensional blood vessel image and the three-dimensional lesion image are displayed as a complete image, that is, when there are blood vessels and lesions in the target tissue, the steps shown in Figure 2 and 5 are simultaneously performed, so as to obtain a three-dimensional image, as shown in Figure 3 and 4 The three-dimensional image contains the three-dimensional blood vessel image and the three-dimensional lesion image, and the positional relationship between the blood vessels and the lesions in the three-dimensional image reflects the positional relationship between the real blood vessels and the real lesions. In the three-dimensional image, the arterial blood vessels, the venous blood vessels, and the lesions are differentially displayed, that is, the three structures can be distinguished by color, brightness, or other graphical elements, and the simplest is that the three structures have different colors. It can be seen that this method can be used in the use scenarios of surgical tasks under laparoscopic ultrasound guidance, such as liver surgery, kidney tumor resection, lung tumor resection, uterine tumor resection, and adnexal tumor resection, and provides great convenience for the surgeon.

[0088] In addition to the three-dimensional splicing of blood vessels, in some embodiments, the vessels can also be spliced, such as the splicing of bile ducts and other structures. The method provided by the present application is very suitable for anatomical structures that are long, have many branches, and are inconvenient for doctors to observe. The method extracts the anatomical structures from the ultrasound images and splices them into three-dimensional images, so that the doctors can intuitively see the anatomical structures, which is beneficial to various types of surgeries. As shown in Figure 6 The processor 40 can perform three-dimensional splicing of the vessels, which can specifically include the following steps:

[0089] Step 2'', the processor 40 identifies the vessel structure in the ultrasound image of each target region to obtain a sub-three-dimensional vessel image of each target region. The specific method of identifying the vessel structure to obtain the sub-three-dimensional vessel image is the same as that of identifying the blood vessel structure to obtain the sub-three-dimensional blood vessel image in step 2 of the foregoing embodiment, that is, the "blood vessel structure" in the content of step 2 is changed to "vessel structure", and the "sub-three-dimensional blood vessel image" is changed to "sub-three-dimensional vessel image". Here, no further description is given. Step 2'', step 2', and step 2 can be performed simultaneously in pairs, or all three can be performed simultaneously.

[0090] Step 4", the processor 40 determines the position information of the vessel structure in the ultrasound image of each target region based on the position change information of the ultrasound probe when the ultrasound probe scans the plurality of target regions. Similarly, the determination of the position information of the vessel structure can use the same method as the determination of the position information of the blood vessel structure in the aforementioned step 4, that is, replace "blood vessel structure" in the content of the aforementioned step 4 with "vessel structure", and the details are not repeated here. Step 4", step 4' and step 4 can be performed simultaneously in pairs, or all three can be performed simultaneously.

[0091] Step 5", the processor 40 splices the sub-three-dimensional vessel images of each target region to obtain a three-dimensional vessel image of the plurality of target regions based on the determined position information of the vessel structure in the ultrasound image of each target region. Similarly, the splicing of the sub-three-dimensional vessel images of each target region to obtain a three-dimensional vessel image can use the same method as the splicing of the sub-three-dimensional blood vessel images of each target region to obtain a three-dimensional blood vessel image in the aforementioned step 5, that is, replace "blood vessel structure", "sub-three-dimensional blood vessel image" and "three-dimensional blood vessel image" in the content of the aforementioned step 5 with "vessel structure", "sub-three-dimensional vessel image" and "three-dimensional vessel image", respectively, and the details are not repeated here. Step 5", step 5' and step 5 can be performed simultaneously in pairs, or all three can be performed simultaneously.

[0092] After obtaining the three-dimensional vessel image, the processor 40 can display the three-dimensional vessel image, for example, display the three-dimensional vessel image in the display interface of the ultrasound imaging device, display the three-dimensional vessel image in the display interface of the laparoscope device, etc. In the present embodiment, two or all of the three-dimensional blood vessel image, the three-dimensional lesion image and the three-dimensional vessel image are displayed as a complete image. In the present embodiment, if the target tissue has blood vessels, lesions and vessels, the three-dimensional images of the three are displayed, and if only two or one of them are present, only the three-dimensional images of the two or one are displayed. Similarly, the color of the vessels in the three-dimensional vessel image can be different from that of the blood vessels and lesions, so as to be displayed differently.

[0093] Based on the above background, the present application provides an ultrasound data three-dimensional reconstruction system under laparoscopic ultrasound, which is a three-dimensional reconstruction tool for a large range of blood vessels and lesions, and provides a three-dimensional display of blood vessels and lesions that can reflect the entire target tissue such as the operation related region of the liver for doctors. The specific process can be that the doctor uses a laparoscopic ultrasound probe with a positioning device to scan multiple times at different positions in the relevant region, obtains ultrasound data of liver tumors and liver blood vessels, obtains tumor and blood vessel contours through tumor and blood vessel segmentation, the system automatically performs three-dimensional reconstruction and splicing to obtain three-dimensional data of blood vessels and lesions in the entire region, and simultaneously combines color Doppler blood flow signals (signal intensity, direction, flow rate, etc.), identifies the blood vessels as arteries or veins, and marks the blood vessels according to the type and displays them in real time in the window, thereby providing great convenience and support for the doctor to perform the operation.

[0094] The difference, beneficial effects and advantages of the present scheme and the prior art are that the prior art is limited by the limitations of laparoscopic ultrasound operation itself, cannot provide a sufficient range of three-dimensional blood vessel model, still needs the doctor to observe repeatedly by hand, estimates according to imagination, and does not have the ability to prompt the branch attribute of the blood vessel according to the blood flow direction, the reliability is low, and the clinical significance is limited. The present scheme can provide a doctor with a large range of lesion, blood vessel tree three-dimensional reconstruction by splicing, and different color display of the blood vessel tree combined with blood flow data, so that the doctor can more comprehensively observe the lesion, blood vessel three-dimensional space state of the operation related liver region, and can combine the branch attribute of the blood vessel color marking to better perform puncture or resection operation planning and implementation.

[0095] This document describes various exemplary embodiments with reference to the drawings. However, those skilled in the art will recognize that changes and modifications can be made to the exemplary embodiments without departing from the scope hereof. For example, various operational steps and components for carrying out the operational steps can be implemented in different ways depending upon the particular application or applications, or to accomplish the same or similar functions of the system in conjunction with any number of other cost functions associated with the operation of the system (e.g., one or more steps can be deleted, modified, or combined with other steps).

[0096] In addition, as understood by those skilled in the art, the principles herein can be reflected in a computer program product on a computer readable storage medium preloaded with computer readable program code. Any tangible, non-transitory computer readable storage medium can be used, including magnetic storage devices (hard disk, floppy disk, etc.), optical storage devices (CD-ROM, DVD, Blu Ray disc, etc.), flash memory, and / or the like. These computer program instructions can be loaded onto a general purpose computer, a special purpose computer, or other programmable communication device to form a machine, so that these instructions executed on the computer or other programmable communication device can generate a device that implements the specified function. These computer program instructions can also be stored in a computer readable storage medium, which can instruct the computer or other programmable communication device to operate in a specific way, so that the instructions stored in the computer readable storage medium can form a manufactured product, including an implementation device that implements the specified function. Computer program instructions can also be loaded onto a computer or other programmable communication device to execute a series of operational steps to generate a computer-implemented process, so that the instructions executed on the computer or other programmable device can provide steps for implementing the specified function.

[0097] While the principles of the present disclosure have been illustrated in various embodiments, many modifications of structure, arrangement, proportions, elements, materials, and components which are specifically adapted to specific environments and / or operations can be employed without departing from the principles and scope of the disclosure. Such modifications and other changes or modifications are intended to be included within the scope of the present disclosure.

[0098] The foregoing detailed description has been described with reference to various embodiments. However, those skilled in the art will recognize that changes and modifications can be made thereto without departing from the scope of the present disclosure. Accordingly, the present disclosure is to be considered as illustrative and not restrictive, and all changes and modifications that come within the scope of the present disclosure are therefore intended to be embraced therein. Likewise, the advantages provided by the various embodiments will be recognized by those of ordinary skill in the art and implementations of the concepts described herein will be readily made. Furthermore, benefits, other advantages, and solutions to problems have been described above with regard to various embodiments. However, the benefits, advantages, solutions to problems and any element(s) that can cause any of such should not be construed as critical, required or essential. The terms "comprises", "comprising", or any other variations thereof are to be construed as non-exclusive, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. Further, no language or term should be construed as indicating any non-claimed element as essential to the practice of the disclosure.

[0099] Those skilled in the art will recognize that many modifications can be made to the details of the above-described embodiments without departing from the underlying principles of the present disclosure. The scope of the present disclosure should, therefore, be determined only from the following claims.

Claims

1. A laparoscopic ultrasound imaging method, characterized in that: include: Acquiring an ultrasonic image of each of the plurality of target areas obtained by ultrasonically scanning the plurality of target areas of the target tissue; identifying a vascular structure in the ultrasound image of each target area to obtain a sub-three-dimensional vascular image of each target area; Acquiring a color Doppler image or a vector blood flow image of each target area; Acquiring, based on the color Doppler image or the vector blood flow image of each target area, a vessel type of a blood vessel structure in an ultrasound image of each target area, wherein the vessel type includes at least an arterial vessel type and a venous vessel type; acquiring position change information of the ultrasound probe when performing ultrasound scanning on the multiple target areas, and determining position information of the blood vessel structure in the ultrasound image of each target area based on the position change information; Based on the determined position information of the vascular structure in the ultrasound image of each target area, stitching the sub-three-dimensional vascular images of each target area to obtain and display a three-dimensional vascular image of the multiple target areas, and differentially displaying the vascular types in the three-dimensional vascular image; The differentially displaying the blood vessel types in the three-dimensional blood vessel image includes: Performing differentiated display for the three-dimensional vascular image based on the vascular type of the vascular structure in the ultrasound image of each target area; or, Based on the color Doppler image of each target area and the position information of the vascular structure in the determined ultrasound image of each target area, a sub-three-dimensional vascular type image of each target area is obtained by three-dimensional reconstruction, and the sub-three-dimensional vascular type image is superimposed on the three-dimensional vascular image for display.

2. A laparoscopic ultrasound imaging method, characterized in that: include: Acquiring an ultrasonic image of each of the plurality of target areas obtained by ultrasonically scanning the plurality of target areas of the target tissue; identifying a vascular structure in the ultrasound image of each target area to obtain a sub-three-dimensional vascular image of each target area; Determining a blood vessel type of a blood vessel structure in the ultrasound image of each target area, wherein the blood vessel type includes at least an arterial blood vessel type and a venous blood vessel type; acquiring position change information of the ultrasound probe when performing ultrasound scanning on the multiple target areas, and determining position information of the blood vessel structure in the ultrasound image of each target area based on the position change information; Based on the determined position information of the vascular structure in the ultrasound image of each target area, the sub-three-dimensional vascular images of each target area are spliced ​​to obtain and display a three-dimensional vascular image of the multiple target areas, and the vascular types are differentially displayed in the three-dimensional vascular image.

3. The method according to claim 1 or 2, wherein: The ultrasound image of each target area is a three-dimensional ultrasound image; and identifying the vascular structure in the ultrasound image of each target area to obtain a sub-three-dimensional vascular image of each target area includes: The blood vessel structure in the three-dimensional ultrasound image is detected and segmented to obtain a sub-three-dimensional blood vessel image of each target area.

4. The method according to claim 1 or 2, wherein: The ultrasound image of each target area is a multi-frame two-dimensional ultrasound image; and identifying the vascular structure in the ultrasound image of each target area to obtain a sub-three-dimensional vascular image of each target area includes at least one of the following: reconstructing the multiple frames of two-dimensional ultrasound images into three-dimensional ultrasound images based on the position change information, detecting and segmenting the blood vessel structure in the three-dimensional ultrasound image, and obtaining sub-three-dimensional blood vessel images of each target area; displaying the multiple frames of two-dimensional ultrasound images, obtaining the vascular structures in the multiple frames of two-dimensional ultrasound images in response to a user's marking operation on the vascular structures in the displayed multiple frames of two-dimensional ultrasound images, and obtaining sub-three-dimensional vascular images of the target areas based on the vascular structures in the multiple frames of two-dimensional ultrasound images; The vascular structure of each frame of the two-dimensional ultrasound image in the multi-frame two-dimensional ultrasound image is detected and segmented to obtain the vascular structure in the multi-frame two-dimensional ultrasound image, and a sub-three-dimensional vascular image of each target area is obtained based on the vascular structure in the multi-frame two-dimensional ultrasound image.

5. The method according to claim 2, wherein Determining the blood vessel type of the blood vessel structure in the ultrasound image of each target area includes: Acquiring a color Doppler image or a vector blood flow image of each target area; The vessel type of the vessel structure in the ultrasound image of each target area is acquired according to the color Doppler image or the vector blood flow image of each target area.

6. The method according to claim 1 or 5, wherein: After acquiring the color Doppler image or vector blood flow image of each target area, the method further includes: According to the color Doppler image or the vector blood flow image of each target area, blood flow information of the blood vessel structure in the ultrasound image of each target area is obtained, and the blood flow information includes: blood flow intensity and / or blood flow velocity.

7. The method according to claim 1 or 2, wherein: The acquiring of position change information of the ultrasound probe when performing ultrasound scanning on the multiple target areas includes: The position change information of the ultrasound probe is acquired by using a position sensor configured on the ultrasound probe.

8. The method according to claim 4, wherein The obtaining of the sub-three-dimensional vascular images of each target area based on the vascular structures in the multiple frames of two-dimensional ultrasound images includes: Based on the obtained vascular structures in the multi-frame two-dimensional ultrasound images of the target areas and the determined position information of the vascular structures in the multi-frame two-dimensional ultrasound images of the target areas, three-dimensional reconstruction is performed to obtain sub-three-dimensional vascular images of the target areas.

9. The method according to claim 1 or 2, wherein: The step of stitching the sub-three-dimensional vascular images of the target areas based on the determined position information of the vascular structures in the ultrasound images of the target areas to obtain a three-dimensional vascular image of the multiple target areas and displaying the image includes: Based on the sub-three-dimensional vascular images of each target area and the determined position information of the vascular structure in the ultrasound image of each target area, the sub-three-dimensional vascular images of each target area are spliced ​​into a three-dimensional vascular image, and the three-dimensional vascular image is displayed.

10. The method according to claim 1 or 2, wherein: Also includes: Identifying the lesion structure in the ultrasound image of each target area to obtain a sub-three-dimensional lesion image of each target area; determining, based on position change information of the ultrasound probe when performing ultrasound scanning on the multiple target areas, position information of the lesion structure in the ultrasound image of each target area; splicing the sub-three-dimensional lesion images of the target areas based on the determined position information of the lesion structures in the ultrasound images of the target areas to obtain a three-dimensional lesion image of the multiple target areas; The three-dimensional lesion image is displayed.

11. The method according to claim 1 or 2, wherein: The target tissue is liver tissue.

12. The method according to claim 2, wherein The differentially displaying the blood vessel types in the three-dimensional blood vessel image includes: Performing differentiated display for the three-dimensional vascular image based on the vascular type of the vascular structure in the ultrasound image of each target area; or, Based on the color Doppler image of each target area and the position information of the vascular structure in the determined ultrasound image of each target area, a sub-three-dimensional vascular type image of each target area is obtained by three-dimensional reconstruction, and the sub-three-dimensional vascular type image is superimposed on the three-dimensional vascular image for display.

13. The method according to any one of claims 1 to 12, characterized in that displaying the three-dimensional blood vessel image on a display interface of an ultrasonic imaging device; and / or, The three-dimensional blood vessel image is displayed on a display interface of a laparoscope device.

14. The method according to claim 13, wherein Displaying the three-dimensional blood vessel image on a display interface of the laparoscope device includes: The three-dimensional blood vessel image is superimposed and displayed on the optical image displayed on the display interface of the laparoscope device.

15. An ultrasonic imaging device, characterized in that: include: An ultrasonic probe, which is equipped with a position sensor, wherein the position sensor is used to obtain real-time position information of the ultrasonic probe; a transmitting and receiving circuit, configured to control the ultrasonic probe to transmit ultrasonic waves to the target tissue and receive ultrasonic echo signals; A processor, configured to execute the method according to any one of claims 1 to 14.

16. A computer program product comprising a computer program and / or instructions, characterized in that When the computer program and / or instructions are executed by a processor, the method according to any one of claims 1 to 14 is implemented.

Images