Laparoscopic ultrasonic imaging method and ultrasonic imaging equipment

Abstract

According to the laparoscopic ultrasonic imaging method and the ultrasonic imaging equipment provided by the invention, blood vessel structures in ultrasonic images of a plurality of target areas are identified, and sub-three-dimensional blood vessel images of the target areas are obtained; determining the blood vessel type of the blood vessel structure in the ultrasonic image of each target area; acquiring position change information when the ultrasonic probe performs ultrasonic scanning on each target area, and determining position information of a vascular structure in the ultrasonic image of each target area based on the position change information; and based on the determined position information of the blood vessel structure, splicing the sub-three-dimensional blood vessel images of the target areas to obtain a three-dimensional blood vessel image and displaying the three-dimensional blood vessel image, and carrying out differential display on the blood vessel types in the three-dimensional blood vessel image. Therefore, a doctor can more comprehensively observe the three-dimensional space state of a focus and a blood vessel in a related operation area, and can better carry out puncture or resection operation planning and implementation.

Description

Technical Field

[0001] The present invention relates to the field of medical instruments, and in particular to a laparoscopic ultrasonic imaging method and ultrasonic imaging equipment. Background Art

[0002] With the development of minimally invasive surgical techniques, especially laparoscopic surgical techniques, laparoscopic surgical techniques have been increasingly used to treat many abdominal organ diseases. During laparoscopic surgical treatment of abdominal organ diseases including liver cancer, laparoscopic ultrasound (LUS) is often required for diagnostic evaluation and auxiliary guidance. Because LUS can help doctors see structures below the surface and perform palpation, it is a good auxiliary guidance image paradigm in laparoscopic surgery.

[0003] Currently, laparoscopic ultrasound imaging can provide two-dimensional ultrasound section images. During the operation, the doctor needs to rely on imagination to form three-dimensional cognition based on the two-dimensional ultrasound section images in order to provide good auxiliary guidance, which requires a high level of experience and ability of the doctor.

[0004] In summary, there is a certain demand for three-dimensional reconstruction imaging of laparoscopic ultrasound data. Summary of the invention

[0005] In view of the above problems, the present application provides a laparoscopic ultrasonic imaging method and an ultrasonic imaging device, which are described in detail below.

[0006] According to the first aspect, an embodiment provides a laparoscopic ultrasound imaging method, comprising:

[0007] Acquire an ultrasonic image of each of the multiple target areas obtained by ultrasonically scanning the multiple target areas of the target tissue;

[0008] Identifying the vascular structure in the ultrasound image of each target area to obtain a sub-three-dimensional vascular image of each target area;

[0009] Acquiring a color Doppler image or a vector blood flow image of each target area;

[0010] Acquire the blood vessel type of the blood vessel structure in the ultrasound image of each target area according to the color Doppler image or the vector blood flow image of each target area, wherein the blood vessel type at least includes an arterial blood vessel type and a venous blood vessel type;

[0011] Acquiring position change information of the ultrasound probe when performing ultrasound scanning on the multiple target areas, and determining position information of the blood vessel structure in the ultrasound image of each target area based on the position change information;

[0012] Based on the determined position information of the vascular structure in the ultrasound image of each target area, stitching the sub-three-dimensional vascular images of each target area to obtain and display a three-dimensional vascular image of the multiple target areas, and differentially displaying the vascular types in the three-dimensional vascular image;

[0013] The differentially displaying the blood vessel types in the three-dimensional blood vessel image includes:

[0014] Performing differentiated display for the three-dimensional vascular image based on the vascular type of the vascular structure in the ultrasound image of each target area; or,

[0015] Based on the color Doppler images of the target areas and the determined position information of the vascular structures in the ultrasound images of the target areas, three-dimensional reconstruction is performed to obtain sub-three-dimensional vascular type images of the target areas, and the sub-three-dimensional vascular type images are superimposed on the three-dimensional vascular images for display.

[0016] According to the second aspect, an embodiment provides a laparoscopic ultrasound imaging method, comprising:

[0017] Acquire an ultrasonic image of each of the multiple target areas obtained by ultrasonically scanning the multiple target areas of the target tissue;

[0018] Identifying the vascular structure in the ultrasound image of each target area to obtain a sub-three-dimensional vascular image of each target area;

[0019] Determine the blood vessel type of the blood vessel structure in the ultrasound image of each target area, wherein the blood vessel type at least includes an arterial blood vessel type and a venous blood vessel type;

[0020] Acquiring position change information of the ultrasound probe when performing ultrasound scanning on the multiple target areas, and determining position information of the blood vessel structure in the ultrasound image of each target area based on the position change information;

[0021] Based on the determined position information of the vascular structure in the ultrasound image of each target area, the sub-three-dimensional vascular images of each target area are spliced ​​to obtain and display a three-dimensional vascular image of the multiple target areas, and the vascular types are differentially displayed in the three-dimensional vascular image.

[0022] According to a third aspect, an embodiment provides an ultrasonic imaging device, comprising:

[0023] An ultrasonic probe, which is equipped with a position sensor, wherein the position sensor is used to obtain real-time position information of the ultrasonic probe;

[0024] A transmitting and receiving circuit, used for controlling the ultrasonic probe to transmit ultrasonic waves to the target tissue and receive ultrasonic echo signals;

[0025] A processor, configured to execute any of the methods described above.

[0026] According to a fourth aspect, an embodiment provides a computer program product, comprising a computer program and / or instructions, wherein the computer program and / or instructions implement any of the above methods when executed by a processor.

[0027] According to the laparoscopic ultrasonic imaging method and ultrasonic imaging device of the above-mentioned embodiment, the vascular structures in the ultrasonic images of multiple target areas are identified to obtain the sub-three-dimensional vascular images of each target area; the vascular type of the vascular structure in the ultrasonic image of each target area is determined; the position change information of the ultrasonic probe when performing ultrasonic scanning on each target area is obtained, and the position information of the vascular structure in the ultrasonic image of each target area is determined based on the position change information; and then based on the determined position information of the vascular structure, the sub-three-dimensional vascular images of each target area are spliced ​​to obtain a three-dimensional vascular image and display it, and the vascular type can also be displayed differently in the three-dimensional vascular image, so that the doctor can more comprehensively observe the three-dimensional spatial state of the lesions and blood vessels in the operation-related area, and better plan and implement the puncture or resection operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 A structural block diagram of an embodiment of an ultrasonic imaging device provided by the present invention;

[0029] Figure 2 A flow chart of an embodiment of a laparoscopic ultrasonic imaging method provided by the present invention;

[0030] Figure 3 A schematic diagram of a three-dimensional blood vessel image obtained by stitching together sub-three-dimensional blood vessel images of multiple target areas;

[0031] Figure 4 A schematic diagram of differentially displaying blood vessel types in a three-dimensional blood vessel image;

[0032] Figure 5 A flow chart of another embodiment of the laparoscopic ultrasonic imaging method provided by the present invention;

[0033] Figure 6 The present invention provides a flow chart of another embodiment of the laparoscopic ultrasonic imaging method. DETAILED DESCRIPTION

[0034] The present invention is further described in detail below by specific embodiments in conjunction with the accompanying drawings. Wherein similar elements in different embodiments adopt associated similar element numbers. In the following embodiments, many detailed descriptions are for making the present application better understood. However, those skilled in the art can easily recognize that some features can be omitted in different situations, or can be replaced by other elements, materials, methods. In some cases, some operations related to the present application are not shown or described in the specification, this is to avoid the core part of the present application being overwhelmed by too much description, and for those skilled in the art, it is not necessary to describe these related operations in detail, and they can fully understand the related operations according to the description in the specification and the general technical knowledge in the art.

[0035] In addition, the features, operations or characteristics described in the specification can be combined in any appropriate manner to form various implementations. At the same time, the steps or actions in the method description can also be interchanged or adjusted in a manner that is obvious to those skilled in the art. Therefore, the various sequences in the specification and the drawings are only for the purpose of clearly describing a certain embodiment and are not meant to be a required sequence, unless otherwise specified that a certain sequence must be followed.

[0036] The serial numbers of the components in this document, such as "first", "second", etc., are only used to distinguish the objects described and do not have any order or technical meaning. The "connection" and "coupling" mentioned in this application, unless otherwise specified, include direct and indirect connections (couplings).

[0037] The present invention obtains ultrasonic data of multiple target areas (such as laparoscopic ultrasonic data) by performing multiple ultrasonic scans on the target tissue, reconstructs the lesions and blood vessels in three dimensions based on the ultrasonic data, and performs three-dimensional splicing based on the navigation positioning information during the ultrasonic scan, and performs color display in combination with the blood flow information. This technical solution can provide doctors with a large-scale three-dimensional display of vascular trees and lesions during laparoscopic surgery, and provide blood flow direction information, so that doctors no longer need to rely on spatial imagination. When performing puncture or resection, they can more clearly understand the spatial position relationship between liver blood vessels and lesions in the intraoperative state, and understand the branch properties of the vascular tree, so that puncture or resection planning and surgical implementation are more convenient. This is explained in detail below through some embodiments.

[0038] like Figure 1 As shown, the ultrasonic imaging device includes an ultrasonic probe 10 , a transmitting and receiving circuit 20 , an echo processing module 30 and a processor 40 .

[0039] The ultrasonic probe 10 is used to transmit ultrasonic waves to the target tissue A, and to receive corresponding ultrasonic echo signals to obtain ultrasonic data, such as two-dimensional ultrasonic data or three-dimensional ultrasonic data. In some specific embodiments, the ultrasonic probe 10 includes a plurality of array elements, which are used to realize the mutual conversion between electrical pulse signals and ultrasonic waves, so as to realize the transmission of ultrasonic waves to the target tissue A and the reception of corresponding ultrasonic echo signals. The array element can transmit ultrasonic waves according to the excitation electrical signal, or convert the received ultrasonic waves into electrical signals. Therefore, each array element can be used to transmit ultrasonic waves to the target tissue A, and can also be used to receive ultrasonic echoes returned by the tissue. When performing ultrasonic detection, which array elements are used to transmit ultrasonic waves and which array elements are used to receive ultrasonic waves can be controlled by the transmission sequence and the reception sequence, or the array elements can be controlled to transmit ultrasonic waves or receive ultrasonic echoes in time slots. All array elements participating in ultrasonic emission can be excited by electrical signals at the same time, so as to transmit ultrasonic waves at the same time; or the array elements participating in ultrasonic emission can also be excited by several electrical signals with a certain time interval, so as to continuously transmit ultrasonic waves with a certain time interval.

[0040] The transmitting and receiving circuit 20 is used to control the ultrasonic probe 10 to transmit ultrasonic waves and receive ultrasonic echo signals. For example, the transmitting and receiving circuit 20 is used to control the ultrasonic probe 10 to transmit ultrasonic waves to the target tissue A on the one hand, and to control the ultrasonic probe 10 to receive ultrasonic echo signals reflected by the target tissue A on the other hand. In some specific embodiments, the transmitting and receiving circuit 20 is used to generate a transmitting sequence and a receiving sequence, and output them to the ultrasonic probe 10. The transmitting sequence is used to control part or all of the multiple array elements in the ultrasonic probe 10 to transmit ultrasonic waves to the target tissue A. The parameters of the transmitting sequence include the number of array elements used for transmission and ultrasonic transmission parameters (such as amplitude, frequency domain, number of wave transmissions, transmission interval, transmission angle, waveform and / or focus position, etc.). The receiving sequence is used to control part or all of the multiple array elements to receive the echo of the ultrasonic wave after passing through the target tissue. The parameters of the receiving sequence include the number of array elements used for reception and the receiving parameters of the echo (such as reception angle, depth, etc.). The ultrasonic parameters in the transmitting sequence and the echo parameters in the receiving sequence are different depending on the purpose of the ultrasonic echo or the image generated according to the ultrasonic echo.

[0041] The echo processing module 30 is used to process the ultrasonic echo signal received by the ultrasonic probe 10, for example, filtering, amplifying, beamforming, etc., to obtain ultrasonic image data. In a specific embodiment, the echo processing module 30 can output the ultrasonic image data to the processor 40, or store the ultrasonic image data in a memory first, and when it is necessary to perform calculations based on the ultrasonic image data, the processor 40 reads the ultrasonic image data from the memory. Those skilled in the art should understand that in some embodiments, when it is not necessary to filter, amplify, beamform, etc. the ultrasonic echo signal, the echo processing module 30 can also be omitted. In some embodiments, part or even all of the functions of the echo processing module 30 can also be completed by the processor 40, that is, the echo processing module 30 can be a part of the processor 40.

[0042] The processor 40 is used to obtain ultrasound image data and use relevant algorithms to obtain the required parameters or images. The processor 40 in some embodiments of the present invention includes but is not limited to a central processing unit (CPU), a microcontroller unit (MCU), a field programmable gate array (FPGA), and a digital signal processing (DSP) device for interpreting computer instructions and processing data in computer software. In some embodiments, the processor 40 is used to execute various computer applications in a computer-readable storage medium, thereby realizing various functions of the ultrasound imaging device.

[0043] In some embodiments, the ultrasonic imaging device may further include a human-computer interaction device 50. The human-computer interaction device 50 is used for human-computer interaction, that is, receiving user input and outputting visual information; it may receive user input by using a keyboard, operation button, mouse, trackball, etc., or a touch screen integrated with a display; it may output visual information by using a display. The display may be used to display information, such as displaying parameters and images calculated by the processor 40. Those skilled in the art should understand that in some embodiments, the ultrasonic imaging device itself may not have an integrated display, but may be connected to an external display to display information through the external display.

[0044] It should be noted that Figure 1 The structure is only for illustration and may also include Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown. Figure 1 Each component shown in the figure may be implemented by hardware and / or software.

[0045] The ultrasound probe 10 of this embodiment is equipped with a position sensor, which is used to obtain real-time position information of the ultrasound probe 10 .

[0046] The processor 40 can control the ultrasonic imaging device to perform laparoscopic ultrasonic imaging. The specific process is as follows: Figure 2 As shown, the following steps are included:

[0047] Step 1, the processor 40 obtains an ultrasonic image of each target area in the multiple target areas obtained by ultrasonically scanning the multiple target areas of the target tissue. In the present embodiment, the ultrasonic probe 10 is provided with a position sensor, and the position sensor is used to detect the position of the ultrasonic probe 10 to obtain the position change information of the ultrasonic probe 10. The position sensor may be a magnetic positioning device, such as a magnetic navigation system and a magnetic positioning sensor. However, the position sensor may not be limited to a magnetic positioning device, and other positioning devices with the same or similar positioning capabilities may also be used, such as inertial navigation and inertial measurement positioning sensors, or optical navigation and positioning sensors, etc. The preferred installation method of the position sensor is to be built into the laparoscopic ultrasonic probe 10, that is, the ultrasonic probe 10 is built with a position sensor. Of course, in other embodiments, the position sensor may also be installed on the outer casing of the ultrasonic probe 10 using an external fixing device.

[0048] The target tissue can be various tissues of a living body, such as the uterus, heart, thyroid, ovary, kidney, liver, etc. This embodiment is described by taking the target tissue as liver tissue as an example. Medical staff can use a laparoscopic ultrasound probe with a built-in electromagnetic positioning device to scan the liver and obtain ultrasound images of multiple target areas of the target tissue, thereby confirming the target tumor and vascular information of related areas based on these ultrasound images. Among them, the ultrasound image of the target area can be a three-dimensional ultrasound image or a multi-frame two-dimensional ultrasound image. Medical staff can obtain these ultrasound images in B mode, C mode, contrast mode, or elastic mode, which are explained below.

[0049] Medical personnel can start ultrasound data acquisition by pressing buttons or touching the screen on the ultrasound probe 10, and also start the position sensor, for example, to collect ultrasound data in the liver area. When collecting ultrasound data, the ultrasound probe 10 can be controlled to move from the head side to the foot side of the patient in multiple target areas ( Figure 3The example shows two target areas (D1 and D2) for scanning and acquisition, obtaining lesions (as shown in the green area in the figure), blood vessel contours (as shown in the red area in the figure) and positioning information, and collecting color Doppler blood flow signals (C mode) to obtain blood flow direction and blood volume. Ultrasonic data acquisition can specifically collect 3D ultrasonic data, automatically reconstruct and generate 3D data (three-dimensional ultrasonic images), such as Freehand 3D scanning and reconstruction based on the positioning navigation system, and 3D ultrasonic acquisition using a volume probe. Of course, two-dimensional ultrasonic data can also be collected. The collection and scanning technique of medical staff can be to control the ultrasonic probe 10 to scan and collect from the patient's head side to the foot side, with the purpose of obtaining the scanning direction as a reference for the identification of vascular types (arteries / veins). However, the scanning and acquisition techniques to achieve this purpose are not limited to this, and can also include methods such as from the patient's foot side to the head side, from the patient's left side (right side) to the right side (left side), or free scanning and automatic judgment combined with positioning information.

[0050] It can be seen that the present invention can utilize the ultrasound images obtained by doctors during routine laparoscopic ultrasound examinations for subsequent processing, and ultimately obtain a relatively large three-dimensional vascular image without the need for doctors to perform additional scanning operations.

[0051] Step 2, the processor 40 identifies the vascular structure in the ultrasound image of each target area to obtain the sub-three-dimensional vascular image of each target area. The first choice can be to directly perform 3D detection and segmentation on the three-dimensional ultrasound image (3D ultrasound data) to obtain the vascular model. Of course, it is also possible to first perform tracing detection frame by frame based on a single section, and then perform 3D reconstruction based on the positioning information. That is, the ultrasound image of the target area can be a three-dimensional ultrasound image (3D ultrasound data) or a multi-frame two-dimensional ultrasound image, so the specific process of obtaining the sub-three-dimensional vascular image of the target area is different, and examples are given below to illustrate them respectively.

[0052] In one embodiment, the ultrasound image of each target area is a three-dimensional ultrasound image. In this case, the processor 40 can directly detect and segment the vascular structure in the three-dimensional ultrasound image to obtain the sub-three-dimensional vascular image of each target area. This is equivalent to detecting and segmenting the vascular image from the three-dimensional ultrasound image of each target area, thereby obtaining the sub-three-dimensional vascular image of each target area. The processor 40 automatically detects and segments the vascular structure in the three-dimensional ultrasound image, which can be implemented in a variety of ways, and three examples are given below for illustration.

[0053] The first implementation method is to detect and segment the vascular structure in the three-dimensional ultrasound image based on the traditional image processing segmentation algorithm. For example, the processor 40 uses filtering, edge detection and / or threshold segmentation algorithms to perform threshold division based on the image pixels, grayscale and other information in the three-dimensional ultrasound image, thereby segmenting the vascular structure in the three-dimensional ultrasound image. The filtering, edge detection, threshold segmentation and other algorithms can specifically adopt the Roberts algorithm, Prewitt algorithm, Otsu method (OTSU) and the like.

[0054] The second implementation method is to detect and segment the vascular structure in the three-dimensional ultrasound image based on the semantic segmentation algorithm of traditional machine learning. For example, the idea of ​​splitting the image into small blocks (patching) and then classifying can be used. This implementation method is mainly divided into two steps: the processor 40 first extracts features from the image blocks of the three-dimensional ultrasound image. Specifically, the PCA (Principal Component Analysis) algorithm, LDA (Linear Discriminant Analysis) algorithm or deep neural network can be used to extract features from the image blocks of the three-dimensional ultrasound image, establish feature vectors, and then classify the feature vectors. Specifically, the k-NN algorithm (k nearest neighbor algorithm), random forest (Random Forest) or SVM (Support Vector Machine) algorithm can be used to classify the feature vectors; the processor 40 then pieces the classified image blocks back into the entire image to obtain the segmentation result (vascular structure).

[0055] The third implementation method is to detect and segment the vascular structure in the three-dimensional ultrasound image based on the semantic segmentation algorithm of deep learning. For example, a supervised learning strategy can be adopted in advance. By stacking modules such as convolutional layers, pooling layers, upsampling layers, deconvolution layers and fully connected layers, a deep neural network is established, and the real segmentation annotation information is made into a mask image (mask) of the same size as the input image (three-dimensional ultrasound image) as supervision information, so that the neural network model can learn and output the area that needs to be segmented. The two-dimensional segmentation network used can adopt the common Mask R-CNN, U-Net, Solo, etc. In this way, the processor 40 only needs to input the three-dimensional ultrasound image into the trained neural network model to obtain the area that needs to be segmented (that is, the vascular structure) output by the neural network model.

[0056] After the processor 40 detects and segments the vascular structure in the three-dimensional ultrasound image through any one of these implementations, it can perform three-dimensional reconstruction to obtain sub-three-dimensional vascular images of each target area.

[0057] In another embodiment, the ultrasound images of each target area are multi-frame two-dimensional ultrasound images (such as ultrasound B images). In this embodiment, there are also multiple ways to obtain the sub-three-dimensional blood vessel images of each target area, and three of them are also cited below for illustration.

[0058] In the first method, the processor 40 reconstructs the multi-frame two-dimensional ultrasound images into a three-dimensional ultrasound image based on the position change information. The position change information of the ultrasound probe 10 when scanning the target area is known, that is, the orientation and movement trajectory of the ultrasound probe 10 are known, and the relative position relationship between the ultrasound probe 10 and each pixel point of the two-dimensional ultrasound image obtained by its scanning is also known, so a three-dimensional ultrasound image can be reconstructed based on the two. This is also a common three-dimensional imaging method. Then the processor 40 detects and segments the vascular structure in the three-dimensional ultrasound image to obtain the sub-three-dimensional vascular image of each target area. After reconstructing the multi-frame two-dimensional ultrasound image into a three-dimensional ultrasound image, it is the same as the previous embodiment, so the specific implementation method of obtaining the sub-three-dimensional vascular image of each target area mentioned in the previous embodiment can be adopted, that is, the processor 40 can use the aforementioned first, second or third implementation method to detect and segment the vascular structure in the three-dimensional ultrasound image, thereby obtaining the sub-three-dimensional vascular image of each target area, which will not be repeated here.

[0059] In the second method, the processor 40 displays the multi-frame two-dimensional ultrasound images through the display of the human-computer interaction device 50, and in response to the user's marking operation on the vascular structure in the displayed multi-frame two-dimensional ultrasound images, obtains the vascular structure in the multi-frame two-dimensional ultrasound images, and obtains the sub-three-dimensional vascular images of each target area based on the vascular structure in the multi-frame two-dimensional ultrasound images. That is, the display will display the multi-frame two-dimensional ultrasound images of the target area, and the user will mark the blood vessels on the two-dimensional ultrasound images, wherein the marking operation can be manual tracing or marking with regular graphics (such as circles, ellipses, rectangles, etc.), so that the processor 40 knows the vascular structure and its position on each two-dimensional ultrasound image, and can generate a sub-three-dimensional vascular image based on each vascular structure and its position, for example, based on the position of each vascular structure in the two-dimensional ultrasound image, the image and the positional relationship between each frame of the two-dimensional ultrasound image, perform three-dimensional reconstruction to obtain the sub-three-dimensional vascular image. This method is equivalent to semi-automatically obtaining the sub-three-dimensional vascular image.

[0060] In a third method, the processor 40 detects and segments the vascular structure of each frame of the two-dimensional ultrasound image in the multiple frames of the two-dimensional ultrasound image to obtain the vascular structure in the multiple frames of the two-dimensional ultrasound image. Compared with the aforementioned second method, this method does not require user participation and automatically detects and segments the vascular structure, so that the processor 40 knows the vascular structure and its position, and can generate a sub-3D vascular image based on each vascular structure and its position. This method is equivalent to automatically obtaining a sub-3D vascular image.

[0061] In this method, the vascular structure in the two-dimensional ultrasound image is detected and segmented, and the specific implementation method adopted can be the same as the aforementioned detection and segmentation of the vascular structure in the three-dimensional ultrasound image, such as the aforementioned first, second or third implementation method. That is, the processor 40 can use filtering, edge detection and / or threshold segmentation algorithms to perform threshold division based on image pixels, grayscale and other information in the two-dimensional ultrasound image, so as to segment the vascular structure in the two-dimensional ultrasound image. The processor 40 can also first extract features from the image blocks of the two-dimensional ultrasound image, establish feature vectors, and then classify the feature vectors, and then piece the classified image blocks back into the entire image to obtain the segmentation result (vascular structure). The processor 40 can also input the two-dimensional ultrasound image into a trained neural network model to obtain the area that needs to be segmented (that is, the vascular structure) output by the neural network model.

[0062] In the second and third modes, the processor 40 obtains the sub-three-dimensional vascular images of each target area based on the vascular structures in the multi-frame two-dimensional ultrasound images. Specifically, the processor 40 can obtain the sub-three-dimensional vascular images of each target area by three-dimensional reconstruction based on the obtained vascular structures in the multi-frame two-dimensional ultrasound images of each target area and the determined position information of the vascular structures in the multi-frame two-dimensional ultrasound images of each target area. The position information of the vascular structure in the two-dimensional ultrasound image of the target area can be determined based on the position change information, as shown in step 4.

[0063] Each target area has blood vessels, and this step is equivalent to extracting the three-dimensional images of the blood vessels in each target area through ultrasound images, which is convenient for subsequent splicing.

[0064] Step 3: The processor 40 determines the vessel type of the vascular structure in the ultrasound image of each target area, wherein the vessel type includes at least an arterial vessel type and a venous vessel type. That is, the processor 40 determines which vessels in each target area are arterial vessels and which vessels are venous vessels.

[0065] The ultrasonic scan performed by the user in step 1 can be performed in the B+C mode, that is, the ultrasonic image obtained in step 1 is a color Doppler image (including the ultrasonic B image obtained in the B mode and the blood flow image obtained in the C mode). Of course, the ultrasonic scan performed by the user in step 1 can also be performed in the vector blood flow mode, that is, the ultrasonic image obtained in step 1 is a vector blood flow image. In any case, the ultrasonic B image and blood flow information can be obtained. Therefore, the processor 40 can obtain the color Doppler image or the vector blood flow image of each target area; according to the color Doppler image or the vector blood flow image of each target area, the vascular type of the vascular structure in the ultrasonic image of each target area is obtained. Specifically, the processor 40 can obtain the blood flow information of the vascular structure in the ultrasonic image of each target area according to the color Doppler image or the vector blood flow image of each target area, and the blood flow information includes: blood flow intensity and / or blood flow velocity; in some embodiments, the blood flow information can also include blood flow direction, etc. Then, the processor 40 determines the vascular type of the vascular structure in the ultrasonic image of each target area according to the blood flow information of the vascular structure in the ultrasonic image of each target area. In some embodiments, the processor 40 can also obtain the scanning direction of the ultrasound probe as a reference for determining the type of blood vessel. The blood flow intensity and velocity in arteries and veins are quite different, and based on this information, it can be determined which blood vessels in the image are arteries and which are veins.

[0066] Determining the vascular type of the vascular structure can be specifically achieved through a preset model / algorithm. Considering that the more information the model / algorithm inputs, the more accurate the vascular type result given, the ultrasound image, the sub-three-dimensional vascular image of the target area, etc. can also be input into the model / algorithm. For example, if the ultrasound image of each target area is a three-dimensional ultrasound image, the processor 40 can input the ultrasound image of the target area (such as three-dimensional image point cloud data), the sub-three-dimensional vascular image of the target area (such as three-dimensional image point cloud data), and the blood flow information (at least one of signal strength, direction, and flow rate) of the vascular structure in the ultrasound image of the target area into a pre-trained multimodal three-dimensional vascular type recognition model or into a preset multimodal three-dimensional vascular type recognition algorithm to obtain the vascular type of the vascular structure in the ultrasound image of the target area output by the multimodal three-dimensional vascular type recognition model or the multimodal three-dimensional vascular type recognition algorithm.

[0067] The multimodal three-dimensional blood vessel type recognition model / algorithm may adopt one of the following two methods:

[0068] A multimodal three-dimensional blood vessel type recognition model / algorithm is a classification model / algorithm based on traditional machine learning. The classification model / algorithm establishes a feature vector based on the image pixels, grayscale and other information of the ultrasound image of the target area and the sub-three-dimensional blood vessel image of the target area. For example, PCA or LDA algorithm is used to process the image pixels, grayscale and other information of the ultrasound image of the target area and the sub-three-dimensional blood vessel image of the target area to establish a feature vector. At the same time, text data (such as blood flow information of the blood vessel structure) is also processed into a feature vector, and then the two feature vectors are fused, and then the fused feature vector is classified. Specifically, algorithms such as k-NN, Random Forest, or SVM can be used to classify the fused feature vector, so as to output the classification result, that is, the blood vessel structure is classified to obtain the type of each blood vessel structure (artery or vein).

[0069] Another multimodal three-dimensional vascular type recognition model / algorithm is a classification algorithm based on deep learning. This classification model / algorithm usually adopts a supervised learning strategy. By stacking modules such as convolutional layers, pooling layers, upsampling layers, deconvolution layers, and fully connected layers, a deep neural network is established. The real classification results with manual annotations are used as supervision information. The neural network takes images or point cloud data as input, that is, the ultrasound image of the target area and the sub-three-dimensional vascular image of the target area are input into the model / algorithm. The graphic feature vector is obtained through model operation, and the text feature vector obtained based on the blood flow information of the vascular structure is input into the model / algorithm for learning and outputting the classification result. That is, the vascular structure is classified to obtain the type of each vascular structure (artery or vein). Of course, the text feature vector can also be processed into array data of the same dimension as the image or point cloud data, spliced ​​with the image or point cloud data, input into the neural network for learning, and output the classification result.

[0070] For another example, if the ultrasound image of each target area is a multi-frame two-dimensional ultrasound image, the processor 40 can input the multi-frame two-dimensional ultrasound image of the target area, the vascular structure segmented in the two-dimensional ultrasound image (such as a segmentation mask), and the blood flow information (at least one of signal strength, direction, and flow velocity) of the vascular structure in the ultrasound image of the target area into a pre-trained multimodal two-dimensional blood vessel type recognition model or into a preset multimodal two-dimensional blood vessel type recognition algorithm to obtain the blood vessel type of the vascular structure in the ultrasound image of the target area output by the multimodal two-dimensional blood vessel type recognition model or the multimodal two-dimensional blood vessel type recognition algorithm.

[0071] Similarly, the multimodal two-dimensional blood vessel type recognition model / algorithm can adopt one of the following two methods:

[0072] A multimodal two-dimensional vascular type recognition model / algorithm is a classification model / algorithm based on traditional machine learning. The classification model / algorithm establishes a feature vector based on the image pixels, grayscale and other information of the two-dimensional ultrasound image of the target area and the vascular structure in the two-dimensional ultrasound image. For example, the PCA or LDA algorithm is used to process the image pixels, grayscale and other information of the two-dimensional ultrasound image of the target area and the vascular structure in the two-dimensional ultrasound image to establish a feature vector. At the same time, text data (such as blood flow information of the vascular structure) is also processed into a feature vector, and then the two feature vectors are fused, and then the fused feature vector is classified. Specifically, algorithms such as k-NN, Random Forest, or SVM can be used to classify the fused feature vector, so as to output the classification result.

[0073] Another multimodal two-dimensional vascular type recognition model / algorithm is a classification algorithm based on deep learning. This classification model / algorithm usually adopts a supervised learning strategy, and establishes a deep neural network by stacking modules such as convolutional layers, pooling layers, upsampling layers, deconvolution layers, and fully connected layers. The real classification results with manual annotations are used as supervision information, and the neural network is input with images or point cloud data, that is, the two-dimensional ultrasound image of the target area and the vascular structure in the two-dimensional ultrasound image are input into the model / algorithm, and the graphic feature vector is obtained through model operation. Then, the text feature vector obtained based on the blood flow information of the vascular structure is input into the model / algorithm for learning and outputting the classification result. Of course, the text feature vector can also be processed into array data of the same dimension as the two-dimensional ultrasound image, spliced ​​with the image, input into the neural network for learning, and output the classification result.

[0074] Step 4, the processor 40 obtains the position change information of the ultrasonic probe 10 when the ultrasonic scanning of the multiple target areas is performed. For example, when the ultrasonic scanning of the multiple target areas is performed, the processor 40 uses the position sensor configured on the ultrasonic probe 10 to obtain the position change information of the ultrasonic probe 10. The position change information of the ultrasonic probe 10 when scanning the target area is known, for example, the direction and movement trajectory of the ultrasonic probe 10 are known, for the same spatial coordinate system, the position of the ultrasonic probe 10 is known, and the plane where the ultrasonic image obtained by scanning each position is located is also known. Therefore, the processor 40 can determine the position information of the vascular structure in the ultrasonic image of each target area based on the position change information, that is, the position of the ultrasonic probe 10 when scanning the ultrasonic image is known, and the relative position relationship between the vascular structure and the ultrasonic probe in the ultrasonic image is known, so the position information of the vascular structure in the ultrasonic image can be determined based on these two information. These position relationships can be based on the same spatial coordinate system, such as the spatial coordinate system based on the position sensor, which is convenient for calculating the position of the vascular structure in the ultrasonic image of each target area (for example, determining the coordinates of the vascular structure in the spatial coordinate system).

[0075] Step 5: The processor 40 stitches the sub-3D vascular images of each target area based on the position information of the vascular structure in the ultrasound image of each target area to obtain a 3D vascular image of the multiple target areas and displays it. It can be seen that the main method of stitching the sub-3D vascular images is to automatically stitch them according to the position based on the positioning information. Specifically, the processor 40 stitches the sub-3D vascular images of each target area in the multiple target areas into a 3D vascular image based on the sub-3D vascular images of each target area and the position information of the vascular structure in the ultrasound image of each target area. Figure 3 An example is given of splicing two sub-3D blood vessel images in target regions D1 and D2 into a 3D blood vessel image of a larger region D3, and then displaying the 3D blood vessel image (eg Figure 3 3D vascular image on the right).

[0076] After the three-dimensional vascular image is obtained by splicing, its posture can also be corrected to make the three-dimensional vascular image have no traces of splicing as much as possible and improve the accuracy of the vascular position in the image. The processor 40 can correct the posture of the three-dimensional vascular image based on the vascular tree structure. For example, the processor 40 can specifically correct the posture of the three-dimensional vascular image according to the vascular centerline to align the vascular tree structure. For another example, the processor 40 can specifically align each sub-three-dimensional vascular image in the three-dimensional vascular image based on the vascular contour of the splicing surface. For another example, after the three-dimensional vascular image is displayed, the position of each sub-three-dimensional vascular image in the displayed three-dimensional vascular image can be edited, that is, medical staff are allowed to manually splice and adjust the position of multiple sub-three-dimensional vascular images based on observation and understanding. In this way, the smoothness and accuracy of the three-dimensional vascular image can be further improved.

[0077] Since a laparoscope and ultrasound are used in laparoscopic ultrasound examination, the processor 40 can display the three-dimensional vascular image in the display interface of the ultrasound imaging device, or in the display interface of the laparoscope device, and of course, can also display the three-dimensional vascular image in the display interface of both devices. The display interface of the laparoscope device usually displays an optical image taken by the laparoscope, such as a real-time video, a picture, etc. The three-dimensional vascular image displayed on the display interface of the laparoscope device can be displayed on the same screen as the optical image, or can be superimposed, such as superimposing the three-dimensional vascular image on the optical image displayed in the display interface of the laparoscope device. In this way, the user can see the real-time video inside the target tissue, and can also see the three-dimensional image of the blood vessels blocked by the tissue, which provides great convenience for doctors whether doing an examination or an operation.

[0078] like Figure 4As shown, the processor 40 can also display the blood vessel types differently in the three-dimensional blood vessel image, such as highlighting the arterial blood vessels to remind the doctor to pay attention to these arterial blood vessels during surgery. In this way, the user can see which blood vessels are arteries and which blood vessels are veins from the three-dimensional blood vessel image. The processor 40 can display the blood vessel types differently in the three-dimensional blood vessel image in a variety of specific implementations, and two examples are given below for illustration.

[0079] In the first method, the processor 40 can perform differentiated display for the three-dimensional vascular image based on the vascular type of the vascular structure in the ultrasound image of each target area, that is, directly perform differentiated display on the vascular structure in the three-dimensional vascular image, for example, the vascular structure of the arterial vascular type and the vascular structure of the venous vascular type have different colors in the three-dimensional vascular image.

[0080] In the second method, if Figure 4 As shown, the processor 40 can reconstruct the sub-three-dimensional vascular type image of each target area based on the color Doppler image of each target area and the position information of the vascular structure in the ultrasound image of each target area, and superimpose the sub-three-dimensional vascular type image on the three-dimensional vascular image for display. In the sub-three-dimensional vascular type image, the vascular structure of different vascular types has different colors. In this way, each sub-three-dimensional vascular type image is superimposed on the three-dimensional vascular image for display, and the user can see the arterial blood vessels and venous blood vessels represented by different colors. Specifically, in the case where the ultrasound image is a two-dimensional ultrasound image, after the processor 40 identifies the vascular structure (vascular segmentation mask) of the two-dimensional ultrasound image in step 2, the vascular structure of the two-dimensional ultrasound image can be colored, and then based on the position information of the vascular structure in the ultrasound image of each target area, the sub-three-dimensional vascular type image with color of each target area is reconstructed. Of course, the processor 40 can also directly use the C-mode blood flow image in the color Doppler image to extract the image of the vascular structure therein, and reconstruct the extracted images of the vascular structure based on the position information of the vascular structure in the ultrasound image of each target area to obtain the sub-three-dimensional vascular type image with color of the target area.

[0081] In some embodiments, in step 1, the doctor triggers an ultrasound scan, and an ultrasound image of a target area can be collected within a preset time (for example, about 3 seconds); the doctor triggers an ultrasound scan again to obtain an ultrasound image of another target area. The doctor can perform ultrasound scans as many times as he wants. After multiple scans, ultrasound images of multiple target areas can be obtained, and subsequent steps can be executed to obtain three-dimensional vascular images of the entire blood vessels in these target areas.

[0082] In another embodiment, in step 1, the doctor triggers an ultrasound scan, and an ultrasound image of a target area can be acquired within a preset time (e.g., about 3 seconds); the doctor triggers an ultrasound scan again, and an ultrasound image of another target area can be acquired. The processor 40 can execute the ultrasound images of the target area obtained by the two scans. Figure 2 The processing shown is performed to obtain a three-dimensional blood vessel image and display it, that is, steps 1-5 are completed. Afterwards, the doctor may continue to scan the ultrasound images of other target areas, that is, the processor 40 may obtain the ultrasound image of the target area currently being scanned, and then identify the vascular structure in the ultrasound image of the current target area to obtain a sub-3D vascular image of the current target area (the specific process is the same as step 2, except that the number of target areas is different, which will not be described in detail), determine the vascular type of the vascular structure in the ultrasound image of the current target area (the specific process is the same as step 3, except that the number of target areas is different, which will not be described in detail), obtain the position change information of the ultrasound probe when the previous target area and the current target area are ultrasonically scanned, and determine the position information of the vascular structure in the ultrasound image of the current target area based on the position change information (the specific process is the same as step 4, except that it is limited to the previous and next target areas, which will not be described in detail), and based on the position information of the vascular structure of the current target area, splice the sub-3D vascular image of the current target area into the currently displayed 3D vascular image, and update the 3D vascular image on the display interface, that is, if the display interface currently displays a 3D vascular image, after obtaining the sub-3D vascular image through the aforementioned steps, the sub-3D vascular image can be spliced ​​into the existing 3D vascular image, thereby updating the 3D vascular image. This is equivalent to scanning multiple target areas at the beginning to obtain and display three-dimensional vascular images. Every time the doctor scans the ultrasound image of a target area, a sub-three-dimensional vascular image of the target area is added to the three-dimensional vascular image displayed on the display interface. The doctor can see the results of his real-time scanning and real-time stitching, and get feedback, which is conducive to quickly scanning the area he wants.

[0083] Taking into account the limited field of view of laparoscopic ultrasonic imaging, the ultrasonic images obtained from each scan can only show some local blood vessels, and are two-dimensional, which is not conducive to the doctor's grasp of the location of lesions and blood vessels from a global perspective, resulting in time-consuming and labor-intensive surgery. One scan can obtain an ultrasonic image of a target area. The present invention processes the ultrasonic images of multiple target areas obtained by multiple scans through the above method, thereby splicing together a three-dimensional image of blood vessels in a relatively large area, which greatly facilitates doctors' inspections and surgeries. In addition to three-dimensional splicing of blood vessels, of course, lesions (such as kidney tumors, lung tumors, uterine tumors, adnexal tumors, etc.) can also be three-dimensionally spliced, such as Figure 5 As shown, the processor 40 may specifically perform three-dimensional stitching of the lesions including the following steps:

[0084] Step 2': the processor 40 identifies the lesion structure (such as Figure 3 and Figure 4 The green area in the ultrasound image) is used to obtain a sub-3D lesion image of each target area. The specific method used to identify the lesion structure to obtain the sub-3D lesion image is the same as that used in step 2 of the aforementioned embodiment to identify the vascular structure to obtain the sub-3D vascular image, that is, the "vascular structure" in the content of the aforementioned step 2 is changed to "lesion structure", and the "sub-3D vascular image" is changed to "sub-3D lesion image", which will not be repeated here. Step 2' and step 2 can be performed simultaneously, that is, the processor 40 can identify the vascular structure in the ultrasound image of each target area to obtain the sub-3D vascular image of each target area, and identify the lesion structure in the ultrasound image of each target area to obtain the sub-3D lesion image of each target area.

[0085] In step 4', the processor 40 determines the position information of the lesion structure in the ultrasound image of each target area based on the position change information of the ultrasound probe when the multiple target areas are ultrasonically scanned. Similarly, the position information of the lesion structure can be determined in the same way as the position information of the blood vessel structure in the aforementioned step 4, that is, the "blood vessel structure" in the content of the aforementioned step 4 is changed to "lesion structure", which will not be elaborated here. Step 4' and step 4 can be performed simultaneously, that is, the processor 40 can determine the position information of the blood vessel structure in the ultrasound image of each target area and the position information of the lesion structure based on the position change information.

[0086] Step 5', the processor 40 splices the sub-three-dimensional lesion images of each target area based on the position information of the lesion structure in the ultrasound image of each target area to obtain a three-dimensional lesion image of the multiple target areas. Similarly, the splicing of the sub-three-dimensional lesion images of each target area to obtain a three-dimensional lesion image can be done in the same way as the splicing of the sub-three-dimensional blood vessel images of each target area to obtain a three-dimensional blood vessel image in the aforementioned step 5, that is, the "blood vessel structure" in the aforementioned step 5 is changed to "lesion structure", "sub-three-dimensional blood vessel image" is changed to "sub-three-dimensional lesion image", and "three-dimensional blood vessel image" is changed to "three-dimensional lesion image", which will not be repeated here. Step 5' and step 5 can be performed simultaneously, that is, the processor 40 can splice the sub-three-dimensional lesion images of each target area based on the position information of the lesion structure in the ultrasound image of each target area to obtain a three-dimensional lesion image of the multiple target areas; splicing the sub-three-dimensional blood vessel images of each target area to obtain a three-dimensional blood vessel image based on the position information of the blood vessel structure in the ultrasound image of each target area.

[0087] After obtaining the three-dimensional lesion image, the processor 40 can display the three-dimensional lesion image, for example, displaying the three-dimensional lesion image in the display interface of the ultrasound imaging device, displaying the three-dimensional lesion image in the display interface of the laparoscope device, etc. In this embodiment, the three-dimensional blood vessel image and the three-dimensional lesion image are displayed as a complete image, that is, when there are blood vessels and lesions in the target tissue, the three-dimensional blood vessel image and the three-dimensional lesion image are displayed as a complete image. Figure 2 and 5 The steps shown in FIG. 1 are used to obtain a three-dimensional image, such as Figure 3 and 4 As shown in the three-dimensional images on the left and right sides, this three-dimensional image contains a three-dimensional blood vessel image and a three-dimensional lesion image, and the positional relationship between the blood vessels and lesions reflects the position of the real blood vessels and lesions. In this three-dimensional image, arterial vessels, venous vessels and lesions are displayed differently, that is, these three structures can be distinguished by color, brightness or other graphic elements. The simplest is that the three colors are different. It can be seen that this method can use laparoscopic ultrasound guidance in surgical tasks under laparoscopic ultrasound guidance, such as liver surgery, kidney tumor resection, lung tumor resection, uterine tumor resection, adnexal tumor resection and other usage scenarios, which provides great convenience for doctors' surgery.

[0088] In addition to three-dimensional splicing of blood vessels, some embodiments can also splice blood vessels, such as splicing structures such as bile ducts. The method provided by the present invention is very suitable for such anatomical structures that are long and have many branches and are difficult for doctors to observe. They can be extracted from ultrasound images and then spliced ​​in three dimensions, so that doctors can see them intuitively, which is conducive to performing various types of surgeries. Figure 6 As shown, the processor 40 may specifically perform the following steps on the three-dimensional splicing of the blood vessels:

[0089] In step 2", the processor 40 identifies the vascular structure in the ultrasound image of each target area to obtain a sub-3D vascular image of each target area. The specific method used to identify the vascular structure to obtain the sub-3D vascular image is the same as that used to identify the vascular structure to obtain the sub-3D vascular image in step 2 of the aforementioned embodiment, that is, the "vascular structure" in the content of the aforementioned step 2 is changed to "vascular structure", and the "sub-3D vascular image" is changed to "sub-3D vascular image", which will not be described in detail here. Step 2", step 2' and step 2 can be performed simultaneously in pairs, or all three can be performed simultaneously.

[0090] In step 4", the processor 40 determines the position information of the vascular structure in the ultrasound image of each target area based on the position change information of the ultrasound probe when the multiple target areas are ultrasonically scanned. Similarly, the position information of the vascular structure can be determined in the same way as the position information of the vascular structure in the aforementioned step 4, that is, the "vascular structure" in the content of the aforementioned step 4 is changed to "vascular structure", which will not be elaborated here. Step 4", step 4' and step 4 can be performed simultaneously, or all three can be performed simultaneously.

[0091] In step 5", the processor 40 splices the sub-three-dimensional vascular images of each target area based on the determined position information of the vascular structure in the ultrasound image of each target area to obtain a three-dimensional vascular image of the multiple target areas. Similarly, the three-dimensional vascular images of each target area are spliced ​​to obtain a three-dimensional vascular image in the same way as the three-dimensional vascular images of each target area are spliced ​​to obtain a three-dimensional vascular image in the aforementioned step 5, that is, the "vascular structure" in the aforementioned step 5 is changed to "vascular structure", the "sub-three-dimensional vascular image" is changed to "sub-three-dimensional vascular image", and the "three-dimensional vascular image" is changed to "three-dimensional vascular image", which will not be described in detail here. Step 5", step 5' and step 5 can be performed simultaneously in pairs, or all three can be performed simultaneously.

[0092] After obtaining the three-dimensional vascular image, the processor 40 can display the three-dimensional vascular image, for example, displaying the three-dimensional vascular image in the display interface of the ultrasound imaging device, displaying the three-dimensional vascular image in the display interface of the laparoscopic device, etc. In this embodiment, two or all of the three-dimensional blood vessel image, the three-dimensional lesion image and the three-dimensional vascular image are displayed as a complete image. In this embodiment, if there are blood vessels, lesions and blood vessels in the target tissue, the three-dimensional images of these three are displayed, and if there are only two or one of them, only the three-dimensional images of two or one of them are displayed. Similarly, the color of the blood vessels in the three-dimensional vascular image can be different from that of the blood vessels and lesions for differential display.

[0093] Based on the above background, the present invention proposes a 3D reconstruction system for ultrasound data under laparoscopic ultrasound, which is a tool for 3D reconstruction of blood vessels and lesions in a large range, and provides doctors with a 3D display of blood vessels and lesions that can reflect the entire target tissue, such as the liver surgery-related area. The specific process can be that the doctor uses a laparoscopic ultrasound probe with a positioning device to scan multiple times at different positions in the relevant area to obtain ultrasound data of liver tumors and liver blood vessels, and obtains the contours of tumors and blood vessels through tumor and blood vessel segmentation. The system automatically performs 3D reconstruction and splicing to obtain 3D data of blood vessels and lesions in the entire area. At the same time, combined with color Doppler blood flow signals (signal strength, direction, flow velocity, etc.), the blood vessels are identified as arteries or veins, and the blood vessels are color-coded according to the type, and displayed in real time in the window, which provides great convenience and support for doctors to perform surgery.

[0094] The difference, beneficial effects and advantages of this solution and the existing technology are: the existing technology is limited by the limitations of laparoscopic ultrasound operation itself, and cannot provide a three-dimensional vascular model of sufficient range. Doctors still need to observe repeatedly manually and estimate based on imagination, and they do not have the ability to indicate the attributes of vascular branches according to the direction of blood flow. The reliability is low and the clinical significance is limited. This solution can provide doctors with a large range of lesions and vascular tree three-dimensional reconstructions through splicing, and display the vascular tree in different colors in combination with blood flow data, so that doctors can more comprehensively observe the lesions and blood vessels in the liver area related to the operation. The three-dimensional spatial state, and can be combined with the branch attributes marked by vascular color to better plan and implement puncture or resection operations.

[0095] This document is described with reference to various exemplary embodiments. However, those skilled in the art will recognize that changes and modifications may be made to the exemplary embodiments without departing from the scope of this document. For example, various operating steps and components for performing the operating steps may be implemented in different ways (e.g., one or more steps may be deleted, modified, or incorporated into other steps) depending on the specific application or considering any number of cost functions associated with the operation of the system.

[0096] In addition, as will be appreciated by those skilled in the art, the principles of the present invention may be reflected in a computer program product on a computer-readable storage medium, which is pre-installed with a computer-readable program code. Any tangible, non-temporary computer-readable storage medium may be used, including magnetic storage devices (hard disks, floppy disks, etc.), optical storage devices (CD-ROMs, DVDs, Blu Ray disks, etc.), flash memory, and / or the like. These computer program instructions may be loaded onto a general-purpose computer, a special-purpose computer, or other programmable communication device to form a machine, so that the instructions executed on the computer or other programmable communication device may generate a device that implements a specified function. These computer program instructions may also be stored in a computer-readable memory, which may instruct a computer or other programmable communication device to operate in a specific manner, so that the instructions stored in the computer-readable memory may form a manufactured product, including an implementation device that implements a specified function. The computer program instructions may also be loaded onto a computer or other programmable communication device, so that a series of operating steps are performed on the computer or other programmable device to generate a computer-implemented process, so that the instructions executed on the computer or other programmable device may provide steps for implementing a specified function.

[0097] Although the principles of this invention have been shown in various embodiments, many modifications of structures, arrangements, proportions, elements, materials and components particularly suitable for specific environments and operational requirements can be used without departing from the principles and scope of this invention. The above modifications and other changes or amendments will be included in the scope of this invention.

[0098] The foregoing specific description has been described with reference to various embodiments. However, those skilled in the art will recognize that various modifications and changes can be made without departing from the scope of the present disclosure. Therefore, the consideration of the present disclosure will be illustrative rather than restrictive, and all these modifications will be included in its scope. Similarly, the advantages, other advantages and solutions to the problems of various embodiments have been described above. However, the benefits, advantages, solutions to the problems and any elements that can produce these, or make them more clear, should not be interpreted as critical, necessary or necessary. The term "include" and any other variants used in this article are all non-exclusive inclusions, so that the process, method, article or device including the list of elements not only includes these elements, but also includes other elements that are not explicitly listed or do not belong to the process, method, system, article or device. In addition, the term "coupled" and any other variants used in this article refer to physical connections, electrical connections, magnetic connections, optical connections, communication connections, functional connections and / or any other connections.

[0099] Those skilled in the art will appreciate that many changes may be made to the details of the above-described embodiments without departing from the basic principles of the invention. Therefore, the scope of the invention should be determined from the following claims.

Claims

1. A laparoscopic ultrasonic imaging method, characterized in that: include: Acquire an ultrasonic image of each of the multiple target areas obtained by ultrasonically scanning the multiple target areas of the target tissue; Identifying the vascular structure in the ultrasound image of each target area to obtain a sub-three-dimensional vascular image of each target area; Acquiring a color Doppler image or a vector blood flow image of each target area; Acquire the blood vessel type of the blood vessel structure in the ultrasound image of each target area according to the color Doppler image or the vector blood flow image of each target area, wherein the blood vessel type at least includes an arterial blood vessel type and a venous blood vessel type; Acquiring position change information of the ultrasound probe when performing ultrasound scanning on the multiple target areas, and determining position information of the blood vessel structure in the ultrasound image of each target area based on the position change information; Based on the determined position information of the vascular structure in the ultrasound image of each target area, stitching the sub-three-dimensional vascular images of each target area to obtain and display a three-dimensional vascular image of the multiple target areas, and differentially displaying the vascular types in the three-dimensional vascular image; The differentially displaying the blood vessel types in the three-dimensional blood vessel image includes: Performing differentiated display for the three-dimensional vascular image based on the vascular type of the vascular structure in the ultrasound image of each target area; or, Based on the color Doppler images of the target areas and the determined position information of the vascular structures in the ultrasound images of the target areas, three-dimensional reconstruction is performed to obtain sub-three-dimensional vascular type images of the target areas, and the sub-three-dimensional vascular type images are superimposed on the three-dimensional vascular images for display.

2. A laparoscopic ultrasonic imaging method, characterized in that: include: Acquire an ultrasonic image of each of the multiple target areas obtained by ultrasonically scanning the multiple target areas of the target tissue; Identifying the vascular structure in the ultrasound image of each target area to obtain a sub-three-dimensional vascular image of each target area; Determine the blood vessel type of the blood vessel structure in the ultrasound image of each target area, wherein the blood vessel type at least includes an arterial blood vessel type and a venous blood vessel type; Acquiring position change information of the ultrasound probe when performing ultrasound scanning on the multiple target areas, and determining position information of the blood vessel structure in the ultrasound image of each target area based on the position change information; Based on the determined position information of the vascular structure in the ultrasound image of each target area, the sub-three-dimensional vascular images of each target area are spliced ​​to obtain and display a three-dimensional vascular image of the multiple target areas, and the vascular types are differentially displayed in the three-dimensional vascular image.

3. The method according to claim 1 or 2, characterized in that The ultrasound image of each target area is a three-dimensional ultrasound image; and identifying the vascular structure in the ultrasound image of each target area to obtain a sub-three-dimensional vascular image of each target area includes: The blood vessel structure in the three-dimensional ultrasound image is detected and segmented to obtain sub-three-dimensional blood vessel images of each target area.

4. The method according to claim 1 or 2, characterized in that: The ultrasound image of each target area is a multi-frame two-dimensional ultrasound image; and the identifying of the vascular structure in the ultrasound image of each target area to obtain the sub-three-dimensional vascular image of each target area includes at least one of the following: reconstructing the multiple frames of two-dimensional ultrasound images into three-dimensional ultrasound images based on the position change information, detecting and segmenting the blood vessel structure in the three-dimensional ultrasound image, and obtaining sub-three-dimensional blood vessel images of each target area; Displaying the multiple frames of two-dimensional ultrasound images, obtaining the blood vessel structures in the multiple frames of two-dimensional ultrasound images in response to a user's marking operation on the blood vessel structures in the displayed multiple frames of two-dimensional ultrasound images, and obtaining the sub-three-dimensional blood vessel images of the target areas based on the blood vessel structures in the multiple frames of two-dimensional ultrasound images; The vascular structure of each frame of the two-dimensional ultrasonic image in the multiple frames of the two-dimensional ultrasonic image is detected and segmented to obtain the vascular structure in the multiple frames of the two-dimensional ultrasonic image, and the sub-three-dimensional vascular image of each target area is obtained based on the vascular structure in the multiple frames of the two-dimensional ultrasonic image.

5. The method according to claim 2, characterized in that Determining the blood vessel type of the blood vessel structure in the ultrasound image of each target area includes: Acquiring a color Doppler image or a vector blood flow image of each target area; The blood vessel type of the blood vessel structure in the ultrasound image of each target area is acquired according to the color Doppler image or the vector blood flow image of each target area.

6. The method according to claim 1 or 5, characterized in that After acquiring the color Doppler image or the vector blood flow image of each target area, the method further includes: According to the color Doppler image or the vector blood flow image of each target area, blood flow information of the blood vessel structure in the ultrasound image of each target area is acquired, and the blood flow information includes: blood flow intensity and / or blood flow velocity.

7. The method according to claim 1 or 2, characterized in that: The obtaining of position change information of the ultrasonic probe when performing ultrasonic scanning on the multiple target areas includes: The position change information of the ultrasonic probe is acquired by using a position sensor configured on the ultrasonic probe.

8. The method according to claim 4, characterized in that The obtaining of the sub-three-dimensional vascular images of each target area based on the vascular structures in the multiple frames of two-dimensional ultrasound images includes: Based on the obtained vascular structures in the multi-frame two-dimensional ultrasound images of the target areas and the determined position information of the vascular structures in the multi-frame two-dimensional ultrasound images of the target areas, three-dimensional reconstruction is performed to obtain sub-three-dimensional vascular images of the target areas.

9. The method according to claim 1 or 2, characterized in that: The step of stitching the sub-three-dimensional vascular images of each target area based on the determined position information of the vascular structure in the ultrasound image of each target area to obtain a three-dimensional vascular image of the plurality of target areas and displaying the image includes: Based on the sub-three-dimensional blood vessel images of each target area and the determined position information of the blood vessel structure in the ultrasound image of each target area, the sub-three-dimensional blood vessel images of each target area in the multiple target areas are spliced ​​into a three-dimensional blood vessel image, and the three-dimensional blood vessel image is displayed.

10. The method according to claim 1 or 2, characterized in that: Also includes: Identifying the lesion structure in the ultrasound image of each target area to obtain a sub-three-dimensional lesion image of each target area; Determining the position information of the lesion structure in the ultrasound image of each target area based on the position change information of the ultrasound probe when performing ultrasound scanning on the multiple target areas; Based on the determined position information of the lesion structure in the ultrasound image of each target area, splicing the sub-three-dimensional lesion images of each target area to obtain a three-dimensional lesion image of the multiple target areas; The three-dimensional lesion image is displayed.

11. The method according to claim 1 or 2, characterized in that: The target tissue is liver tissue.

12. The method according to claim 2, characterized in that The differentially displaying the blood vessel types in the three-dimensional blood vessel image includes: Performing differential display for the three-dimensional vascular image based on the vascular type of the vascular structure in the ultrasound image of each target area; or, Based on the color Doppler images of the target areas and the determined position information of the vascular structures in the ultrasound images of the target areas, three-dimensional reconstruction is performed to obtain sub-three-dimensional vascular type images of the target areas, and the sub-three-dimensional vascular type images are superimposed on the three-dimensional vascular images for display.

13. The method according to any one of claims 1 to 12, characterized in that Displaying the three-dimensional blood vessel image in a display interface of an ultrasonic imaging device; and / or, The three-dimensional blood vessel image is displayed on a display interface of a laparoscope device.

14. The method according to claim 13, characterized in that The displaying of the three-dimensional blood vessel image in the display interface of the laparoscope device comprises: The three-dimensional blood vessel image is superimposed and displayed on the optical image displayed on the display interface of the laparoscope device.

15. An ultrasonic imaging device, characterized in that: include: An ultrasonic probe, which is equipped with a position sensor, wherein the position sensor is used to obtain real-time position information of the ultrasonic probe; A transmitting and receiving circuit, used for controlling the ultrasonic probe to transmit ultrasonic waves to the target tissue and receive ultrasonic echo signals; A processor, configured to execute the method according to any one of claims 1 to 14.

16. A computer program product comprising a computer program and / or instructions, characterized in that: When the computer program and / or the instructions are executed by a processor, the method according to any one of claims 1 to 14 is implemented.

Images