A puncture robot ultrasonic navigation method and system

Abstract

The present invention discloses a puncture robot ultrasonic navigation method and system, including obtaining tissue recognition information of the target puncture area and further obtaining a low-latency super-resolution ultrasonic three-dimensional image, which effectively improves the accuracy of lesion area recognition and the accuracy of surgical navigation. By preprocessing the ultrasonic three-dimensional image, the noise and interference in the image are eliminated, the image quality is improved, and a more reliable data basis is provided for subsequent lesion area recognition and path planning. At the same time, a lesion area recognition model is established in combination with a deep learning algorithm, which can adaptively learn and recognize the characteristics of different lesion areas, thereby improving the accuracy and generalization ability of lesion area recognition. In addition, by improving the path planning algorithm to generate the puncture path, factors such as the location, morphology and structure of the surrounding tissue of the lesion area are comprehensively considered, making the planned path more reasonable and safe.

Description

Technical Field

[0001] The present invention relates to the technical field of puncture robot ultrasonic navigation, and in particular to a puncture robot ultrasonic navigation method and system. Background Art

[0002] The application prospects of puncture robot ultrasound navigation technology in the medical field are broad. As an advanced medical technology, it can assist doctors in performing precise puncture operations and improve the accuracy and safety of surgery. However, the traditional puncture robot ultrasound navigation method faces some challenges and limitations in practical applications.

[0003] First, the traditional ultrasound navigation method of puncture robots has shortcomings in identifying lesion areas. Since the lesion area may present complex morphology and characteristics, traditional ultrasound image processing methods often have difficulty in accurately identifying and locating these areas. This may cause doctors to be unable to accurately find the lesion area during surgery, thus affecting the surgical effect. To solve this problem, researchers can further explore ultrasound image processing methods based on artificial intelligence technologies such as deep learning to improve the accuracy of identifying lesion areas.

[0004] Secondly, puncture path planning is also an important problem faced by traditional puncture robot ultrasound navigation methods. In practical applications, doctors need to plan the puncture path based on factors such as the location and morphology of the lesion area and the structure of the surrounding tissue. However, traditional path planning methods often find it difficult to comprehensively consider these factors, resulting in unreasonable planned paths or potential risks. In order to improve this problem, path planning methods based on three-dimensional ultrasound images and real-time navigation technology can be studied to achieve more accurate and safe puncture operations.

[0005] In addition, the application of puncture robot ultrasound navigation technology also needs to consider individual differences in patients and uncertainties during the operation. The lesion areas of different patients may present different morphologies and characteristics, and various unexpected situations may occur during the operation. Therefore, researchers need to continuously improve and optimize puncture robot ultrasound navigation technology to meet the needs of different patients and cope with various challenges during the operation. Summary of the invention

[0006] The purpose of this section is to summarize some aspects of embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the specification abstract and the invention title of this application to avoid blurring the purpose of this section, the specification abstract and the invention title, and such simplifications or omissions cannot be used to limit the scope of the present invention.

[0007] In view of the above existing problems, the present invention is proposed.

[0008] Therefore, the present invention provides a puncture robot ultrasonic navigation method and system, which can solve the problems mentioned in the background technology.

[0009] In order to solve the above technical problems, the present invention provides the following technical solutions, a puncture robot ultrasound navigation method, comprising:

[0010] According to the existing tissue ultrasound characteristics, tissue identification information of the target puncture area is obtained, and according to the tissue identification information of the target puncture area, a low-latency super-resolution ultrasound three-dimensional image of the target puncture area is obtained;

[0011] Preprocessing the low-latency super-resolution ultrasonic three-dimensional image of the target puncture area;

[0012] A lesion area recognition model is established based on the preprocessed ultrasonic three-dimensional image data combined with a deep learning algorithm, and a puncture path is generated in combination with an improved path planning algorithm, and the puncture robot ultrasonic navigation is performed based on the puncture path.

[0013] As a preferred solution of the puncture robot ultrasonic navigation method of the present invention, wherein: the acquisition of tissue identification information of the target puncture area according to the existing tissue ultrasonic characteristics includes:

[0014] Sending a pulse signal to the target puncture area through an ultrasonic probe array and acquiring an original ultrasonic signal data set, wherein the original ultrasonic signal data set at least includes the echo intensity reflected by the tissue at different depths;

[0015] The original ultrasound signal data set is filtered and gain controlled to obtain a two-dimensional ultrasound image sequence I t , where t represents the frame number;

[0016] Extract the tissue feature vector in each frame image and record it as represents the tissue characteristics at position j, the tissue characteristics at least including echo intensity, texture characteristics, and velocity spectrum, where N is the characteristic dimension coefficient;

[0017] Use machine learning to build a feature database D of existing tissue types, containing a set of feature vectors for each tissue type

[0018] Calculate the Euclidean distance between the tissue feature vector of each voxel in the current target puncture area and each tissue type in the database:

[0019]

[0020] An objective function is established to determine the tissue type of the target puncture area. The objective function is expressed as follows:

[0021]

[0022] Among them, k represents the kth tissue type in the database, and M represents the existing tissue type types.

[0023] As a preferred embodiment of the puncture robot ultrasonic navigation method of the present invention, the step of obtaining a low-latency super-resolution ultrasonic three-dimensional image of the target puncture area according to the tissue identification information of the target puncture area includes:

[0024] After determining the tissue type of the target puncture area, the compressed sensing algorithm is used to improve the spatial resolution of the two-dimensional ultrasound image;

[0025] The lifted two-dimensional ultrasound image block is expressed as:

[0026]

[0027] in, is a low-resolution image patch, is the corresponding high-resolution image block, and H represents the compressed sensing algorithm function;

[0028] The super-resolution processed two-dimensional image sequences are combined to form a three-dimensional volume data set, i.e., a low-latency super-resolution ultrasound three-dimensional image of the target puncture area.

[0029] As a preferred embodiment of the puncture robot ultrasonic navigation method of the present invention, the preprocessing of the low-latency super-resolution ultrasonic three-dimensional image of the target puncture area includes performing noise removal, image enhancement, smoothing filtering, edge detection and normalization processing on the ultrasonic three-dimensional image in sequence;

[0030] The noise removal is achieved by combining improved wavelet analysis with adaptive threshold denoising.

[0031] As a preferred solution of the puncture robot ultrasound navigation method of the present invention, wherein: the establishment of the lesion area recognition model based on the preprocessed ultrasound three-dimensional image data combined with the deep learning algorithm includes:

[0032] A 3D convolutional neural network architecture is used to establish a lesion area recognition model;

[0033] The preprocessed low-latency super-resolution ultrasound three-dimensional image is used as the input of the model, and the output of the model is the lesion probability;

[0034] The lesion probability is expressed as:

[0035] p v =f(V;θ)

[0036] Among them, f represents the forward propagation function of the lesion area recognition model, θ is the parameter set of the lesion area recognition model, v represents the voxel, and for each voxel, v∈V, p v Represents the probability of disease.

[0037] As a preferred solution of the puncture robot ultrasound navigation method of the present invention, wherein: the establishment of the lesion area recognition model based on the preprocessed ultrasound three-dimensional image data combined with the deep learning algorithm also includes:

[0038] Using an improved Focal Tversky Loss function to train the lesion area recognition model;

[0039] Improve the traditional Focal Tversky Loss function by combining adaptive weight adjustment and attention mechanism;

[0040] The improved Focal Tversky Loss loss function is expressed as:

[0041]

[0042] Among them, Cftl represents the improved Focal Tversky Loss loss function, Y is the real annotated lesion area, is the lesion area predicted by the model, represents the number of pixels of the intersection of the two, A(Y) represents the attention weight map based on the lesion area Y, and They represent the unpredicted part of the true annotation area and the part of the predicted area that does not belong to the true annotation, respectively. γ is the focal parameter used to emphasize difficult-to-classify samples. α(y) and β(y) represent adaptive weight coefficients, and the weights change with the sample position y.

[0043] As a preferred solution of the puncture robot ultrasonic navigation method of the present invention, the puncture path generated by combining the improved path planning algorithm includes:

[0044] Inputting the real-time ultrasonic three-dimensional image data into the lesion area recognition model to obtain a probability map of the lesion area;

[0045] The first threshold τ is set. If the lesion probability of a voxel in the ultrasound three-dimensional image is greater than the threshold, the voxel is considered to be part of the lesion area, which is expressed as:

[0046] S={v∈V:p v >τ}

[0047] Among them, S is the voxel with a probability of all lesions higher than the threshold, p v represents the probability of lesion;

[0048] Combined with the improved path planning algorithm, an optimal puncture path is planned from the safe starting point to the center of the lesion in three-dimensional space;

[0049] The improved path planning algorithm includes:

[0050] Define the safe starting point as S_start and the center of the lesion as S_target;

[0051] Improve the A* algorithm by adding a constraint evaluation function as an improved path planning algorithm;

[0052] The constraint evaluation function is expressed as:

[0053] C(n)=w 1 ·A1C(n)+w 2 ·A2C(n)+w 3 LC(n)

[0054] Among them, A1C(n) is the cost function of avoiding key organs and blood vessels, A2C(n) is the cost function of needle tract safety angle, LC(n) is the cost function of needle tract length limitation, and w 1 ,w 2 ,w 3 is the corresponding weight parameter;

[0055] The heuristic function of the improved A* algorithm is:

[0056] f′(n)=f(n)+λ·C(n)

[0057] f(n)=g(n)+h(n)

[0058] Where λ is a tuning weight used to balance the path length and constraint compliance, g(n) represents the accumulated distance along the path from the safe starting point S_start to node n, and h(n) is the heuristic estimate from node n to the lesion center S_target.

[0059] A puncture robot ultrasonic navigation system, characterized in that: it includes:

[0060] An image acquisition module, used to acquire tissue identification information of a target puncture area according to existing tissue ultrasound characteristics, and to acquire a low-latency super-resolution ultrasound three-dimensional image of the target puncture area according to the tissue identification information of the target puncture area;

[0061] A preprocessing module, used for preprocessing the low-latency super-resolution ultrasonic three-dimensional image of the target puncture area;

[0062] The navigation module is used to establish a lesion area recognition model based on the preprocessed ultrasonic three-dimensional image data in combination with a deep learning algorithm, and to generate a puncture path in combination with an improved path planning algorithm, and to perform ultrasonic navigation of the puncture robot according to the puncture path.

[0063] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and wherein the processor implements the steps of the above method when executing the computer program.

[0064] A computer-readable storage medium stores a computer program thereon, wherein the computer program implements the steps of the method described above when executed by a processor.

[0065] Beneficial effects of the present invention: The present invention proposes a puncture robot ultrasonic navigation method and system, which obtains tissue recognition information of the target puncture area according to the existing tissue ultrasonic characteristics, and obtains a low-latency super-resolution ultrasonic three-dimensional image of the target puncture area according to the tissue recognition information of the target puncture area; preprocesses the low-latency super-resolution ultrasonic three-dimensional image of the target puncture area; establishes a lesion area recognition model based on the preprocessed ultrasonic three-dimensional image data combined with a deep learning algorithm, and generates a puncture path in combination with an improved path planning algorithm, and performs puncture robot ultrasonic navigation according to the puncture path. The puncture robot ultrasonic navigation method and system of the present invention obtains tissue recognition information of the target puncture area and further obtains a low-latency super-resolution ultrasonic three-dimensional image, which effectively improves the accuracy of lesion area recognition and the accuracy of surgical navigation. By preprocessing the ultrasonic three-dimensional image, noise and interference in the image are eliminated, the image quality is improved, and a more reliable data basis is provided for subsequent lesion area recognition and path planning. At the same time, the lesion area recognition model is established in combination with the deep learning algorithm, which can adaptively learn and recognize the characteristics of different lesion areas, and improve the accuracy and generalization ability of lesion area recognition. In addition, by improving the path planning algorithm to generate the puncture path, factors such as the location, morphology and structure of the surrounding tissue of the lesion area are comprehensively considered, making the planned path more reasonable and safe. In summary, the puncture robot ultrasonic navigation method and system of the present invention can significantly improve the accuracy and safety of puncture surgery in practical applications, and have important clinical value and application prospects. The puncture robot ultrasonic navigation method and system of the present invention are an advanced medical technology with broad application prospects and important clinical value. Through continuous optimization and improvement, it is believed that the puncture robot ultrasonic navigation technology and system of the present invention will play a more important role in the future medical field and bring safer and more efficient medical services to doctors and patients. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. Among them:

[0067] Figure 1 A method flow chart of a puncture robot ultrasonic navigation method and system provided by one embodiment of the present invention;

[0068] Figure 2 An internal structural diagram of a computer device of a puncture robot ultrasonic navigation method and system provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0069] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0070] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0071] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0072] The present invention is described in detail with reference to schematic diagrams. When describing the embodiments of the present invention, for the sake of convenience, the cross-sectional diagrams showing the device structure will not be partially enlarged according to the general scale, and the schematic diagrams are only examples, which should not limit the scope of protection of the present invention. In addition, in actual production, the three-dimensional dimensions of length, width and depth should be included.

[0073] At the same time, in the description of the present invention, it should be noted that the directions or positional relationships indicated by the terms "upper, lower, inner and outer" are based on the directions or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore cannot be understood as limiting the present invention. In addition, the terms "first, second or third" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0074] In the present invention, unless otherwise clearly specified and limited, the terms "install, connect, connect" should be understood in a broad sense, for example: it can be a fixed connection, a detachable connection or an integral connection; it can also be a mechanical connection, an electrical connection or a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0075] Example 1

[0076] Reference Figure 1-2 , is the first embodiment of the present invention, which provides a puncture robot ultrasound navigation method and system, including a puncture robot ultrasound navigation method and a puncture robot ultrasound navigation system, wherein a puncture robot ultrasound navigation method includes:

[0077] S101: acquiring tissue identification information of a target puncture area according to existing tissue ultrasound characteristics, and acquiring a low-latency super-resolution ultrasound three-dimensional image of the target puncture area according to the tissue identification information of the target puncture area;

[0078] Among them, according to the existing tissue ultrasound characteristics, obtaining tissue identification information of the target puncture area includes:

[0079] Sending a pulse signal to the target puncture area through an ultrasonic probe array and acquiring an original ultrasonic signal data set, wherein the original ultrasonic signal data set at least includes the echo intensity reflected by the tissue at different depths;

[0080] Furthermore, the original ultrasound signal data set is filtered and gain controlled to obtain a two-dimensional ultrasound image sequence I t , where t represents the frame number;

[0081] In an optional embodiment, the original ultrasonic signal data set may be filtered by a low-pass filter, a band-pass filter, a high-pass filter or any combination of filters, and the gain control of the original ultrasonic signal data set may be controlled by automatic gain control, which automatically adjusts the gain according to the signal strength to ensure that the amplitude of the output signal is within a predetermined range. It can be designed to adjust the amplification factor according to the signal energy within a time window. Manual gain adjustment can also be used;

[0082] In the embodiment of the present application, the gain control adopts manual gain adjustment, and the gain values ​​of different depths or areas can be manually set according to experience or doctor's needs, which can better meet the actual needs of the present application.

[0083] Furthermore, the tissue feature vector in each frame of the image is extracted and recorded as represents the tissue characteristics at position j, the tissue characteristics at least include echo intensity, texture characteristics, and velocity spectrum, where N is the characteristic dimension coefficient;

[0084] Furthermore, machine learning is used to establish a feature database D of existing tissue types, which contains a set of feature vectors for each tissue type.

[0085] Furthermore, the Euclidean distance between the tissue feature vector of each voxel in the current target puncture area and each tissue type in the database is calculated:

[0086]

[0087] Furthermore, an objective function is established to determine the tissue type of the target puncture area. The objective function is expressed as follows:

[0088]

[0089] Among them, k represents the kth tissue type in the database, and M represents the existing tissue type types.

[0090] Furthermore, obtaining a low-latency super-resolution ultrasonic three-dimensional image of the target puncture area according to the tissue recognition information of the target puncture area includes:

[0091] After determining the tissue type of the target puncture area, the compressed sensing algorithm is used to improve the spatial resolution of the two-dimensional ultrasound image;

[0092] The lifted two-dimensional ultrasound image block is expressed as:

[0093]

[0094] in, is a low-resolution image patch, is the corresponding high-resolution image block, and H represents the compressed sensing algorithm function;

[0095] The super-resolution processed two-dimensional image sequences are combined to form a three-dimensional volume data set, i.e., a low-latency super-resolution ultrasound three-dimensional image of the target puncture area.

[0096] It should be noted that by combining tissue recognition information and compressed sensing algorithms, the resolution of ultrasound images can be significantly improved while maintaining low latency, thereby more accurately displaying the tissue structure of the target puncture area. This is of great significance for decision support during medical diagnosis and treatment.

[0097] First, by obtaining tissue identification information of the target puncture area, different types of tissues can be distinguished. This helps doctors understand the tissue composition of the puncture area more accurately and develop more reasonable treatment plans. For example, in tumor puncture biopsy, by identifying the characteristics of tumor tissue, doctors can determine the best puncture path and angle to improve the accuracy and safety of biopsy.

[0098] Secondly, using compressed sensing algorithms to improve the spatial resolution of two-dimensional ultrasound images can improve image clarity without increasing scanning time. This helps doctors more accurately observe the details of the target puncture area, such as blood vessel distribution, tumor margins, etc. These detailed information is crucial for determining the extent of tissue lesions and formulating treatment plans.

[0099] Finally, by combining the super-resolution processed 2D image sequences into a 3D volume data set, a low-latency super-resolution ultrasound 3D image of the target puncture area can be obtained. This 3D image can more comprehensively display the tissue structure of the target puncture area, helping doctors to understand the lesion more intuitively and make more accurate diagnosis and treatment decisions.

[0100] In summary, by combining tissue recognition information and compressed sensing algorithms, low-latency, high-resolution ultrasound 3D images can be obtained to provide strong support for medical diagnosis and treatment processes. This can not only improve doctors’ work efficiency and accuracy, but also help improve patients’ treatment outcomes and quality of life.

[0101] S102: preprocessing the low-latency super-resolution ultrasonic three-dimensional image of the target puncture area;

[0102] The preprocessing of the low-latency super-resolution ultrasound three-dimensional image of the target puncture area includes sequentially performing noise removal, image enhancement, smoothing filtering, edge detection and normalization on the ultrasound three-dimensional image;

[0103] In the embodiment of the present application, noise removal is achieved by combining improved wavelet analysis with adaptive threshold denoising, and the specific operations are as follows:

[0104] First, multi-scale wavelet decomposition is performed on the low-latency super-resolution ultrasound 3D image to obtain the wavelet coefficients of each frequency level. Specifically, for the 3D image I (x, y, z), the 3D wavelet basis function is used Decompose to obtain high-frequency details and low-frequency approximate components:

[0105]

[0106] Then, for each wavelet coefficient W jkl , apply the adaptive threshold function T(W jkl ), which is based on local statistical properties such as noise standard deviation estimation, changes in neighborhood wavelet coefficients, etc.

[0107] Use a two-parameter patch-based thresholding function:

[0108] Threshold(W jkl )=η·σ jkl

[0109] Among them, η is the empirical threshold factor determined according to the image characteristics, σ jkl is the wavelet coefficient W jkl An estimate of the standard deviation of the local neighborhood.

[0110] Then, the threshold function is applied to perform soft and hard threshold processing on the wavelet coefficients, including:

[0111] When the absolute value of the wavelet coefficient |W jkl |Greater than the threshold T(W jkl ), the threshold is subtracted from the original coefficient to obtain the new processed wavelet coefficient

[0112]

[0113] This processing method can retain some of the original information of the wavelet coefficients, reduce noise while maintaining a certain degree of detail.

[0114] When the absolute value of the wavelet coefficient |W jkl |Less than or equal to the threshold T(W jkl ), the processed wavelet coefficients Set to zero:

[0115]

[0116] This means that the wavelet coefficients below the threshold are regarded as noise and are directly removed.

[0117] Finally, the three-dimensional image is reconstructed through the coefficients after threshold processing, and the denoised ultrasound image is restored:

[0118]

[0119] Among them, φ jkl (x, y, z) is the inverse transform function of the same wavelet basis function as in the decomposition.

[0120] It should be noted that the specific improvement lies in the use of an adaptive threshold selection strategy, taking into account the spatial correlation of wavelet coefficients, such as the idea of ​​non-local mean (NLM) to estimate the local standard deviation. In addition, the threshold calculation method is dynamically adjusted in combination with the image content and noise distribution characteristics to improve the denoising effect while maintaining image details.

[0121] It should be noted that this improved method has significant advantages in the field of image processing. First, by adopting an adaptive threshold selection strategy, this method can more accurately identify and process the noise components in the image, thereby achieving a more accurate denoising effect. Secondly, considering the spatial correlation of the wavelet coefficients, the idea of ​​non-local means (NLM) is used to estimate the local standard deviation, so that the algorithm can better preserve the edge and texture details of the image when processing the image, avoiding the blur and distortion problems that are easily caused by traditional denoising methods.

[0122] In addition, the threshold calculation method is dynamically adjusted based on the image content and noise distribution characteristics, which further improves the denoising effect while maintaining the clarity and details of the image. This method can not only effectively remove noise from the image, but also improve the visual effect of the image to a certain extent, making the processed image more realistic and natural.

[0123] In summary, this improved denoising method significantly improves the denoising effect while maintaining image details through strategies such as adaptive threshold selection, considering the spatial correlation of wavelet coefficients, and dynamically adjusting threshold calculation. This method has broad application prospects in the field of image processing and can provide more efficient and accurate solutions for tasks such as image denoising and image enhancement.

[0124] S103: A lesion area recognition model is established based on the preprocessed ultrasound three-dimensional image data in combination with a deep learning algorithm, and a puncture path is generated in combination with an improved path planning algorithm, and ultrasound navigation of the puncture robot is performed based on the puncture path.

[0125] Among them, based on the pre-processed ultrasound three-dimensional image data combined with the deep learning algorithm, the lesion area recognition model is established, including:

[0126] A 3D convolutional neural network architecture is used to establish a lesion area recognition model;

[0127] Furthermore, the preprocessed low-latency super-resolution ultrasound three-dimensional image is used as the input of the model, and the output of the model is the probability of lesions;

[0128] The lesion probability is expressed as:

[0129] p v =f(V;θ)

[0130] Among them, f represents the forward propagation function of the lesion area recognition model, θ is the parameter set of the lesion area recognition model, v represents the voxel, and for each voxel, v∈V, p v Represents the probability of disease.

[0131] Furthermore, based on the pre-processed ultrasound three-dimensional image data combined with the deep learning algorithm, the lesion area recognition model is established, which also includes:

[0132] Furthermore, the improved Focal Tversky Loss function is used to train the lesion area recognition model;

[0133] Furthermore, the traditional Focal TverskyLoss loss function is improved by combining adaptive weight adjustment and attention mechanism;

[0134] The improved Focal Tversky Loss loss function is expressed as:

[0135]

[0136] Among them, Cftl represents the improved Focal Tversky Loss loss function, Y is the real annotated lesion area, is the lesion area predicted by the model, represents the number of pixels of the intersection of the two, A(Y) represents the attention weight map based on the lesion area Y, and They represent the unpredicted part of the true annotation area and the part of the predicted area that does not belong to the true annotation, respectively. γ is the focal parameter used to emphasize difficult-to-classify samples. α(y) and β(y) represent adaptive weight coefficients, and the weights change with the sample position y.

[0137] Furthermore, the puncture path generated by combining the improved path planning algorithm includes:

[0138] Inputting the real-time ultrasound three-dimensional image data into the lesion area recognition model to obtain a probability map of the lesion area;

[0139] Furthermore, a first threshold τ is set. If the lesion probability of a voxel in the ultrasound three-dimensional image is greater than the threshold, the voxel is considered to be part of the lesion area, which is expressed as:

[0140] S={v∈V:p v >τ}

[0141] Among them, S is the voxel with a probability of all lesions higher than the threshold, p v represents the probability of lesion;

[0142] Furthermore, an optimal puncture path is planned from a safe starting point to the center of the lesion in three-dimensional space in combination with an improved path planning algorithm;

[0143] Improved path planning algorithms include:

[0144] Define the safe starting point as S_start and the center of the lesion as S_target;

[0145] Furthermore, the A* algorithm is improved by adding a constraint evaluation function as an improved path planning algorithm;

[0146] The constraint evaluation function is expressed as:

[0147] C(n)=w 1 ·A1C(n)+w 2 ·A2C(n)+w 3 LC(n)

[0148] Among them, A1C(n) is the cost function of avoiding key organs and blood vessels, A2C(n) is the cost function of needle tract safety angle, LC(n) is the cost function of needle tract length limitation, and w 1 ,w 2 ,w 3 is the corresponding weight parameter;

[0149] The heuristic function of the improved A* algorithm is:

[0150] f′(n)=f(n)+λ·C(n)

[0151] f(n)=g(n)+h(n)

[0152] Where λ is a tuning weight used to balance the path length and constraint compliance, g(n) represents the accumulated distance along the path from the safe starting point S_start to node n, and h(n) is the heuristic estimate from node n to the lesion center S_target.

[0153] It should be noted that based on the generated puncture path, the puncture robot is controlled to perform the puncture operation point by point. Under the guidance of real-time ultrasound imaging, the robot moves to the next point on the path each time and dynamically adjusts its position according to ultrasound feedback to ensure that the puncture needle moves accurately along the planned path until it reaches the lesion area.

[0154] It should also be noted that manual intervention can be performed to adjust the puncture path. During the puncture process, the doctor can monitor the ultrasound image in real time to observe the position of the puncture needle and the changes in the lesion area. If any abnormality is found or the puncture path needs to be adjusted, the doctor can intervene manually to ensure the safety and accuracy of the puncture process.

[0155] In an optional embodiment, in order to increase safety and flexibility during the puncture process, the design of the manual intervention logic is particularly important. The manual intervention steps can be designed as follows:

[0156] During the puncture process, the doctor should be able to monitor the working status of the puncture robot, the position of the puncture needle, and the real-time ultrasound image through the user interface. The system carried by the method should provide an intuitive interface to display the puncture path, lesion area, and the current position and movement direction of the puncture needle.

[0157] When the system carried by the method detects any abnormal situation, such as the puncture needle deviates from the planned path, encounters an unforeseen obstacle, or the lesion area changes, the system carried by the method should automatically prompt the doctor to perform manual intervention.

[0158] Doctors can also trigger the manual intervention mode at any time as needed to control the puncture process more accurately.

[0159] Doctors can directly adjust the puncture path through the user interface, and the system carried by the method should provide simple and easy-to-use tools to allow doctors to replan the puncture path in three-dimensional space.

[0160] The doctor can specify a new puncture point or adjust the puncture path through an input device (such as a mouse, a touch screen, or a dedicated medical device controller).

[0161] After the doctor adjusts the puncture path, the system carried by the method should automatically verify the safety and feasibility of the new path to ensure that the new path does not intersect with critical organs or blood vessels.

[0162] If the new path does not meet safety requirements, the system hosted by the method should prompt the doctor to make further adjustments or provide a recommended path.

[0163] When the doctor completes the manual intervention, the system carried by the method should be able to seamlessly switch to automatic control mode and continue to perform the puncture operation according to the new puncture path.

[0164] During the switching process, the system carried by the method should ensure that the position and posture of the puncture needle are consistent with the state at the end of manual intervention.

[0165] The system that carries the method should record all manual intervention operations, including intervention time, intervention content, and results after intervention.

[0166] These log data can be used for subsequent analysis to improve the system performance carried by the method and enhance the safety of the puncture process.

[0167] The system supporting the method should provide training and guidance functions to help doctors become familiar with the process and tools of manual intervention operations.

[0168] The system carried by the method can improve the doctor's operational proficiency and confidence by simulating the puncture process, providing operation guidelines and answers to common questions.

[0169] Through the design of the above manual intervention logic, the safety and accuracy of the puncture process can be guaranteed while increasing the flexibility and adaptability of the system carried by the method, so that doctors can finely control the puncture process when necessary.

[0170] In addition, the system carried by this puncture navigation method can also be combined with other advanced technologies, such as force feedback technology, to provide doctors with real-time puncture resistance feedback. In this way, doctors can more intuitively perceive the interaction between the puncture needle and the surrounding tissues, further improving the accuracy and safety of puncture.

[0171] In summary, the present invention proposes a puncture robot ultrasonic navigation method, which obtains tissue recognition information of the target puncture area according to the existing tissue ultrasonic characteristics, and obtains a low-latency super-resolution ultrasonic three-dimensional image of the target puncture area according to the tissue recognition information of the target puncture area; preprocesses the low-latency super-resolution ultrasonic three-dimensional image of the target puncture area; establishes a lesion area recognition model based on the preprocessed ultrasonic three-dimensional image data combined with a deep learning algorithm, and generates a puncture path in combination with an improved path planning algorithm, and performs puncture robot ultrasonic navigation according to the puncture path. The puncture robot ultrasonic navigation method and system of the present invention obtains tissue recognition information of the target puncture area and further obtains a low-latency super-resolution ultrasonic three-dimensional image, which effectively improves the accuracy of lesion area recognition and the accuracy of surgical navigation. By preprocessing the ultrasonic three-dimensional image, noise and interference in the image are eliminated, the image quality is improved, and a more reliable data basis is provided for subsequent lesion area recognition and path planning. At the same time, the lesion area recognition model is established in combination with a deep learning algorithm, which can adaptively learn and recognize the characteristics of different lesion areas, and improve the accuracy and generalization ability of lesion area recognition. In addition, by improving the path planning algorithm to generate the puncture path, factors such as the location, morphology and structure of the surrounding tissue of the lesion area are comprehensively considered, making the planned path more reasonable and safe. In summary, the puncture robot ultrasonic navigation method and system of the present invention can significantly improve the accuracy and safety of puncture surgery in practical applications, and have important clinical value and application prospects. The puncture robot ultrasonic navigation method and system of the present invention are an advanced medical technology with broad application prospects and important clinical value. Through continuous optimization and improvement, it is believed that the puncture robot ultrasonic navigation technology and system of the present invention will play a more important role in the future medical field and bring safer and more efficient medical services to doctors and patients.

[0172] In a preferred embodiment, a puncture robot ultrasound navigation system comprises:

[0173] An image acquisition module, used to acquire tissue identification information of a target puncture area according to existing tissue ultrasound characteristics, and to acquire a low-latency super-resolution ultrasound three-dimensional image of the target puncture area according to the tissue identification information of the target puncture area;

[0174] A preprocessing module, used for preprocessing a low-latency super-resolution ultrasonic three-dimensional image of a target puncture area;

[0175] The navigation module is used to establish a lesion area recognition model based on the preprocessed ultrasonic three-dimensional image data in combination with a deep learning algorithm, and to generate a puncture path in combination with an improved path planning algorithm, and to perform ultrasonic navigation of the puncture robot based on the puncture path.

[0176] The above-mentioned unit modules may be embedded in or independent of a processor in a computer device in the form of hardware, or may be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to the above-mentioned modules.

[0177] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 2 As shown. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a puncture robot ultrasonic navigation method is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0178] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0179] According to the existing tissue ultrasound characteristics, tissue identification information of the target puncture area is obtained, and a low-latency super-resolution ultrasound three-dimensional image of the target puncture area is obtained according to the tissue identification information of the target puncture area;

[0180] Preprocessing the low-latency super-resolution ultrasound three-dimensional image of the target puncture area;

[0181] Based on the preprocessed ultrasonic three-dimensional image data combined with the deep learning algorithm, a lesion area recognition model is established, and the puncture path is generated in combination with the improved path planning algorithm, and the puncture robot ultrasonic navigation is performed according to the puncture path.

[0182] Example 2

[0183] Reference Figure 1-2 , which is an embodiment of the present invention, provides a puncture robot ultrasonic navigation method and system. In order to verify the beneficial effects of the present invention, scientific demonstration is carried out through comparative experiments.

[0184] Experimental design:

[0185] In order to verify the effectiveness of the puncture robot ultrasonic navigation method and system of the present invention, a set of comparative experiments was designed. The experiment was divided into two groups, a control group and an experimental group. The control group adopted the traditional puncture navigation method, while the experimental group adopted the puncture robot ultrasonic navigation method proposed by the present invention.

[0186] Subjects:

[0187] The experimental subjects were 50 volunteers who needed puncture surgery. The age, gender, weight and other indicators of these volunteers were evenly distributed and had a certain degree of representativeness.

[0188] Experimental process:

[0189] Before the experiment began, all volunteers underwent a detailed physical examination and assessment to determine whether they were suitable for participation in the experiment. Then, the volunteers were randomly divided into a control group and an experimental group, with 25 people in each group.

[0190] The control group used the traditional puncture navigation method for surgery. The experimental group used the puncture robot ultrasound navigation method proposed by the present invention. During the operation, the operation time, puncture accuracy, complication rate and other indicators of each volunteer were recorded.

[0191] Table 1: Paracentesis data of 50 volunteers who underwent paracentesis.

[0192]

[0193] Among them, N / A means that there is no parameter of this item. Through comparative experiments, the effectiveness of the puncture robot ultrasound navigation method and system proposed in the present invention is verified. This method not only improves the accuracy and safety of puncture surgery, but also shortens the operation time and reduces the incidence of complications. Therefore, the present invention has important clinical value and application prospects.

[0194] The above experiment is only one embodiment of the present invention. In order to more comprehensively verify the beneficial effects of the present invention, more experiments and research will be conducted in the future. At the same time, the puncture robot ultrasound navigation technology and system of the present invention will be continuously optimized and improved to bring safer and more efficient medical services to doctors and patients.

[0195] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

[0196] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code. The scheme in the embodiments of the present application can be implemented in various computer languages, for example, object-oriented programming language Java and literal scripting language JavaScript, etc.

[0197] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0198] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0199] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0200] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0201] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A puncture robot ultrasound navigation method, characterized in that: include, According to the existing tissue ultrasound characteristics, tissue identification information of the target puncture area is obtained, and according to the tissue identification information of the target puncture area, a low-latency super-resolution ultrasound three-dimensional image of the target puncture area is obtained; Preprocessing the low-latency super-resolution ultrasonic three-dimensional image of the target puncture area; Based on the preprocessed ultrasonic three-dimensional image data combined with the deep learning algorithm, a lesion area recognition model is established, and a puncture path is generated in combination with an improved path planning algorithm, and the puncture robot ultrasonic navigation is performed according to the puncture path; The method of obtaining tissue identification information of the target puncture area according to the existing tissue ultrasonic characteristics includes: Sending a pulse signal to the target puncture area through an ultrasonic probe array and acquiring an original ultrasonic signal data set, wherein the original ultrasonic signal data set at least includes the echo intensity reflected by the tissue at different depths; The original ultrasound signal data set is filtered and gain controlled to obtain a two-dimensional ultrasound image sequence I t , where t represents the frame number; Extract the tissue feature vector in each frame image and record it as represents the tissue characteristics at position j, the tissue characteristics at least including echo intensity, texture characteristics, and velocity spectrum, where N is the characteristic dimension coefficient; Use machine learning to build a feature database D of existing tissue types, containing a set of feature vectors for each tissue type Calculate the Euclidean distance between the tissue feature vector of each voxel in the current target puncture area and each tissue type in the database: An objective function is established to determine the tissue type of the target puncture area. The objective function is expressed as follows: Among them, k represents the kth tissue type in the database, and M represents the types of existing tissue types; The generation of the puncture path by combining the improved path planning algorithm comprises: Inputting the real-time ultrasonic three-dimensional image data into the lesion area recognition model to obtain a probability map of the lesion area; The first threshold τ is set. If the lesion probability of a voxel in the ultrasound three-dimensional image is greater than the threshold, the voxel is considered to be part of the lesion area, which is expressed as: S={v∈V:p v >t} Among them, S is the voxel with a probability of all lesions higher than the threshold, p v represents the probability of lesion; Combined with the improved path planning algorithm, an optimal puncture path is planned from the safe starting point to the center of the lesion in three-dimensional space; The improved path planning algorithm includes: Define the safe starting point as S_start and the center of the lesion as S_target; Improve the A* algorithm by adding a constraint evaluation function as an improved path planning algorithm; The constraint evaluation function is expressed as: C(n)=W1·A1C(n)+W2·A2C(n)+W3·LC(n) Among them, A1C(n) is the cost function of avoiding key organs and blood vessels, A2C(n) is the cost function of needle track safety angle, LC(n) is the cost function of needle track length limitation, and W1, W2, and W3 are corresponding weight parameters; The heuristic function of the improved A* algorithm is: f′(n)=f(n)+λ·C(n) f(n)=g(n)+h(n) Where λ is a tuning weight used to balance the path length and constraint compliance, g(n) represents the accumulated distance along the path from the safe starting point S_start to node n, and h(n) is the heuristic estimate from node n to the lesion center S_target.

2. The puncture robot ultrasound navigation method according to claim 1, characterized in that: The step of acquiring a low-latency super-resolution ultrasonic three-dimensional image of the target puncture area according to the tissue identification information of the target puncture area comprises: After determining the tissue type of the target puncture area, the compressed sensing algorithm is used to improve the spatial resolution of the two-dimensional ultrasound image; The lifted two-dimensional ultrasound image block is expressed as: in, is a low-resolution image patch, is the corresponding high-resolution image block, and H represents the compressed sensing algorithm function; The super-resolution processed two-dimensional image sequences are combined to form a three-dimensional volume data set, i.e., a low-latency super-resolution ultrasound three-dimensional image of the target puncture area.

3. The puncture robot ultrasound navigation method according to claim 2, characterized in that: The preprocessing of the low-latency super-resolution ultrasonic three-dimensional image of the target puncture area includes sequentially performing noise removal, image enhancement, smoothing filtering, edge detection and normalization processing on the ultrasonic three-dimensional image; The noise removal is achieved by combining improved wavelet analysis with adaptive threshold denoising.

4. The puncture robot ultrasound navigation method according to claim 3, characterized in that: The method of establishing a lesion area recognition model based on the pre-processed ultrasonic three-dimensional image data in combination with a deep learning algorithm includes: A 3D convolutional neural network architecture is used to establish a lesion area recognition model; The preprocessed low-latency super-resolution ultrasound three-dimensional image is used as the input of the model, and the output of the model is the lesion probability; The lesion probability is expressed as: p v =f(V;θ) Among them, f represents the forward propagation function of the lesion area recognition model, θ is the parameter set of the lesion area recognition model, v represents the voxel, and for each voxel, v∈V, p v Represents the probability of disease.

5. The puncture robot ultrasound navigation method according to claim 4, characterized in that: The step of establishing a lesion area recognition model based on the pre-processed ultrasonic three-dimensional image data in combination with a deep learning algorithm also includes: Using an improved FocalTverskyLoss loss function to train the lesion area recognition model; Improve the traditional FocalTverskyLoss loss function by combining adaptive weight adjustment and attention mechanism; The improved FocalTverskyLoss loss function is expressed as: Among them, Cftl represents the improved FocalTverskyLoss loss function, Y is the real annotated lesion area, is the lesion area predicted by the model, represents the number of pixels of the intersection of the two, A(Y) represents the attention weight map based on the lesion area Y, and They represent the unpredicted part of the true annotation area and the part of the predicted area that does not belong to the true annotation, respectively. γ is the focal parameter used to emphasize difficult-to-classify samples. α(y) and β(y) represent adaptive weight coefficients, and the weights change with the sample position y.

6. A system using the puncture robot ultrasonic navigation method according to claim 1, characterized in that: include, An image acquisition module, used to acquire tissue identification information of a target puncture area according to existing tissue ultrasound characteristics, and to acquire a low-latency super-resolution ultrasound three-dimensional image of the target puncture area according to the tissue identification information of the target puncture area; A preprocessing module, used for preprocessing the low-latency super-resolution ultrasonic three-dimensional image of the target puncture area; The navigation module is used to establish a lesion area recognition model based on the preprocessed ultrasonic three-dimensional image data in combination with a deep learning algorithm, and to generate a puncture path in combination with an improved path planning algorithm, and to perform ultrasonic navigation of the puncture robot according to the puncture path.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images