Optical excitation guided planning method and system based on photoacoustic imaging and storage medium

Abstract

The present application relates to the technical field of medical imaging processing, in particular to a light excitation guide planning method and system based on photoacoustic imaging and a storage medium, comprising the following steps: planning a first path from an entrance end of a urethral region to a target end of a prostate region; acquiring photoacoustic signal data of a flexible optical fiber during the travel of the flexible optical fiber according to the first path in real time; determining position information of a front end of the flexible optical fiber in the photoacoustic signal data, and reconstructing a photoacoustic image in the photoacoustic signal data; registering the first path from the medical image to the photoacoustic image, and determining guide and correction information of the travel process in the registered photoacoustic image by using the first path and the position information of the front end of the flexible optical fiber. The present application pre-plans a guide path for introducing the flexible optical fiber into the prostate region through the urethra, and accurately positions the front end of the flexible optical fiber by using the guide path combined with ultrasonic guidance, so as to accurately guide the flexible optical fiber during the travel process.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical imaging processing, in particular to a light excitation guidance planning method and system based on photoacoustic imaging and a storage medium. BACKGROUND

[0002] Current medical lesion identification relies heavily on imaging methods, that is, medical images such as magnetic resonance images, ultrasound images, and photoacoustic images are used to identify lesions. Taking the identification of lesions in the prostate region as an example, magnetic resonance imaging, ultrasound imaging, and photoacoustic imaging of the prostate region can be used to identify lesions in the prostate region alone or in combination with multi-modal identification of lesions in the prostate region. Whether the imaging is used alone or in combination, high-quality imaging results are required. Among them, photoacoustic imaging of the prostate region requires a light excitation module (such as a flexible optical fiber) to irradiate the prostate region from the transurethral region to obtain high-quality photoacoustic imaging.

[0003] Therefore, the guidance of the light excitation module in photoacoustic imaging is very important.

[0004] Currently, it is very difficult to guide the light excitation module without a field of view. SUMMARY

[0005] The present application aims to provide a light excitation guidance planning method based on photoacoustic imaging to solve the technical problem of the difficulty of guiding the light excitation module without a field of view in the prior art.

[0006] To solve the above technical problems, the present application specifically provides the following technical solutions:

[0007] A light excitation guidance planning method based on photoacoustic imaging, comprising the following steps:

[0008] Image segmentation is performed on the medical image of the prostate tissue to distinguish the prostate region, the urethral region, and the surrounding tissue region in the image;

[0009] According to the medical image, the surrounding tissue region is taken as an obstacle point, and a first path from the inlet end of the urethral region to the target end of the prostate region is planned;

[0010] The photoacoustic signal data of the flexible optical fiber during the travel according to the first path is obtained in real time;

[0011] The position information of the front end of the flexible optical fiber is determined in the photoacoustic signal data, and a photoacoustic image is reconstructed in the photoacoustic signal data;

[0012] The first path is registered into the photoacoustic image from the medical image, and the guidance and correction information of the travel process is determined by using the first path and the position information of the front end of the flexible optical fiber in the registered photoacoustic image.

[0013] As a preferred scheme of the present application, the method for image segmentation of the medical image of the prostate tissue comprises:

[0014] The prostate region, the urethral region and the surrounding tissue region are segmented in the medical image by using three U-net networks, wherein the three U-net networks are respectively a prostate segmentation network, a urethral segmentation network and a surrounding tissue segmentation network;

[0015] The three U-net networks are cross-optimized and trained by using the output results of the three U-net networks, wherein:

[0016] The prostate region in the medical image is processed as a background region according to the prostate region obtained by the prostate segmentation network, to form a prostate region mask image;

[0017] The urethral region in the medical image is processed as a foreground region according to the urethral region obtained by the urethral segmentation network, to form a urethral region mask image;

[0018] The surrounding tissue region in the medical image is processed as a foreground region according to the surrounding tissue region obtained by the surrounding tissue segmentation network, to form a surrounding region mask image;

[0019] The prostate region is segmented in the urethral region mask image and the surrounding region mask image by using the prostate segmentation network, and a prostate segmentation consistency constraint is established between the medical image, the urethral region mask image and the surrounding region mask image, and the prostate segmentation network is trained based on the prostate segmentation consistency constraint to form an optimal prostate segmentation network;

[0020] The prostate segmentation consistency constraint is: ;

[0021] The optimal prostate segmentation network is: ;

[0022] In the formula, is the segmented prostate region in the medical image, is the segmented prostate region in the urethral region mask image, is the segmented prostate region in the surrounding region mask image, and G is the medical image and UNet is the U-Net network.

[0023] ​The urethra segmentation network is used to segment the urethra region in the prostate region mask image and the surrounding region mask image respectively, and a urethra segmentation consistency constraint is established between the medical image, the prostate region mask image and the surrounding region mask image, and the urethra segmentation network is trained based on the urethra segmentation consistency constraint to form an optimal urethra segmentation network;

[0024] The urethra segmentation consistency constraint is For: ;

[0025] The optimal urethra segmentation network is: ;

[0026] In the formula, The urethra region segmented in the medical image is urethra region segmented in the prostate region mask image, The urethra region segmented in the prostate region mask image is urethra region segmented in the surrounding region mask image; The urethra region segmented in the prostate region mask image is urethra region segmented in the surrounding region mask image;

[0027] The surrounding tissue segmentation network is used to segment the surrounding tissue region in the prostate region mask image and the urethra region mask image respectively, and a surrounding tissue segmentation consistency constraint is established between the medical image, the prostate region mask image and the urethra region mask image, and the surrounding tissue segmentation network is trained based on the surrounding tissue segmentation consistency constraint to form an optimal surrounding tissue segmentation network;

[0028] The surrounding tissue segmentation consistency constraint is For: ;

[0029] The optimal surrounding tissue segmentation network is: ;

[0030] In the formula, The surrounding tissue region segmented in the medical image is urethra region segmented in the prostate region mask image, The surrounding tissue region segmented in the prostate region mask image is urethra region segmented in the surrounding region mask image, The surrounding tissue region segmented in the prostate region mask image is urethra region segmented in the surrounding region mask image, The L2 norm is.

[0031] As a preferred scheme of the present application, the planning method of the first path comprises:

[0032] The multiple optimization objectives for planning the first path comprise:

[0033] The distance maximization objective between the flexible optical fiber front end and the surrounding tissue region , In the formula, The coordinate value of the flexible optical fiber front end at the i-th path point in the first path is is the coordinate value of the point with the shortest distance to the urethral region in the jth surrounding tissue region, n is all path points in the first path, k is the total number of surrounding tissue regions, is the Euclidean distance operation formula, and max is a maximization identifier;

[0034] Minimization of distance between flexible optical fiber front end and prostate region , , wherein, is the coordinate value of the flexible optical fiber front end at the nth path point in the first path, is the coordinate value of the point with the shortest distance to the urethral region in the prostate region, and min is a minimization identifier;

[0035] Minimization of flexible optical fiber front end travel distance , , wherein, is the coordinate value of the flexible optical fiber front end at the ith path point in the first path, is the coordinate value of the flexible optical fiber front end at the ith+1 path point in the first path;

[0036] Smoothness of flexible optical fiber front end travel , , wherein, is the coordinate value of the flexible optical fiber front end at the ith path point in the first path, is the coordinate value of the flexible optical fiber front end at the ith+1 path point in the first path, is the coordinate value of the flexible optical fiber front end at the ith-1 path point in the first path;

[0037] The urethral region is taken as a solution space, and multi-objective optimization solving algorithm is used to optimize and solve , , and to obtain the first path, and the first path is subjected to smoothing processing.

[0038] As a preferred scheme of the present application, the method for determining the position information of the flexible optical fiber in the photoacoustic signal data comprises:

[0039] The flexible optical fiber front end is subjected to travel simulation training in a human body model, and photoacoustic signals of the flexible optical fiber front end at known points are collected by using an intrarectal ultrasonic probe array to obtain photoacoustic signals with positioning information;

[0040] A fitting relationship between the photoacoustic signals and the positioning information is established by using a convolutional neural network to obtain a positioning model: , wherein, P is the position coordinate of the flexible optical fiber front end predicted by the positioning model, signal is the photoacoustic signal, and CNN is the convolutional neural network;

[0041] Loss function of the positioning model is: , wherein, is the true value of the position coordinate of the front end of the flexible optical fiber, is the position coordinate of the front end of the flexible optical fiber decomposed from the photoacoustic signal by using a mathematical method, is the mean square error, is a hyperparameter;

[0042] The method for decomposing the position coordinate from the photoacoustic signal by using a mathematical method comprises:

[0043] The urethral region, the prostate region, and the surrounding tissue region are taken as a search space grid;

[0044] In each grid point , the theoretical propagation time of the sound wave from to the lth array element in the transrectal ultrasound probe array is calculated , wherein, is the sound speed, is the position coordinate of the lth array element;

[0045] The signal received by the lth array element is time-delay compensated to obtain ;

[0046] All the compensated array element signals are superimposed to obtain , wherein m is the total number of array elements;

[0047] The energy of the superimposed signal is calculated to obtain , and the grid point corresponding to the maximum value of is taken as the position coordinate of the front end of the flexible optical fiber.

[0048] As a preferred scheme of the present application, the method for reconstructing a photoacoustic image from photoacoustic signal data comprises:

[0049] The photoacoustic signal data is subjected to laser intensity compensation, filtering, and peak envelope detection through Hilbert transform conversion;

[0050] The photoacoustic image is reconstructed from the photoacoustic signal data by using a beamforming or delay-and-sum method array ultrasound imaging algorithm.

[0051] As a preferred scheme of the present application, the method for registering the first path from the medical image to the photoacoustic image comprises:

[0052] The medical image is taken as a reference image, and the photoacoustic image is taken as a floating image; the medical image and the photoacoustic image are registered and fused to obtain a photoacoustic image containing the first path after registration.

[0053] As a preferred scheme of the present application, the method for determining the guidance and correction information of the advancing process in the registered photoacoustic image by using the position information of the first path and the flexible optical fiber front end comprises:

[0054] When the position coordinates of the flexible optical fiber front end are on the first path, the next path point is taken as the guidance information for the flexible optical fiber front end to continue advancing.

[0055] When the position coordinates of the flexible optical fiber front end deviate from the first path, the path point with the closest distance between the first path and the flexible optical fiber front end is taken as the correction information for the flexible optical fiber front end to continue advancing.

[0056] As a preferred scheme of the present application, the flexible optical fiber front end is loaded with a columnar diffuse light source.

[0057] As a preferred scheme of the present application, the present application provides a photoacoustic imaging-based light excitation guidance planning system applied to a photoacoustic imaging-based light excitation guidance planning method, and the system comprises:

[0058] The first data processing unit performs image segmentation on the medical image of the prostate tissue to distinguish the prostate region, the urethral region and the surrounding tissue region in the image.

[0059] The path planning unit plans a first path from the inlet end of the urethral region to the target end of the prostate region according to the medical image, taking the surrounding tissue region as the obstacle point.

[0060] The photoacoustic imaging unit is used for acquiring photoacoustic signal data of the flexible optical fiber in the advancing process according to the first path in real time.

[0061] The second data processing unit is used for determining the position information of the flexible optical fiber front end in the photoacoustic signal data and reconstructing a photoacoustic image in the photoacoustic signal data.

[0062] The image registration unit is used for registering the first path from the medical image to the photoacoustic image and determining the guidance and correction information of the advancing process in the registered photoacoustic image by using the position information of the first path and the flexible optical fiber front end.

[0063] As a preferred scheme of the present application, the present application provides a computer readable storage medium, and the computer readable storage medium stores computer execution instructions.

[0064] Compared with the prior art, the present application has the following beneficial effects:

[0065] The present application pre-plans a guiding path for a flexible optical fiber used in photoacoustic imaging to be introduced into a prostate site through a urethra, uses the guiding path to guide the travel process of the flexible optical fiber, improves safety, and uses ultrasound guidance technology to accurately position the front end of the flexible optical fiber and master the relative position between the front end of the flexible optical fiber and the guiding path, so as to provide accurate guiding information for the travel process of the flexible optical fiber by using the planned path. BRIEF DESCRIPTION OF DRAWINGS

[0066] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the following embodiment or prior art description will be briefly introduced. Obviously, the drawings in the following description are only exemplary, and other drawings can be obtained by the provided drawings without creative labor for those skilled in the art.

[0067] Figure 1 The light excitation module guiding path planning method flow chart provided for the embodiment of the present application;

[0068] Figure 2 The light excitation module guiding path planning system diagram provided for the embodiment of the present application;

[0069] Figure 3 The medical image segmentation network structure block diagram provided for the embodiment of the present application;

[0070] Figure 4 The flexible optical fiber schematic diagram provided for the embodiment of the present application. DETAILED DESCRIPTION

[0071] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the protection scope of the present application.

[0072] As Figure 1 shown, the present application provides a light excitation guiding planning method based on photoacoustic imaging, including the following steps:

[0073] Image segmentation is performed on the medical image of the prostate tissue to distinguish the prostate region, urethra region and surrounding tissue region in the image;

[0074] According to the medical image, a first path from the urethra region to the prostate region is planned with the surrounding tissue region as an obstacle point, which is essentially a planned path for guiding the travel of the flexible optical fiber;

[0075] Real-time acquisition of photoacoustic signal data of the flexible optical fiber during the running of the first path, the photoacoustic signal data is usually acquired by using a transrectal ultrasound probe array;

[0076] Determination of the position information of the front end of the flexible optical fiber in the photoacoustic signal data, and reconstruction of a photoacoustic image in the photoacoustic signal data;

[0077] Registration of the first path to the photoacoustic image according to the medical image, and determination of the guiding and correction information of the running in the registered photoacoustic image according to the first path and the position information of the front end of the flexible optical fiber.

[0078] In the embodiment, the following data is mainly acquired:

[0079] The first data is the medical image data of the prostate tissue, which is subjected to object analysis to obtain the condition of the target tissue, and a path is planned according to the condition.

[0080] The second data is the running data of the flexible optical fiber. In essence, it is the analysis and processing of multiple image data, and the deviation between the flexible optical fiber and the planned path during the running is grasped, so as to design a guiding method with correction function for the optical fiber. The method can take the constructed human model or the model of the entire prostate part as the object, or other tissue models (also taking the surrounding tissue as the obstacle point, planning the path from the inlet end to the target end).

[0081] In addition, the method can also be applied to the planning of mechanical operation paths in other cases without vision.

[0082] In the present application, when the photoacoustic imaging is introduced into the light excitation module (flexible optical fiber), first, the individual structure of the operation object is acquired, and then the optimal guiding path of introducing the flexible optical fiber into the prostate area through the urethra is formulated according to the individual structure of the operation object.

[0083] After the optimal guiding path suitable for the individual structure is formulated in the present application, the running of the flexible optical fiber during the introduction process is guided, and photoacoustic signals are acquired during the running. The photoacoustic signals contain light source positioning information, that is, the position information of the front end of the flexible optical fiber. The position information of the front end of the flexible optical fiber is determined from the photoacoustic signals, so as to judge whether the flexible optical fiber has deviated from the planned first path, and to provide running correction information for the deviated flexible optical fiber according to the deviation shown in the image.

[0084] The present application first needs to master the individual structure characteristics when planning before the flexible optical fiber guide, therefore, the image acquisition is carried out to the individual operation object first, and the structure, position information such as prostate region, urethral region and surrounding tissue region (such as ejaculatory duct, bladder and other tissues) are mastered. Therefore, the present application carries out the tissue region segmentation in the image before the flexible optical fiber guide, as follows:

[0085] The method for segmenting the prostate region, urethral region and surrounding tissue region in the medical image (such as MRI) includes:

[0086] As shown in Figure 3 , the prostate region, urethral region and surrounding tissue region are segmented in the medical image by using three U-net networks, wherein the three Unet networks are prostate segmentation network, urethral segmentation network and surrounding tissue segmentation network respectively;

[0087] The output results of the three U-net networks are used for cross-optimization training of the three U-net networks, wherein:

[0088] The prostate region obtained by the prostate segmentation network is processed as a background region to form a prostate region mask image;

[0089] The urethral region obtained by the urethral segmentation network is processed as a foreground region to form a urethral region mask image;

[0090] The surrounding tissue region obtained by the surrounding tissue segmentation network is processed as a foreground region to form a surrounding region mask image;

[0091] The prostate segmentation network is used to segment the prostate region in the urethral region mask image and the surrounding region mask image respectively, and the prostate segmentation consistency constraint is established between the medical image, the urethral region mask image and the surrounding region mask image, and the prostate segmentation network is trained based on the prostate segmentation consistency constraint to form an optimal prostate segmentation network;

[0092] The prostate segmentation consistency constraint is: ;

[0093] The optimal prostate segmentation network is:

[0094] In the formula, is the segmented prostate region in the medical image, is the segmented prostate region in the urethral region mask image, is the segmented prostate region in the surrounding region mask image, and G is the medical image, and UNet is the U-Net network.​​

[0095] The application first establishes three segmentation networks directly applied to the medical image original graph, corresponding to obtain the prostate region segmentation result, the urethra region segmentation result and the surrounding tissue region segmentation result, which has high segmentation efficiency directly on the original graph, but has limited segmentation accuracy due to the interference between regions of the original graph. Therefore, the application optimizes the segmentation accuracy performance of the three segmentation networks directly applied to the medical image original graph.

[0096] For example, when optimizing the prostate segmentation network, the segmentation results of the other two segmentation networks are used to form two mask images on the medical image original graph, which eliminates the interference of the other two regions on the prostate region, and the two mask images are used for prostate region segmentation to improve the segmentation effect. Through prostate segmentation consistency constraint , the segmentation result of the original prostate segmentation network is forced to approach the segmentation result of the two mask images, that is the expected image original graph segmentation result approaches the urethra region mask image segmentation result, the expected original graph segmentation result approaches the surrounding tissue region mask image segmentation result, so that the initial prostate segmentation network can obtain the prostate segmentation result close to the mask image on the image original graph, improve the segmentation accuracy, and at the same time retain the original efficiency performance.

[0097] The urethra segmentation network is used to segment the urethra region in the prostate region mask image and the surrounding region mask image, and a urethra segmentation consistency constraint is established between the medical image, the prostate region mask image and the surrounding region mask image. The urethra segmentation network is trained based on the urethra segmentation consistency constraint to form an optimal urethra segmentation network.

[0098] The urethra segmentation consistency constraint is . ;

[0099] The optimal urethra segmentation network is .

[0100] In the formula, is the segmented urethra region in the medical image, is the segmented urethra region in the prostate region mask image, is the segmented urethra region in the surrounding region mask image.

[0101] Through the urethra segmentation consistency constraint , the segmentation result of the original urethra segmentation network is forced to approach the segmentation result of the two mask images, that is the expected image original graph segmentation result approaches the prostate region mask image segmentation result, The goal is to make the segmentation result of the original image closer to the segmentation result of the masked image of the surrounding tissue region, so that the initial urethral segmentation network can obtain a urethral region segmentation result on the original image that is close to that on the masked image, thereby improving the segmentation accuracy while retaining the original efficiency performance.

[0102] The surrounding tissue segmentation network was used to segment the surrounding tissue region in the prostate region masked image and the urethra region masked image, respectively. A consistency constraint on the surrounding tissue segmentation was established between the medical image, the prostate region masked image and the urethra region masked image. The surrounding tissue segmentation network was trained based on the consistency constraint to form the optimal surrounding tissue segmentation network.

[0103] Surrounding tissue segmentation consistency constraints for: ;

[0104] The optimal surrounding tissue segmentation network is: ;

[0105] In the formula, The surrounding tissue region segmented in medical imaging. The surrounding tissue region is re-segmented from the masked image of the prostate region. To segment the surrounding tissue region in an image masked for the urethral region. It is an L2 norm.

[0106] Consistency constraint based on surrounding tissue segmentation This forces the segmentation results of the original surrounding tissue segmentation network to be closer to the segmentation results of the two masked images, i.e. The goal is to make the segmentation results of the original image closer to those of the masked image of the prostate region. The goal is to make the segmentation results of the original image closer to those of the masked image of the urethral region, so that the initial surrounding tissue segmentation network can obtain segmentation results of the surrounding tissue region on the original image that are close to those on the masked image, thereby improving segmentation accuracy while retaining the original efficiency performance.

[0107] After obtaining the individual organizational structure, this invention uses a multi-objective optimization approach to plan the first path, as follows:

[0108] The planning methods for the first path include:

[0109] The multiple optimization objectives for determining the first path include:

[0110] Maximize the distance between the flexible fiber front end and the surrounding tissue area , In the formula, Xi is the coordinate value of the flexible optical fiber front end at the i-th path point in the first path, Xj is the coordinate value of the point in the j-th surrounding tissue region with the shortest distance to the urethral region, n is the total number of path points in the first path, and k is the total number of surrounding tissue regions, D is the Euclidean distance operation formula, and max is the maximization identifier;

[0111] Objective The safety optimization objective is characterized, so that the flexible optical fiber is as far away from the surrounding tissue region as possible during the travel, thereby ensuring safety.

[0112] Flexible optical fiber front end and prostate region distance minimization objective , , wherein, Xi is the coordinate value of the flexible optical fiber front end at the n-th path point in the first path, Xj is the coordinate value of the point in the prostate region with the shortest distance to the urethral region, and min is the minimization identifier;

[0113] Objective The objective achievement optimization objective is characterized, so that the flexible optical fiber is as close to the prostate region as possible during the travel, thereby ensuring that the targeted light irradiation on the prostate region is completed.

[0114] Flexible optical fiber front end travel distance minimization objective , , wherein, Xi is the coordinate value of the flexible optical fiber front end at the i-th path point in the first path, Xi+1 is the coordinate value of the flexible optical fiber front end at the i+1-th path point in the first path;

[0115] Objective The distance optimization objective is characterized, so that the flexible optical fiber reaches the prostate region as quickly as possible during the travel, thereby ensuring import efficiency.

[0116] Flexible optical fiber front end travel smoothness objective , , wherein, Xi is the coordinate value of the flexible optical fiber front end at the i-th path point in the first path, Xi+1 is the coordinate value of the flexible optical fiber front end at the i+1-th path point in the first path, Xi-1 is the coordinate value of the flexible optical fiber front end at the i-1-th path point in the first path;

[0117] Objective The path smoothness optimization objective is characterized, so that the flexible optical fiber reaches the prostate region as smoothly as possible during the travel, thereby ensuring path continuity and safety.

[0118] Using the urethral region as the solution space, a multi-objective optimization algorithm is employed to solve the problem. , , and The first path is obtained by optimization and then smoothed.

[0119] This invention is achieved through , , and Planning the first path can yield a safe, smooth, and efficient path.

[0120] In guiding the flexible optical fiber using a planned path, this invention requires real-time monitoring of the fiber's position. To avoid adverse effects from the positioning sensor during photoacoustic imaging of the prostate region—such as: metal components reflecting ultrasound waves, generating bright echoes and acoustic shadows; sensor encapsulation material mismatch with tissue acoustic impedance causing acoustic wave scattering and resulting in acoustic artifacts; sensor light-shielding metal components blocking the excitation light path, leading to a decrease in local light energy density; and stray reflections from the sensor surface reflecting laser light to non-target areas, generating unexpected photoacoustic signals and thus causing photoacoustic interference—this invention abandons the sensor positioning method and adopts an end-to-end position prediction model using a neural network to directly predict the position coordinates of the fiber tip based on the photoacoustic signal, as detailed below:

[0121] Methods for determining the location information of flexible optical fibers from photoacoustic signal data include:

[0122] Walking simulation training was conducted on a human body model at the front end of a flexible optical fiber. At the same time, the photoacoustic signal of the flexible optical fiber front end at a known point was collected by a transrectal ultrasound probe array to obtain a photoacoustic signal with positioning information.

[0123] A localization model is obtained by using a convolutional neural network to establish a fitting relationship between photoacoustic signals and localization information. In the formula, P is the position coordinate of the flexible optical fiber front end predicted by the positioning model, signal is the photoacoustic signal, and CNN is the convolutional neural network.

[0124] Loss function of localization model for: In the formula, This represents the true coordinates of the flexible optical fiber's front end. The coordinates of the flexible optical fiber front end are obtained by mathematical decomposition of the photoacoustic signal. Mean square error, For hyperparameters;

[0125] This invention utilizes simulation training to accumulate model training data, and then correlates and maps photoacoustic signals and positioning information, thereby enabling the prediction of positioning information from an input photoacoustic signal. In the training of the positioning model, a constraint term is added to the loss function. Even when the fitting relationship between the photoacoustic signal and the positioning information fails, the model can output a light source position that is mathematically solved from the photoacoustic signal. The light source position is located at the front end of the optical fiber, which is equivalent to directly decomposing the optical fiber front end position from the photoacoustic signal. In other words, the positioning model will not become invalid even when the fitting relationship between the photoacoustic signal and the positioning information fails. The two output results of the model complement each other and improve the robustness of the positioning model.

[0126] Among them, the methods for decomposing position coordinates from photoacoustic signals using mathematical methods include:

[0127] The urethral region, prostate region, and surrounding tissue region are used as the search space grid.

[0128] At each grid point In the process of calculating sound waves from Theoretical propagation time to the l-th element in the transrectal ultrasound probe array ,in, For the speed of sound, The coordinates of the l-th element are given.

[0129] The signal received by the l-th element Time delay compensation is obtained ;

[0130] The compensated signals of all array elements are superimposed to obtain In the formula, m is the total number of array elements;

[0131] Calculate superimposed signals Energy obtained and will The grid point corresponding to the maximum value is used as the coordinate of the flexible fiber front end.

[0132] Methods for reconstructing photoacoustic images from photoacoustic signal data include:

[0133] The photoacoustic signal data is subjected to laser intensity compensation, filtering, and peak detection through Hilbert transform.

[0134] Photoacoustic images are reconstructed from photoacoustic signal data using beamforming or delay summation array ultrasound imaging algorithms.

[0135] Beamforming is a technology that enhances signals in a specific direction and suppresses interference from other directions by combining multiple sensor signals, widely used in radar, sonar, wireless communication and other fields. Delay and sum is the most basic implementation method, which adjusts the time delay of the signal to align and superimpose the signal at a specific angle, thereby enhancing the energy of the target direction.

[0136] The method for registering the first path from the medical image to the photoacoustic image comprises:

[0137] Taking the medical image as a reference image and the photoacoustic image as a floating image, the medical image and the photoacoustic image are registered and fused to obtain a photoacoustic image containing the first path after registration.

[0138] In the photoacoustic image after registration, the first path and the position information of the front end of the flexible optical fiber are used to determine the guidance and correction information of the travel process, which comprises:

[0139] When the position coordinates of the front end of the flexible optical fiber are on the first path, the next path point is taken as the guidance information for guiding the front end of the flexible optical fiber to continue traveling;

[0140] When the position coordinates of the front end of the flexible optical fiber deviate from the first path, the path point closest to the front end of the flexible optical fiber is taken as the correction information for correcting the deviation process of the front end of the flexible optical fiber and guiding it to restore to the first path to continue traveling.

[0141] The front end of the flexible optical fiber is loaded with a columnar diffuse light source, as shown in Figure 4 .

[0142] As shown in Figure 2 , the application provides a light excitation guidance planning system based on photoacoustic imaging, which is applied to a light excitation guidance planning method based on photoacoustic imaging, and the system comprises:

[0143] A first data processing unit performs image segmentation on the medical image of the prostate tissue to distinguish the prostate region, the urethral region and the surrounding tissue region in the image;

[0144] A path planning unit plans a first path from the entrance end of the urethral region to the target end of the prostate region according to the medical image, taking the surrounding tissue region as an obstacle point;

[0145] A photoacoustic imaging unit is used to acquire photoacoustic signal data of the flexible optical fiber in the travel process according to the first path in real time;

[0146] A second data processing unit is used to determine the position information of the front end of the flexible optical fiber in the photoacoustic signal data and to reconstruct a photoacoustic image in the photoacoustic signal data.

[0147] A registration correction unit is configured to register the first path into the photoacoustic image from the medical image, and determine the guidance and correction information of the travel process of the flexible optical fiber front end in the registered photoacoustic image by using the first path and the position information of the flexible optical fiber front end.

[0148] The application provides a computer readable storage medium, and the computer readable storage medium stores computer execution instructions.

[0149] The application pre-plans a guide path of a flexible optical fiber used in photoacoustic imaging and introduced into a prostate part through a urethra, guides the travel process of the flexible optical fiber by using the guide path, improves safety, and accurately positions the flexible optical fiber front end by using an ultrasound guidance technology, and grasps the relative position between the flexible optical fiber front end and the guide path, so as to provide accurate guide information for the travel process of the flexible optical fiber by using the planned path.

[0150] The above examples are only exemplary embodiments of the application, and are not used to limit the application, and the protection scope of the application is defined by the claims. Those skilled in the art can make various modifications or equivalent replacements to the application within the spirit and protection scope of the application, and the modification or equivalent replacement is also regarded as falling within the protection scope of the application.

Claims

1. A photoexcitation-guided planning method based on photoacoustic imaging, characterized in that, Includes the following steps: Image segmentation is performed on medical images of prostate tissue to distinguish the prostate region, urethral region, and surrounding tissue region in the image; Based on the medical images, using the surrounding tissue area as obstacle points, a first path is planned from the inlet end of the urethral region to the target end of the prostate region; Real-time acquisition of photoacoustic signal data of the flexible optical fiber as it travels along the first path; The position information of the flexible optical fiber front end is determined from the photoacoustic signal data, and the photoacoustic image is reconstructed from the photoacoustic signal data; The first path is registered from the medical image to the photoacoustic image, and the guidance and correction information for the travel process is determined in the registered photoacoustic image using the position information of the first path and the flexible optical fiber front end.

2. The photoexcitation-guided planning method based on photoacoustic imaging according to claim 1, characterized in that: Methods for image segmentation of medical images of prostate tissue include: Three U-net networks were used to segment the prostate region, urethra region, and surrounding tissue region in medical images. The three U-net networks are the prostate segmentation network, urethra segmentation network, and surrounding tissue segmentation network, respectively. The three U-net networks are cross-optimized and trained using their outputs, where: Based on the prostate region obtained from the prostate segmentation network, the prostate region in the medical image is processed into a background region to form a prostate region masking image; Based on the urethral region obtained from the urethral segmentation network, the urethral region in the medical image is processed into the foreground region to form a urethral region masking image; Based on the surrounding tissue region obtained from the surrounding tissue segmentation network, the surrounding tissue region in the medical image is processed into the foreground region to form a surrounding region masking image; The prostate region was segmented in the urethral region masked image and the surrounding region masked image using a prostate segmentation network. A prostate segmentation consistency constraint was established between the medical image, the urethral region masked image and the surrounding region masked image. The prostate segmentation network was trained based on the prostate segmentation consistency constraint to form the optimal prostate segmentation network. Prostate segmentation consistency constraint for: ; The optimal prostate segmentation network is: ; In the formula, This refers to the prostate region segmented in medical imaging. This is a segmented prostate region in an image masked for the urethral region. The prostate region is segmented from the image to mask the surrounding area. G represents the medical image, and UNet is the U-Net network. The urethral region was segmented in the prostate region masked image and the surrounding region masked image using a urethral segmentation network. A urethral segmentation consistency constraint was established between the medical image, the prostate region masked image and the surrounding region masked image. The urethral segmentation network was trained based on the urethral segmentation consistency constraint to form the optimal urethral segmentation network. Urethral segmentation consistency constraint for: ; The optimal urethral segmentation network is: ; In the formula, This refers to the segmented urethral region in medical imaging. This refers to the segmented urethral region in a masked image of the prostate region. The urethral region is segmented from the image to mask the surrounding area; The surrounding tissue segmentation network was used to segment the surrounding tissue region in the prostate region masked image and the urethra region masked image, respectively. A consistency constraint on the surrounding tissue segmentation was established between the medical image, the prostate region masked image and the urethra region masked image. The surrounding tissue segmentation network was trained based on the consistency constraint to form the optimal surrounding tissue segmentation network. Surrounding tissue segmentation consistency constraints for: ; The optimal surrounding tissue segmentation network is: ; In the formula, The surrounding tissue area segmented in medical imaging. The surrounding tissue region is re-segmented from the masked image of the prostate region. To segment the surrounding tissue region in the masked image of the urethral region. It is an L2 norm.

3. The photoexcitation-guided planning method based on photoacoustic imaging according to claim 2, characterized in that: The planning method for the first path includes: The multiple optimization objectives for determining the first path include: Maximizing the distance between the flexible fiber front end and the surrounding tissue area , In the formula, Let be the coordinates of the i-th path point of the flexible optical fiber front end in the first path. Let be the coordinates of the point in the j-th surrounding tissue region that has the shortest distance to the urethral region, n be the total number of points in the first path, and k be the total number of surrounding tissue regions. This is the Euclidean distance expression, where max is the maximization identifier; Minimize the distance between the flexible optical fiber front end and the prostate region , In the formula, Here are the coordinates of the nth path point in the first path of the flexible optical fiber front end. The coordinates of the point in the prostate region that has the shortest distance to the urethral region are given; min is the minimum identifier. Minimize the travel distance of the flexible optical fiber front end , In the formula, Let be the coordinates of the i-th path point of the flexible optical fiber front end in the first path. The coordinates of the (i+1)th path point of the flexible optical fiber front end in the first path; Flexible optical fiber front end traveling smooth target , In the formula, Let be the coordinates of the i-th path point of the flexible optical fiber front end in the first path. Let be the coordinates of the (i+1)th path point of the flexible optical fiber front end in the first path. The coordinates of the (i-1)th path point of the flexible optical fiber front end in the first path; Using the urethral region as the solution space, a multi-objective optimization algorithm is employed to solve the problem. , , and The first path is obtained by optimization and then smoothed.

4. The photoexcitation-guided planning method based on photoacoustic imaging according to claim 3, characterized in that: Methods for determining the location information of flexible optical fibers from photoacoustic signal data include: Walking simulation training was conducted on a human body model at the front end of a flexible optical fiber. At the same time, the photoacoustic signal of the flexible optical fiber front end at a known point was collected by a transrectal ultrasound probe array to obtain a photoacoustic signal with positioning information. A localization model is obtained by using a convolutional neural network to establish a fitting relationship between photoacoustic signals and localization information. In the formula, P is the position coordinate of the flexible optical fiber front end predicted by the positioning model, signal is the photoacoustic signal, and CNN is the convolutional neural network. Loss function of localization model for: In the formula, This represents the true coordinates of the flexible optical fiber's front end. The coordinates of the flexible optical fiber front end are obtained by mathematical decomposition from the photoacoustic signal. Mean square error, For hyperparameters; Among them, the methods for decomposing position coordinates from photoacoustic signals using mathematical methods include: The urethral region, prostate region, and surrounding tissue region are used as the search space grid. At each grid point In the process of calculating sound waves from Theoretical propagation time to the l-th element in the transrectal ultrasound probe array ,in, For the speed of sound, The coordinates of the l-th element are given. The signal received by the l-th element Time delay compensation is obtained ; The compensated signals of all array elements are superimposed to obtain In the formula, m is the total number of array elements; Calculate superimposed signals Energy obtained and will The grid point corresponding to the maximum value is used as the coordinate of the flexible fiber front end.

5. The photoexcitation-guided planning method based on photoacoustic imaging according to claim 4, characterized in that: Methods for reconstructing photoacoustic images from photoacoustic signal data include: The photoacoustic signal data is subjected to laser intensity compensation, filtering, and peak detection through Hilbert transform. Photoacoustic images are reconstructed from photoacoustic signal data using beamforming or delay summation array ultrasound imaging algorithms.

6. The photoexcitation-guided planning method based on photoacoustic imaging according to claim 5, characterized in that: The method for registering the first path from the medical image to the photoacoustic image includes: Using a medical image as a reference image and a photoacoustic image as a floating image, the medical image and the photoacoustic image are registered and fused to obtain a registered photoacoustic image containing the first path.

7. The photoexcitation-guided planning method based on photoacoustic imaging according to claim 6, characterized in that: The method for determining guidance and correction information for the travel process using the position information of the first path and the flexible optical fiber front end in the registered photoacoustic image includes: When the position coordinates of the flexible fiber front end are on the first path, the next path point is used as guidance information for the flexible fiber front end to continue its journey. When the position coordinates of the flexible fiber front end deviate from the first path, the path point that is closest to the flexible fiber front end is used as correction information for the flexible fiber front end to continue its journey.

8. The photoexcitation-guided planning method based on photoacoustic imaging according to claim 7, characterized in that: The flexible optical fiber is equipped with a columnar diffuse light source at its front end.

9. A photoexcitation-guided planning system based on photoacoustic imaging, characterized in that, The system, applied to the photoexcitation-guided planning method based on photoacoustic imaging as described in any one of claims 1-8, comprises: The first data processing unit performs image segmentation on the medical images of prostate tissue to distinguish the prostate region, urethral region and surrounding tissue region in the image; The path planning unit, based on the medical images, uses the surrounding tissue areas as obstacle points to plan a first path from the inlet end of the urethral region to the target end of the prostate region; The photoacoustic imaging unit is used to acquire photoacoustic signal data in real time as the flexible optical fiber travels along the first path. The second data processing unit is used to determine the position information of the flexible optical fiber front end in the photoacoustic signal data and reconstruct the photoacoustic image in the photoacoustic signal data. The registration correction unit is used to register the first path from the medical image to the photoacoustic image, and to determine the guidance and correction information of the travel process in the registered photoacoustic image using the position information of the first path and the flexible optical fiber front end.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by the processor, implement the photoexcitation-guided planning method based on photoacoustic imaging as described in any one of claims 1-8.

Images