Optical excitation guide planning method and system based on photoacoustic imaging and storage medium

Abstract

The invention relates to the technical field of medical imaging processing, in particular to an optical excitation guide planning method and system based on photoacoustic imaging and a storage medium, and the method comprises the following steps: planning a first path from an inlet end of a urethra region to a target end of a prostate region; acquiring photoacoustic signal data in the advancing process of the flexible optical fiber according to the first path in real time; determining position information of the front end of the flexible optical fiber in the photoacoustic signal data, and reconstructing a photoacoustic image in the photoacoustic signal data; and registering the first path from the medical image to the photoacoustic image, and determining guide and correction information of an advancing process in the registered photoacoustic image by using the first path and the position information of the front end of the flexible optical fiber. A guiding path for guiding the flexible optical fiber to the prostate part through the urethra is planned in advance, the front end of the flexible optical fiber is accurately positioned by combining the guiding path with ultrasonic guiding, and accurate guiding information is provided for the advancing process of the flexible optical fiber.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical imaging processing, in particular to a light excitation guidance planning method and system based on photoacoustic imaging and a storage medium. BACKGROUND

[0002] Current medical lesion identification relies heavily on imaging methods, that is, medical images such as magnetic resonance images, ultrasound images, and photoacoustic images are used to identify lesions. Taking the identification of lesions in the prostate region as an example, magnetic resonance imaging, ultrasound imaging, and photoacoustic imaging of the prostate region can be used to identify lesions in the prostate region alone or in combination with multi-modal identification of lesions in the prostate region. Whether the imaging is used alone or in combination, high-quality imaging results are required. Among them, photoacoustic imaging of the prostate region requires a light excitation module (such as a flexible optical fiber) to irradiate the prostate region from the transurethral region to obtain high-quality photoacoustic imaging.

[0003] Therefore, the guidance of the light excitation module in photoacoustic imaging is very important.

[0004] Currently, it is very difficult to guide the light excitation module without a field of view. SUMMARY

[0005] The present application aims to provide a light excitation guidance planning method based on photoacoustic imaging to solve the technical problem of the difficulty of guiding the light excitation module without a field of view in the prior art.

[0006] To solve the above technical problems, the present application specifically provides the following technical solutions: A light excitation guidance planning method based on photoacoustic imaging, comprising the following steps: Segmenting the medical image of the prostate tissue to distinguish the prostate region, the urethral region, and the surrounding tissue region in the image; According to the medical image, taking the surrounding tissue region as an obstacle point, planning a first path from the inlet end of the urethral region to the target end of the prostate region; Real-time acquisition of photoacoustic signal data of the flexible optical fiber during the travel of the first path; Determine the position information of the front end of the flexible optical fiber in the photoacoustic signal data and reconstruct a photoacoustic image in the photoacoustic signal data; Register the first path from the medical image to the photoacoustic image, and determine the guidance and correction information of the travel process in the registered photoacoustic image using the first path and the position information of the front end of the flexible optical fiber.

[0007] As a preferred scheme of the present application, the method for image segmentation of the medical image of the prostate tissue comprises: The prostate region, the urethral region and the surrounding tissue region are segmented in the medical image by using three U-net networks, wherein the three U-net networks are respectively a prostate segmentation network, a urethral segmentation network and a surrounding tissue segmentation network; The three U-net networks are cross-optimization trained by using the output results of the three U-net networks, wherein: The prostate region in the medical image is processed as a background region according to the prostate region obtained by the prostate segmentation network, to form a prostate region mask image; The urethral region in the medical image is processed as a foreground region according to the urethral region obtained by the urethral segmentation network, to form a urethral region mask image; The surrounding tissue region in the medical image is processed as a foreground region according to the surrounding tissue region obtained by the surrounding tissue segmentation network, to form a surrounding region mask image; The prostate region is segmented in the urethral region mask image and the surrounding region mask image by using the prostate segmentation network, and the prostate segmentation consistency constraint is established between the medical image, the urethral region mask image and the surrounding region mask image, and the prostate segmentation network is trained based on the prostate segmentation consistency constraint to form an optimal prostate segmentation network; The prostate segmentation consistency constraint is: ; The optimal prostate segmentation network is: ; In the formula, is the segmented prostate region in the medical image, is the segmented prostate region in the urethral region mask image, is the segmented prostate region in the surrounding region mask image, and G is the medical image and UNet is the U-Net network; The urethral region is segmented in the prostate region mask image and the surrounding region mask image by using the urethral segmentation network, and the urethral segmentation consistency constraint is established between the medical image, the prostate region mask image and the surrounding region mask image, and the urethral segmentation network is trained based on the urethral segmentation consistency constraint to form an optimal urethral segmentation network; The urethral segmentation consistency constraint is: ; The optimal urethral segmentation network is: ; In the formula, is the segmented urethral region in the medical image, is the segmented urethral region in the prostate region mask image, segmenting the urethra region in the peripheral region mask image; segmenting the peripheral tissue region in the peripheral tissue region mask image respectively in the prostate region mask image and the urethra region mask image, and establishing a peripheral tissue segmentation consistency constraint between the medical image, the prostate region mask image and the urethra region mask image, and training the peripheral tissue segmentation network based on the peripheral tissue segmentation consistency constraint to form an optimal peripheral tissue segmentation network; the peripheral tissue segmentation consistency constraint is: ; the optimal peripheral tissue segmentation network is: ; in the formula, the peripheral tissue region segmented in the medical image, the peripheral tissue region re-segmented in the prostate region mask image, the peripheral tissue region segmented in the urethra region mask image, is an L2 norm formula.

[0008] As a preferred scheme of the present application, the planning method of the first path includes: determining the multi-optimization objectives for planning the first path includes: a distance maximization objective between the flexible optical fiber front end and the peripheral tissue region , in the formula, is the coordinate value of the flexible optical fiber front end at the i-th path point in the first path, is the coordinate value of the point with the shortest distance to the urethra region in the j-th peripheral tissue region, n is all path points in the first path, and k is the total number of peripheral tissue regions, is an Euclidean distance operation formula, and max is a maximization identifier; a distance minimization objective between the flexible optical fiber front end and the prostate region , in the formula, is the coordinate value of the flexible optical fiber front end at the n-th path point in the first path, is the coordinate value of the point with the shortest distance to the urethra region in the prostate region, and min is a minimization identifier; a flexible optical fiber front end travel distance minimization objective , in the formula, is the coordinate value of the flexible optical fiber front end at the i-th path point in the first path, is the coordinate value of the flexible optical fiber front end at the i+1-th path point in the first path; a flexible optical fiber front end travel smoothness objective , , wherein, is a coordinate value of the front end of the flexible optical fiber at an i-th path point in the first path, is a coordinate value of the front end of the flexible optical fiber at an i+1-th path point in the first path, is a coordinate value of the front end of the flexible optical fiber at an i-1-th path point in the first path; taking the urethral region as a solution space, using a multi-objective optimization solution algorithm to solve , , and to obtain the first path, and performing smoothing processing on the first path.

[0009] As a preferred scheme of the present application, the method for determining the position information of the flexible optical fiber in the photoacoustic signal data comprises: performing travel simulation training on the front end of the flexible optical fiber in the human body model, and simultaneously using the intrarectal ultrasonic probe array to collect photoacoustic signals of the front end of the flexible optical fiber at known points to obtain photoacoustic signals with positioning information; using a convolutional neural network to establish a fitting relationship between the photoacoustic signals and the positioning information to obtain a positioning model: , wherein P is the position coordinate of the front end of the flexible optical fiber predicted by the positioning model, signal is the photoacoustic signal, and CNN is the convolutional neural network; the loss function of the positioning model is : , wherein, is the true value of the position coordinate of the front end of the flexible optical fiber, is the position coordinate of the front end of the flexible optical fiber decomposed from the photoacoustic signal by using a mathematical method, is the mean square error, is a hyperparameter; The method for decomposing the position coordinate from the photoacoustic signal by using a mathematical method comprises: taking the urethral region, the prostate region and the surrounding tissue region as a search space grid; in each grid point , calculating the theoretical propagation time of the sound wave from to the l-th element in the intrarectal ultrasonic probe array, wherein, is the sound velocity, is the position coordinate of the l-th element; performing time delay compensation on the signal received by the l-th element to obtain ; superimposing all the compensated element signals to obtain , wherein m is the total number of elements. computing the energy of the superimposed signal , and taking the grid point corresponding to the maximum value as the position coordinate of the front end of the flexible optical fiber.

[0010] As a preferred scheme of the present application, the method for reconstructing a photoacoustic image from photoacoustic signal data comprises: performing laser intensity compensation, filtering, and Hilbert transform conversion for peak envelope detection on the photoacoustic signal data; reconstructing a photoacoustic image from the photoacoustic signal data using a beamforming or delay-and-sum array ultrasonic imaging algorithm.

[0011] As a preferred scheme of the present application, the method for registering a first path from the medical image to the photoacoustic image comprises: taking the medical image as a reference image and the photoacoustic image as a floating image, and performing registration fusion on the medical image and the photoacoustic image to obtain a photoacoustic image containing the first path after registration.

[0012] As a preferred scheme of the present application, the method for determining guidance and correction information of a travel process using the position information of the first path and the front end of the flexible optical fiber in the photoacoustic image after registration comprises: when the position coordinate of the front end of the flexible optical fiber is on the first path, taking the next path point as guidance information for the front end of the flexible optical fiber to continue traveling; when the position coordinate of the front end of the flexible optical fiber deviates from the first path, taking the path point closest to the front end of the flexible optical fiber on the first path as correction information for the front end of the flexible optical fiber to continue traveling.

[0013] As a preferred scheme of the present application, the front end of the flexible optical fiber is loaded with a columnar diffuse light source.

[0014] As a preferred scheme of the present application, the present application provides a light excitation guidance planning system based on photoacoustic imaging, which is applied to a light excitation guidance planning method based on photoacoustic imaging, and the system comprises: a first data processing unit for performing image segmentation on a medical image of prostate tissue to distinguish prostate regions, urethral regions, and surrounding tissue regions in the image; a path planning unit for planning a first path from an inlet end of the urethral region to a target end of the prostate region according to the medical image, taking the surrounding tissue regions as obstacle points; a photoacoustic imaging unit for acquiring photoacoustic signal data of a flexible optical fiber during travel according to the first path in real time; ​​A second data processing unit is configured to determine the position information of the front end of the flexible optical fiber in the photoacoustic signal data, and reconstruct a photoacoustic image in the photoacoustic signal data. An image registration unit is configured to register the first path into the photoacoustic image from the medical image, and determine the guidance and correction information of the travel process in the registered photoacoustic image by using the first path and the position information of the front end of the flexible optical fiber.

[0015] As a preferred scheme of the present application, the present application provides a computer readable storage medium, wherein computer execution instructions are stored in the computer readable storage medium, and when a processor executes the computer execution instructions, a light excitation guidance planning method based on photoacoustic imaging is realized.

[0016] Compared with the prior art, the present application has the following beneficial effects: The present application pre-plans a guide path of the flexible optical fiber used in photoacoustic imaging for introducing into the prostate position through the urethra, guides the travel process of the flexible optical fiber by using the guide path, improves the safety, and accurately positions the front end of the flexible optical fiber by using the ultrasound guidance technology, and grasps the relative position between the front end of the flexible optical fiber and the guide path, so as to provide accurate guide information for the travel process of the flexible optical fiber by using the planned path. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced below. Obviously, the drawings in the following description are only exemplary, and for those skilled in the art, other drawings can also be obtained from the provided drawings without creative labor.

[0018] Figure 1 A light excitation module guide path planning method flow chart is provided for the embodiments of the present application. Figure 2 A light excitation module guide path planning system diagram is provided for the embodiments of the present application. Figure 3 A medical image segmentation network structure block diagram is provided for the embodiments of the present application. Figure 4 A flexible optical fiber schematic diagram is provided for the embodiments of the present application. DETAILED DESCRIPTION

[0019] With reference to the drawings of the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described, obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0020] As shown in the drawings, Figure 1 The present application provides a light excitation guide planning method based on photoacoustic imaging, comprising the following steps: Image segmentation is performed on the medical image of the prostate tissue to distinguish the prostate region, urethral region and surrounding tissue region in the image; According to the medical image, the first path from the urethral region to the prostate region is planned with the surrounding tissue region as the obstacle point, which is essentially a planned path for guiding the flexible optical fiber to travel; Real-time acquisition of photoacoustic signal data of the flexible optical fiber during travel according to the first path, which is usually acquired by a transrectal ultrasound probe array; The position information of the front end of the flexible optical fiber is determined in the photoacoustic signal data, and a photoacoustic image is reconstructed in the photoacoustic signal data; The first path is registered from the medical image to the photoacoustic image, and the guide and correction information of the travel process is determined in the registered photoacoustic image by using the first path and the position information of the front end of the flexible optical fiber.

[0021] In this embodiment, the following data is mainly acquired: First data, medical image data of the prostate tissue, object analysis is performed to obtain the condition of the target tissue, and the path is planned according to the condition.

[0022] Second data, travel data of the flexible optical fiber. Essentially, it is an analysis and processing of multiple image data, and the offset between the flexible optical fiber and the planned path during travel is mastered, so as to design a guide method with correction function for the optical fiber, which can take the constructed human model or the model of the entire prostate part as the object, or other tissue models (also taking the surrounding tissue as the obstacle point, planning the path from the inlet end to the target end).

[0023] In addition, the method can also be applied to mechanical operation path planning in other cases without vision.

[0024] In the present application, photoacoustic imaging is used when introducing the light excitation module (flexible optical fiber), first, the individual structure of the operation object is acquired, and then the optimal guide path for introducing the flexible optical fiber through the urethra into the prostate region is formulated according to the individual structure of the operation object.

[0025] After the optimal guiding path suitable for the individual structure is formulated, the flexible optical fiber is guided during the introduction process, and the photoacoustic signal is acquired during the guiding process. The photoacoustic signal contains the light source positioning information, that is, the position information of the front end of the flexible optical fiber. The position information of the front end of the flexible optical fiber is determined from the photoacoustic signal, so as to determine whether the flexible optical fiber has deviated from the planned first path, and to provide guiding correction information for the deviated flexible optical fiber according to the deviation shown in the image.

[0026] Before guiding the flexible optical fiber, the individual structure characteristics need to be grasped first. Therefore, the image of the operation object is acquired first to grasp the structure and position information of the individual prostate region, urethral region and surrounding tissue region (such as the ejaculatory duct, bladder and other tissues). Therefore, the tissue region is segmented in the image before the flexible optical fiber is introduced, and the specific steps are as follows: The method for segmenting the prostate region, urethral region and surrounding tissue region in the medical image (such as MRI) includes: As shown in Figure 3 , three U-net networks are used to segment the prostate region, urethral region and surrounding tissue region in the medical image, wherein the three Unet networks are respectively a prostate segmentation network, a urethral segmentation network and a surrounding tissue segmentation network; The output results of the three U-net networks are used to cross-optimize and train the three U-net networks, wherein: The prostate region obtained by the prostate segmentation network is processed as a background region in the medical image to form a prostate region mask image; The urethral region obtained by the urethral segmentation network is processed as a foreground region in the medical image to form a urethral region mask image; The surrounding tissue region obtained by the surrounding tissue segmentation network is processed as a foreground region in the medical image to form a surrounding region mask image; The prostate segmentation network is used to segment the prostate region in the urethral region mask image and the surrounding region mask image, and a prostate segmentation consistency constraint is established between the medical image, the urethral region mask image and the surrounding region mask image, and the prostate segmentation network is trained based on the prostate segmentation consistency constraint to form an optimal prostate segmentation network; The prostate segmentation consistency constraint is: ; The optimal prostate segmentation network is: ; In the formula, is the segmented prostate region in the medical image,​ masking the prostate region segmented in the image for the urethra region, masking the prostate region segmented in the image for the surrounding region, G is a medical image, and UNet is a U-Net network. The application first establishes three segmentation networks directly applied to the original medical image, corresponding to the prostate region segmentation result, the urethra region segmentation result and the surrounding tissue region segmentation result. This direct segmentation on the original image is efficient, but the segmentation accuracy is limited due to the interference between regions of the original image. Therefore, the application optimizes the performance of the segmentation accuracy of the three segmentation networks directly applied to the original medical image.

[0027] For example, when optimizing the prostate segmentation network, the segmentation results of the other two segmentation networks are used to form two mask images on the original medical image, which eliminates the interference of the other two regions on the prostate region, and the segmentation effect is improved in the prostate region segmentation. Through prostate segmentation consistency constraint , the segmentation result of the original prostate segmentation network is forced to approach the segmentation result of the two mask images, that is the expected image original segmentation result approaches the urethra region mask image segmentation result, the expected original image segmentation result approaches the surrounding tissue region mask image segmentation result, so that the initial prostate segmentation network can obtain the prostate segmentation result close to the mask image on the original image, improve the segmentation accuracy, and at the same time retain the original efficiency performance.

[0028] The urethra segmentation network is used to segment the urethra region in the prostate region mask image and the surrounding region mask image, and a urethra segmentation consistency constraint is established between the medical image, the prostate region mask image and the surrounding region mask image. The urethra segmentation network is trained based on the urethra segmentation consistency constraint to form an optimal urethra segmentation network. urethra segmentation consistency constraint is: ; The optimal urethra segmentation network is: ; In the formula, is the urethra region segmented in the medical image, is the urethra region segmented in the prostate region mask image, is the urethra region segmented in the surrounding region mask image. Through the urethra segmentation consistency constraint , the segmentation result of the original urethra segmentation network is forced to approach the segmentation result of the two mask images, that is The image original segmentation result is expected to be close to the prostate region mask image segmentation result, The image original segmentation result is expected to be close to the surrounding tissue region mask image segmentation result, so that the initial urethra segmentation network can obtain the urethra region segmentation result close to the mask image on the image original, improve the segmentation accuracy, and meanwhile retain the original efficiency performance.

[0029] The surrounding tissue segmentation network is used to segment the surrounding tissue region in the prostate region mask image and the urethra region mask image respectively, and a surrounding tissue segmentation consistency constraint is established between the medical image, the prostate region mask image and the urethra region mask image, and the surrounding tissue segmentation network is trained based on the surrounding tissue segmentation consistency constraint to form an optimal surrounding tissue segmentation network. The surrounding tissue segmentation consistency constraint For: The optimal surrounding tissue segmentation network is: In the formula, The surrounding tissue region segmented in the medical image, The surrounding tissue region re-segmented in the prostate region mask image, The surrounding tissue region segmented in the urethra region mask image, The L2 norm formula.

[0030] Through the surrounding tissue segmentation consistency constraint , the segmentation result of the original surrounding tissue segmentation network is forced to be close to the segmentation result of the two mask images, that is The image original segmentation result is expected to be close to the prostate region mask image segmentation result, The image original segmentation result is expected to be close to the urethra region mask image segmentation result, so that the initial surrounding tissue segmentation network can obtain the surrounding tissue region segmentation result close to the mask image on the image original, improve the segmentation accuracy, and meanwhile retain the original efficiency performance.

[0031] After obtaining the individual tissue structure, the first path is planned by using a multi-objective optimization method, and the specific process is as follows: The first path planning method comprises: The multi-optimization objectives for planning the first path comprise: The distance maximization objective between the flexible optical fiber front end and the surrounding tissue region , In the formula, The coordinate value of the flexible optical fiber front end at the i-th path point in the first path, ​​is the coordinate value of the point with the shortest distance to the urethral region in the jth surrounding tissue region, n is all path points in the first path, k is the total number of surrounding tissue regions, is the Euclidean distance operation formula, max is the maximization identifier; Objective The safety optimization objective is characterized, so that the flexible optical fiber is as far away from the surrounding tissue region as possible during the travel process, thereby ensuring safety.

[0032] Flexible optical fiber front end and prostate region distance minimization objective , wherein, is the coordinate value of the flexible optical fiber front end at the nth path point in the first path, is the coordinate value of the point with the shortest distance to the urethral region in the prostate region, min is the minimization identifier; Objective The objective achievement optimization objective is characterized, so that the flexible optical fiber is as close to the prostate region as possible during the travel process, thereby ensuring that the prostate region is completed for the purpose of light irradiation.

[0033] Flexible optical fiber front end travel distance minimization objective , wherein, is the coordinate value of the flexible optical fiber front end at the ith path point in the first path, is the coordinate value of the flexible optical fiber front end at the ith+1 path point in the first path; Objective The distance optimization objective is characterized, so that the flexible optical fiber is as fast as possible to reach the prostate region during the travel process, thereby ensuring import efficiency.

[0034] Flexible optical fiber front end travel smoothness objective , wherein, is the coordinate value of the flexible optical fiber front end at the ith path point in the first path, is the coordinate value of the flexible optical fiber front end at the ith+1 path point in the first path, is the coordinate value of the flexible optical fiber front end at the ith-1 path point in the first path; Objective The path smoothness optimization objective is characterized, so that the flexible optical fiber is as smooth as possible to reach the prostate region during the travel process, thereby ensuring path continuity and safety.

[0035] The urethral region is taken as the solution space, and the multi-objective optimization solution algorithm is used to solve , , and An optimization solution is performed to obtain a first path, and the first path is smoothed.

[0036] The present invention 、 、 and Planning the first path can obtain an efficient path that is safe, smooth, and efficient.

[0037] In the process of guiding the flexible optical fiber using the planned path, the present invention needs to grasp the position information of the flexible optical fiber in real time in order to avoid the adverse effects of the positioning sensor in the photoacoustic imaging of the prostate area: for example, the metal component reflects the ultrasonic wave, generating a bright echo and an acoustic shadow, the sensor packaging material and the tissue acoustic impedance mismatch cause the sound wave to scatter and thus produce acoustic artifacts, the sensor light-shielding metal component blocks the excitation light path, resulting in a decrease in the local light energy density, and the stray reflection sensor surface reflects the laser to the non-target area, generating an unexpected photoacoustic signal, thereby generating photoacoustic interference. The increase in fiber rigidity increases the local rigidity of the sensor package, limiting the bending ability of the catheter, and the increase in diameter requires the increase in the catheter diameter to accommodate the sensor, affecting the ability to pass through the urethra. Therefore, the present invention abandons the sensor positioning method and adopts an end-to-end position prediction model formed by a neural network to directly predict the position coordinates of the front end of the optical fiber based on the photoacoustic signal, as follows: The method for determining the position information of the flexible optical fiber in the photoacoustic signal data includes: The flexible optical fiber front end is trained in a simulated movement in a human model, and a transrectal ultrasound probe array is used to collect photoacoustic signals at known points on the flexible optical fiber front end to obtain photoacoustic signals with positioning information. A convolutional neural network is used to establish a fitting relationship between the photoacoustic signal and the positioning information, and a positioning model is obtained: , where P is the position coordinate of the flexible optical fiber front end predicted by the positioning model, signal is the photoacoustic signal, and CNN is the convolutional neural network; Loss function of the positioning model for: , where is the true value of the flexible optical fiber front end position coordinate, The position coordinates of the flexible optical fiber front end are decomposed from the photoacoustic signal using mathematical methods. is the mean square error, is a hyperparameter; The present invention uses simulation training to accumulate model training data and associates the photoacoustic signal with the positioning information, thereby enabling the input photoacoustic signal to predict the positioning information. In the training of the positioning model, a constraint term is added to the loss function. In the case that the fitting relationship between the photoacoustic signal and the positioning information fails, the model can output a light source position solved according to the photoacoustic signal using a mathematical method. The light source position is located at the front end of the optical fiber, which is equivalent to directly decomposing the front end position of the optical fiber from the photoacoustic signal. That is to say, in the case that the fitting relationship between the photoacoustic signal and the positioning information fails, the positioning model will not be invalid. The two result outputs of the model complement each other, thereby improving the robustness of the positioning model.

[0038] Among them, the method of using mathematical methods to decompose the position coordinates in the photoacoustic signal includes: The urethra region, prostate region and surrounding tissue region are used as search space grids; At each grid point In the calculation, the sound wave is Theoretical propagation time to the lth element in the transrectal ultrasound probe array ,in, is the speed of sound, is the position coordinate of the lth array element; The signal received by the lth array element Perform time delay compensation to obtain ; The signals of all compensated array elements are superimposed to obtain , where m is the total number of array elements; Calculate the superposition signal The energy obtained , and The grid point corresponding to the maximum value is used as the position coordinate of the front end of the flexible optical fiber.

[0039] The method for reconstructing a photoacoustic image from photoacoustic signal data includes: The photoacoustic signal data is subjected to laser intensity compensation, filtering, and packet peak detection through Hilbert transform conversion; The photoacoustic image is reconstructed from the photoacoustic signal data using beamforming or delay-sum array ultrasound imaging algorithms.

[0040] Beamforming is a technology that combines multiple sensor signals to enhance signals in a specific direction and suppress interference from other directions. It is widely used in radar, sonar, wireless communications, and other fields. The delay and sum method is the most basic implementation. It adjusts the time delay of the signals to align and superimpose the signals at specific angles, thereby enhancing the energy in the target direction.

[0041] The method of registering the first path from the medical image to the photoacoustic image includes: The medical image is taken as a reference image, and the photoacoustic image is taken as a floating image, the medical image and the photoacoustic image are fused by registration, and a photoacoustic image containing a first path after registration is obtained.

[0042] The method for determining the guidance and correction information of the travel process in the photoacoustic image after registration by using the position information of the first path and the front end of the flexible optical fiber comprises the following steps: When the position coordinates of the front end of the flexible optical fiber are on the first path, the next path point is taken as the guidance information for guiding the front end of the flexible optical fiber to continue traveling; When the position coordinates of the front end of the flexible optical fiber deviate from the first path, the path point with the closest distance between the first path and the front end of the flexible optical fiber is taken as the correction information for correcting the deviation process of the front end of the flexible optical fiber and guiding it to return to the first path to continue traveling.

[0043] The front end of the flexible optical fiber is loaded with a columnar diffuse light source, as shown in Figure 4 .

[0044] As shown in Figure 2 , the present application provides a light excitation guidance planning system based on photoacoustic imaging, which is applied to a light excitation guidance planning method based on photoacoustic imaging, and the system comprises: A first data processing unit performs image segmentation on the medical image of the prostate tissue to distinguish the prostate region, the urethral region and the surrounding tissue region in the image; A path planning unit plans a first path from the inlet end of the urethral region to the target end of the prostate region according to the medical image, taking the surrounding tissue region as an obstacle point; A photoacoustic imaging unit is used to acquire photoacoustic signal data of the flexible optical fiber in the travel process according to the first path in real time; A second data processing unit is used to determine the position information of the front end of the flexible optical fiber in the photoacoustic signal data and to reconstruct a photoacoustic image in the photoacoustic signal data; A registration correction unit is used to register the first path from the medical image to the photoacoustic image and to determine the guidance and correction information of the travel process in the photoacoustic image after registration by using the position information of the first path and the front end of the flexible optical fiber.

[0045] The present application provides a computer readable storage medium, and the computer readable storage medium stores computer execution instructions, when the processor executes the computer execution instructions, a light excitation guidance planning method based on photoacoustic imaging is realized.

[0046] The application pre-plans a guiding path of a flexible optical fiber used in photoacoustic imaging for introduction into a prostate site through a urethra, uses the guiding path to guide the travel process of the flexible optical fiber, improves safety, and uses ultrasound guiding technology to accurately position the front end of the flexible optical fiber and master the relative position between the front end of the flexible optical fiber and the guiding path, so as to provide accurate guiding information for the travel process of the flexible optical fiber by using the planned path.

[0047] The above examples are only exemplary embodiments of the present application and are not intended to limit the present application, and the protection scope of the present application is defined by the claims. Those skilled in the art can make various modifications or equivalent replacements to the present application within the spirit and protection scope of the present application, and such modifications or equivalent replacements are also considered to fall within the protection scope of the present application.

Claims

1. A photoexcitation guided planning method based on photoacoustic imaging, characterized in that, The method comprises the following steps: image segmentation is performed on a medical image of prostate tissue to distinguish a prostate region, a urethra region and a surrounding tissue region in the image; a first path from an inlet end of the urethra region to a target end of the prostate region is planned based on the medical image, with the surrounding tissue region as an obstacle point; photoacoustic signal data of a flexible optical fiber during travel along the first path is acquired in real time; position information of a front end of the flexible optical fiber is determined in the photoacoustic signal data, and a photoacoustic image is reconstructed in the photoacoustic signal data; the first path is registered from the medical image to the photoacoustic image, and guidance and correction information of the travel process is determined in the registered photoacoustic image by using the first path and the position information of the front end of the flexible optical fiber.

2. The photoacoustic imaging-based photoexcitation guidance planning method of claim 1, wherein: The method for image segmentation of a medical image of prostate tissue comprises: three U-net networks are used to correspondingly segment the prostate region, the urethra region and the surrounding tissue region in the medical image, wherein the three U-net networks are respectively a prostate segmentation network, a urethra segmentation network and a surrounding tissue segmentation network; output results of the three U-net networks are used to cross-optimize and train the three U-net networks, wherein: the prostate region obtained by the prostate segmentation network is used to process the prostate region in the medical image as a background region, forming a prostate region mask image; the urethra region obtained by the urethra segmentation network is used to process the urethra region in the medical image as a foreground region, forming a urethra region mask image; the surrounding tissue region obtained by the surrounding tissue segmentation network is used to process the surrounding tissue region in the medical image as a foreground region, forming a surrounding region mask image; the prostate segmentation network is used to segment the prostate region in the urethra region mask image and the surrounding region mask image respectively, and prostate segmentation consistency constraints are established among the medical image, the urethra region mask image and the surrounding region mask image, and the prostate segmentation network is trained based on the prostate segmentation consistency constraints to form an optimal prostate segmentation network; Prostate segmentation consistency constraints are: ; The optimal prostate segmentation network is: ; In the formula, is a prostate region segmented in a urethral region mask image, is a prostate region segmented in a urethral region mask image, is a prostate region segmented in a surrounding region mask image, G is a medical image, and UNet is a U-Net network. the urethra segmentation network is used to segment the urethra region in the prostate region mask image and the surrounding region mask image respectively, and urethra segmentation consistency constraints are established among the medical image, the prostate region mask image and the surrounding region mask image, and the urethra segmentation network is trained based on the urethra segmentation consistency constraints to form an optimal urethra segmentation network; Urethral segmentation consistency constraint To: ; The optimal urethral segmentation network is: ; In the formula, is a urethra region segmented in a medical image, is a urethra region segmented in a prostate region mask image, is a urethra region segmented in a surrounding region mask image; the surrounding tissue segmentation network is used to segment the surrounding tissue region in the prostate region mask image and the urethra region mask image respectively, and surrounding tissue segmentation consistency constraints are established among the medical image, the prostate region mask image and the urethra region mask image, and the surrounding tissue segmentation network is trained based on the surrounding tissue segmentation consistency constraints to form an optimal surrounding tissue segmentation network; Surrounding tissue segmentation consistency constraint To: ; The optimal peripheral tissue segmentation network is: ; wherein is a surrounding tissue region segmented in the medical image, is a surrounding tissue region re-segmented from the prostate region mask image, is a surrounding tissue region segmented in the urethra region mask image, is an L2 norm.

3. The photoacoustic imaging-based photoexcitation guidance planning method of claim 2, wherein: the method for planning the first path comprises: determining multiple optimization objectives for planning the first path comprises: Objective of maximizing distance between flexible optical fiber tip and surrounding tissue regions , , wherein is a coordinate value of the flexible optical fiber tip at the i-th path point in the first path, is a coordinate value of the point in the j-th surrounding tissue region that has the shortest distance to the urethral region, n is the total number of path points in the first path, and k is the total number of surrounding tissue regions, is an Euclidean distance operation, and max is a maximization identifier; Minimizing distance between flexible optical fiber tip and prostate region , , wherein, is a coordinate value of the flexible optical fiber tip at an nth path point in the first path, is a coordinate value of a point in the prostate region that is at a shortest distance from the urethral region, and min is a minimization identifier. Flexible optical fiber front end travel distance minimization objective , , where, is a coordinate value of the flexible optical fiber front end at an i-th waypoint in the first path, is a coordinate value of the flexible optical fiber front end at an i+1-th waypoint in the first path. Smooth target for flexible optical fiber front end travel , , wherein is a coordinate value of the flexible optical fiber front end at an i-th path point in the first path, is a coordinate value of the flexible optical fiber front end at an i+1-th path point in the first path, is a coordinate value of the flexible optical fiber front end at an i-1-th path point in the first path; Taking the urethra region as a solution space, a multi-objective optimization solution algorithm is used to optimize and solve 、 、 and to obtain a first path, and the first path is smoothed.

4. The light excitation guided planning method based on photoacoustic imaging according to claim 3, characterized in that: the method for determining the position information of the flexible optical fiber in the photoacoustic signal data comprises: the front end of the flexible optical fiber is subjected to travel simulation training in a human body model, and photoacoustic signals of the front end of the flexible optical fiber at known points are acquired by using an intra-rectal ultrasound probe array to obtain photoacoustic signals with positioning information; A convolutional neural network is used to establish a fitting relationship between the photoacoustic signal and the positioning information, and a positioning model is obtained: , where P is the position coordinate of the flexible optical fiber front end predicted by the positioning model, signal is the photoacoustic signal, and CNN is the convolutional neural network; Loss function of the positioning model is: , wherein, is the true value of the coordinate of the front end of the flexible optical fiber, is the coordinate of the front end of the flexible optical fiber decomposed from the photoacoustic signal by using a mathematical method, is the mean square error, is a hyperparameter; The method for decomposing the position coordinates in the photoacoustic signal by using a mathematical method comprises: The urethral region, the prostate region, and the surrounding tissue region are taken as a search space grid; In each grid point , the theoretical propagation time of the sound wave from to the lth element in the endorectal ultrasound probe array is calculated , where is the sound speed, is the position coordinate of the lth element; for the signal received by the lth array element time delay compensation is performed to obtain ; All compensated array element signals are superimposed to obtain , wherein m is the total number of array elements. Calculate the superposition signal The energy obtained , and The grid point corresponding to the maximum value is used as the position coordinate of the front end of the flexible optical fiber.

5. The photoacoustic imaging-based photoexcitation guidance planning method of claim 4, wherein: The method for reconstructing a photoacoustic image from the photoacoustic signal data comprises: The photoacoustic signal data is subjected to laser intensity compensation, filtering, and peak envelope detection by Hilbert transform conversion; The photoacoustic image is reconstructed from the photoacoustic signal data by using a beamforming or delay-and-sum array ultrasonic imaging algorithm.

6. The photoacoustic imaging-based photoexcitation guidance planning method of claim 5, wherein: The method for registering the first path from the medical image to the photoacoustic image comprises: The medical image is taken as a reference image, and the photoacoustic image is taken as a floating image, and the medical image and the photoacoustic image are registered and fused to obtain a photoacoustic image containing the first path after registration.

7. The method of claim 6, wherein: The method for determining the guidance and correction information of the travel process by using the first path and the position information of the front end of the flexible optical fiber in the photoacoustic image after registration comprises: When the position coordinates of the front end of the flexible optical fiber are on the first path, the next path point is taken as the guidance information for the front end of the flexible optical fiber to continue traveling; When the position coordinates of the front end of the flexible optical fiber deviate from the first path, the path point closest to the front end of the flexible optical fiber is taken as the correction information for the front end of the flexible optical fiber to continue traveling.

8. The photoacoustic imaging-based photoexcitation guidance planning method of claim 7, wherein: The front end of the flexible optical fiber is loaded with a columnar diffuse light source.

9. A photoacoustic imaging-based photoexcitation guidance planning system, characterized by, The system is applied to the light excitation guidance planning method based on photoacoustic imaging according to any one of claims 1-8, and the system comprises: A first data processing unit performs image segmentation on a medical image of prostate tissue to distinguish a prostate region, a urethral region, and a surrounding tissue region in the image; A path planning unit plans a first path from an inlet end of the urethral region to a target end of the prostate region according to the medical image, with the surrounding tissue region as an obstacle point; A photoacoustic imaging unit is configured to acquire photoacoustic signal data of a flexible optical fiber during travel according to the first path in real time; A second data processing unit is configured to determine position information of a front end of the flexible optical fiber in the photoacoustic signal data and reconstruct a photoacoustic image from the photoacoustic signal data; A registration and correction unit is configured to register the first path from the medical image to the photoacoustic image and determine guidance and correction information of the travel process by using the first path and the position information of the front end of the flexible optical fiber in the photoacoustic image after registration.

10. A computer-readable storage medium, characterized in that The computer readable storage medium stores computer execution instructions, and when the processor executes the computer execution instructions, the method for light excitation guidance planning based on photoacoustic imaging according to any one of claims 1-8 is implemented.

Images