Photoacoustic imaging image processing system and method based on particle swarm optimization

Abstract

The invention provides a photoacoustic imaging image processing system and method based on a particle swarm algorithm. A noise suppression module carries out noise suppression processing on a preprocessed photoacoustic imaging image based on the particle swarm algorithm to obtain a first photoacoustic imaging image; the edge extraction module extracts an edge image from the first photoacoustic imaging image based on an edge detection algorithm; and the detail enhancement module performs detail enhancement on the edge image, and fuses the edge image after detail enhancement and the first photoacoustic imaging image to obtain a second photoacoustic imaging image. Noise of a photoacoustic imaging image is suppressed through a particle swarm algorithm, detail enhancement is performed after an edge image is extracted, effective noise suppression and image detail enhancement are realized, the definition and contrast of the image are improved, and the real-time application requirement is met.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to a photoacoustic imaging image processing system and method based on a particle swarm algorithm. Background Art

[0002] Photoacoustic Imaging (PAI) is a new type of non-invasive and non-ionizing biomedical imaging method developed in recent years. When a pulsed laser is irradiated into biological tissue (thermoacoustic imaging specifically refers to irradiation with a radio frequency pulsed laser), the light absorption domain of the tissue will generate an ultrasonic signal. We call this ultrasonic signal generated by light excitation a photoacoustic signal. The photoacoustic signal generated by biological tissue carries the light absorption characteristic information of the tissue. By detecting the photoacoustic signal, the light absorption distribution image in the tissue can be reconstructed. The image quality of photoacoustic imaging is often interfered by noise, resulting in unclear image details and affecting the accuracy of diagnosis. Existing photoacoustic imaging image processing methods mainly include spatial filtering, frequency domain filtering, and model-based image reconstruction methods. These methods can suppress noise to a certain extent, but often have the following shortcomings:

[0003] Spatial filtering: such as mean filtering and median filtering, although it can effectively suppress noise, it can easily lead to the loss of image details.

[0004] Frequency domain filtering: such as Fourier transform filtering, although it can separate noise and signal, it is not effective in handling the situation where the noise spectrum and signal spectrum overlap.

[0005] Model-based image reconstruction methods: such as compressed sensing and sparse representation, although they can restore image details to a certain extent, have high algorithm complexity and long calculation time, and are not suitable for real-time applications. Summary of the invention

[0006] Based on the above content, the present invention provides a photoacoustic imaging image processing system and method based on particle swarm algorithm, which only solves the technical problems in the prior art such as poor noise suppression of photoacoustic imaging images affecting details.

[0007] A photoacoustic imaging image processing system based on particle swarm algorithm, comprising:

[0008] An image acquisition module, used for acquiring original photoacoustic imaging images of body parts;

[0009] A preprocessing module, connected to the image acquisition module, for preprocessing the original photoacoustic imaging image;

[0010] A noise suppression module, connected to the preprocessing module, for performing noise suppression processing on the preprocessed photoacoustic imaging image based on a particle swarm algorithm to obtain a first photoacoustic imaging image;

[0011] An edge extraction module, connected to the noise suppression module, for extracting an edge image from the first photoacoustic imaging image based on an edge detection algorithm;

[0012] The detail enhancement module is connected to the edge extraction module and is used to perform detail enhancement on the edge image and fuse the detail-enhanced edge image with the first photoacoustic imaging image to obtain a second photoacoustic imaging image.

[0013] Furthermore, the preprocessing module includes:

[0014] A first preliminary denoising unit, using a median filter to perform a first preliminary denoising process on the preprocessed photoacoustic imaging image;

[0015] A second preliminary denoising unit, connected to the first preliminary denoising unit, is used to perform a second preliminary denoising process on the photoacoustic imaging image after the first preliminary denoising process by using a bilateral filter;

[0016] The image preliminary enhancement unit is connected to the second preliminary denoising unit and is used to perform contrast enhancement on the photoacoustic imaging image after the second preliminary denoising process based on a histogram equalization method.

[0017] Furthermore, the noise suppression module includes:

[0018] A particle swarm initialization unit is used to initialize the position and velocity of each particle in the particle swarm;

[0019] A fitness calculation unit, connected to the particle swarm initialization unit, is used to calculate the fitness of each particle based on a defined fitness function;

[0020] A fitness updating unit, connected to the fitness calculating unit, is used to update the individual best position of the particle and the global best position of the particle group according to the fitness of the particle;

[0021] A speed and position updating unit, connected to the fitness updating unit, is used to update the speed and position of each particle according to the individual best position and the global best position;

[0022] An iteration stop judgment unit, connected to the speed and position update unit, is used to judge whether the current iteration meets the preset stop condition and obtain a judgment result;

[0023] The fitness calculation unit is also connected to the iteration stop judgment unit, and is used to continue to use the fitness function to calculate the fitness of each particle after the speed and position are updated when the judgment result is that the preset stop condition is not reached;

[0024] The image noise suppression unit is connected to the iteration stop judgment unit and is used to form an image denoising parameter using the global optimal position of the last output particle group to process the first photoacoustic imaging image.

[0025] Furthermore, the fitness function is a weighted sum function of the peak signal-to-noise ratio and the structural similarity index.

[0026] Furthermore, the detail enhancement module includes:

[0027] An enhancement unit, configured to perform detail enhancement on the edge image based on the edge enhancement parameter, and perform multi-scale detail enhancement on the first photoacoustic imaging image based on the multi-scale Retinex algorithm to form a detail enhancement component;

[0028] An image fusion unit, connected to the enhancement unit, for fusing the detail enhancement component with the first photoacoustic imaging image to obtain a second photoacoustic imaging image;

[0029] a fusion evaluation unit, connected to the image fusion unit, for comparing the first photoacoustic imaging image with the second photoacoustic imaging image to evaluate whether the second photoacoustic imaging image meets the detail enhancement condition, and obtain an evaluation result;

[0030] The enhancement unit is connected to the fusion evaluation unit, and is used to adjust the edge enhancement parameters and the parameters of the multi-scale Retinex algorithm when the evaluation result shows that the second photoacoustic imaging image does not meet the detail enhancement condition, continue to perform detail enhancement on the edge image, and perform multi-scale detail enhancement on the first photoacoustic imaging image to form a detail enhancement component;

[0031] The image output unit is connected to the fusion evaluation unit and is used to output the second photoacoustic imaging image when the evaluation result shows that the second photoacoustic imaging image meets the detail enhancement condition.

[0032] Furthermore, the edge extraction module includes:

[0033] A Gaussian filtering unit, configured to perform filtering processing on the first photoacoustic imaging image using a Gaussian filter;

[0034] A gradient calculation unit, connected to the Gaussian filter unit, is used to calculate the horizontal gradient and vertical gradient of the photoacoustic imaging image using the Sobel operator, and calculate the gradient amplitude and gradient direction of each pixel point according to the horizontal gradient and the vertical gradient;

[0035] The non-maximum suppression unit is connected to the gradient calculation unit, and compares the gradient amplitudes of two pixels adjacent to the pixel along the gradient direction to extract edge pixels;

[0036] A dual threshold detection unit is connected to the non-maximum suppression unit to divide the extracted edge pixels into strong edge pixels and weak edge pixels according to the gradient amplitude;

[0037] The edge connection unit is connected to the dual threshold detection unit and is used to connect the strong edge pixel points and the weak edge pixel points adjacent to the strong edge pixel points to form an edge image.

[0038] Furthermore, the body part captured by the image acquisition module is superficial tissue.

[0039] Furthermore, the system also includes:

[0040] The image acquisition module is also used to acquire camera images of the same body part;

[0041] A multimodal fusion module, connected to the image acquisition module and the detail enhancement module, respectively, for fusing the camera image and the second photoacoustic imaging image to obtain a multimodal fusion image;

[0042] The diagnosis module is connected to the multimodal fusion module and processes the multimodal fusion images based on the pre-trained disease diagnosis model to form a diagnosis report.

[0043] A photoacoustic imaging image diagnosis method based on a particle swarm algorithm, using the aforementioned photoacoustic imaging image processing system based on a particle swarm algorithm, comprising:

[0044] Step A1, collecting original photoacoustic imaging images and camera images of the same body part;

[0045] Step A2, preprocessing the original photoacoustic imaging image;

[0046] Step A3, performing noise suppression processing on the preprocessed photoacoustic imaging image based on a particle swarm algorithm to obtain a first photoacoustic imaging image;

[0047] Step A4, extracting an edge image from the first photoacoustic imaging image based on an edge detection algorithm;

[0048] Step A5: perform detail enhancement on the edge image, and fuse the detail-enhanced edge image with the first photoacoustic imaging image to obtain a second photoacoustic imaging image.

[0049] The beneficial technical effect of the present invention is that the noise of the photoacoustic imaging image is suppressed by the particle swarm algorithm, and the detail is enhanced after the edge image is extracted, so as to achieve effective noise suppression and image detail enhancement, improve the clarity and contrast of the image, and meet the real-time application requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1-5 A schematic diagram of a module of a photoacoustic imaging image processing system based on a particle swarm algorithm according to the present invention;

[0051] Figure 6-11 The present invention is a flowchart of the steps of a photoacoustic imaging image processing method based on a particle swarm algorithm. DETAILED DESCRIPTION

[0052] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0053] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0054] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, but they are not intended to limit the present invention.

[0055] See also Figure 1 The present invention provides a photoacoustic imaging image processing system based on a particle swarm algorithm, comprising:

[0056] An image acquisition module (1), used for acquiring original photoacoustic imaging images of body parts;

[0057] A preprocessing module (2), connected to the image acquisition module (1), for preprocessing the original photoacoustic imaging image;

[0058] A noise suppression module (3) is connected to the preprocessing module (2) and is used to perform noise suppression processing on the preprocessed photoacoustic imaging image based on a particle swarm algorithm to obtain a first photoacoustic imaging image;

[0059] An edge extraction module (4), connected to the noise suppression module (3), for extracting an edge image from the first photoacoustic imaging image based on an edge detection algorithm;

[0060] The detail enhancement module (5) is connected to the edge extraction module (4) and is used to perform detail enhancement on the edge image and fuse the detail-enhanced edge image with the first photoacoustic imaging image to obtain a second photoacoustic imaging image.

[0061] The particle swarm algorithm is used to suppress the noise of the photoacoustic imaging image, and the edge image is extracted for detail enhancement, thereby achieving effective noise suppression and image detail enhancement, improving the image clarity and contrast, and meeting the needs of real-time applications.

[0062] See also Figure 2 , further, the preprocessing module (2) comprises:

[0063] A first preliminary denoising unit (21) performs a first preliminary denoising process on the pre-processed photoacoustic imaging image using a median filter;

[0064] A second preliminary denoising unit (22), connected to the first preliminary denoising unit (21), is used to perform a second preliminary denoising process on the photoacoustic imaging image after the first preliminary denoising process using a bilateral filter;

[0065] The image preliminary enhancement unit (23) is connected to the second preliminary denoising unit (22) and is used to perform contrast enhancement on the photoacoustic imaging image after the second preliminary denoising process based on a histogram equalization method.

[0066] The median filter is a nonlinear filter that effectively removes salt and pepper noise by replacing the value of each pixel with the median value of the pixels in its neighborhood. Pepper noise appears as black dots on the image, i.e., pixel values ​​are 0. Salt noise appears as white dots on the image, i.e., pixel values ​​are the maximum value (e.g., 255 in an 8-bit grayscale image).

[0067] The bilateral filter is a nonlinear filter that can retain the edge details of an image while removing noise. The bilateral filter combines the concepts of spatial neighborhood filtering and grayscale neighborhood filtering. It can maintain important image features such as edges and textures while smoothing the image.

[0068] Histogram equalization: By adjusting the grayscale histogram of the image, the distribution is made more uniform and the image contrast is improved.

[0069] After preprocessing, the photoacoustic imaging image data can be converted into a standard image format (such as PNG, JPEG) for subsequent processing and analysis. The preprocessing step of the photoacoustic imaging image can effectively remove noise, retain image details, and enhance image contrast, providing high-quality input data for subsequent noise suppression and detail enhancement.

[0070] See also Figure 3 , further, the noise suppression module (3) comprises:

[0071] A particle swarm initialization unit (31), used for initializing the position and velocity of each particle in the particle swarm;

[0072] A fitness calculation unit (32), connected to the particle swarm initialization unit (31), is used to calculate the fitness of each particle based on a defined fitness function;

[0073] A fitness updating unit (33), connected to the fitness calculating unit (32), is used to update the individual optimal position of the particle and the global optimal position of the particle group according to the fitness of the particle;

[0074] A speed and position updating unit (34), connected to the fitness updating unit (33), is used to update the speed and position of each particle according to the individual best position and the global best position;

[0075] An iteration stop judgment unit (35), connected to the speed and position update unit (34), is used to judge whether the current iteration meets the preset stop condition and obtain a judgment result;

[0076] The fitness calculation unit (32) is also connected to the iteration stop judgment unit (35) for continuing to use the fitness function to calculate the fitness of each particle after the speed and position are updated when the judgment result is that the preset stop condition is not met;

[0077] An image noise suppression unit (36) is connected to the iteration stop judgment unit (35) and is used to form an image denoising parameter using the global optimal position of the last output particle group to process the first photoacoustic imaging image.

[0078] Particle Swarm Optimization (PSO) is an optimization algorithm based on swarm intelligence. The core of PSO is the particle position and velocity update formula. The following is the general formula for particle position and velocity update:

[0079] The speed update formula is as follows:

[0080]

[0081] The position update formula is as follows:

[0082]

[0083] in, is the new velocity of particle i in dimension d;

[0084] ω is the inertia weight, which controls how well the particle maintains its current velocity;

[0085] c1 and c2 are acceleration coefficients, which are used to balance the effect of particles moving toward the individual optimal position and the global optimal position;

[0086] r1 and r2 are random numbers between [0,1], used to introduce randomness;

[0087] pbest i，d is the individual historical optimal position of particle i in dimension d.

[0088] gbest d is the dimension d of the particle swarm at the global optimal position.

[0089] is the position of particle i in dimension d at the tth iteration.

[0090] Use the fitness function to calculate the fitness of each particle's current position. If the current fitness is better than the particle's historical optimal fitness, update the particle's historical optimal position to the current position, and update the particle's historical optimal fitness value. If the current fitness is better than the global optimal fitness, update the global optimal position to the current position, and update the global optimal fitness.

[0091] Check whether the iteration reaches the stopping criterion (i.e. the preset stopping condition, such as the maximum number of iterations or the fitness no longer changes significantly). If the maximum number of iterations is reached, or the fitness value has not improved significantly within several iterations, the iteration is terminated. The purpose of setting the preset stopping condition is to prevent the algorithm from falling into invalid iterations and improve computational efficiency.

[0092] The particle swarm algorithm can adjust the position and speed of particles in each generation and continuously approach the optimal solution to the problem.

[0093] Furthermore, the fitness function is a weighted sum function of peak signal-to-noise ratio and structural similarity.

[0094] The fitness function is the core of the particle swarm algorithm and is used to evaluate the quality of each particle.

[0095] Peak Signal-to-Noise Ratio (PSNR): PSNR is used to measure the noise suppression effect of an image. The higher the PSNR value, the less noise there is in the image. The calculation of PSNR is based on the mean square error (MSE), which is the average of the squares of the pixel differences between the original image and the processed image. The calculation formula for PSNR is as follows:

[0096]

[0097] in,

[0098] MAX is the maximum value of the image pixel value (for 8-bit images, MAX = 255);

[0099] MSE is the mean square error, and the calculation formula is as follows:

[0100]

[0101] Among them, I(i,j) and K(i,j) are the pixel values ​​of the original image and the processed image at position (i,j), respectively, and M and N are the height and width of the image.

[0102] Structural Similarity (SSIM): SSIM is used to measure the detail preservation of an image. The higher the SSIM value, the better the structural similarity of the images. SSIM measures the similarity between two images in three key aspects: luminance, contrast, and structure. The combination of these three similarities determines the final SSIM value.

[0103] Brightness similarity: evaluated by comparing the average brightness of two images.

[0104] Contrast similarity: evaluated by comparing the contrast (i.e. variance) of two images.

[0105] Structural similarity: Assessing the structural consistency by comparing the covariance of two images.

[0106] The calculation formula of SSIM is:

[0107]

[0108] Among them, C1 and C2 are two constants used to stabilize the denominator;

[0109] For images x and y, the mean value ratio is μ x and μ y ;

[0110] σ xy is the covariance, and are the image x and y variances respectively.

[0111] The fitness is obtained by weighted summing the calculation results of PSNR and SSIM.

[0112] The fitness function is a key component in the particle swarm algorithm (PSO), which is used to evaluate the quality of each particle. For noise suppression and detail enhancement of ultrasound images, the fitness function needs to comprehensively consider the noise suppression effect and detail retention effect of the image. The present invention combines the peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) to design an effective fitness function, which can comprehensively consider the noise suppression effect and detail retention effect, thereby optimizing image processing parameters and improving image clarity and contrast.

[0113] See also Figure 5 , further, the detail enhancement module (5) comprises:

[0114] An enhancement unit (51) is used to perform detail enhancement on the edge image based on the edge enhancement parameter, and to perform multi-scale detail enhancement on the first photoacoustic imaging image based on the multi-scale Retinex algorithm to form a detail enhancement component;

[0115] An image fusion unit (52), connected to the enhancement unit (51), used to fuse the detail enhancement component and the first photoacoustic imaging image to obtain a second photoacoustic imaging image;

[0116] A fusion evaluation unit (53), connected to the image fusion unit (52), is used to compare the first photoacoustic imaging image with the second photoacoustic imaging image to evaluate whether the second photoacoustic imaging image meets the detail enhancement condition, and obtain an evaluation result;

[0117] The enhancement unit (51) is connected to the fusion evaluation unit (53) and is used to adjust the edge enhancement parameters and the parameters of the multi-scale Retinex algorithm when the evaluation result shows that the second photoacoustic imaging image does not meet the detail enhancement condition, continue to perform detail enhancement on the edge image, and perform multi-scale detail enhancement on the first photoacoustic imaging image to form a detail enhancement component;

[0118] The image output unit (54) is connected to the fusion evaluation unit (53) and is used to output the second photoacoustic imaging image when the evaluation result shows that the second photoacoustic imaging image meets the detail enhancement condition.

[0119] For edge images, a high-pass filter can be used to enhance the high-frequency part of the image and highlight the detail information.

[0120] Multi-scale detail enhancement: The multi-scale Retinex algorithm is used to enhance the multi-scale details of the image.

[0121] Detail enhancement filtering is an image processing technique used to enhance the details and edge features in an image, making the image appear clearer and sharper. The intensity of the enhancement needs to be properly controlled to avoid excessive enhancement leading to image distortion.

[0122] The detail enhancement component is fused with the original first PAT image to balance the enhancement effect with the naturalness of the image.

[0123] After the image fusion unit performs fusion, the contrast and brightness of the enhanced second photoacoustic imaging image are adjusted to further improve the visual effect. The purpose of image fusion is to combine the advantages of each image to generate an image containing more information. In the context of detail enhancement filtering, image fusion usually refers to synthesizing the enhanced detail components with the image before enhancement. Usually, a weighted average is performed according to a certain weight, and the choice of weight can be adjusted according to factors such as image quality and detail intensity.

[0124] After fusing to form the second photoacoustic imaging image, the image fusion unit further adjusts parameters such as contrast and brightness of the second photoacoustic imaging image to optimize the visual effect and form a final second photoacoustic imaging image.

[0125] Evaluating the result of detail enhancement is an important step to ensure that the image processing effect meets expectations. The fusion effect can be evaluated by subjective visual evaluation and objective quantitative indicator evaluation.

[0126] Subjective visual evaluation directly observes the enhanced image with human eyes to check whether the details and edges are clear. Compare the images before and after enhancement to observe the effect of detail enhancement. Pay special attention to the key areas in the image (such as edges and parts with rich textures) to check whether the details in these areas have been effectively enhanced. The indicators of quantitative evaluation include but are not limited to one or more of the following indicators: Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index (SSIM), Gradient Magnitude (GM), Information Entropy (Entropy), Contrast Enhancement Index (CEI), Sharpness Index (SI).

[0127] Gradient Magnitude (GM): Calculates the magnitude of image gradients and measures the degree of enhancement of image edges and details.

[0128] Entropy: Evaluates the change in the amount of information in an image. The larger the entropy, the more information the image contains.

[0129] Contrast Enhancement Index (CEI): Measures the enhancement effect of image contrast.

[0130] Sharpness Index (SI): Evaluates the sharpness of an image, usually calculated by image gradient or Laplacian operator.

[0131] The evaluation results of multiple quantitative indicators are combined to form an overall evaluation. The weight of each indicator can be set and a weighted average is performed according to the importance of the specific application scenario. According to the evaluation results of quantitative indicators, analyze whether the effect of detail enhancement meets the expectations. If the evaluation results are not ideal, adjust the parameters and continue to enhance the details.

[0132] See also Figure 4 , further, the edge extraction module (4) comprises:

[0133] A Gaussian filtering unit (41), configured to perform filtering processing on the first photoacoustic imaging image using a Gaussian filter;

[0134] A gradient calculation unit (42), connected to the Gaussian filter unit (41), is used to calculate the horizontal gradient and vertical gradient of the photoacoustic imaging image using a Sobel operator, and calculate the gradient amplitude and gradient direction of each pixel point according to the horizontal gradient and the vertical gradient;

[0135] A non-maximum suppression unit (43) is connected to the gradient calculation unit (42) and compares the gradient amplitudes of two pixel points adjacent to the pixel point along the gradient direction to extract edge pixel points;

[0136] A dual threshold detection unit (44) is connected to the non-maximum suppression unit (43) and divides the extracted edge pixel points into strong edge pixel points and weak edge pixel points according to the gradient amplitude;

[0137] The edge connection unit (45) is connected to the dual threshold detection unit (44) and is used to connect the strong edge pixel points and the weak edge pixel points adjacent to the strong edge pixel points to form an edge image.

[0138] The edge extraction module (4) uses an adaptive edge detection algorithm (Canny edge detection) to extract image edge information and generate an edge image.

[0139] The edge extraction module (4) also converts the photoacoustic imaging image into a grayscale image before the Gaussian filtering unit to simplify subsequent processing. Each pixel value in the grayscale image represents its brightness.

[0140] The Gaussian filter unit uses a Gaussian filter to smooth the image and reduce the interference of noise on edge detection.

[0141] The Sobel operator is used to calculate the gradient of the image in the horizontal direction (Gx) and the vertical direction (Gy). The Sobel operator is a discrete differential operator used to calculate the approximate value of the gradient. Then the gradient magnitude and gradient direction of each pixel are calculated.

[0142] The non-maximum suppression unit scans the gradient amplitude and compares the gradient amplitude of neighboring pixels along the gradient direction. Only the pixels with the local maximum gradient amplitude are retained, and the other pixels are suppressed to zero. This step helps to refine the edge and make it more accurate.

[0143] Apply double threshold: Use two thresholds (high threshold and low threshold) to classify the gradient magnitude. Pixels above the high threshold are marked as strong edges. Pixels below the low threshold are suppressed as non-edges. Pixels between the high and low thresholds are marked as weak edges.

[0144] After double threshold detection, the edge may still be intermittent. By analyzing connectivity, weak edge points are connected to strong edge points to form a complete edge contour. If a weak edge point has at least one adjacent strong edge point, then the weak edge point will be retained as part of the edge. On the contrary, if there are no strong edge points around a weak edge point connected to it, then the weak edge point will be suppressed (set as a background point).

[0145] Edge detection algorithms can effectively identify areas of significant change in images and extract the edges of objects.

[0146] Furthermore, the body part captured by the image acquisition module is superficial tissue.

[0147] Furthermore, the system also includes:

[0148] The image acquisition module (1) is also used to acquire camera images of the same body part;

[0149] A multimodal fusion module (6) is connected to the image acquisition module (1) and the detail enhancement module (5) respectively, and is used to fuse the camera image and the second photoacoustic imaging image to obtain a multimodal fusion image;

[0150] The diagnosis module (7) is connected to the multimodal fusion module (6) and processes the multimodal fusion image based on a pre-trained disease diagnosis model to form a diagnosis report.

[0151] Photoacoustic imaging (PA): using the photoacoustic effect, laser excitation tissue generates ultrasound to form an image, which can provide information about the function and structure of the tissue. High-definition photography (HD) uses visible light imaging to provide high-resolution surface images, which is suitable for imaging surface structures, such as shallow tissues such as skin.

[0152] For superficial tissues (such as skin lesions, superficial tumors, etc.), high-definition camera images can provide high-resolution surface structure and true color information. Photoacoustic imaging can provide functional and structural information of tissues. Combined with high-definition camera images, multimodal information fusion can be achieved to provide more comprehensive diagnostic information.

[0153] Applicable scenarios:

[0154] Skin lesions: such as skin cancer and melanoma, high-definition camera images can provide surface details, and photoacoustic imaging can provide functional information of lesion tissues.

[0155] Superficial tumors: such as breast tumors, thyroid nodules, etc. High-definition video images can provide surface structure, and photoacoustic imaging can provide blood supply and functional information of the tumor.

[0156] Superficial blood vessels: such as varicose veins, superficial vascular lesions, etc. High-definition camera images can display the surface vascular structure, and photoacoustic imaging can display the functional status of blood vessels.

[0157] Not applicable scenarios:

[0158] Deep organs: such as the heart and deep blood vessels, high-definition camera images cannot be acquired, and effective multimodal fusion cannot be performed.

[0159] Internal organs: such as liver, kidneys, etc. High-definition camera images cannot provide effective information about internal tissues.

[0160] See also Figure 6 The present invention also provides a photoacoustic imaging image diagnosis method based on a particle swarm algorithm, using the aforementioned photoacoustic imaging image processing system based on a particle swarm algorithm, comprising:

[0161] Step A1, collecting original photoacoustic imaging images and camera images of the same body part;

[0162] Step A2, preprocessing the original photoacoustic imaging image;

[0163] Step A3, performing noise suppression processing on the preprocessed photoacoustic imaging image based on a particle swarm algorithm to obtain a first photoacoustic imaging image;

[0164] Step A4, extracting an edge image from the first photoacoustic imaging image based on an edge detection algorithm;

[0165] Step A5: perform detail enhancement on the edge image, and fuse the detail-enhanced edge image with the first photoacoustic imaging image to obtain a second photoacoustic imaging image.

[0166] See also Figure 7 , further, step A2 comprises:

[0167] Step A21, using a median filter to perform a first preliminary denoising process on the preprocessed photoacoustic imaging image;

[0168] Step A22, performing a second preliminary denoising process on the photoacoustic imaging image after the first preliminary denoising process using a bilateral filter;

[0169] Step A23: performing contrast enhancement on the photoacoustic imaging image after the second preliminary denoising process based on a histogram equalization method.

[0170] See also Figure 8 , further, step A3 comprises:

[0171] Step A30, initializing the position and velocity of each particle in the particle swarm;

[0172] Step A31, calculating the fitness of each particle based on the defined fitness function;

[0173] Step A32, updating the individual optimal position of the particle and the global optimal position of the particle group according to the fitness of the particle;

[0174] Step A33, updating the speed and position of each particle according to the individual optimal position and the global optimal position;

[0175] Step A34, determine whether the current iteration meets the preset stop condition:

[0176] If yes, go to step A35;

[0177] If not, continue to step A31;

[0178] Step A35, using the global optimal position of the last output particle group to form image denoising parameters to process the first photoacoustic imaging image.

[0179] Furthermore, the fitness function is a weighted sum function of the peak signal-to-noise ratio and the structural similarity index.

[0180] See also Fig. 9 , further, step A4 comprises:

[0181] Step A41, using a Gaussian filter to filter the first photoacoustic imaging image;

[0182] Step A42, using the Sobel operator to calculate the horizontal gradient and vertical gradient of the photoacoustic imaging image, and calculating the gradient amplitude and gradient direction of each pixel point according to the horizontal gradient and the vertical gradient;

[0183] Step A43, comparing the gradient amplitudes of two pixels adjacent to the pixel along the gradient direction to extract edge pixels;

[0184] Step A44, dividing the extracted edge pixels into strong edge pixels and weak edge pixels according to the gradient amplitude;

[0185] Step A45, connecting the strong edge pixel points and the weak edge pixel points adjacent to the strong edge pixel points to form an edge image.

[0186] Furthermore, the body part collected is superficial tissue.

[0187] See also Fig.10 , further, step A5 comprises:

[0188] Step A51, performing detail enhancement on the edge image based on the edge enhancement parameter, and performing multi-scale detail enhancement on the first photoacoustic imaging image based on the multi-scale Retinex algorithm to form a detail enhancement component;

[0189] Step A52, fusing the detail enhancement component and the first photoacoustic imaging image to obtain a second photoacoustic imaging image;

[0190] Step A53, comparing the first photoacoustic imaging image with the second photoacoustic imaging image to evaluate whether the second photoacoustic imaging image meets the detail enhancement condition:

[0191] If satisfied, go to step A54;

[0192] If not satisfied, go to step A55;

[0193] Step A54, outputting a second photoacoustic imaging image;

[0194] Step A55, adjust the edge enhancement parameters and the parameters of the multi-scale Retinex algorithm, and continue with step A51.

[0195] See also Fig.11 ,Further, step A1 also collects camera images of the same body part;

[0196] Step A5 and later also include:

[0197] Step A6, fusing the camera image and the second photoacoustic imaging image to obtain a multimodal fused image;

[0198] Step A7, processing the multimodal fusion image based on the pre-trained disease diagnosis model to form a diagnosis report.

[0199] The above are only preferred embodiments of the present invention, and are not intended to limit the implementation methods and protection scope of the present invention. Those skilled in the art should be aware that all solutions obtained by equivalent substitutions and obvious changes made using the description and illustrations of the present invention should be included in the protection scope of the present invention.

Claims

1. A photoacoustic imaging image processing system based on particle swarm algorithm, characterized in that: include: An image acquisition module, used for acquiring original photoacoustic imaging images of body parts; A preprocessing module, connected to the image acquisition module, for preprocessing the original photoacoustic imaging image; A noise suppression module, connected to the preprocessing module, configured to perform noise suppression processing on the preprocessed photoacoustic imaging image based on a particle swarm algorithm to obtain a first photoacoustic imaging image; An edge extraction module, connected to the noise suppression module, configured to extract an edge image from the first photoacoustic imaging image based on an edge detection algorithm; A detail enhancement module is connected to the edge extraction module and is used to perform detail enhancement on the edge image and fuse the edge image after detail enhancement with the first photoacoustic imaging image to obtain a second photoacoustic imaging image.

2. A photoacoustic imaging image processing system based on particle swarm algorithm as claimed in claim 1, characterized in that: The preprocessing module comprises: A first preliminary denoising unit, using a median filter to perform a first preliminary denoising process on the preprocessed photoacoustic imaging image; a second preliminary denoising unit, connected to the first preliminary denoising unit, and configured to perform a second preliminary denoising process on the photoacoustic imaging image after the first preliminary denoising process by using a bilateral filter; The image preliminary enhancement unit is connected to the second preliminary denoising unit and is used to perform contrast enhancement on the photoacoustic imaging image after the second preliminary denoising process based on a histogram equalization method.

3. The photoacoustic imaging image processing system based on particle swarm algorithm according to claim 1, characterized in that: The noise suppression module comprises: A particle swarm initialization unit is used to initialize the position and velocity of each particle in the particle swarm; A fitness calculation unit, connected to the particle swarm initialization unit, for calculating the fitness of each particle based on a defined fitness function; A fitness updating unit, connected to the fitness calculating unit, for updating the individual best position of the particle and the global best position of the particle group according to the fitness of the particle; A speed and position updating unit, connected to the fitness updating unit, for updating the speed and position of each particle according to the individual optimal position and the global optimal position; An iteration stop judgment unit, connected to the speed and position update unit, used to judge whether the current iteration meets the preset stop condition and obtain a judgment result; The fitness calculation unit is also connected to the iteration stop judgment unit, and is used to continue to use the fitness function to calculate the fitness of each particle after the speed and position are updated when the judgment result is that the preset stop condition is not met; An image noise suppression unit is connected to the iteration stop judgment unit and is used to form an image denoising parameter using the global optimal position of the last output particle group to process the first photoacoustic imaging image.

4. A photoacoustic imaging image processing system based on particle swarm algorithm as claimed in claim 3, characterized in that: The fitness function is a weighted sum function of peak signal-to-noise ratio and structural similarity index.

5. The photoacoustic imaging image processing system based on particle swarm algorithm according to claim 1, characterized in that: The detail enhancement module comprises: an enhancement unit, configured to perform detail enhancement on the edge image based on edge enhancement parameters, and perform multi-scale detail enhancement on the first photoacoustic imaging image based on a multi-scale Retinex algorithm to form a detail enhancement component; an image fusion unit, connected to the enhancement unit, and configured to fuse the detail enhancement component and the first photoacoustic imaging image to obtain the second photoacoustic imaging image; a fusion evaluation unit, connected to the image fusion unit, configured to compare the first photoacoustic imaging image with the second photoacoustic imaging image to evaluate whether the second photoacoustic imaging image meets a detail enhancement condition, and obtain an evaluation result; The enhancement unit is connected to the fusion evaluation unit, and is used for adjusting the edge enhancement parameter and the parameter of the multi-scale Retinex algorithm to continue to perform detail enhancement on the edge image and perform multi-scale detail enhancement on the first photoacoustic imaging image to form the detail enhancement component when the evaluation result shows that the second photoacoustic imaging image does not meet the detail enhancement condition; The image output unit is connected to the fusion evaluation unit, and is used to output the second photoacoustic imaging image when the evaluation result shows that the second photoacoustic imaging image meets the detail enhancement condition.

6. A photoacoustic imaging image processing system based on particle swarm algorithm as claimed in claim 5, characterized in that: The edge extraction module comprises: A Gaussian filtering unit, configured to perform filtering processing on the first photoacoustic imaging image using a Gaussian filter; A gradient calculation unit, connected to the Gaussian filter unit, for calculating the horizontal gradient and the vertical gradient of the photoacoustic imaging image using a Sobel operator, and calculating the gradient amplitude and the gradient direction of each pixel point according to the horizontal gradient and the vertical gradient; A non-maximum suppression unit is connected to the gradient calculation unit, and compares the gradient amplitudes of two pixel points adjacent to the pixel point along the gradient direction to extract edge pixel points; A dual threshold detection unit, connected to the non-maximum suppression unit, divides the extracted edge pixels into strong edge pixels and weak edge pixels according to the gradient amplitude; An edge connection unit is connected to the dual threshold detection unit and is used to connect the strong edge pixel points and the weak edge pixel points adjacent to the strong edge pixel points to form an edge image.

7. The photoacoustic imaging image processing system based on particle swarm algorithm according to claim 1, characterized in that: The body part captured by the image acquisition module is superficial tissue.

8. The photoacoustic imaging image processing system based on particle swarm algorithm according to claim 1, characterized in that: The system also includes: The image acquisition module is also used to acquire camera images of the same body part; a multimodal fusion module, connected to the image acquisition module and the detail enhancement module, respectively, and configured to fuse the camera image and the second photoacoustic imaging image to obtain a multimodal fusion image; The diagnosis module is connected to the multimodal fusion module and processes the multimodal fusion image based on a pre-trained disease diagnosis model to form a diagnosis report.

9. A photoacoustic imaging image diagnosis method based on particle swarm algorithm, characterized in that: A photoacoustic imaging image processing system based on a particle swarm algorithm as described in any one of claims 1 to 8, comprising: Step A1, collecting original photoacoustic imaging images and camera images of the same body part; Step A2, preprocessing the original photoacoustic imaging image; Step A3, performing noise suppression processing on the preprocessed photoacoustic imaging image based on a particle swarm algorithm to obtain a first photoacoustic imaging image; Step A4, extracting an edge image from the first photoacoustic imaging image based on an edge detection algorithm; Step A5: perform detail enhancement on the edge image, and fuse the detail-enhanced edge image with the first photoacoustic imaging image to obtain a second photoacoustic imaging image.

Images