Endoscope image processing method based on photodynamic imaging and endoscope imaging system

Abstract

The invention relates to the technical field of image processing, in particular to an endoscope image processing method based on photodynamic imaging and an endoscope imaging system.The endoscope image processing method based on photodynamic imaging comprises the steps that a focus image is collected through a photodynamic imaging endoscope, and preprocessing, three-dimensional reconstruction, combined reaction and curative effect database analysis are conducted; configuring an image processing module according to the request, and docking three-dimensional display; according to the method, the technical problem that the imaging request cannot be quickly and accurately responded due to the fact that the speed of endoscopic image processing through empirical data is low is solved, and the effects that the endoscopic image is finely processed, the photodynamic reaction features are extracted and the imaging request cannot be quickly and accurately responded are achieved. The technical effects that the optimal image enhancement parameter is adaptively searched, the imaging request is quickly responded, and the image targeting enhancement is performed are achieved, so that the focus area is accurately and visually displayed in a three-dimensional manner, and the output quality of the endoscope image is remarkably improved are achieved.

Description

Technical Field

[0001] The present invention relates to the technical field related to image processing, and in particular to an endoscopic image processing method and an endoscopic imaging system based on photodynamic imaging. Background Art

[0002] Photodynamic therapy (PDT) refers to the use of photosensitizers and light of a specific wavelength to activate the production of reactive oxygen species, thereby destroying diseased cells, and using light of a specific wavelength to excite photosensitive substances in tissues to produce fluorescence or other optical signals, thereby achieving high-resolution imaging of tissue structure and function; photodynamic imaging refers to the synchronous visualization output during the photodynamic therapy process. Commonly, endoscopic images are used to observe the changes in diseased tissues under the action of photodynamic therapy in real time. However, deep learning through empirical data for endoscopic image processing is time-consuming and, in practical applications, it is impossible to quickly respond to imaging requests issued by the application end.

[0003] In summary, the prior art has a technical problem that the speed of endoscopic image processing based on empirical data is slow, resulting in an inability to quickly and accurately respond to imaging requests. Summary of the invention

[0004] The present application provides an endoscopic image processing method and an endoscopic imaging system based on photodynamic imaging, aiming to solve the technical problem in the prior art that the speed of endoscopic image processing based on empirical data is slow, resulting in the inability to quickly and accurately respond to imaging requests.

[0005] In view of the above problems, the technical solution to implement this application is:

[0006] In a first aspect of the present application, an endoscopic image processing method based on photodynamic imaging is provided, wherein the method comprises: using an endoscopic device with photodynamic imaging technology to collect a lesion area to obtain original image information, performing image preprocessing on the original image information, and three-dimensionally reconstructing to obtain a three-dimensional restored structure, wherein the original image information includes image information corresponding to the photodynamic reaction information;

[0007] Based on the photodynamic reaction database, P initialization reaction data sets are set according to the reaction type, and the initialization reaction data sets include the injection situation of the photodynamic agent, the irradiation condition of the light source, and the exposure time;

[0008] Based on the treatment effect evaluation database, Q historical reaction data sets are set according to the treatment effect, and the historical reaction data sets include image information of the early stage of treatment, image information of the middle stage of treatment, and image information of the late stage of treatment;

[0009] After receiving the imaging request, based on the P initialization response data sets, an initial state of the endoscope image processing module is configured, and a three-dimensional display terminal corresponding to the three-dimensional restoration structure is connected;

[0010] Extracting photodynamic reaction features based on image information corresponding to the photodynamic reaction information in the preprocessed image information, wherein the photodynamic reaction features include fluorescence intensity features, signal distribution features, and edge features;

[0011] Connecting to the treatment effect evaluation database, combining the photodynamic reaction characteristics to perform simultaneous evaluation and prediction of treatment effects, obtaining prediction and evaluation results, and judging whether image targeted enhancement is needed based on the prediction and evaluation results;

[0012] If image targeted enhancement is required, at the starting window time point in the preset time window, the image targeted enhancement instruction is activated in combination with the imaging request, and an iterative optimization search is performed in the optimization space corresponding to the Q historical reaction data sets to obtain an image targeted enhancement parameter;

[0013] Through the one-time image targeting enhancement parameters, the P initialization reaction data sets corresponding to the initial state of the endoscopic image processing module are cross-updated to obtain P test reaction data sets. The cycle continues until the termination condition is met, and image targeting enhancement is performed at the cutoff window time point in the preset time window and visualized output is performed on the three-dimensional display terminal.

[0014] In a second aspect of the present application, an endoscopic imaging system is provided, wherein the endoscopic imaging system comprises: a memory for storing executable instructions; and a processor for executing the executable instructions stored in the memory to implement the steps of the endoscopic image processing method based on photodynamic imaging as described in any one of the first aspects.

[0015] In summary, one or more technical solutions provided in the present application realize the refined processing of endoscopic images, extract photodynamic response characteristics, adaptively search for optimal image enhancement parameters, quickly respond to imaging requests and perform targeted image enhancement, thereby accurately displaying the lesion area in three-dimensional visualization, and significantly improving the technical effect of the output quality of endoscopic images. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A schematic flow chart of an endoscopic image processing method based on photodynamic imaging is provided for this application;

[0017] Figure 2 A schematic flow chart of obtaining characteristic distribution calculation results in an endoscopic image processing method based on photodynamic imaging is provided for this application;

[0018] Figure 3 A structural schematic diagram of an endoscopic imaging system is provided for this application.

[0019] Description of the reference numerals: processor 21 , memory 22 , input device 23 , output device 24 . DETAILED DESCRIPTION

[0020] Embodiment 1

[0021] The present application is described in detail below with reference to the accompanying drawings. Figure 1 As shown, the present application provides an endoscopic image processing method based on photodynamic imaging, wherein the method comprises:

[0022] S1: using an endoscope device with photodynamic imaging technology to collect the lesion area, obtain original image information, perform image preprocessing on the original image information, and three-dimensionally reconstruct to obtain a three-dimensional restored structure, wherein the original image information includes image information corresponding to the photodynamic reaction information;

[0023] S2: Based on the photodynamic reaction database, P initialization reaction data sets are set according to the reaction type, and the initialization reaction data sets include the injection situation of the photodynamic agent, the irradiation conditions of the light source, and the exposure time;

[0024] S3: Based on the treatment effect evaluation database, Q historical response data sets are set according to the treatment effect, wherein the historical response data sets include image information of the early stage of treatment, image information of the middle stage of treatment, and image information of the late stage of treatment;

[0025] In actual applications, it is found that due to the influence of the complex internal environment of the organism, the photodynamic imaging images collected by the endoscope may have problems such as noise, low contrast, unclear details, etc., which directly affect the doctor's accurate judgment of the lesion range, degree of lesions and treatment effect; therefore, it is necessary to process the original images of the endoscope, such as noise suppression, smoothing, brightness and contrast adjustment, etc., to improve the basic quality of the image;

[0026] However, conventional endoscopic image processing methods often fail to fully utilize the specific information of photodynamic reactions, such as changes in fluorescence intensity and distribution of photodynamic agents. Conventional endoscopic image processing uses all empirical data as training data for deep learning. The endoscopic image processing of the trained model takes a long time and cannot quickly respond to imaging requests from the application end.

[0027] Based on this, the present application accurately identifies and strengthens the fluorescence signal of the photodynamic reaction, reduces the interference of irrelevant signals, and at the same time, realizes the accurate display of the three-dimensional structure of the lesion, and can quantitatively analyze the distribution of the photodynamic agent in the three-dimensional space. According to individual differences and real-time changes during treatment, combined with the photodynamic reaction database and the treatment effect evaluation database, the image processing parameters are dynamically optimized, and the image enhancement is flexibly performed according to the actual situation of each patient, and the optimal image enhancement parameters are adaptively found, and the imaging request is quickly responded to and the image is targeted. Enhancement, including, the lesion area is the target area of ​​photodynamic therapy, and image preprocessing refers to basic image processing, such as noise suppression, smoothing, brightness and contrast adjustment, etc.;

[0028] Using an endoscope equipped with a photodynamic imaging function to accurately observe and capture images of the lesion area to obtain original image information, wherein the original image information includes a fluorescent signal reflecting the photodynamic reaction and other related structural information; performing image preprocessing operations on the collected original image information, such as noise removal, contrast enhancement, brightness correction, color balance adjustment, etc., to facilitate subsequent accurate identification of photodynamic reaction characteristics;

[0029] Using multiple images collected continuously, three-dimensional reconstruction is performed using three-dimensional animation modeling software (such as 3DStudio Max (software name), Maya (software name) or other modeling software) to construct a three-dimensional structural model of the lesion area, which helps to observe the lesion from different angles and depths and determine its boundary, shape and relationship with surrounding tissues;

[0030] Based on the photodynamic reaction database, P different initialization reaction data sets are set according to the reaction type, wherein each initialization reaction data set includes information such as the type, dosage, injection method of a specific photodynamic agent, and the wavelength, power, irradiation time and mode of the light source; based on the treatment effect evaluation database, Q historical reaction data sets are set according to the treatment effect, wherein each historical reaction data set records the image information of the entire process from the early, middle and late stages of treatment, including physiological index data such as changes in tumor volume, angiogenesis, and inflammatory response degree;

[0031] It should be noted that the reaction types include, but are not limited to, light-induced fluorescence reaction (fluorescence reaction emitted by a specific molecule after being excited by light of a specific wavelength, and the corresponding fluorescence signal can be used to detect and locate the target substance), light-induced quenching reaction (the fluorescence intensity of certain substances changes (such as quenching) after being exposed to light), photodynamic oxidation reaction (in photodynamic therapy, certain photosensitizers can produce oxidative stress under light, leading to cell or tissue damage), photoinduced electron transfer reaction (involving the electron transfer process of photosensitizers under light, which helps to understand the mechanism of action of photosensitizers and thus optimize their application in photodynamic imaging), photoisomerization reaction (some molecules undergo structural changes (isomerization) under light, and this reaction data helps to analyze the stability and activity of molecules in photodynamic imaging), photodynamic drug release reaction (some photosensitizers or drug carriers can release drugs under specific light, which is used to evaluate the efficiency and specificity of drug release);

[0032] S4: after receiving the imaging request, based on the P initialization response data sets, configuring the initial state of the endoscope image processing module, and connecting to the three-dimensional display terminal corresponding to the three-dimensional restoration structure;

[0033] S5: extracting photodynamic reaction features based on image information corresponding to the photodynamic reaction information in the preprocessed image information, wherein the photodynamic reaction features include fluorescence intensity features, signal distribution features, and edge features;

[0034] Based on the different photodynamic agents and light source parameters corresponding to the P initialization reaction data sets, the working parameters of the endoscope image processing module are adjusted to accurately identify and analyze the corresponding photodynamic reaction characteristics; the endoscope image processing module is connected to the three-dimensional display terminal to ensure that the processed image information can be transmitted to the three-dimensional display interface in real time, so as to intuitively observe and analyze the three-dimensional restored structure;

[0035] Based on the pre-processed image information, locate the key areas that reflect the photodynamic reaction; segment the parts related to the photodynamic reaction, focus on the image information of the fluorescence signal part, and further quantitatively analyze the fluorescence intensity value of each pixel point or area of ​​interest in the image, and reveal the activity of the photodynamic reaction by calculating parameters such as average intensity, maximum intensity, and intensity distribution curve; analyze the spatial distribution law of the fluorescence signal, such as concentration, diffusion range, morphological characteristics, etc. The signal distribution characteristics are used to judge the permeability and distribution uniformity of the drug in the lesion; detect the clarity of the fluorescence signal boundary, and use the edge detection algorithm to extract the boundary contour between the lesion area and the normal tissue to determine whether the effective range of the photodynamic reaction and the lesion boundary are clear;

[0036] The extracted fluorescence intensity features, signal distribution features, and edge features are integrated into the three-dimensional restoration structure to achieve visual annotation and dynamic presentation of the photodynamic response features on the three-dimensional restoration structure; on the three-dimensional display terminal, representative response features are highlighted through color coding, transparency adjustment, etc., to provide technical support for more accurate diagnosis and treatment planning.

[0037] S6: connecting to the treatment effect evaluation database, performing simultaneous evaluation and prediction of treatment effect in combination with the photodynamic reaction characteristics, obtaining a prediction and evaluation result, and judging whether image targeted enhancement is needed according to the prediction and evaluation result;

[0038] S7: If image targeted enhancement is required, at a starting window time point in the preset time window, the image targeted enhancement instruction is activated in combination with the imaging request, and an iterative optimization search is performed in the optimization space corresponding to the Q historical reaction data sets to obtain an image targeted enhancement parameter;

[0039] S8: Through the one-time image targeting enhancement parameters, cross-update the P initialization reaction data sets corresponding to the initial state of the endoscopic image processing module to obtain P test reaction data sets, continue the cycle until the termination condition is met, perform image targeting enhancement at the cutoff window time point in the preset time window and perform visual output on the three-dimensional display terminal.

[0040] The collected photodynamic reaction image information of the current lesion area is connected with the treatment effect evaluation database; the current treatment effect is synchronously evaluated and predicted by using a machine learning or deep learning model, combined with the photodynamic reaction characteristics, to obtain a prediction evaluation result, which includes the treatment effect level, the degree of lesion improvement, potential risk assessment, etc.; based on the prediction evaluation result, it is determined whether it is necessary to further improve the image quality for more accurate evaluation. If the prediction result shows that the treatment is insufficient or the condition is complicated, and the image information is not clear enough, the image targeted enhancement process is started;

[0041] At a starting window time point within a preset time window, an image targeting enhancement instruction activated by an imaging request is received; in an optimization space formed by Q historical response data sets, an iterative optimization algorithm (such as a genetic algorithm, a particle swarm optimization, etc.) is used to find a primary image targeting enhancement parameter corresponding to the optimization space, involving gain adjustment, filter parameter adjustment, and contrast / brightness optimization;

[0042] After obtaining the corresponding image targeting enhancement parameters after an iterative optimization, the P initialization reaction data sets in the initial state of the image targeting enhancement parameters and the endoscopic image processing module are compared, and each set is cross-updated to form a new P test reaction data set, aiming to simulate the change of the image under different enhancement strategies, so as to find the best enhancement scheme; continue to repeat the above optimization and update process until the preset termination conditions are reached, such as reaching the predetermined image quality standard, the upper limit of the number of iterations or the convergence of the optimization index; at the cutoff window time point of the preset time window, the original image is enhanced with the determined optimal image targeting enhancement parameters; after the processed enhanced image is three-dimensionally reconstructed, it is displayed in real time on the three-dimensional display terminal, and the refresh frequency corresponding to the preset time window is not less than 30fps (less than 30fps will cause a decrease in visual fluency and affect the observation of endoscopic image details), so as to quickly find the optimal image enhancement strategy in a short time and significantly improve the output quality of the endoscopic image.

[0043] Furthermore, an iterative optimization search is performed in the optimization space corresponding to the Q historical reaction data sets. The method of the present application includes:

[0044] S71: In the endoscope image processing module, performing historical poor quality scoring corresponding to historical evaluation results according to the treatment effect evaluation database to obtain M poor quality scoring data sets;

[0045] S72: extracting N core poor quality parameter features by comparing the M poor quality score data sets, and setting the search probability by the M poor quality scores corresponding to the M poor quality score data sets;

[0046] S73: Based on the N core inferior parameter features, the M inferior score data sets and the search probability, data clustering is performed again according to data association to establish an optimization space.

[0047] In the endoscope image processing module, the evaluation results of the historical treatment cases are counted and analyzed according to the stored treatment effect evaluation database, wherein each historical treatment record is evaluated in detail, and the historical inferior quality score of each case is calculated based on a preset evaluation system (such as phototoxicity, side effects, complications, etc.); the historical inferior quality scores of all cases are aggregated to form M inferior quality score data sets, each set representing a specific historical treatment case;

[0048] For the above M inferior score data sets, in-depth mining and data analysis are carried out to find N core inferior parameter features that are significantly related to the treatment effect, wherein the core inferior parameter features include inferior parameter features corresponding to phototoxicity (active oxygen generation capacity of different photosensitizers after light activation, degree of cell damage, range of tissue necrosis, etc.), inferior parameter features corresponding to side effects (recording adverse reactions that may occur during and after photodynamic therapy, such as skin photosensitivity, pain, edema, bleeding, infection, etc.), and inferior parameter features corresponding to complications (complication cases specific to different tissue layers (such as muscle layer, etc.));

[0049] According to the score in each inferior score data set, the corresponding search probability is set. Under normal circumstances, cases with higher inferior scores (i.e., cases with poor effects or greater side effects) will be given a higher search probability because they need to be paid attention to during the optimization process; for the extracted N core inferior parameter features, based on the data correlation between the N core inferior parameter features and the historical inferior scores, the intrinsic connection between the features and the correlation with the treatment effect are analyzed;

[0050] Clustering algorithms (such as K-means, hierarchical clustering, DBSCAN, etc.) are used to re-divide the data to form several clusters with similar characteristics and poor quality scoring characteristics, and an optimization space based on historical data is constructed. The optimization space contains various possible combinations of image processing parameters and the corresponding historical performances, which provides a basic framework for the subsequent iterative optimization search; in the constructed optimization space, according to the set search probability weights, appropriate optimization algorithms (such as genetic algorithms, particle swarm optimization, simulated annealing, etc.) are used to perform iterative optimization searches; the goal of the search is to find a set of optimal photodynamic response parameter combinations so that the endoscopic image processing module can better highlight the lesion characteristics and improve image quality in subsequent processing.

[0051] Furthermore, based on the N core inferior parameter features, the M inferior scoring data sets and the search probability, data clustering is performed again according to data association, and the method of the present application includes:

[0052] S731: performing an optimal space search configuration based on the M low-quality score data sets and the search probability to obtain a search blocking index parameter, wherein the search blocking index parameter includes a search minimum interval index parameter and a blocking search boundary constraint index parameter;

[0053] S732: Performing an optimization configuration on the optimization space by using the search blocking index parameter;

[0054] S733: Based on the N core inferior parameter features, feature distribution calculation is performed in combination with the M inferior score data sets, and secondary optimization configuration is performed on the optimization space according to the feature distribution calculation results.

[0055] Using the obtained M inferior score data sets and the corresponding search probabilities, the optimization space is searched and configured. The search configuration refers to setting the search path and priority according to the inferior score of each historical case and the probability of being searched, and determining the direction and intensity of the search; defining the search blocking index parameter, which means avoiding falling into the area with high inferior scores (i.e., poor effect) during the optimization process, and tending to search in the direction of low inferior scores (i.e., better effect). The search blocking index parameter includes the search minimum interval index parameter and the blocking search boundary constraint index parameter. The search minimum interval index parameter is used to set the minimum distance between adjacent parameter combinations during the search process to prevent over-intensive search; the blocking search boundary constraint index parameter is used to limit the search range to avoid entering the obviously invalid parameter area;

[0056] According to the search barrier index parameters, a rough optimization configuration is performed on the optimization space, and the initial step length, search starting point and boundary conditions of the search are set to form a preliminary search path and strategy; based on the N core inferior parameter characteristics, the characteristic distribution of the M inferior score data sets is calculated, and the distribution law of each characteristic variable in different score intervals is analyzed; according to the results of the characteristic distribution calculation, a more detailed and accurate secondary configuration of the optimization space is performed, and further, according to the trend and density distribution of the characteristic variables in different score intervals, the search focus and direction are adjusted, so that the search is more concentrated on those parameter areas that are strongly related to excellent treatment effects. At this time, it also involves adjusting the search step length, updating the search starting point or changing the constraints of the search boundary, so that the optimal solution or approximate optimal solution can be found faster in the optimization process;

[0057] By combining the poor quality scores of historical data, core parameter characteristics and their correlation, as well as search probability and other factors, the configuration of the optimization space is gradually refined and optimized, so as to effectively find the optimal parameter setting scheme in photodynamic imaging endoscopic image processing.

[0058] Furthermore, based on the N core inferior parameter features, the feature distribution is calculated in combination with the M inferior score data sets, and the optimization space is configured for secondary optimization according to the feature distribution calculation results. The method of this application includes:

[0059] S7331: Taking the N core inferior parameter features as the center, performing distribution feature analysis based on the first inferior score data set, and constructing a first feature distribution matrix;

[0060] S7332: traverse the M low-quality rating data sets to obtain a first feature distribution matrix, a second feature distribution matrix, ..., an Mth feature distribution matrix;

[0061] S7333: Based on the first feature distribution matrix, the second feature distribution matrix, ... the Mth feature distribution matrix, obtain the feature distribution calculation result.

[0062] Extract N parameters closely related to the treatment effect from the M inferior score data sets as core inferior parameter features; take the first inferior score data set as an example, conduct detailed distribution feature analysis for the N core parameter features, such as calculating the mean, variance, skewness, kurtosis and other statistics of each parameter, or draw histograms, scatter plots and other charts to intuitively show the parameter distribution, and construct a first feature distribution matrix by comparing the parameter distribution. The first feature distribution matrix reflects the numerical distribution and mutual relationship of each core parameter feature under the first inferior score data set;

[0063] Traverse the remaining M-1 low-quality rating data sets, perform the above distribution feature analysis on each set, and construct corresponding feature distribution matrices, i.e., the second feature distribution matrix to the Mth feature distribution matrix. Each feature distribution matrix is ​​a multidimensional array, wherein the rows of the feature distribution matrix represent different core parameter features, and the columns of the feature distribution matrix represent the instance distribution of the corresponding features in a specific low-quality rating data set;

[0064] After obtaining M feature distribution matrices, the distribution patterns, variation trends and relationships of each core parameter feature at different scoring levels are explored; based on the results of feature distribution calculation, the optimization space is finely adjusted. For example, if a core parameter feature is determined to show a good therapeutic effect within a specific scoring interval, then this parameter area will be considered when the optimization algorithm is searching for optimization; through in-depth analysis of the feature distribution matrix, the search strategy can be optimized on the basis of the original optimization space, such as adjusting the search step size, priority, or adding additional constraints, in order to quickly approach the optimal solution and improve the efficiency of endoscopic image processing.

[0065] Furthermore, if Figure 2 As shown, based on the first feature distribution matrix, the second feature distribution matrix, ... the Mth feature distribution matrix, the feature distribution calculation result is obtained, and the method of the present application includes:

[0066] S73331: vertically stacking the first characteristic distribution matrix, the second characteristic distribution matrix, ... the Mth characteristic distribution matrix according to the damage parameter characteristics and synthesizing a first damage overall characteristic distribution matrix;

[0067] S73332: stack the first feature distribution matrix, the second feature distribution matrix, ... the Mth feature distribution matrix horizontally according to the concurrent parameter feature and synthesize them into a first concurrent overall feature distribution matrix;

[0068] S73333: Calculate a first characteristic distribution characteristic factor through the first damage overall characteristic distribution matrix and the first concurrent overall characteristic distribution matrix;

[0069] S73334: Calculate a second characteristic distribution characteristic factor with reference to the first characteristic distribution characteristic factor, and obtain the characteristic distribution calculation result through the first characteristic distribution characteristic factor and the second characteristic distribution characteristic factor.

[0070] According to the injury parameter characteristics (such as the wavelength, power density, illumination duration and other related light source parameter characteristics of different light sources (laser, LED, etc.) corresponding to phototoxicity and side effects), the first feature distribution matrix to the Mth feature distribution matrix are stacked vertically, and for each sample, the data of each feature distribution matrix in the injury parameter dimension are superimposed in sequence to form a multidimensional array or long vector, so as to construct the first injury overall feature distribution matrix; similarly, according to the concurrent parameter characteristics (such as the wavelength, power density, illumination duration and other related light source parameter characteristics of different light sources (laser, LED, etc.) corresponding to complications), the first feature distribution matrix to the Mth feature distribution matrix are stacked horizontally, and the feature distribution matrices under different concurrent states under the same injury parameter are merged by column to form a new two-dimensional matrix, namely, the first concurrent overall feature distribution matrix;

[0071] By analyzing the first damage overall characteristic distribution matrix and the first concurrent overall characteristic distribution matrix, extracting and calculating the first characteristic distribution characteristic factor reflecting the internal laws and characteristics, wherein the first characteristic distribution characteristic factor includes various indicators in statistics, such as mean, variance, covariance, entropy, etc.; referring to the first characteristic distribution characteristic factor that has been obtained, using similar or related calculation methods, digging and calculating the second characteristic distribution characteristic factor from other perspectives or at a deeper level, so as to fully understand and quantify the characteristic distribution;

[0072] Combine the first characteristic distribution characteristic factor with the second characteristic distribution characteristic factor, and perform weighted fusion on the first characteristic distribution characteristic factor and the second characteristic distribution characteristic factor through objective weighting methods such as the coefficient of variation method to calculate and obtain the characteristic distribution calculation result. The influence of damage and concurrent state on the characteristic distribution is fully considered, and the quantitative accuracy and comprehensiveness of the characteristic distribution are improved through multi-perspective and in-depth mining.

[0073] Further, by using the primary image targeting enhancement parameters, the P initialization reaction data sets corresponding to the initial state of the endoscopic image processing module are cross-updated to obtain P test reaction data sets, and the cycle is continued until the termination condition is met. The method of the present application includes:

[0074] S81: cross-update a first initialization reaction data set among P initialization reaction data sets corresponding to the initial state of the endoscopic image processing module to obtain a first test reaction data set, and repeat to obtain P test reaction data sets;

[0075] S82: generating salient change features based on the P test response data sets, wherein the salient change features include contrast features, image texture features, and color characteristics;

[0076] S83: Determine whether to continue the loop optimization according to the termination condition corresponding to the preset optimization target.

[0077] Based on the first initialization reaction data set among the P initialization reaction data sets corresponding to the initial state of the endoscopic image processing module, the one-time image targeting enhancement parameters are applied to perform image targeting enhancement. Image targeting enhancement includes adjusting various attributes of the image, such as brightness, contrast, sharpness, color balance, etc., to improve the image quality and lesion recognition ability. The processed results constitute the first test reaction data set.

[0078] After that, repeat P-1 to adjust the remaining (P-1) initialization reaction data sets one by one, and generate a new test reaction data set after each processing until the cross-update of all P sets is completed; perform feature extraction on the P test reaction data sets obtained after the cross-update to generate prominent change features, which include but are not limited to contrast features (by comparing the grayscale difference of the image before and after the processing), image texture features (such as edge clarity, patch distribution, etc.), and color characteristics (such as changes in hue and saturation, etc.);

[0079] Based on the preset optimization goal, the generated prominent change features are used for evaluation; it is determined whether the current P test reaction data sets meet the preset termination conditions, which may be reaching a certain image quality improvement standard, feature improvement threshold or iteration number limit, etc.; if the termination conditions are not met, the new or adjusted image targeted enhancement parameters are used to perform a new round of cross-update optimization on all the initialization reaction data sets until the termination conditions are met.

[0080] Furthermore, the P initialization reaction data sets corresponding to the initial state of the endoscopic image processing module are cross-updated to obtain P test reaction data sets. The present application also includes:

[0081] After activating the image targeting enhancement command, synchronous monitoring is performed to acquire real-time image sequences;

[0082] Receive demand information, and perform image enhancement and distortion comparison against the real-time image sequence to determine whether to issue an interrupt instruction;

[0083] If the interruption instruction is issued, cross-update backtracking is performed using the time information of the interruption instruction.

[0084] Start the endoscope image processing module and activate the image targeting enhancement instruction. While the enhancement instruction is activated, synchronously monitor and acquire the image sequence collected by the endoscope in real time; receive and cache continuous image frames from the endoscope in real time to form a real-time image sequence; receive specific demand information proposed by the user, which includes requirements for detail retention, processing speed, etc.; perform image enhancement distortion comparison between the test reaction data set currently being cross-updated and the received real-time image sequence, calculate the difference between the enhancement effect and the original image, and whether it meets the requirements in the demand information.

[0085] If the image enhancement process is found to be excessively distorted and does not meet the diagnostic requirements, it will automatically determine whether to issue an interruption instruction; when an interruption instruction is received, the specific time information of the interruption instruction is recorded; this time information is used as a reference point to achieve rapid backtracking during the cross-update process. Specifically, according to the recorded time point, the last valid test reaction data set state before the interruption is restored, or the corresponding intermediate result is selected according to the timestamp to ensure that the image processing process can be restarted from the correct position, avoiding further propagation and enhancement of invalid results, ensuring that various emergencies can be flexibly responded to when the image processing module is cross-updated, and effectively guaranteeing image quality.

[0086] In summary, the beneficial effects of the embodiments of the present application are:

[0087] 1. By integrating the photodynamic response database and the treatment effect evaluation database, and using advanced image preprocessing technology and three-dimensional reconstruction algorithms, it is possible to generate a three-dimensional restored structure of the lesion area and achieve real-time tracking of the photodynamic therapy effect.

[0088] 2. Quickly find the optimal enhancement parameters within the preset time window and perform targeted image enhancement, thereby dynamically adjusting the output effect of the display terminal.

[0089] 3. After activating the image targeted enhancement command, the real-time image sequence is monitored synchronously; the demand information is received, and the image enhancement distortion is compared with the real-time image sequence to determine whether to issue an interruption command; if an interruption command is issued, cross-update backtracking is performed through the moment information of the interruption command. It ensures that the image processing process can be restarted from the correct position to avoid further propagation and enhancement of invalid results, and ensures that various emergencies can be flexibly responded to when the image processing module is cross-updated, effectively ensuring image quality.

[0090] Embodiment 2

[0091] like Figure 3 FIG. 1 is a schematic diagram of the structure of an endoscope imaging system provided in an embodiment of the present application. Figure 3 The endoscopic imaging system shown is only an example and should not limit the functions and scope of use of the embodiments of the present invention. Figure 3 As shown, the endoscope imaging system includes a processor 21, a memory 22, an input device 23 and an output device 24; wherein the number of the processor 21 can be one or more, Figure 3 Taking a processor 21 as an example, the processor 21, the memory 22, the input device 23 and the output device 24 may be connected via a bus or other means. Figure 3 The example of connecting through bus is taken in the following.

[0092] The memory 22 can be used to store software programs, computer executable programs and modules, such as program instructions / modules corresponding to the endoscopic image processing method based on photodynamic imaging in the embodiment of the present application. The processor 21 executes various functional applications and data processing of the computer device by running the software programs, instructions and modules stored in the memory 22, that is, realizing the above-mentioned endoscopic image processing method based on photodynamic imaging.

[0093] It should be noted that the above-mentioned sequence of the embodiments of the present application is only for description and does not represent the advantages and disadvantages of the embodiments. And the above-mentioned specific embodiments of this specification are described. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0094] In summary, any step can be stored as a computer instruction or program in an unlimited computer memory, and can be called and recognized by an unlimited computer processor, and no unnecessary restrictions are made here.

[0095] Furthermore, the above-mentioned technical scheme only reflects the preferred technical scheme of the technical scheme of the embodiment of the present application. Some changes that may be made to some parts thereof by technicians in this technical field all reflect the novel principles of the embodiment of the present application. Obviously, technicians in this field can make various changes and modifications to the present application without departing from the scope of the present application.

Claims

1. An endoscopic image processing method based on photodynamic imaging, characterized in that: The method comprises: An endoscopic device using photodynamic imaging technology collects the lesion area to obtain original image information, performs image preprocessing on the original image information, and performs three-dimensional reconstruction to obtain a three-dimensional restored structure, wherein the original image information includes image information corresponding to the photodynamic reaction information; Based on the photodynamic reaction database, P initialization reaction data sets are set according to the reaction type, and the initialization reaction data sets include the injection situation of the photodynamic agent, the irradiation condition of the light source, and the exposure time; Based on the treatment effect evaluation database, Q historical reaction data sets are set according to the treatment effect, and the historical reaction data sets include image information of the early stage of treatment, image information of the middle stage of treatment, and image information of the late stage of treatment; After receiving the imaging request, based on the P initialization response data sets, an initial state of the endoscope image processing module is configured, and a three-dimensional display terminal corresponding to the three-dimensional restoration structure is connected; Extracting photodynamic reaction features based on image information corresponding to the photodynamic reaction information in the preprocessed image information, wherein the photodynamic reaction features include fluorescence intensity features, signal distribution features, and edge features; Connecting to the treatment effect evaluation database, combining the photodynamic reaction characteristics to perform simultaneous evaluation and prediction of the treatment effect, obtaining the prediction and evaluation results, and judging whether image targeted enhancement is needed based on the prediction and evaluation results; If image targeted enhancement is required, at the starting window time point in the preset time window, the image targeted enhancement instruction is activated in combination with the imaging request, and an iterative optimization search is performed in the optimization space corresponding to the Q historical reaction data sets to obtain an image targeted enhancement parameter; Through the one-time image targeting enhancement parameters, the P initialization reaction data sets corresponding to the initial state of the endoscopic image processing module are cross-updated to obtain P test reaction data sets. The cycle continues until the termination condition is met, and image targeting enhancement is performed at the cutoff window time point in the preset time window and visualized output is performed on the three-dimensional display terminal.

2. The endoscopic image processing method based on photodynamic imaging according to claim 1, characterized in that: Performing iterative optimization search in the optimization space corresponding to the Q historical reaction data sets, the method comprises: In the endoscopic image processing module, historical poor quality scores corresponding to historical evaluation results are performed according to the treatment effect evaluation database to obtain M poor quality score data sets; According to the M low-quality score data sets, N core low-quality parameter features are extracted, and the search probability is set according to the M low-quality scores corresponding to the M low-quality score data sets; Based on the N core inferior parameter features, the M inferior score data sets and the search probability, data clustering is performed again according to data association to establish an optimization space.

3. The endoscopic image processing method based on photodynamic imaging according to claim 2, characterized in that: Based on the N core inferior parameter features, the M inferior scoring data sets and the search probability, data clustering is performed again according to data association to establish an optimization space, and the method includes: Performing an optimal space search configuration based on the M low-quality score data sets and the search probability to obtain a search blocking index parameter, wherein the search blocking index parameter includes a search minimum interval index parameter and a blocking search boundary constraint index parameter; Perform an optimization configuration on the optimization space by using the search barrier index parameter; Based on the N core inferior parameter features, feature distribution calculation is performed in combination with the M inferior score data sets, and a secondary optimization configuration is performed on the optimization space according to the feature distribution calculation result.

4. The endoscopic image processing method based on photodynamic imaging according to claim 3, characterized in that: Based on the N core inferior parameter features, in combination with the M inferior scoring data sets, feature distribution calculation is performed, and secondary optimization configuration is performed on the optimization space according to the feature distribution calculation result. The method includes: Taking the N core inferior parameter features as the center, performing distribution feature analysis based on the first inferior score data set, and constructing a first feature distribution matrix; Traversing the M low-quality rating data sets to obtain a first feature distribution matrix, a second feature distribution matrix, ... an Mth feature distribution matrix; Based on the first feature distribution matrix, the second feature distribution matrix, ... the Mth feature distribution matrix, the feature distribution calculation result is obtained.

5. The endoscopic image processing method based on photodynamic imaging according to claim 4, characterized in that: Based on the first feature distribution matrix, the second feature distribution matrix, ... the Mth feature distribution matrix, obtaining the feature distribution calculation result, the method comprising: According to the damage parameter characteristics, the first characteristic distribution matrix, the second characteristic distribution matrix, ... the Mth characteristic distribution matrix are stacked vertically to synthesize a first damage overall characteristic distribution matrix; According to the concurrent parameter characteristics, the first feature distribution matrix, the second feature distribution matrix, ... the Mth feature distribution matrix are stacked horizontally and synthesized into a first concurrent overall feature distribution matrix; Calculating a first characteristic distribution characteristic factor through the first damage overall characteristic distribution matrix and the first concurrent overall characteristic distribution matrix; The second characteristic distribution characteristic factor is calculated with reference to the first characteristic distribution characteristic factor, and the characteristic distribution calculation result is obtained through the first characteristic distribution characteristic factor and the second characteristic distribution characteristic factor.

6. The endoscopic image processing method based on photodynamic imaging according to claim 1, characterized in that: By using the primary image targeting enhancement parameters, cross-updating P initialization reaction data sets corresponding to the initial state of the endoscopic image processing module to obtain P test reaction data sets, and continuing the cycle until a termination condition is met, the method includes: Cross-update a first initialization reaction data set among P initialization reaction data sets corresponding to the initial state of the endoscopic image processing module to obtain a first test reaction data set, and repeat to obtain P test reaction data sets; Based on the P test response data sets, generating prominent change features, the prominent change features including contrast features, image texture features, and color characteristics; According to the termination condition corresponding to the preset optimization target, determine whether it is necessary to continue the loop optimization.

7. The endoscopic image processing method based on photodynamic imaging according to claim 6, characterized in that: Cross-updating P initialization reaction data sets corresponding to the initial state of the endoscopic image processing module to obtain P test reaction data sets, the method further comprising: After activating the image targeting enhancement command, synchronous monitoring is performed to acquire real-time image sequences; Receive demand information, and perform image enhancement and distortion comparison against the real-time image sequence to determine whether to issue an interrupt instruction; If the interruption instruction is issued, cross-update backtracking is performed using the time information of the interruption instruction.

8. An endoscopic imaging system, characterized in that: The endoscopic imaging system comprises: A memory for storing executable instructions; The processor is used to implement the steps of the endoscopic image processing method based on photodynamic imaging as described in any one of claims 1 to 7 when executing the executable instructions stored in the memory.

Images