Confocal microendoscope image processing method and device, medium and terminal

Abstract

The invention discloses a confocal microendoscope image processing method and device, a medium and a terminal. The method comprises the following steps: acquiring a reference optical fiber end face image and a plurality of frames of test optical fiber end face images; aligning each frame of test optical fiber end face image by using odd-even line alignment and / or line-by-line alignment; and positioning a first fiber core point of the reference optical fiber end face image and a second fiber core point of the alignment image, and fitting a distortion correction model based on a matching result of the two so as to realize distortion correction of the optical fiber end face image. According to the method, odd-even row alignment is carried out on images with unobvious dislocation, odd-even row and line-by-line alignment is carried out on images with obvious dislocation in sequence, then fiber core point matching is carried out on the aligned images and a standard optical fiber end face image when the aligned images leave a factory, and therefore a distortion correction model is fitted to correct a subsequent optical fiber end face image. According to the method, the processing efficiency is considered while the alignment and distortion correction effects are ensured, the image processing precision is improved, and clearer and more reliable image basis is provided for clinical diagnosis.

Description

Technical Field

[0001] The present invention relates to the field of confocal microendoscopy, and in particular to a confocal microendoscopy image processing method, device, medium and terminal. Background Art

[0002] A confocal endoscope is a medical device that can be inserted into the human body through channels such as a gastroscope and colonoscope to obtain local histological images to achieve accurate diagnosis of tiny lesions, gastrointestinal lesions and early gastrointestinal cancer.

[0003] During the use of confocal endoscopes, due to hardware and algorithm problems, misalignment and distortion are inevitable, such as Figure 1 As shown, it is necessary to correct the misalignment and distortion, otherwise the detected image will not match the actual shape of the object. If such an image is used in clinical practice, it will provide users with wrong information, which will lead to incorrect diagnosis results. Summary of the Invention

[0004] The present invention provides a confocal microendoscopic image processing method, device, medium and terminal, which solve the above-mentioned technical problems.

[0005] The technical solution of the present invention to solve the above technical problems is as follows: A first aspect of an embodiment of the present invention provides a confocal microendoscopy image processing method, comprising the following steps: Step 1: Obtain a reference fiber end face image of the confocal endoscope when it leaves the factory and a multi-frame test fiber end face image in the current scene; Step 2, aligning each frame of the test optical fiber end face image using a preset odd-even row alignment method and / or a preset row-by-row alignment method to generate an optical fiber alignment image; Step 3: locating a first core point of the reference optical fiber end face image and a second core point of the optical fiber alignment image, and fitting a distortion correction model based on a matching result of the first core point and the second core point; Step 4: Correct the real-time optical fiber end face image under the same scene based on the distortion correction model to generate an optimized image.

[0006] A second aspect of an embodiment of the present invention provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the above-mentioned confocal microendoscopy image processing method is implemented.

[0007] A third aspect of an embodiment of the present invention provides a confocal microendoscopy image processing terminal, comprising a computer-readable storage medium and a processor, wherein the processor implements the steps of the above-mentioned confocal microendoscopy image processing method when executing the computer program on the computer-readable storage medium.

[0008] A fourth aspect of the embodiments of the present invention provides a confocal microendoscopic image processing device, comprising an acquisition module, an alignment module, a model fitting module, and a distortion correction module. The acquisition module is used to acquire the reference optical fiber end face image of the confocal microendomicroscope when it leaves the factory and the multi-frame test optical fiber end face image in the current scene; The alignment module is used to align each frame of the test optical fiber end face image using a preset odd-even row alignment method and / or a preset row-by-row alignment method to generate an optical fiber alignment image; The model fitting module is used to locate the first core point of the reference optical fiber end face image and the second core point of the optical fiber alignment image, and fit the distortion correction model based on the matching result of the first core point and the second core point; The distortion correction module is used to correct the real-time optical fiber end face image in the same scene based on the distortion correction model to generate an optimized image.

[0009] Beneficial effects of the present invention: The embodiments of the present invention provide a confocal microendoscopic image processing method, device, medium and terminal, which adopts an odd-even row alignment method for optical fiber end face images with less obvious misalignment, and adopts an odd-even row alignment and a row-by-row alignment method in sequence for optical fiber end face images with more obvious misalignment, and then matches the core points of the aligned image with the factory reference optical fiber end face image, thereby fitting a distortion correction model to correct subsequent optical fiber end face images, taking into account processing efficiency while ensuring the alignment and distortion correction effects, improving the accuracy of image processing, and providing clearer and more reliable imaging basis for clinical diagnosis.

[0010] In order to make the above-mentioned objects, features and advantages of the invention more obvious and easy to understand, preferred embodiments of the present invention are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0012] Figure 1 The optical fiber end face image in the prior art has obvious dislocation and distortion; Figure 2 1 is a flow chart of a confocal microendoscopy image processing method provided in Example 1; Figure 3 is an image of the optical fiber end face after alignment and distortion correction using the method of Example 1; Figure 4 is a schematic structural diagram of a confocal microendoscopic image processing device provided in Example 2; Figure 5 This is a schematic diagram of the structure of the confocal microendoscopy image processing terminal provided in Example 3. DETAILED DESCRIPTION

[0013] In order to make the purpose, technical solution and beneficial technical effects of the present invention more clear, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described in this specification are only for the purpose of explaining the present invention and are not intended to limit the present invention.

[0014] Figure 2 : is a flow chart of the confocal microendoscopy image processing method provided in Example 1, comprising the following steps: Step 1: Obtain a reference fiber end face image of the confocal endoscope when it leaves the factory and a multi-frame test fiber end face image in the current scene; Step 2, aligning each frame of the test optical fiber end face image using a preset odd-even row alignment method and / or a preset row-by-row alignment method to generate an optical fiber alignment image; Step 3: locating a first core point of the reference optical fiber end face image and a second core point of the optical fiber alignment image, and fitting a distortion correction model based on a matching result of the first core point and the second core point; Step 4: Correct the real-time optical fiber end face image under the same scene based on the distortion correction model to generate an optimized image.

[0015] The above embodiment provides a confocal microendoscopic image processing method, which adopts the odd-even row alignment method for the fiber end face image with less obvious misalignment, and adopts the odd-even row alignment and row-by-row alignment methods in sequence for the fiber end face image with more obvious misalignment. The aligned image is then matched with the factory reference fiber end face image for fiber core point matching, so as to fit the distortion correction model to correct the subsequent fiber end face images. While ensuring the alignment and distortion correction effects, the processing efficiency is taken into account, the accuracy of image processing is improved, and a clearer and more reliable image basis is provided for clinical diagnosis.

[0016] The specific technical solutions and technical effects of the above steps are described below through specific embodiments.

[0017] Exemplarily, in a preferred embodiment, two image databases can be set up, one of which is a permanent database that stores the reference fiber end face image and its fiber end face data when the confocal microendoscopy is shipped from the factory, mainly including the first pixel coordinates of each first fiber core point in the reference fiber end face image and the first fiber core center coordinates of all first fiber core points. This is because the fiber end face image when shipped from the factory is a standard image that can be used as a reference benchmark with no or low distortion. At the same time, the coordinates of each first fiber core point have been accurately marked and can be retrieved through a pre-stored coordinate file, thereby serving as a benchmark for comparison with the real-time fiber end face image to achieve distortion correction. The other database is a continuously updated temporary database used to store the original fiber end face images collected in real time and the individual fiber end face images during odd-even row alignment and / or row-by-row alignment.

[0018] As is well known in the art, laser confocal systems employ a point-by-point imaging approach. The laser scans the target line by line within the scanning plane, while the photomultiplier tube outputs a voltage signal representing the intensity of the fluorescence excited by the target. When the laser spot begins scanning a line, the horizontal galvanometer assembly emits a line synchronization pulse to indicate the start of the line scan. The sampling system begins collecting data for each line, starting with the line synchronization signal. Finally, the host computer's image display software stitches the sampled data from each line sequentially into an image, resulting in a complete image frame.

[0019] The line synchronization pulse emitted by the horizontal galvanometer assembly is used to indicate the start of scanning each line. If each line of data is collected using this signal, theoretically the spliced ​​image should be consistent with the actual scene. However, due to errors in the synchronization pulse generation circuit, image acquisition module, etc. in the system, the two are not completely matched. Therefore, when data is collected at the beginning of each line, the actual scanning position of the laser is not completely aligned in the vertical direction of the scan, which will cause the final image to deviate from the actual scene in the vertical direction. Therefore, it is necessary to use the preset odd-even row alignment method and / or the preset row-by-row alignment method in step 2 to align the test fiber end face image of each frame.

[0020] Exemplarily, in a preferred embodiment, a preset odd-even row alignment method is used to align each frame of the test fiber end face image, specifically: Extracting odd-numbered rows of pixels from the image of the end face of the test optical fiber to form a first image, and extracting even-numbered rows of pixels to form a second image, and calculating a pixel misalignment distance between the first image and the second image; The test optical fiber end face images are aligned respectively according to the pixel misalignment distance, so that the center points of the first image and the second image are aligned or the end points on one side are aligned.

[0021] If the misalignment is not obvious, the above-mentioned odd-even row alignment method can achieve the alignment effect. However, for the misalignment of the fiber end face image, the alignment effect of the above-mentioned method alone is difficult to achieve the required accuracy. Therefore, it is necessary to continue to perform the row-by-row alignment method on this basis. For example, in a preferred embodiment, a preset row-by-row alignment method is used to align each frame of the test fiber end face image, specifically: Storing multiple frames of test optical fiber end face images in a temporary database, and calculating the cumulative sum corresponding to each row of data in the multiple frames of test optical fiber end face images through an independent storage space; Calculating the mean of the cumulative sum of each row according to the number of frames of the test optical fiber end face image, and performing the same number of forward and reverse pixel shift processing on the mean to generate multiple rows of expanded data corresponding to each row of data; A new round of second test fiber end face images stored in the temporary database is obtained, and a target offset of each row of data in the second test fiber end face image is calculated based on the corresponding row expansion data, so as to align the second test fiber end face images row by row according to the target offset, thereby improving the alignment effect.

[0022] The above embodiment first sets up a temporary database. For example, if N frames of images are to be collected, in addition to setting up N blocks of frame data storage space, an independent cumulative space is also set up. This cumulative space is used to calculate the cumulative sum of each row of data in multiple frames of test fiber end face images. Then, the cumulative sum mean of each row is calculated, that is, the cumulative sum / N, and the same number of forward and reverse pixel offset processing is performed on the mean, such as performing 0-L unit offsets in the forward and reverse directions respectively, to obtain a total of 2L+1 rows of expanded data. When a new round of second test fiber end face images is stored in the temporary database, the difference between each row of data of the second test fiber end face image and each corresponding row of expanded data is calculated, and the minimum difference is used as the target offset of the row data, thereby obtaining the target offset of each row of data. Then, each row of data is offset according to the corresponding target offset to obtain the row-by-row alignment result. This row-by-row alignment method has a larger computational complexity than the odd-even row alignment method, but it can achieve row-by-row alignment, so the alignment effect is better.

[0023] In the above preferred embodiment, the test fiber end face image stored in the temporary database is the test fiber end face image processed by the preset odd-even row alignment method. In other embodiments, the above row-by-row alignment method can also be used alone. In this case, the test fiber end face image stored in the temporary database is the original test fiber end face image, and the subsequent row-by-row processing method is the same, which will not be repeated here.

[0024] A preferred embodiment of the confocal endomicroscopy image processing method further includes an alignment method selection step, specifically: Sequentially adopting a preset even-odd row alignment method and a preset row-by-row alignment method; if the target offset corresponding to the preset row-by-row alignment method is less than a first preset value, all subsequent optical fiber end face images under the same scene adopt the preset even-odd row alignment method; If the target offset is greater than a second preset value, a warning instruction is generated to remind the technician to check the current model parameters of the preset odd-even row alignment method; If the target offset is between a first preset value and a second preset value, subsequent optical fiber end face images in the same scene all adopt the preset odd-even row alignment method and the preset row-by-row alignment method in sequence; the second preset value is greater than the first preset value.

[0025] In the above preferred embodiment, the parity alignment and row-by-row alignment methods can be used separately or in combination. For example, if the parity row + row-by-row method is adopted, any target offset calculated row by row is less than the first preset value, which means that the parity row alignment scheme has achieved the alignment effect in this scenario, that is, the misalignment in this scenario is small. In the same scenario, only the parity row alignment scheme can be used in the future. Here, the first preset value is set based on empirical values. If the target offset is greater than the second preset value (the second preset value is greater than the first preset value), it means that the parity row alignment has no effect. At this time, an early warning can be issued to remind the technician to check whether the current model parameters of the parity row alignment method are correct. Here, the second preset value can be set to only use the row-by-row alignment method to calculate the offset value. If the target offset is between the first preset value and the second preset value, the parity row + row-by-row alignment scheme is adopted, taking into account both the alignment effect and efficiency.

[0026] As known to those skilled in the art, confocal endoscopes generally use sampling at equal time intervals during scanning. Due to the reciprocating and sinusoidal characteristics of the resonant mirror during scanning, and the angular velocity of the sinusoidal characteristic during scanning is slow at the edges and fast in the middle, the overall distortion is caused by stretching at both ends and compression in the middle. For example, a normal image of the end face of an optical fiber is a perfect circle, while a distorted image may be an ellipse. The distortion causes the obtained image to be inconsistent with the actual shape of the object. If such an image is used in clinical practice, it will provide users with incorrect information, which in turn leads to incorrect diagnostic results. Therefore, distortion correction needs to be performed based on alignment.

[0027] In a preferred embodiment, step 3 of fitting the distortion correction model specifically includes: The optical fiber end face data corresponding to the reference optical fiber end face image is obtained, including the first fiber core center coordinates and the first pixel coordinates of each first fiber core point. As described above, the data can be obtained by reading the factory coordinate file.

[0028] The optical fiber alignment image is then filtered, and image processing techniques such as edge detection method, connected region analysis method, template matching method, etc. are used to identify each second fiber core point, and the second pixel coordinates of each second fiber core point are recorded, and the second fiber core center coordinates are generated.

[0029] Comparing the first fiber core center coordinates with the second fiber core center coordinates, and since the images are aligned row by row, searching for matching points of the second fiber core point in the fiber alignment result in the reference fiber end face image according to each row number, and establishing a mapping relationship to form a plurality of point pairs; A displacement vector is calculated for each point pair based on the coordinates, and a displacement function of the fiber alignment image relative to the reference fiber endface image is fitted using these displacement vectors to serve as the distortion correction model. In a preferred embodiment, this displacement function can employ a polynomial distortion model, which has a small number of parameters and is computationally efficient, making it well-suited for rapid and stable global correction of a large number of fiber core points.

[0030] Finally, the aligned pixels are adjusted or remapped according to the fitted distortion correction model to generate an optimized image, such as Figure 3 In other embodiments, the distortion correction model can also correct the real-time optical fiber end face image after row-by-row alignment in the same scene to generate an optimized image, which will not be described in detail here.

[0031] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0032] An embodiment of the present invention further provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-mentioned confocal microendoscopy image processing method.

[0033] Figure 4 Schematic diagram of the structure of the confocal microendoscopic image processing device provided in Example 2. Figure 4 As shown, it includes an acquisition module 100, an alignment module 200, a model fitting module 300 and a distortion correction module 400. The acquisition module 100 is used to acquire the reference fiber end face image of the confocal microendoscope when it leaves the factory and the multi-frame test fiber end face image in the current scene; The alignment module 200 is used to align each frame of the test optical fiber end face image using a preset odd-even row alignment method and / or a preset row-by-row alignment method to generate an optical fiber alignment image; The model fitting module 300 is used to locate the first core point of the reference optical fiber end face image and the second core point of the optical fiber alignment image, and fit the distortion correction model based on the matching result of the first core point and the second core point; The distortion correction module 400 is used to correct the real-time optical fiber end face image in the same scene based on the distortion correction model to generate an optimized image.

[0034] The above embodiment provides a confocal microendoscopic image processing device, which adopts the odd-even row alignment method for the fiber end face image with less obvious misalignment, and adopts the odd-even row alignment and row-by-row alignment methods in sequence for the fiber end face image with more obvious misalignment. Then, the aligned image and the factory reference fiber end face image are matched with the fiber core points, so as to fit the distortion correction model to correct the subsequent fiber end face images. While ensuring the alignment and distortion correction effects, the processing efficiency is taken into account, the accuracy of image processing is improved, and a clearer and more reliable image basis is provided for clinical diagnosis.

[0035] In a preferred embodiment, the alignment module 200 includes a parity alignment unit, and the parity alignment unit specifically includes: A first calculation unit is configured to extract odd-numbered rows of pixels from the image of the end face of the test optical fiber to form a first image, and extract even-numbered rows of pixels to form a second image, and calculate a pixel misalignment distance between the first image and the second image; The first alignment unit is configured to align the test optical fiber end face images respectively according to the pixel offset distance, so that the center points of the first image and the second image are aligned or the end points on one side are aligned.

[0036] In a preferred embodiment, the alignment module 200 includes a row-by-row alignment unit, and the row-by-row alignment unit specifically includes: A second calculation unit is used to store the multiple frames of test optical fiber end face images in a temporary database, and calculate the cumulative sum corresponding to each row of data in the multiple frames of test optical fiber end face images through an independent storage space; an offset unit, configured to calculate a mean of the accumulated sums of each row according to the number of frames of the test optical fiber end face image, and perform the same number of forward and reverse pixel offset processes on the mean to generate a plurality of rows of expanded data corresponding to each row of data; The second alignment unit is used to obtain a new round of second test fiber end face images stored in the temporary database, and calculate the target offset of each row of data in the second test fiber end face image according to the corresponding row expansion data, so as to align the second test fiber end face images row by row according to the target offset.

[0037] Exemplarily, in a preferred embodiment, the second alignment unit is used to obtain each row of data of the second test optical fiber end face image, calculate the difference between it and each corresponding row of expanded data, and use the minimum difference as the target offset of the row data.

[0038] Exemplarily, in a preferred embodiment, the image processing device further includes an alignment method selection module for sequentially adopting a preset odd-even row alignment method and a preset row-by-row alignment method; if the target offset corresponding to the preset row-by-row alignment method is less than a first preset value, subsequent optical fiber end face images under the same scene all adopt the preset odd-even row alignment method; if the target offset is greater than a second preset value, an early warning instruction is generated to remind technicians to check the current model parameters of the preset odd-even row alignment method; if the target offset is between the first preset value and the second preset value, subsequent optical fiber end face images under the same scene all adopt the preset odd-even row alignment method and the preset row-by-row alignment method in sequence; the second preset value is greater than the first preset value.

[0039] Exemplarily, in a preferred embodiment, the model fitting module 300 specifically includes: an acquisition unit, configured to acquire optical fiber end face data corresponding to the reference optical fiber end face image, including the first fiber core center coordinates and the first pixel coordinates of each first fiber core point; an identification unit, configured to filter the optical fiber alignment image, identify the second pixel coordinates of each second fiber core point, and generate the second fiber core center coordinates; A search unit, configured to locate the coordinates of the center of the first fiber core and the center of the second fiber core, search for matching points of the second fiber core point in the reference optical fiber end face image according to each row number, and establish a mapping relationship to form a plurality of point pairs; A fitting unit is used to calculate the displacement vector of each point pair according to the coordinates, and fit the displacement function of the optical fiber alignment image relative to the reference optical fiber end face image through a plurality of displacement vectors to serve as the distortion correction model.

[0040] An embodiment of the present invention also provides a confocal microendoscopy image processing terminal, comprising a computer-readable storage medium and a processor, wherein the processor implements the steps of the above-mentioned confocal microendoscopy image processing method when executing the computer program on the computer-readable storage medium. Figure 5 Schematic diagram of the structure of the confocal microendoscopic image processing terminal provided by Example 3 of the present invention, as shown in FIG. Figure 5As shown, the confocal microendoscopic image processing terminal 8 of this embodiment includes: a processor 80, a readable storage medium 81, and a computer program 82 stored in the readable storage medium 81 and executable on the processor 80. When the processor 80 executes the computer program 82, the steps in the above-mentioned various method embodiments are implemented, for example Figure 2 Alternatively, when the processor 80 executes the computer program 82, the functions of the modules in the above-mentioned device embodiments are realized, for example Figure 4 Functionality of the modules shown.

[0041] Exemplarily, the computer program 82 may be divided into one or more modules, which are stored in the readable storage medium 81 and executed by the processor 80 to implement the present invention. The one or more modules may be a series of computer program instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of the computer program 82 in the confocal endomicroscopy image processing terminal 8.

[0042] The confocal microendoscopic image processing terminal 8 may include, but is not limited to, a processor 80 and a readable storage medium 81. Those skilled in the art will understand that Figure 5 It is only an example of the confocal microendoscopic image processing terminal 8 and does not constitute a limitation of the confocal microendoscopic image processing terminal 8. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the confocal microendoscopic image processing terminal may also include a power management module, an operation processing module, input and output devices, network access equipment, a bus, etc.

[0043] The processor 80 may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0044] The readable storage medium 81 can be an internal storage unit of the confocal endomicroscope image processing terminal 8, such as a hard disk or memory of the confocal endomicroscope image processing terminal 8. The readable storage medium 81 can also be an external storage device of the confocal endomicroscope image processing terminal 8, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the confocal endomicroscope image processing terminal 8. Furthermore, the readable storage medium 81 can also include both the internal storage unit of the confocal endomicroscope image processing terminal 8 and an external storage device. The readable storage medium 81 is used to store the computer program and other programs and data required by the confocal endomicroscope image processing terminal. The readable storage medium 81 can also be used to temporarily store data that has been output or is about to be output.

[0045] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0046] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0047] Those skilled in the art will appreciate that the units and method steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0048] In the embodiments provided by the present invention, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are merely illustrative. For example, the division of the modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0049] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0050] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0051] The present invention is not limited to what is described in the specification and embodiments, and additional advantages and modifications will be readily apparent to those skilled in the art. Therefore, the present invention is not limited to the specific details, representative devices, and illustrative examples shown and described herein without departing from the spirit and scope of the general concept defined by the claims and their equivalents.

Claims

1. A confocal microendoscopy image processing method, characterized in that: The following steps are involved: Step 1: Obtain a reference fiber end face image of the confocal endoscope when it leaves the factory and a multi-frame test fiber end face image in the current scene; Step 2, aligning each frame of the test optical fiber end face image using a preset odd-even row alignment method and / or a preset row-by-row alignment method to generate an optical fiber alignment image; Step 3: locating a first core point of the reference optical fiber end face image and a second core point of the optical fiber alignment image, and fitting a distortion correction model based on a matching result of the first core point and the second core point; Step 4: Correct the real-time optical fiber end face image under the same scene based on the distortion correction model to generate an optimized image.

2. The confocal endomicroscopy image processing method according to claim 1, characterized in that: The preset odd-even row alignment method is used to align each frame of the test fiber end face image, specifically: Extracting odd-numbered rows of pixels from the image of the end face of the test optical fiber to form a first image, and extracting even-numbered rows of pixels to form a second image, and calculating a pixel misalignment distance between the first image and the second image; The test optical fiber end face images are aligned respectively according to the pixel misalignment distance, so that the center points of the first image and the second image are aligned or the end points on one side are aligned.

3. The confocal endomicroscopy image processing method according to claim 1, characterized in that: Each frame of the test fiber end face image is aligned using a preset row-by-row alignment method, specifically: Storing multiple frames of test optical fiber end face images in a temporary database, and calculating the cumulative sum corresponding to each row of data in the multiple frames of test optical fiber end face images through an independent storage space; Calculating the mean of the cumulative sum of each row according to the number of frames of the test optical fiber end face image, and performing the same number of forward and reverse pixel shift processing on the mean to generate multiple rows of expanded data corresponding to each row of data; A new round of second test fiber end face images stored in the temporary database is obtained, and a target offset of each row of data in the second test fiber end face image is calculated based on the corresponding row expansion data, so as to align the second test fiber end face images row by row according to the target offset.

4. The confocal endomicroscopy image processing method according to claim 3, characterized in that: Each row of data of the second test optical fiber end face image is obtained, the difference between the row of data and the corresponding row of expanded data is calculated, and the minimum difference is used as the target offset of the row of data.

5. The confocal endomicroscopy image processing method according to claim 3, characterized in that: The test optical fiber end face image stored in the temporary database is the original test optical fiber end face image or the test optical fiber end face image processed by a preset odd-even row alignment method.

6. The confocal endomicroscopy image processing method according to any one of claims 1 to 5, characterized in that: It also includes the alignment method selection step, specifically: Sequentially adopting a preset even-odd row alignment method and a preset row-by-row alignment method; if the target offset corresponding to the preset row-by-row alignment method is less than a first preset value, all subsequent optical fiber end face images under the same scene adopt the preset even-odd row alignment method; If the target offset is greater than a second preset value, a warning instruction is generated to remind the technician to check the current model parameters of the preset odd-even row alignment method; If the target offset is between the first preset value and the second preset value, subsequent optical fiber end face images in the same scene all adopt the preset odd-even row alignment method and the preset row-by-row alignment method in sequence; The second preset value is greater than the first preset value.

7. The confocal endomicroscopy image processing method according to claim 6, characterized in that: Step 3: Fit the distortion correction model, specifically: Acquire optical fiber end face data corresponding to the reference optical fiber end face image, including the first fiber core center coordinates and the first pixel coordinates of each first fiber core point; filtering the optical fiber alignment image, identifying the second pixel coordinates of each second fiber core point, and generating the second fiber core center coordinates; Locating the coordinates of the center of the first fiber core and the center of the second fiber core, searching for matching points of the second fiber core point in the reference optical fiber end face image according to each row number, and establishing a mapping relationship to form a plurality of point pairs; The displacement vector of each point pair is calculated according to the coordinates, and a displacement function of the optical fiber alignment image relative to the reference optical fiber end face image is fitted through a plurality of displacement vectors to serve as the distortion correction model.

8. A confocal microendoscopic image processing device, based on the method according to any one of claims 1 to 7, characterized in that: Including acquisition module, alignment module, model fitting module and distortion correction module, The acquisition module is used to acquire the reference optical fiber end face image of the confocal microendomicroscope when it leaves the factory and the multi-frame test optical fiber end face image in the current scene; The alignment module is used to align each frame of the test optical fiber end face image using a preset odd-even row alignment method and / or a preset row-by-row alignment method to generate an optical fiber alignment image; The model fitting module is used to locate the first core point of the reference optical fiber end face image and the second core point of the optical fiber alignment image, and fit the distortion correction model based on the matching result of the first core point and the second core point; The distortion correction module is used to correct the real-time optical fiber end face image in the same scene based on the distortion correction model to generate an optimized image.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the confocal microendoscopy image processing method according to any one of claims 1 to 7 are implemented.

10. A confocal microendoscopic image processing terminal, comprising a computer-readable storage medium and a processor, characterized in that: When the processor executes the computer program on the computer-readable storage medium, the steps of the confocal endomicroscopy image processing method according to any one of claims 1 to 7 are implemented.

Images