Confocal microendoscopy image processing method, device, medium and terminal

Abstract

The application discloses a confocal endomicroscopy image processing method, device, medium and terminal, and the method comprises the following steps: acquiring a reference optical fiber end face image and a plurality of test optical fiber end face images; aligning each test optical fiber end face image by using odd-even row alignment and / or row-by-row alignment; positioning a first fiber core point of the reference optical fiber end face image and a second fiber core point of the aligned image, and fitting a distortion correction model based on the matching result of the two, so as to realize distortion correction of the optical fiber end face image. The application uses odd-even row alignment on images with inconspicuous misalignment, and uses odd-even row alignment and row-by-row alignment on images with obvious misalignment in turn, and then matches the fiber core points of the aligned images and the reference optical fiber end face image at the time of leaving the factory, so as to fit the distortion correction model to correct subsequent optical fiber end face images. The processing efficiency is taken into account while ensuring the alignment and distortion correction effect, the image processing precision is improved, and clearer and more reliable image data are provided for clinical diagnosis.

Description

Technical Field

[0001] This invention relates to the field of confocal microscopy, and more particularly to a confocal microscopy image processing method, apparatus, medium, and terminal. Background Technology

[0002] Confocal endoscopy is a medical device that can be inserted into the human body through channels such as gastroscopes and colonoscopes to obtain local histological images, enabling precise diagnosis of small lesions, gastrointestinal diseases, and early gastrointestinal cancers.

[0003] During use, confocal endoscopes inevitably experience misalignment and distortion due to hardware and algorithm limitations. Figure 1 As shown, misalignment and distortion need to be corrected; otherwise, the detected image will not match the actual shape of the object. If such an image is applied clinically, it will provide users with incorrect information, leading to incorrect diagnostic results. Summary of the Invention

[0004] This invention provides a method, apparatus, medium, and terminal for processing images in confocal microscopy endoscopes, which solves the technical problems mentioned above.

[0005] The technical solution of the present invention to solve the above-mentioned technical problems is as follows: The first aspect of the present invention provides a confocal endoscopic microscopy image processing method, comprising the following steps:

[0006] Step 1: Obtain the reference fiber end face image of the confocal endoscope at the time of its manufacture and multiple test fiber end face images in the current scene.

[0007] Step 2: Align each frame of the test fiber end face image using a preset odd-even row alignment method and / or a preset line-by-line alignment method to generate a fiber alignment image;

[0008] Step 3: Locate the first core point of the reference fiber end face image and the second core point of the fiber alignment image, and fit a distortion correction model based on the matching result of the first core point and the second core point;

[0009] Step 4: Correct the real-time fiber end-face image under the same scene based on the distortion correction model to generate an optimized image.

[0010] A second aspect of the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the confocal microscopy endoscopic image processing method described above.

[0011] The third aspect of the embodiment of the present application provides a confocal endomicroscopy image processing terminal, comprising a computer readable storage medium and a processor, wherein the processor implements the steps of the confocal endomicroscopy image processing method described above when executing a computer program on the computer readable storage medium.

[0012] The fourth aspect of the embodiment of the present application provides a confocal endomicroscopy image processing device, comprising an acquisition module, an alignment module, a model fitting module and a distortion correction module,

[0013] The acquisition module is configured to acquire a reference fiber end face image of the confocal endomicroscopy when leaving the factory and a plurality of test fiber end face images under a current scene;

[0014] The alignment module is configured to align each of the test fiber end face images by using a preset odd-even row alignment method and / or a preset row-by-row alignment method, and generate a fiber alignment image;

[0015] The model fitting module is configured to locate a first fiber core point of the reference fiber end face image and a second fiber core point of the fiber alignment image, and fit a distortion correction model based on a matching result of the first fiber core point and the second fiber core point;

[0016] The distortion correction module is configured to correct a real-time fiber end face image under the same scene based on the distortion correction model, and generate an optimized image.

[0017] The embodiment of the present application provides a confocal endomicroscopy image processing method, device, medium and terminal, which uses an odd-even row alignment method for a fiber end face image with inconspicuous misplacement, and sequentially uses the odd-even row alignment method and the row-by-row alignment method for a fiber end face image with obvious misplacement, and then matches the fiber core points of the aligned image and the reference fiber end face image when leaving the factory, so as to fit a distortion correction model to correct subsequent fiber end face images, which guarantees the alignment and distortion correction effects while taking into account the processing efficiency, improves the image processing precision, and provides clearer and more reliable image basis for clinical diagnosis.

[0018] In order to make the above objectives, characteristics and advantages of the present application more apparent, clear and easy to understand, the following will describe the preferred embodiments of the present application in detail, and the accompanying drawings will be referred to, as follows. BRIEF DESCRIPTION OF DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments, and it should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as a limitation to the scope, and for those skilled in the art, other related drawings can also be obtained without paying creative labor on the basis of these drawings.

[0020] Figure 1 is the fiber end face image in the prior art with obvious misalignment and distortion;

[0021] Figure 2 is a flowchart of the confocal endomicroscopy image processing method provided in Embodiment 1;

[0022] Figure 3 is the fiber end face image after alignment and distortion correction by the method in Embodiment 1;

[0023] Figure 4 is a structural diagram of the confocal endomicroscopy image processing device provided in Embodiment 2;

[0024] Figure 5 is a structural diagram of the confocal endomicroscopy image processing terminal provided in Embodiment 3. DETAILED DESCRIPTION

[0025] In order to make the objectives, technical solutions and beneficial technical effects of the present application clearer, the present application is further described in detail below in combination with the drawings and specific embodiments. It should be understood that the specific embodiments described in the present specification are only for the purpose of explaining the present application, and are not intended to limit the present application.

[0026] Figure 2 is a flowchart of the confocal endomicroscopy image processing method provided in Embodiment 1, which includes the following steps:

[0027] Step 1, obtaining a reference fiber end face image of a confocal endomicroscopy when it leaves the factory and a plurality of test fiber end face images under a current scene;

[0028] Step 2, aligning each of the test fiber end face images by using a preset odd-even row alignment method and / or a preset row-by-row alignment method to generate fiber alignment images;

[0029] Step 3, locating a first core point of the reference fiber end face image and a second core point of the fiber alignment image, and fitting a distortion correction model based on the matching results of the first core point and the second core point;

[0030] Step 4, correcting real-time fiber end face images under the same scene based on the distortion correction model to generate optimized images.

[0031] The above embodiment provides a confocal endomicroscopy image processing method, for the fiber end face image with inconspicuous misalignment, the odd-even row alignment method is adopted, for the fiber end face image with obvious misalignment, the odd-even row alignment and the row-by-row alignment method are adopted in turn, then the aligned image and the reference fiber end face image at the factory are matched with the fiber core points, so that the subsequent fiber end face image is corrected by the distortion correction model, the processing efficiency is considered while the alignment and distortion correction effects are ensured, the image processing precision is improved, and clearer and more reliable image basis is provided for clinical diagnosis.

[0032] The specific technical solutions and technical effects of the above steps are described below through specific embodiments.

[0033] For example, in a preferred embodiment, two image databases can be provided, one is a permanent database, which stores the reference fiber end face image and the fiber end face data of the confocal endomicroscopy at the factory, mainly including the first pixel coordinates of each first fiber core point in the reference fiber end face image and the first fiber core center coordinates of all first fiber core points. This is because the fiber end face image at the factory is a standard image, which can be used as a reference for no distortion or low distortion, and the coordinates of each first fiber core point have been accurately marked, which can be retrieved through the pre-stored coordinate file, so as to compare with the real-time fiber end face image and realize distortion correction. The other database is a temporary database for continuously updating, which is used to store the original fiber end face image collected in real time and each fiber end face image in the odd-even row alignment and / or row-by-row alignment.

[0034] As known by those skilled in the art, the laser confocal system adopts a point-by-point imaging scheme, the laser scans the target row by row in the scanning plane, and at the same time, the photomultiplier tube outputs a voltage signal representing the fluorescence intensity of the target. When the laser spot starts scanning a row, the horizontal galvanometer assembly sends out a row synchronization pulse to indicate the starting time of row scanning. The sampling system starts collecting data for each row based on the row synchronization signal as the starting time, and finally the image display software of the upper computer splices each row of sampling data in order to obtain a complete image, thereby obtaining a complete image.

[0035] The row synchronization pulse sent out by the horizontal galvanometer assembly is used to indicate the start of scanning each row, if the data of each row is collected based on this signal, theoretically, the spliced image should be consistent with the actual scene, but due to the errors of the synchronization pulse generation circuit, image acquisition module and other systems, the two do not match completely, so when collecting data for each row, the real scanning position of the laser in the vertical direction of scanning does not completely align, which will cause the final image to deviate from the real scene in the vertical direction, therefore, the preset odd-even row alignment method and / or the preset row-by-row alignment method of step 2 are used to align each frame of the test fiber end face image.

[0036] For example, in a preferred embodiment, a preset odd-even row alignment method is used to align each frame of the test fiber end face image, specifically as follows:

[0037] The odd rows of pixels of the test fiber end face image are extracted to form a first image, and the even rows of pixels are extracted to form a second image, and the pixel misalignment distance of the first image and the second image is calculated;

[0038] The test fiber end face image is aligned by the pixel misalignment distance, so that the center point positions of the first image and the second image are aligned or the one-side end point positions are aligned.

[0039] If the fiber end face image is not obviously misaligned, the above-mentioned odd-even row alignment method can achieve the alignment effect, but for the fiber end face image with obvious misalignment, the alignment effect of the above-mentioned method is difficult to meet the precision requirement, so it is necessary to continue to perform the row-by-row alignment method on this basis. For example, in a preferred embodiment, a preset row-by-row alignment method is used to align each frame of the test fiber end face image, specifically as follows:

[0040] Multiple frames of test fiber end face images are stored in a temporary database, and the cumulative sum corresponding to each row of data in the multiple frames of test fiber end face images is calculated through an independent storage space;

[0041] The mean value of each row of cumulative sum is calculated according to the number of frames of the test fiber end face image, and the same number of forward and reverse pixel offset processing is performed on the mean value to generate multiple row expansion data corresponding to each row of data;

[0042] A second test fiber end face image stored in the temporary database in a new round is obtained, and the target offset of each row of data in the second test fiber end face image is calculated according to the corresponding row expansion data, so that the row-by-row alignment of the second test fiber end face image is performed according to the target offset, thereby improving the alignment effect.

[0043] The above embodiment firstly sets up a temporary database, such as collecting N frames of images, in addition to setting up N blocks of frame data storage space, an independent accumulation space is also set up, which is used to calculate the accumulation sum of each row of data in the multi-frame test fiber end face image. Then the average of the accumulation sum of each row, that is, the accumulation sum / N, is calculated, and the same number of forward and reverse pixel offset processing is performed on the average, such as 0-L unit offset is performed forward and backward respectively, a total of 2L+1 row expansion data is obtained. When the temporary database stores a new round of second test fiber end face image, the difference between each row of data of the second test fiber end face image and the corresponding each row expansion data is calculated, and the minimum difference is taken as the target offset of the row data, so as to obtain the target offset of each row of data, and then each row of data is offset according to the corresponding target offset, that is, the row-by-row alignment result is obtained. Compared with the odd-even row alignment method, the row-by-row alignment method has larger calculation amount, but can realize each row alignment, so the alignment effect is better.

[0044] In the above preferred embodiment, the test fiber end face image stored in the temporary database is the test fiber end face image processed by the preset odd-even row alignment method. In other embodiments, the above row-by-row alignment method can also be used alone, at this time, the test fiber end face image stored in the temporary database is the original test fiber end face image, and the subsequent row-by-row processing method is the same, which will not be repeated here.

[0045] The confocal endomicroscopy image processing method of a preferred embodiment further comprises an alignment method selection step, specifically:

[0046] The preset odd-even row alignment method and the preset row-by-row alignment method are used in turn, if the target offset corresponding to the preset row-by-row alignment method is less than a first preset value, the subsequent fiber end face images under the same scene all use the preset odd-even row alignment method;

[0047] If the target offset is greater than a second preset value, a warning instruction is generated to remind the technician to check the current model parameters of the preset odd-even row alignment method;

[0048] If the target offset is between the first preset value and the second preset value, the subsequent fiber end face images under the same scene all use the preset odd-even row alignment method and the preset row-by-row alignment method in turn; the second preset value is greater than the first preset value.

[0049] In the preferred embodiments described above, the odd-even alignment and the row-by-row alignment method can be used alone or in combination. For example, if the odd-even row + row-by-row method is used, if any target offset calculated by the row-by-row method is less than a first preset value, it indicates that the odd-even row alignment scheme has achieved the alignment effect in this scenario, i.e., the misalignment is small in this scenario, and in the same scenario, only the odd-even row alignment scheme can be used subsequently. Here, the first preset value is set by experience. If the target offset is greater than a second preset value (the second preset value is greater than the first preset value), it indicates that the odd-even row alignment has no effect, and at this time, a warning can be given to remind the technician to check whether the current model parameters of the odd-even row alignment method are correct. Here, the second preset value can be set to the offset value calculated by using only the row-by-row alignment method. If the target offset is between the first preset value and the second preset value, the odd-even row + row-by-row alignment scheme is used, which takes into account the alignment effect and efficiency.

[0050] As known by those skilled in the art, confocal endoscopic scanning generally uses equal time interval sampling. Due to the reciprocating and sinusoidal characteristics in the resonant mirror scanning process, and the angular velocity of the sinusoidal characteristics is slow at the edge and fast in the middle during scanning, resulting in a distortion of stretching at both ends and compression in the middle. For example, a normal fiber end face image is a circle, but the distorted image may be an ellipse. The distortion causes the obtained image to be inconsistent with the actual shape of the object. Such an image, if applied in clinical practice, will provide incorrect information to the user, and thus lead to incorrect diagnostic results. Therefore, distortion correction is needed on the basis of alignment.

[0051] In one preferred embodiment, step 3 fits a distortion correction model, specifically comprising:

[0052] The fiber end face data corresponding to the reference fiber end face image is obtained, including the first core center coordinates and the first pixel coordinates of each first core point. As described above, the coordinates file at the time of factory shipment can be read to obtain the data.

[0053] Then, the fiber alignment image is filtered, and image processing techniques such as edge detection method, connected region analysis method, template matching method, etc. are used to identify each second core point, and the second pixel coordinates of each second core point are recorded, and the second core center coordinates are generated.

[0054] The first core center coordinates and the second core center coordinates are compared, and since the image is aligned row by row, the matching points of the second core points in the fiber alignment result in the reference fiber end face image can be searched according to the row numbers, and a mapping relationship is established to form a plurality of point pairs.

[0055] The displacement vectors of each point pair are calculated according to coordinates, and a displacement function of the fiber alignment image relative to the reference fiber end face image is fitted by a plurality of displacement vectors, to serve as the distortion correction model. In a preferred embodiment, the displacement function can adopt a polynomial distortion model, which has fewer parameters and is computationally efficient, and is very suitable for fast and stable global correction of a large number of core points.

[0056] Finally, each pixel point after row-by-row alignment is adjusted or remapped according to the fitted distortion correction model, to generate an optimized image, as shown in Figure 3 In other embodiments, the distortion correction model can also correct a real-time fiber end face image after row-by-row alignment under the same scene to generate an optimized image, which will not be described here.

[0057] It should be understood that the size of the serial number of each step in the above embodiments does not mean the order of execution, and the execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0058] The embodiments of the present application also provide a computer readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the confocal endomicroscopy image processing method described above.

[0059] Figure 4 is a structural schematic diagram of the confocal endomicroscopy image processing device provided in Embodiment 2, as shown in Figure 4 includes an acquisition module 100, an alignment module 200, a model fitting module 300, and a distortion correction module 400,

[0060] The acquisition module 100 is configured to acquire a reference fiber end face image of a confocal endomicroscopy when it leaves the factory and a plurality of test fiber end face images under a current scene;

[0061] The alignment module 200 is configured to align each of the test fiber end face images by using a preset odd-even row alignment method and / or a preset row-by-row alignment method, to generate a fiber alignment image;

[0062] The model fitting module 300 is configured to locate a first core point of the reference fiber end face image and a second core point of the fiber alignment image, and fit a distortion correction model based on a matching result of the first core point and the second core point;

[0063] The distortion correction module 400 is configured to correct a real-time fiber end face image under the same scene based on the distortion correction model, to generate an optimized image.

[0064] The above embodiment provides a confocal endomicroscopy image processing device, for the fiber end face image with inconspicuous misalignment, the odd-even row alignment method is adopted, for the fiber end face image with obvious misalignment, the odd-even row alignment and the row-by-row alignment method are adopted in turn, then the aligned image and the reference fiber end face image at the factory are matched in terms of the fiber core points, so that the distortion correction model is fitted to correct the subsequent fiber end face image, the processing efficiency is taken into account while the alignment and distortion correction effects are ensured, the image processing precision is improved, and clearer and more reliable image basis is provided for clinical diagnosis.

[0065] In a preferred embodiment, the alignment module 200 comprises an odd-even alignment unit, which specifically comprises:

[0066] A first calculation unit is configured to extract odd rows of pixels of a test fiber end face image to form a first image, extract even rows of pixels to form a second image, and calculate a pixel misalignment distance between the first image and the second image.

[0067] A first alignment unit is configured to align the test fiber end face image by the pixel misalignment distance, so that the center points of the first image and the second image are aligned or the one-side end points are aligned.

[0068] In a preferred embodiment, the alignment module 200 comprises a row-by-row alignment unit, which specifically comprises:

[0069] A second calculation unit is configured to store a plurality of test fiber end face images in a temporary database, and calculate an accumulated sum corresponding to each row of data in the plurality of test fiber end face images by using an independent storage space.

[0070] An offset unit is configured to calculate a mean value of the accumulated sum of each row according to the number of frames of the test fiber end face image, and perform a same number of forward and reverse pixel offset processing on the mean value to generate a plurality of row expansion data corresponding to each row of data.

[0071] A second alignment unit is configured to obtain a second test fiber end face image stored in the temporary database in a new round, calculate a target offset amount of each row of data in the second test fiber end face image according to the corresponding row expansion data, and align the second test fiber end face image row by row according to the target offset amount.

[0072] For example, in a preferred embodiment, the second alignment unit is configured to obtain each row of data of the second test fiber end face image, calculate a difference value between each row of data and each corresponding row expansion data, and take the minimum difference value as the target offset amount of the row of data.

[0073] In an exemplary preferred embodiment, the image processing device further comprises an alignment method selection module configured to sequentially adopt a preset odd-even row alignment method and a preset row-by-row alignment method, and if a target offset corresponding to the preset row-by-row alignment method is less than a first preset value, the preset odd-even row alignment method is adopted for subsequent fiber end face images in the same scene; if the target offset is greater than a second preset value, a warning instruction is generated to remind a technician to check the current model parameters of the preset odd-even row alignment method; and if the target offset is between the first preset value and the second preset value, the preset odd-even row alignment method and the preset row-by-row alignment method are sequentially adopted for subsequent fiber end face images in the same scene; and the second preset value is greater than the first preset value.

[0074] In an exemplary preferred embodiment, the model fitting module 300 specifically comprises:

[0075] An acquisition unit configured to acquire fiber end face data corresponding to the reference fiber end face image, including a first fiber core center coordinate and a first pixel coordinate of each first fiber core point;

[0076] An identification unit configured to filter the fiber alignment image, identify a second pixel coordinate of each second fiber core point, and generate a second fiber core center coordinate;

[0077] A search unit configured to locate the first fiber core center coordinate and the second fiber core center coordinate, search for matching points of the second fiber core points in the reference fiber end face image according to row numbers, and establish a mapping relationship to form a plurality of point pairs;

[0078] A fitting unit configured to calculate a displacement vector of each point pair according to coordinates, and fit a displacement function of the fiber alignment image relative to the reference fiber end face image through a plurality of displacement vectors, to serve as the distortion correction model.

[0079] The embodiment of the present application also provides a confocal endomicroscopy image processing terminal, which comprises a computer readable storage medium and a processor, and the processor implements the steps of the confocal endomicroscopy image processing method when executing a computer program on the computer readable storage medium. Figure 5 is a structural schematic diagram of the confocal endomicroscopy image processing terminal provided in Embodiment 3 of the present application, as Figure 5 shown, the confocal endomicroscopy image processing terminal 8 of this embodiment comprises a processor 80, a readable storage medium 81, and a computer program 82 stored in the readable storage medium 81 and executable on the processor 80. The processor 80 implements the steps in each of the method embodiments when executing the computer program 82, for example Figure 2The processor 80 implements the functions of the modules in the above-mentioned apparatus embodiments when executing the computer program 82. For example, the processor 80 implements the functions of the modules shown in FIG. 2 when executing the computer program 82. Figure 4 The processor 80 implements the functions of the modules in the above-mentioned apparatus embodiments when executing the computer program 82. For example, the processor 80 implements the functions of the modules shown in FIG. 2 when executing the computer program 82.

[0080] For example, the computer program 82 can be divided into one or more modules stored in the readable storage medium 81 and executed by the processor 80 to complete the present application. The one or more modules can be a series of computer program instruction segments capable of completing a specific function, which are used to describe the execution process of the computer program 82 in the confocal endomicroscopy image processing terminal 8.

[0081] The confocal endomicroscopy image processing terminal 8 can include, but is not limited to, the processor 80 and the readable storage medium 81. Those skilled in the art can understand that the confocal endomicroscopy image processing terminal 8 can include more or fewer components than those shown, or combine certain components, or include different components, for example, the confocal endomicroscopy image processing terminal can also include a power management module, an operation processing module, an input / output device, a network access device, a bus, etc. Figure 5 The confocal endomicroscopy image processing terminal 8 shown is only an example and does not constitute a limitation on the confocal endomicroscopy image processing terminal 8, which can include more or fewer components than those shown, or combine certain components, or include different components, for example, the confocal endomicroscopy image processing terminal can also include a power management module, an operation processing module, an input / output device, a network access device, a bus, etc.

[0082] The processor 80 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic components, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0083] The readable storage medium 81 can be an internal storage unit of the confocal endomicroscopy image processing terminal 8, for example, a hard disk or a memory of the confocal endomicroscopy image processing terminal 8. The readable storage medium 81 can also be an external storage device of the confocal endomicroscopy image processing terminal 8, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the confocal endomicroscopy image processing terminal 8. Further, the readable storage medium 81 can include both the internal storage unit and the external storage device of the confocal endomicroscopy image processing terminal 8. The readable storage medium 81 is used to store the computer program and other programs and data required by the confocal endomicroscopy image processing terminal. The readable storage medium 81 can also be used to temporarily store data that has been output or will be output.

[0084] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of each functional unit and module is exemplified, and in actual application, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of software functional unit. In addition, the specific name of each functional unit and module is only for easy distinction, and does not limit the protection scope of the present application. The specific working process of the unit and module in the above system can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0085] In the above embodiments, the description of each embodiment has its own emphasis, and the parts not described or recorded in detail in a certain embodiment can be referred to the related description of other embodiments.

[0086] Those of ordinary skill in the art can realize that the units and method steps of each example described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0087] In the embodiments of the present application, it should be understood that the disclosed apparatus / terminal device and method can be implemented in other manners. For example, the embodiments of the apparatus / terminal device described above are merely schematic, and the division of the modules or units is merely logical function division, and there can be another division manner in actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between different units, or the logical couplings or communication connections between different units, can be indirect couplings or communication connections through some interfaces, devices or units.

[0088] The units described as separated components can or can not be physically separated, and the components displayed as units can or can not be physical units, i.e., can be located in one place or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purposes of the embodiments of the present application.

[0089] In addition, each functional unit in the various embodiments of the present application can be integrated in one processing unit, or each unit can exist physically as a separate unit, or two or more units can be integrated in one unit. The integrated unit can be implemented in the form of hardware, or in the form of a software functional unit.

[0090] The present application is not limited to the description and the embodiments described in the specification, and those skilled in the art can easily implement other advantages and modifications, and therefore the present application is not limited to the specific details, representative devices and examples of the drawings shown and described herein.

Claims

1. A confocal endomicroscopy image processing method, characterized by, The method comprises the following steps: Step 1, obtaining a reference fiber end face image of a confocal endomicroscopy when leaving factory and a plurality of test fiber end face images under a current scene; Step 2, aligning each of the test fiber end face images by using a preset odd-even row alignment method and a preset row-by-row alignment method to generate a fiber alignment image; Step 3, locating a first core point of the reference fiber end face image and a second core point of the fiber alignment image, and fitting a distortion correction model based on a matching result of the first core point and the second core point; Step 4, correcting a real-time fiber end face image under the same scene based on the distortion correction model to generate an optimized image; Further comprising an alignment method selection step, specifically: sequentially using the preset odd-even row alignment method and the preset row-by-row alignment method, if a target offset corresponding to the preset row-by-row alignment method is less than a first preset value, then subsequent fiber end face images under the same scene all use the preset odd-even row alignment method; if the target offset is greater than a second preset value, then generating an early warning instruction to remind a technician to check current model parameters of the preset odd-even row alignment method; if the target offset is between the first preset value and the second preset value, then subsequent fiber end face images under the same scene all sequentially use the preset odd-even row alignment method and the preset row-by-row alignment method; the second preset value is greater than the first preset value.

2. The method of claim 1, wherein the method further comprises: aligning each of the test fiber end face images by using the preset odd-even row alignment method, specifically: extracting odd rows of pixels of a test fiber end face image to form a first image, and extracting even rows of pixels to form a second image, and calculating a pixel misalignment distance between the first image and the second image; aligning the test fiber end face image by using the pixel misalignment distance to align the center point positions or the one-side end point positions of the first image and the second image.

3. The method of claim 1, wherein the method further comprises: aligning each of the test fiber end face images by using the preset row-by-row alignment method, specifically: storing a plurality of test fiber end face images into a temporary database, and calculating an accumulated sum corresponding to each row of data in the plurality of test fiber end face images by using an independent storage space; calculating a mean value of the accumulated sum of each row according to the number of frames of the test fiber end face images, and performing forward and reverse pixel offset processing on the mean value to generate a plurality of row expansion data corresponding to each row of data; obtaining a second test fiber end face image stored in the temporary database in a new round, and calculating a target offset of each row of data in the second test fiber end face image according to the corresponding row expansion data, to align the second test fiber end face image row by row according to the target offset.

4. The method of claim 3, wherein the method further comprises: obtaining each row of data of the second test fiber end face image, calculating a difference value between each row of data and each corresponding row expansion data, and taking the minimum difference value as the target offset of the row of data.

5. The method of claim 3, wherein the method further comprises: The test fiber end face images stored in the temporary database are original test fiber end face images or test fiber end face images processed by the preset odd-even row alignment method.

6. The confocal endomicroscopy image processing method according to any one of claims 1-5, wherein, Step 3 of fitting the distortion correction model, specifically: The fiber end face data corresponding to the reference fiber end face image is acquired, including a first fiber core center coordinate and a first pixel coordinate of each first fiber core point; The fiber alignment image is filtered, a second pixel coordinate of each second fiber core point is identified, and a second fiber core center coordinate is generated; The first fiber core center coordinate and the second fiber core center coordinate are located, and a matching point of the second fiber core point in the reference fiber end face image is searched according to each row number, and a mapping relationship is established, forming a plurality of point pairs; A displacement vector of each point pair is calculated according to the coordinates, and a displacement function of the fiber alignment image relative to the reference fiber end face image is fitted through a plurality of displacement vectors, so as to serve as the distortion correction model.

7. A confocal endomicroscopy image processing device based on the method of any one of claims 1-6, characterized in that, The method comprises an acquisition module, an alignment module, a model fitting module and a distortion correction module, The acquisition module is configured to acquire a reference fiber end face image of a confocal microendoscope when the confocal microendoscope is shipped and a plurality of test fiber end face images in a current scene; The alignment module is configured to align each of the test fiber end face images by using a preset odd-even row alignment method and a preset row-by-row alignment method, and generate a fiber alignment image; The model fitting module is configured to locate a first fiber core point of the reference fiber end face image and a second fiber core point of the fiber alignment image, and fit a distortion correction model based on a matching result of the first fiber core point and the second fiber core point; The distortion correction module is configured to correct a real-time fiber end face image in the same scene based on the distortion correction model, and generate an optimized image.

8. A computer readable storage medium storing a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the confocal microendoscope image processing method of any one of claims 1-6.

9. A confocal endomicroscopy image processing terminal comprising a computer readable storage medium and a processor, characterized in that, The processor executes the computer program on the computer readable storage medium to implement the steps of the confocal microendoscope image processing method of any one of claims 1-6.

Images