A data generation method for correcting endoscopic probe rotation non-uniformity

Abstract

This invention discloses a data generation method for correcting rotational nonuniformity of endoscopic probes, relating to the field of medical image processing technology. The method includes: selecting an acquired image composed of multiple scan line data; performing a random image transformation on the acquired image to obtain a reference image; simulating diverse nonuniform rotational distortion patterns using a mathematical model to randomly synthesize an offset vector corresponding to the number of line data in the reference image; applying the offset vector to the reference image, obtaining corresponding line data based on the offset value at each position, and arranging the newly arranged line data to form a distorted image; and generating a distorted-reference image data pair with known labels from the reference image and the distorted image. This invention can meet the quantitative and diverse requirements for training data in nonuniform rotational distortion correction at extremely low cost, making the training set more complete and exhibiting good applicability and generalization for correcting endoscopic probe imaging of different modalities and systems.

Description

Technical Field

[0001] This invention relates to the field of medical image processing, and more particularly to a data generation method for correcting the rotational inhomogeneity of endoscopic probes. Background Technology

[0002] Endoscopic scanning probes are advanced medical devices used for examination, diagnosis, and treatment. They are inserted into the human body through natural cavities or tiny incisions to help doctors make accurate diagnoses, monitor conditions, and develop treatment plans. Commonly used imaging technologies include optical coherence tomography (OCT), ultrasound, photoacoustics, and fluorescence, making them an indispensable tool in modern medical diagnosis.

[0003] Endoscopic scanning probes typically employ a rotational drive for 360° lateral scanning, making them more suitable for imaging and treatment of luminal structures (such as blood vessels, airways, and digestive tracts). By coordinating axial linear translation (pull-back), three-dimensional scanning information can be obtained. However, when an external motor rotates and drives the endoscopic probe to scan within the cavity, friction occurs between the probe and transmission leads and the protective sheath. This, along with torque transmission losses and minor motor speed errors, results in non-uniform rotational distortion (NURD) of the acquired signal. The scanned linear data shifts and misaligns, leading to image distortion.

[0004] Specifically, such as Figure 1 As shown, the flexible endoscope probe 102 enters the body cavity 104 under the protection of the protective sleeve 103. The motor drive system 101 drives the flexible endoscope probe 102 to rotate within the body cavity 104 to acquire image data. The translation stage 105 moves axially to retract the flexible endoscope probe 102, and continuous rotational scanning is used to obtain three-dimensional sequence data. During this process, due to various factors such as the friction between the flexible endoscope probe 102 and the protective sleeve 103, the torque transmission loss during the rotation of the flexible endoscope probe 102, and the non-uniform rotation of the motor drive system, non-uniform rotational distortion of the endoscope probe occurs, resulting in image distortion.

[0005] Non-uniform rotational distortion can be mitigated by improving the probe structure; however, hardware improvements bring disadvantages such as high cost, complex design, large size, and field of view obstruction. Therefore, considering the cost and advantages of algorithmic correction, some post-processing algorithms for correcting non-uniform rotational distortion in endoscopy have been gradually developed.

[0006] With the rapid development of artificial intelligence technology, data-driven non-uniform rotational distortion correction methods have significantly outperformed traditional methods in terms of correction accuracy and processing speed. Liao et al. (Medical Image Analysis 77(2022):102355) first proposed a data-driven deep learning method for correcting non-uniform rotational distortion of endoscopic OCT probes, and its correction performance significantly surpassed previous methods. Zhang et al. (Biomedical Optics Express 15.1(2024):319-335) developed a deep learning correction method, which improved the correction performance while increasing the processing speed to real-time. Data-driven non-uniform rotational distortion correction methods have demonstrated their potential for correction of rotational scans in clinical endoscopy.

[0007] However, the aforementioned neural network models for non-uniform rotational distortion correction require a large amount of high-quality and diverse labeled distorted image-reference image data pairs for model training. These pairs learn the ability to transform distorted images to reference images (considered distortion-free) to achieve correction. Constructing these distorted image-reference image data pairs requires pre-collecting a large number of endoscopic image sequences to extract labeled training data pairs. Obtaining the corresponding offset annotations between distorted and reference images is difficult. Furthermore, to ensure the model's generalization ability and adapt to the non-uniform rotational distortion patterns under various imaging settings, probe designs, and application scenarios, diverse endoscopic system data needs to be collected, which is costly and time-consuming. The quantity of training data and the diversity of non-uniform rotational distortion patterns it contains affect the correction performance of the correction model. Incorrectly corrected imaging results interfere with doctors' diagnostic and treatment strategies, increasing their psychological burden.

[0008] Therefore, those skilled in the art are dedicated to developing a data generation method for endoscopic probe rotation non-uniformity correction, which can infinitely generate diverse distortion-reference pairing data required for training non-uniform rotation distortion correction models. This data is used by neural network models to predict the distortion offset from the distorted image to the reference image, thereby achieving low-cost, high-precision non-uniform rotation distortion correction. Summary of the Invention

[0009] In view of the above-mentioned deficiencies of the prior art, the technical problem to be solved by the present invention is how to infinitely generate diverse distortion-reference image data pairs required for training non-uniform rotation distortion correction models.

[0010] To achieve the above objectives, the present invention provides a data generation method for correcting the rotational non-uniformity of endoscopic probes, the method comprising the following steps:

[0011] Step 1: Select an image to be acquired, which consists of multiple scan line data;

[0012] Step 2: Perform random image transformation on the acquired image to obtain the reference image;

[0013] Step 3: Simulate diverse non-uniform rotational distortion patterns using a mathematical model, and randomly synthesize an offset vector corresponding to the number of line data in the reference image;

[0014] Step 4: Apply the offset vector to the reference image, obtain the corresponding line data according to the offset value of each position, and form a distorted image from the newly arranged line data. Generate a distorted-reference image data pair with known labels from the reference image and the distorted image.

[0015] Furthermore, the random image transformation in step 2 includes: random up / down / left / right flipping, random overall scrolling along the x-axis and y-axis of the image, adding random noise, and image filtering.

[0016] Furthermore, step 3 also includes:

[0017] Step 3.1: Construct a sine function and set reasonable ranges for the amplitude, frequency, and initial phase of the sine function;

[0018] Step 3.2: Generate n sine functions with random amplitude, frequency, and initial phase within a reasonable range;

[0019] Step 3.3: Generate n random weights, and the sum of the n random weights is 1;

[0020] Step 3.4: Multiply each of the n sine functions by its corresponding random weight and add 1 to obtain the first derivative of the offset vector;

[0021] Step 3.5: Integrate the first derivative to obtain the correspondence between the data position indices of the first line;

[0022] Step 3.6: Apply a random overall offset value to the first line data position index correspondence to obtain the second line data position index correspondence;

[0023] Step 3.7: Subtract the line data position index value from the second line data position index correspondence, and round the result to the nearest integer.

[0024] Step 3.8: Obtain the offset vector corresponding to the simulated reference image to the distorted image, and use it as the offset error label of the distorted-reference image data pair.

[0025] Furthermore, the amplitude range in step 3.1 determines the maximum offset displacement of the line data, while the frequency and initial phase range determine the complexity of the offset vector.

[0026] Furthermore, in step 3.4, when there is no offset distortion, the first derivative is 1, and when there is offset distortion, the first derivative is not 1.

[0027] Furthermore, in the first line data position index correspondence relationship and the second line data position index correspondence relationship, when the slope is 1, it means that the line data at the same index position of the distorted image and the reference image correspond one-to-one, and there is no rotational uneven distortion; when the slope is not 1, it means that the line data at the same position of the distorted image and the reference image do not correspond, and the distorted image has undergone non-uniform rotation relative to the reference image, resulting in line data offset misalignment.

[0028] Furthermore, the set range of the random overall offset value includes the overall maximum offset range caused by non-uniform rotation.

[0029] Furthermore, by repeating steps 3.2 to 3.8, the offset vectors that conform to diverse non-uniform rotation laws can be generated indefinitely.

[0030] Furthermore, step 4 also includes: for the line data position index of the reference image, obtaining the offset value of the offset vector corresponding to the index position, adding the offset value to the index value to obtain the line data position index after the current line data position is offset, filling the offset line data into the current line data position, and performing the operation to fill each position in sequence to obtain the offset distorted image.

[0031] Furthermore, by repeating steps 1 to 4, the distortion-reference image data pairs required for training the non-uniform rotation distortion correction model can be generated indefinitely. These data pairs are used by the neural network model to predict the distortion offset from the distorted image to the reference image.

[0032] Compared with the prior art, the present invention has at least the following beneficial technical effects:

[0033] This invention constructs a mathematical model to simulate the effects of diverse non-uniform rotational distortion on a reference image, thereby synthesizing distortion-reference image training data pairs required for training a non-uniform rotational distortion correction network. This method can generate an unlimited number of distortion-reference image pairs that conform to diverse non-uniform rotational distortion patterns and have known labels, eliminating the need for manual collection and labeling of large amounts of diverse endoscopic probe imaging data. This saves significant manpower, time, and money, and meets the quantitative and diverse requirements for non-uniform rotational distortion correction training data at extremely low cost, resulting in a more complete training set. It exhibits good applicability and generalization for correcting endoscopic probe imaging of different modalities and systems.

[0034] The following will further explain the concept, specific structure, and technical effects of the present invention in conjunction with the accompanying drawings, so as to fully understand the purpose, features, and effects of the present invention. Attached Figure Description

[0035] Figure 1 This is a schematic diagram illustrating the problem of uneven rotation of the endoscopic probe;

[0036] Figure 2 This is a flowchart of a preferred embodiment of the present invention;

[0037] Figure 3 This is a flowchart of a preferred embodiment of the method for synthesizing offset vectors according to the present invention;

[0038] Figure 4 This is a schematic diagram of a preferred embodiment of the present invention, showing a combination of sine functions.

[0039] Figure 5 This is a schematic diagram of the first derivative of a preferred embodiment of the present invention;

[0040] Figure 6 This is a schematic diagram of the first-line data position index correspondence in a preferred embodiment of the present invention;

[0041] Figure 7 This is a schematic diagram of the second-line data position index correspondence in a preferred embodiment of the present invention;

[0042] Figure 8 This is a schematic diagram of the offset vector of a preferred embodiment of the present invention;

[0043] Figure 9 This is a schematic diagram showing the results of a preferred embodiment of the present invention. Detailed Implementation

[0044] The following description, with reference to the accompanying drawings, illustrates several preferred embodiments of the present invention to make its technical content clearer and easier to understand. The present invention can be embodied in many different forms, and the scope of protection of the present invention is not limited to the embodiments mentioned herein.

[0045] In the accompanying drawings, components with the same structure are indicated by the same numerical designation, and components with similar structures or functions are indicated by similar numerical designations. The dimensions and thicknesses of each component shown in the drawings are arbitrary, and the present invention does not limit the dimensions and thicknesses of each component. To make the illustrations clearer, the thickness of some components has been appropriately exaggerated in the drawings.

[0046] This embodiment provides a data generation method for correcting the rotational non-uniformity of endoscopic probes, such as... Figure 2 As shown, it includes the following steps:

[0047] S100. Select an image to be acquired. The acquired image consists of multiple scan line data.

[0048] S200. The acquired image is treated as a reference image through random image transformation; random image transformation includes: randomly flipping the image up, down, left, and right, randomly scrolling the image along the x-axis and y-axis (rows or columns that have exceeded the last position will be scrolled to the first position), adding random noise, and image filtering.

[0049] S300: Simulate diverse non-uniform rotational distortion patterns using a mathematical model, and randomly synthesize an offset vector corresponding to the number of reference image line data.

[0050] S400: Apply the offset vector to the reference image. Obtain the corresponding line data based on the offset value at each position. The newly arranged line data forms the distorted image, thereby generating distorted-reference image paired data with known labels. Specifically: For the line data position index of the reference image, obtain the offset value of the offset vector corresponding to the index position. Add the offset value to the index value to obtain the line data position index after the offset of the current line data position. Fill the current line data position with the offset line data. After filling each position sequentially, the distorted image after offset is obtained.

[0051] By repeating this process, a variety of distortion-reference pairing data can be generated indefinitely to train the non-uniform rotation distortion correction model. This data is used by the neural network model to predict the distortion offset from the distorted image to the reference image, thereby achieving low-cost, high-precision non-uniform rotation distortion correction.

[0052] like Figure 3 As shown, S300 specifically includes the following steps:

[0053] S301. Construct a sine function and set a reasonable range for its amplitude, frequency, and initial phase. The amplitude range determines the maximum offset displacement of the line data, and the frequency and initial phase ranges determine the complexity of the offset vector. In this embodiment, the amplitude range is [0, 20], the frequency range is [0, 40], and the initial phase range is [0, 2π].

[0054] S302. Generate n sine functions f with random amplitude A, frequency f, and initial phase ψ within the range. k (x) = A·sin(2π·f·x+ψ), where k∈[1,n], and in this embodiment, the value of n is set to [50, 100]. Figure 4 The diagram shown is a schematic of the generated combination of sine functions.

[0055] S303. Generate n random weights, the sum of the n random weights is 1, expressed by the formula:

[0056] S304. Multiply each of the n sine functions by its corresponding random weight and add 1 to obtain the first derivative F(x) of the offset vector, expressed by the formula: like Figure 5 As shown, the first derivative is 1 when there is no offset distortion, and it is not 1 when there is offset distortion.

[0057] S305. Integrate the first derivative to obtain the correspondence between the line data position indices of the simulated distortion-reference image, i.e., the correspondence between the first line data position indices, such as... Figure 6 As shown. When the distorted image has no offset distortion relative to the reference image, the data of the same index position lines of the distorted and reference images correspond one-to-one, that is, the slope is 1; when the distorted image has offset distortion relative to the reference image, the data of the same position lines of the distorted and reference images do not correspond, that is, the slope is not 1.

[0058] S306. Apply a random overall offset value within a reasonable range to the simulated distortion-reference image line data position index correspondence to obtain the second line data position index correspondence, such as... Figure 7 As shown; the range of random overall offset values ​​should include the maximum overall offset range caused by non-uniform rotation under normal circumstances.

[0059] S307. Subtract the line data position index value from the second line data position index correspondence and round it to the nearest integer.

[0060] S308. Obtain the offset vector corresponding to the simulated reference image to the distorted image, and use it as the offset error label of the distorted-reference image data pair, such as... Figure 8 As shown.

[0061] By repeating S302 to S308, offset vectors that conform to diverse non-uniform rotation laws can be generated indefinitely.

[0062] According to the method provided in this embodiment, training pairing data generated in a low-cost manner is used for data-driven neural network training of endoscopic non-uniform rotation correction. Figure 9 As shown, Figure (a) is the maximum en-face projection of the original endoscopic OCT probe image sequence before correction, and Figure (b) is the maximum en-face projection of the corrected image sequence. By comparing Figure (a) and Figure (b), it can be seen that the low-cost, diverse distortion-reference paired image data generated in this embodiment can be effectively used to train the correction network and alleviate image distortion.

[0063] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A data generation method for endoscope probe rotation non-uniformity correction, characterized by, The method includes the following steps: Step 1: Select an image to be acquired, which consists of multiple scan line data; Step 2: Perform random image transformation on the acquired image to obtain a reference image; Step 3: Simulate diverse non-uniform rotational distortion patterns using a mathematical model, and randomly synthesize an offset vector corresponding to the number of line data points in the reference image; Step 3 further includes: Step 3.1: Construct a sine function and set reasonable ranges for the amplitude, frequency, and initial phase of the sine function; Step 3.2: Generate n sine functions with random amplitude, frequency, and initial phase within a reasonable range; Step 3.3: Generate n random weights, and the sum of the n random weights is 1; Step 3.4: Multiply each of the n sine functions by its corresponding random weight and add 1 to obtain the first derivative of the offset vector; Step 3.5: Integrate the first derivative to obtain the correspondence between the data position indices of the first line; Step 3.6: Apply a random overall offset value to the first line data position index correspondence to obtain the second line data position index correspondence; Step 3.7: Subtract the line data position index value from the second line data position index correspondence, and round the result to the nearest integer. Step 3.8: Obtain the offset vector corresponding to the simulated reference image to the distorted image, as the offset error label of the distorted-reference image data pair; Step 4: Apply the offset vector to the reference image, obtain the corresponding line data according to the offset value of each position, and form the distorted image from the newly arranged line data. Generate the distorted-reference image data pair with known labels based on the reference image and the distorted image.

2. The data generation method for endoscope probe rotation non-uniformity correction according to claim 1, wherein, The random image transformation in step 2 includes: random up / down / left / right flipping, random overall scrolling along the x and y axes of the image, adding random noise, and image filtering.

3. The data generation method for endoscope probe rotation non-uniformity correction according to claim 1, wherein, In step 3.1, the amplitude range determines the maximum offset displacement of the line data, and the frequency and initial phase range determine the complexity of the offset vector.

4. The data generation method for endoscope probe rotation non-uniformity correction according to claim 1, wherein, In step 3.4, the first derivative is 1 when there is no offset distortion, and is not 1 when there is offset distortion.

5. The data generation method for correcting endoscopic probe rotation non-uniformity as described in claim 1, characterized in that, In the first line data position index correspondence and the second line data position index correspondence, when the slope is 1, it means that the line data at the same index position of the distorted image and the reference image correspond one-to-one, and there is no rotational uneven distortion; when the slope is not 1, it means that the line data at the same position of the distorted image and the reference image do not correspond, and the line data is offset and misaligned due to non-uniform rotation of the distorted image relative to the reference image.

6. The data generation method for correcting endoscopic probe rotation non-uniformity as described in claim 1, characterized in that, The set range of the random overall offset value includes the maximum overall offset range caused by non-uniform rotation.

7. The data generation method for correcting endoscopic probe rotation non-uniformity as described in claim 1, characterized in that, By repeating steps 3.2 to 3.8, the offset vectors that conform to diverse non-uniform rotation laws can be generated indefinitely.

8. The data generation method for correcting endoscopic probe rotation non-uniformity as described in claim 1, characterized in that, Step 4 further includes: for the line data position index of the reference image, obtaining the offset value of the offset vector corresponding to the index position, adding the offset value to the index value to obtain the line data position index after the current line data position is offset, filling the offset line data into the current line data position, and performing the operation to fill each position in sequence to obtain the offset distorted image.

9. The data generation method for correcting endoscopic probe rotation non-uniformity as described in claim 1, characterized in that, By repeating steps 1 to 4, the distortion-reference image data pairs required for training the non-uniform rotation distortion correction model can be generated indefinitely, which are used by the neural network model to predict the distortion offset from the distorted image to the reference image.

Images