Data processing platform and method of confocal microendoscope

Abstract

The invention discloses a confocal microendoscope data processing platform and method, the platform is based on an FPGA controller and an ARM processor, the FPGA controller comprises a BRAM memory, a control module, a distortion correction module and a data packaging module, and the ARM processor comprises a storage module, a state updating thread and an image transmission thread. According to the invention, based on an FPGA + ARM heterogeneous computing platform, address partitioning and management are carried out on the BRAM according to the control parameter type, the system design is simplified, and the integration level is high, so that the equipment reliability is improved, and the system time delay is reduced; meanwhile, image processing of distortion correction is carried out at an embedded FPGA end, the processing process is simple and efficient, original data collection can be carried out at a high speed, the image quality is guaranteed, meanwhile, only qualified images after distortion processing are uploaded to an upper computer through an ARM processor, the transmission bandwidth between an endoscope host and an upper computer server is reduced, and the transmission efficiency is improved. And the transmission reliability is improved.

Description

Technical Field

[0001] This invention relates to the field of confocal microendoscopy, and more particularly to a data processing platform and method for confocal microendoscopy. Background Technology

[0002] Confocal microscopy offers micrometer-level resolution, enabling real-time cell imaging. It provides rapid and accurate diagnostic services requiring in vivo biopsies and has the potential to replace traditional endoscopic biopsies and pathological examinations in the future, demonstrating a very broad clinical application prospect.

[0003] The opto-electro-mechanical system of a confocal microendoscopy system is complex, as is its data processing. Furthermore, the lasers, galvanometers, and focusing motors involved in data transmission between opto-electro-mechanical modules are low-speed interface modules, requiring relatively low-performance controllers; a general-purpose microcontroller can effectively control them. However, the voltage signal output from the photomultiplier tube requires high-speed sampling to produce a satisfactory image, necessitating an image acquisition circuit with an FPGA as the core controller to meet data acquisition requirements. For example, in a system using an 8K galvanometer as the horizontal galvanometer and outputting a 1024-pixel horizontal image, the sampling rate for the 1024-point horizontal image is 8000 × 2 × 1024 = 16.4 MHz. To eliminate the image distortion unique to galvanometer scanning systems, at least double oversampling is generally required, i.e., a sampling rate greater than 32.8 MHz. Such a high sampling rate is beyond the capabilities of a microcontroller and is typically achieved using an FPGA sampling circuit. However, the data bandwidth between the microcontroller and the FPGA is limited by the microcontroller interface, which cannot meet the transmission requirements of high-resolution, high-frame-rate images, thus limiting the performance improvement of the confocal microendoscopic system. Summary of the Invention

[0004] This invention provides a data processing platform and method for confocal microendoscopy, which solves the technical problems mentioned above.

[0005] The technical solution to solve the above-mentioned technical problems is as follows: A first aspect of the present invention provides a data processing method for a confocal microendoscopy system, applied to an FPGA controller, the FPGA controller including a BRAM memory, the method comprising the following steps: Step 1: After the confocal endoscope is powered on, it obtains the initial control values ​​of each preset parameter from the ARM processor and stores the initial control values ​​in the preset address of the BRAM memory. Step 2: Recall the initial control value from the corresponding address in the BRAM memory and control the corresponding peripheral devices in sequence to generate the original image of the tissue to be detected, and store the real-time control value of at least one target preset parameter in the BRAM memory during the control process. Step 3: Perform distortion correction on the original image to generate a corrected image; Step 4: Cache and encapsulate the corrected image according to the image data type, and send the encapsulated image data to the host computer via the ARM processor.

[0006] A second aspect of this invention provides a data processing platform for a confocal microendoscopy system, based on an FPGA controller. The FPGA controller includes a BRAM memory, a control module, a distortion correction module, and a data encapsulation module. The BRAM memory is used to partition and store at least one control value of a preset parameter, the control value including an initial control value and a real-time control value; The control module is used to call the initial control values ​​of each preset parameter from the corresponding preset address of the BRAM memory and control the corresponding peripheral devices in sequence to generate the original image of the tissue to be detected, and store the real-time control value of at least one target preset parameter in the BRAM memory during the control process. The distortion correction module is used to correct the distortion of the original image and generate a corrected image; The data encapsulation module is used to cache and encapsulate the corrected image according to the image data type, and send the encapsulated image data to the host computer via the ARM processor.

[0007] The present invention has the following beneficial effects: It provides a data processing platform and method for a confocal microendoscopy system, based on a heterogeneous computing platform of FPGA+ARM. The address partitioning and management of the FPGA's BRAM memory according to the control parameter type simplifies the system design, increases the integration, thereby improving device reliability and reducing system latency. At the same time, the distortion correction image processing is performed on the embedded FPGA, which is simple and efficient, enabling raw data acquisition at a high rate and ensuring image quality. Furthermore, only the qualified images after distortion correction are uploaded to the host computer through the ARM processor, reducing the transmission bandwidth between the endoscope host and the host server and improving transmission reliability.

[0008] To make the above-mentioned objects, features and advantages of the invention more apparent and understandable, preferred embodiments of the invention are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0009] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 This is a schematic flowchart of the data processing method for a confocal microendoscopy provided in Embodiment 1; Figure 2 This is a schematic diagram of the data processing platform for the confocal microendoscopy provided in Embodiment 2. Detailed Implementation

[0011] To make the objectives, technical solutions, and beneficial effects of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described in this specification are merely for explaining the invention and are not intended to limit the invention.

[0012] Figure 1 This is a flowchart illustrating a data processing method for a confocal microendoscopy system provided in Embodiment 1, as shown below. Figure 1 As shown, it includes the following steps: Step 1: After the confocal endoscope is powered on, it obtains the initial control values ​​of each preset parameter from the ARM processor and stores the initial control values ​​in the preset address of the BRAM memory. Step 2: Recall the initial control value from the corresponding address in the BRAM memory and control the corresponding peripheral devices in sequence to generate the original image of the tissue to be detected, and store the real-time control value of at least one target preset parameter in the BRAM memory during the control process. Step 3: Perform distortion correction on the original image to generate a corrected image; Step 4: Cache and encapsulate the corrected image according to the image data type, and send the encapsulated image data to the host computer via the ARM processor.

[0013] The above embodiments provide a data processing method for a confocal microendoscopy system. Based on an FPGA+ARM heterogeneous processing platform, it simplifies system design, achieves high integration, thereby improving device reliability and reducing system latency. Simultaneously, image processing with distortion correction is performed on the embedded FPGA, making the process simple and efficient. This allows for high-speed acquisition of raw data, ensuring image quality. Furthermore, only the distortion-corrected, qualified images are uploaded to the host computer via the ARM processor, reducing the transmission bandwidth between the endoscope host and the host server and improving transmission reliability.

[0014] The steps of the above method will be described in detail below through specific embodiments.

[0015] In one embodiment, the FPGA controller includes several control modules, exemplarily such as... Figure 2As shown, the system includes a horizontal galvanometer control module 1021, a vertical galvanometer control module 1022, a photomultiplier tube (PMT) control module 1023, a stepper motor control module 1024, and a laser control module 1025. Specifically, the horizontal galvanometer control module 1021 is connected to a rapidly moving horizontal galvanometer drive board, enabling the light to scan rapidly back and forth in the horizontal direction; the vertical galvanometer control module 1022 is connected to a slowly moving vertical galvanometer drive board, enabling the light to scan in the vertical direction, working in conjunction with the horizontal galvanometer to obtain a two-dimensional planar image; the photomultiplier tube (PMT) control module 1023 is connected to the PMT drive board and is used to acquire the fluorescence signal generated by the tissue to be tested; the stepper motor control module 1024 is connected to the motor drive board and is used to adjust the motor position to focus the coupling lens; the laser control module 1025 is connected to the laser and is used to control the laser beam emitted by the laser.

[0016] In one specific embodiment, after the confocal microendoscopy is powered on, the laser emits a laser source, and the motor drive board drives the motor to adjust the position to achieve focusing of the coupling objective. After optical path transmission through optical devices including the galvanometer and coupling objective, the light reaches the tissue to be examined. The fluorescence signal generated by the tissue is transmitted to the pinhole camera controlled by the PMT drive board to obtain fluorescence image data. The photomultiplier tube corresponding to the fluorescence image data outputs an analog voltage signal, which is sampled by the AD converter and mapped to the original image through spatial pixels. Distortion correction is then performed. The corrected image is then cached and encapsulated according to the image data type. The encapsulated data is transmitted to the ARM processor 200 via the internal high-speed interface in DMA mode. The ARM processor then sends the data to the host computer via the network using the TCP / IP protocol. In this way, only distortion-free images after distortion correction are cached and encapsulated. Distorted images are not encapsulated and uploaded, thereby reducing the transmission bandwidth between the heterogeneous processing platform and the host server.

[0017] For example, in a preferred embodiment, the BRAM memory includes multiple parameter areas for storing different parameter categories, thereby facilitating the storage of different categories of parameters at corresponding addresses, enabling the various control modules of the ARM processor and FPGA to perform data retrieval and transmission very accurately for implementing different functions.

[0018] For example, the parameter categories include laser parameters, galvanometer parameters, stepper motor position parameters, photomultiplier tube parameters, distortion correction parameters, and / or data encapsulation parameters.

[0019] Specifically, laser parameters include laser power values. By controlling the laser power, the brightness of the light source can be adjusted to improve image quality.

[0020] The galvanometer parameters include the voltage values ​​of the horizontal and vertical galvanometers, which adjust the deflection angle of the galvanometers. The horizontal galvanometer scans from the leftmost pixel in the field of view to the right, then from the rightmost pixel to the left, completing one round trip. During this process, the vertical galvanometer position increases by one step unit. Once the vertical step size reaches the set number of rows for the vertical image resolution, the vertical galvanometer position decreases by one step unit after each subsequent horizontal scan. With the cooperation of the horizontal and vertical galvanometers, the laser periodically scans the designated area, achieving confocal area imaging.

[0021] The stepper motor position parameters include the stepper motor's real-time position, position adjustment step size, frequency, and motor direction. The position adjustment step size controls the motor's single movement distance, the frequency controls the number of times the motor moves, and the direction controls the direction of the motor's movement, which can include both positive and negative directions. This allows for precise control of the motor position, improving the focusing effect of the coupling lens and enhancing image quality.

[0022] The parameters of a photomultiplier tube include its gain, which is the control voltage of the PMT driver board. By adjusting the voltage value, the electron multiplication efficiency and signal amplification factor of the photomultiplier tube can be controlled, thereby improving the fluorescence signal acquisition effect.

[0023] The distortion correction parameters include the enable switch state of a preset distortion correction algorithm. This enable switch is used to control the on or off of the distortion correction algorithm under different operating modes. For example, when the device is powered on in normal operating mode, the enable switch state is "1", and the distortion correction algorithm is on; in debugging mode or other special needs, the enable switch state is "0", and the distortion correction algorithm can be manually controlled to be turned off.

[0024] The data encapsulation parameters include several image data types used to represent image characteristics, including normal images, vertically spaced stripe images, horizontally spaced stripe images, checkerboard images, horizontal grayscale images, or vertical grayscale images, etc. After classifying the uploaded image data by data type in the data encapsulation parameter area, the host computer can perform different data processing for different data types, and can also determine the cause of data problems based on different data types when problems occur. Specifically, for example... Figure 2As shown, the photomultiplier tube outputs an analog voltage signal, which is sampled by an AD converter to generate the original image and undergo distortion correction. The distortion-corrected image data is then sent to a data encapsulation module for encapsulation, followed by transmission to the ARM processor, and finally to a host computer for display via the network. To test for abnormal data transmission, the encapsulation parameters can be adjusted so that the encapsulated image includes not only the normally corrected image but also image data with vertically spaced stripes, horizontally spaced stripes, checkerboard patterns, horizontal grayscale, and vertical grayscale. If the image data with vertically spaced stripes, horizontally spaced stripes, checkerboard patterns, horizontal grayscale, or vertical grayscale appears normal to the host computer, then the data transmission is normal. In this case, the normally corrected image can be used for subsequent image diagnostic analysis or image display on the host computer. Otherwise, it is necessary to analyze and locate the data transmission problem based on the abnormal phenomenon. After the problem is resolved, the normally corrected image can then be used for subsequent image diagnostic analysis or image display on the host computer.

[0025] For example, existing technologies in the art typically use a uniform speed drive method to control stepper motors, that is, sending N square waves at a fixed period T to drive the stepper motor to move N step angles. However, when the stepper motor moves at a uniform speed, since there is no smooth acceleration and deceleration process when the motor starts and stops, the focusing mechanism will be subjected to a large acceleration when the motor stops. This will cause the coupling lens to vibrate randomly within the reverse gap and stop at an uncertain position, resulting in unstable focusing of the system and ultimately blurry images. Therefore, in a preferred embodiment, the stepper motor is controlled by accelerating first, then maintaining a uniform speed, and finally decelerating. In the acceleration phase, acceleration starts from 0 and increases and then decreases; in the uniform speed phase, it runs at the maximum speed during acceleration; in the deceleration phase, acceleration decreases and then increases, thereby enabling the laser to be accurately coupled to the end face of the imaging probe and improving the focusing effect.

[0026] In the above control process, the period control formula for the square wave signal corresponding to the acceleration or deceleration phase is as follows: , The period control formula for the square wave signal corresponding to the uniform velocity phase is: , Where T is the pulse period, Max is the square wave control coefficient (Max = 2A), A is the number of acceleration / deceleration steps of the motor, and n is the number of square wave signals transmitted within the pulse period T. To achieve the above motor motion control process, in addition to storing the stepper motor's position parameters, the BRAM memory also needs to store the stepper motor's motion parameters, including the stepper motor's motion state, acceleration / deceleration step size, and square wave control coefficient. The motion state includes uniform motion, acceleration / deceleration, and combined states used for debugging the motion control scheme. Furthermore, when partitioning the BRAM memory, the stepper motor motion parameters also need to be partitioned independently to improve parameter retrieval efficiency and accuracy.

[0027] To achieve the desired data processing effect, the preferred embodiment of the present invention further sets the transmission direction of the above-mentioned parameters, and the corresponding address of the BRAM memory can be set based on the data transmission direction of different parameters. For example, the galvanometer parameter area, photomultiplier tube parameter area, and distortion correction parameter area transmit data unidirectionally to their respective control modules; the laser parameter area, stepper motor position parameter area, stepper motor motion parameter area, and data encapsulation parameter area transmit data bidirectionally with their respective control modules. The purpose of unidirectional transmission is to drive peripherals, while the purpose of bidirectional transmission is not only to drive peripherals but also to continuously save data into the ARM processor, providing a basis for subsequent fault diagnosis.

[0028] Specifically, in unidirectional data transmission, each control module only retrieves data from the BRAM memory (hereinafter referred to as BRAM) and does not send new data back to the BRAM. For example, after the device powers on, the galvanometer parameter area receives the galvanometer parameters transmitted from the ARM processor to the BRAM. Then, the peripheral galvanometer driver board retrieves the galvanometer parameters from the BRAM's galvanometer parameter area to drive the galvanometer to move. In bidirectional data transmission, each control module not only retrieves data from the BRAM but also sends new data back to the BRAM. For example, after the device powers on, the stepper motor parameter area (including the position parameter area and motion parameter area) receives the stepper motor parameters transmitted from the ARM to the BRAM. Then, the peripheral motor driver board retrieves the corresponding stepper motor parameters from the BRAM's stepper motor parameter area to drive the stepper motor to move. At the same time, the parameters during the stepper motor's movement are also sent back to the BRAM's stepper motor parameter area in real time, and then from the BRAM's stepper motor parameter area into the ARM for storage.

[0029] For example, in a preferred embodiment, to more rationally call the various parameters, a pre-allocation step of the storage address of the BRAM memory is further included, specifically: Obtain the transmission configuration information for each type of parameter. This transmission configuration information includes at least the data transmission direction, the number of parameters, and the data update frequency. The data transmission direction indicates whether the parameter is transmitted unidirectionally or bidirectionally; the number of parameters indicates the number of control parameters involved in each control module; and the data update frequency indicates the frequency at which each control module calls the corresponding control parameter or sends real-time data back to the BRAM memory.

[0030] Then, based on the transmission configuration information, the predicted data volume for a preset time range under different control modes is calculated, and address configuration information for the parameter area corresponding to each type of parameter is generated. The address configuration information includes the data access frequency and the maximum predicted data volume. Specifically, the control modes include normal detection mode, fault analysis mode, and debugging mode. Each type of parameter has the same or different transmission configuration information under different modes. For example, the number of parameter calls, frequency, data transmission flow direction, and number of parameters can differ under different modes. This can be achieved by constructing a preset mapping table.

[0031] For example, a large amount of historical data from confocal microscopy endoscopes under different usage scenarios and control modes can be collected. This historical data can be used to train a neural network model to establish a predictive model. This predictive model calculates the amount of predicted data that needs to be stored in each parameter area within a preset time range under each control mode, and takes the maximum value as the address configuration information. For example, the preset time range can be a complete endoscope inspection cycle, or the maximum continuous working time of the endoscope in the historical data. The data access frequency is comprehensively evaluated by the control module's first access frequency to the corresponding parameter address in the BRAM memory and the second access frequency of the corresponding parameter address in the BRAM memory by the status update thread in the ARM processor. This determines whether each parameter area is accessed at a high, medium, or low frequency, thus allowing for reasonable partitioning in the BRAM memory and preventing conflicts.

[0032] Finally, based on the address configuration information, the storage addresses of each parameter area in the BRAM memory are pre-allocated, that is, a corresponding storage space and the BRAM block where each type of control parameter is located are allocated.

[0033] The above preferred embodiment, by rationally dividing the addresses of different parameter areas in the BRAM memory, not only facilitates the accurate and rapid data calling and transmission of various control modules of the ARM processor and FPGA to achieve different functions, but also avoids conflicts and optimizes storage performance.

[0034] In another preferred embodiment, the method further includes a preset address update step, which specifically involves: obtaining the current utilization rate of the storage address corresponding to each parameter area or the current transmission flow direction of the parameter category corresponding to each parameter area, and adjusting the pre-allocated storage address in real time according to the current utilization rate and / or the current transmission flow direction when the preset conditions are met. For example, the adjustment is made when the current utilization rate exceeds the corresponding maximum threshold or is less than the minimum threshold, and / or the transmission flow direction of the parameter changes, such as changing from unidirectional to bidirectional, to further optimize the storage performance of the BRAM memory.

[0035] As those skilled in the art know, the amount of data in the original sampled image is generally large, which puts significant pressure on data transmission and poses a high risk of data packet loss. To minimize data transmission pressure, a lower sampling rate is generally used for the original data to control the amount of data and reduce transmission pressure. However, horizontal galvanometers typically use resonant mirrors, which operate by reciprocating along a rotation axis at a certain angle with a sinusoidal velocity—zero at both ends and highest in the middle. If data is collected at equal time intervals and the sampled data is directly stitched into an image, it will cause lateral distortion, resulting in stretching deformation at both ends and compression deformation in the middle. Therefore, distortion correction is necessary. Distortion correction requires a large amount of original sampled data; reducing the amount of original sampled data will limit the effectiveness of image distortion correction and affect image quality. To resolve the contradiction between data transmission and image quality conversion, a preferred embodiment includes a distortion correction module in the FPGA controller. This module corrects the distortion of the original image, and the method specifically includes the following steps: Establish a distortion correction mapping relationship, which includes the correction reference value corresponding to each sampling point under different scanning distortion modes; Acquire a sinusoidal scan image, perform image processing and analysis on the sinusoidal scan image, generate distortion correction parameter values, and generate a target scan distortion pattern based on the distortion correction parameter values; The distortion correction mapping relationship is queried, the target correction reference value corresponding to the sampling point at different position is obtained according to the target scanning distortion mode, and the distortion correction mapping is performed on each sampling point in the sinusoidal scanning image based on the target correction reference value, and a corrected image is generated.

[0036] The preferred embodiment described above constructs multiple distortion correction parameters (such as the center column information and horizontal scaling factor of the scanned image) and corresponding distortion mode judgment conditions based on a large amount of historical data to characterize the degree and morphology of image distortion. For example, when the distortion correction parameters of a sinusoidal scanned image meet the first distortion mode judgment condition, the sinusoidal scanned image is in the first distortion mode; otherwise, it is in the second or third distortion mode, and so on. Simultaneously, a corresponding distortion correction mapping relationship is established for each scanning distortion mode, thereby directly and quickly performing distortion correction on sampling points at different locations in the current sinusoidal scanned image within the FPGA controller. This preserves the original large amount of sampled image data, eliminating the risk of packet loss, and ensures the distortion correction effect without reducing the data volume, effectively guaranteeing the image quality after distortion correction. Furthermore, after distortion correction, valid images that meet the image quality requirements are directly uploaded to the host computer, while invalid images that do not meet the requirements are not uploaded. This reduces the transmission bandwidth between the FPGA chip and the host computer server, improving transmission reliability.

[0037] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0038] Figure 2 This is a schematic diagram of the data processing platform for the confocal microendoscopy provided in Embodiment 2, as shown below. Figure 2 As shown, the FPGA controller includes a BRAM memory 101, a control module 102, a distortion correction module 103, and a data encapsulation module 104. The BRAM memory 101 is used to partition and store at least one control value of a preset parameter, the control value including an initial control value and a real-time control value; The control module 102 is used to call the initial control values ​​of each preset parameter from the corresponding preset address of the BRAM memory and control the corresponding peripheral devices in sequence to generate the original image of the tissue to be detected, and store the real-time control value of at least one target preset parameter in the BRAM memory during the control process. The distortion correction module 103 is used to perform distortion correction on the original image and generate a corrected image; The data encapsulation module 104 is used to cache and encapsulate the corrected image according to the image data type, and send the encapsulated image data to the host computer via the ARM processor 200.

[0039] The above embodiments provide a data processing platform for confocal microendoscopy based on an FPGA+ARM heterogeneous processing platform, which simplifies system design, increases integration, improves device reliability, and reduces system latency. Simultaneously, distortion correction image processing is performed on the embedded FPGA, making the process simple and efficient. This allows for high-speed raw data acquisition, ensuring image quality. Furthermore, only distortion-corrected, qualified images are uploaded to the host computer via the ARM processor, reducing the transmission bandwidth between the endoscope host and the host server and improving transmission reliability.

[0040] For example, such as Figure 1 The data processing platform shown also includes an ARM processor 200 connected to the FPGA controller 100. The ARM processor 200 includes a storage module 201, a status update thread 202, and an image transmission thread 203. The storage module 201 is used to store at least one control value of preset parameters during the use of the confocal microendoscopy. The status update thread 202 is used to send the initial control value of the preset parameter to the BRAM memory and / or read the real-time control value of at least one target preset parameter from the BRAM memory during the use of the confocal microendoscopy. The image transmission thread 203 is used to receive the encapsulated image data and send it to the host computer. In a preferred embodiment, for example, the status update thread 202 sends the initial control value to the BRAM memory 101 via the AXI-Lite bus, the data encapsulation module 104 transmits the encapsulated image data to the ARM processor 200 via an internal high-speed interface using DMA, and the image transmission thread 203 sends it to the host computer via a network using the TCP / IP protocol. Thus, by storing the control values ​​of each control module during the last operation of the confocal microendoscopy system in the storage module 201, not only can initial values ​​be selected and sent to the BRAM memory for quick restart of the confocal microendoscopy system, but the stored data can also be used to quickly find the cause of any malfunction or operational error, further improving the effectiveness of the confocal endoscope.

[0041] In a preferred embodiment, the distortion correction module 103 specifically includes: A construction unit is used to establish a distortion correction mapping relationship, which includes the correction reference value corresponding to each sampling point under different scanning distortion modes; An image acquisition unit is used to acquire a sinusoidal scan image, perform image processing and analysis on the sinusoidal scan image, calculate distortion parameter values, and generate a target scan distortion pattern based on the distortion parameter values. The distortion parameter values ​​include center column information and a horizontal scaling factor. The correction unit is used to query the distortion correction mapping relationship, obtain the target correction reference value corresponding to the sampling point at different position according to the target scanning distortion mode, and perform distortion correction mapping on each sampling point in the sinusoidal scanning image based on the target correction reference value to generate a corrected image.

[0042] In a preferred embodiment, the system further includes an address pre-allocation module, the address pre-allocation module comprising: An information acquisition unit is used to acquire transmission configuration information for each type of parameter, wherein the transmission configuration information includes at least the data transmission flow direction, the number of parameters, and the data update frequency; The prediction unit is used to calculate the amount of predicted data within a preset time range under different control modes based on the transmission configuration information, and to generate address configuration information for the parameter area corresponding to each type of parameter. The address configuration information includes the data access frequency and the maximum amount of predicted data. The pre-allocation unit is used to pre-allocate the storage addresses of each parameter area in the BRAM memory according to the address configuration information.

[0043] In a preferred embodiment, the system further includes an address update module, which is used to obtain the current utilization rate of the storage address corresponding to each parameter area or the current transmission flow direction of the parameter category corresponding to each parameter area, and to adjust the pre-allocated storage address in real time according to the current utilization rate and / or the current transmission flow direction when a preset condition is met.

[0044] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0045] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0046] Those skilled in the art will recognize that the units and method steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0047] In the embodiments provided by this invention, it should be understood that the disclosed apparatus / terminal devices and methods can be implemented in other ways. For example, the apparatus / terminal device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0048] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0049] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0050] The present invention is not limited to the description in the specification and embodiments, and thus other advantages and modifications can be readily realized by those skilled in the art. Therefore, the present invention is not limited to the specific details, representative devices and illustrated examples shown and described herein without departing from the spirit and scope of the general concept as defined by the claims and their equivalents.

Claims

1. A data processing method of a confocal microendoscope, applied to an FPGA controller, the FPGA controller comprising a BRAM memory, characterized in that, The method comprises the following steps: Step 1, after the confocal microscopic endoscope is powered on, initial control values of preset parameters are obtained from an ARM processor, and the initial control values are stored in a preset address of a BRAM memory; Step 2, initial control values are called from the corresponding address of the BRAM memory, and corresponding peripherals are controlled in sequence to generate an original image of a tissue to be detected, and real-time control values of at least one target preset parameter in the control process are stored in the BRAM memory; Step 3, distortion correction is performed on the original image to generate a corrected image; Step 4, the corrected image is cached and packaged according to an image data type, and packaged image data is sent to an upper computer through the ARM processor.

2. The data processing method of the confocal microscopic endoscope according to claim 1, wherein, The BRAM memory comprises a plurality of parameter areas for storing different parameter categories, the parameter categories comprising laser parameters, galvanometer parameters, stepper motor position parameters, photomultiplier parameters, distortion correction parameters and / or data packaging parameters; The laser parameters comprise a laser power value; The galvanometer parameters comprise voltage values of horizontal and vertical galvanometers; The stepper motor position parameters comprise real-time positions, position adjustment steps, frequencies and motor directions of the stepper motor; The photomultiplier parameters comprise a gain of the photomultiplier; The distortion correction parameters comprise an enable switch state of a preset distortion correction algorithm; The data packaging parameters comprise a plurality of image data types for representing image features.

3. The data processing method of a confocal endomicroscope according to claim 2, wherein, The parameter categories further comprise stepper motor motion parameters, the stepper motor motion parameters comprising a motion state, a motor acceleration / deceleration step and a square wave control coefficient of the stepper motor; In the focusing process of the confocal microscopic endoscope, the stepper motor is controlled by a square wave signal to perform continuous actions of acceleration, uniform speed and deceleration to adjust the position of a coupling lens, so that the laser is accurately coupled to the end face of an imaging probe; the period control formula of the square wave signal corresponding to the acceleration or deceleration stage is: ， The period control formula of the square wave signal corresponding to the uniform speed stage is: ， Wherein, T is a pulse period, Max is a square wave control coefficient, and Max = 2A, A is a motor acceleration / deceleration step number, and n is the number of square wave signals sent in the pulse period T.

4. The data processing method of the confocal microscopic endoscope according to claim 2, wherein The galvanometer parameter area unidirectionally transmits data to horizontal and vertical galvanometer control modules; The photomultiplier parameter area unidirectionally transmits data to a photomultiplier control module; The distortion correction parameter area unidirectionally transmits data to a distortion correction module; The laser parameter area bidirectionally transmits data with a laser control module; The stepper motor position parameter area and the stepper motor motion parameter area respectively bidirectionally transmit data with a stepper motor control module; The data packaging parameter area bidirectionally transmits data with a data packaging module.

5. The data processing method of a confocal endomicroscope according to any one of claims 1 to 4, characterized in that, Further comprising a storage address pre-assignment step, specifically: Transmission configuration information of each type of parameter is obtained, the transmission configuration information at least comprising data transmission direction, parameter quantity and data update frequency; Predicted data amounts in preset time ranges under different control modes are calculated according to the transmission configuration information, and address configuration information of each type of parameter corresponding to the parameter area is generated, the address configuration information comprising data access frequency and maximum predicted data amount; According to the address configuration information, a storage address of each parameter area in the BRAM memory is pre-allocated.

6. The data processing method of the confocal microscopic endoscope according to claim 5, wherein Further comprising a preset address updating step, specifically: Obtaining a current utilization rate of each parameter area corresponding storage address or a current transmission flow direction of each parameter area corresponding parameter category, and when a preset condition is met, the pre-allocated storage address is adjusted in real time according to the current utilization rate and / or the current transmission flow direction.

7. The data processing method of the confocal microscopic endoscope according to claim 5, wherein The original image is subjected to distortion correction, specifically including the following steps: A distortion correction mapping relationship is established, and the distortion correction mapping relationship includes a correction reference value corresponding to each sampling point under different scanning distortion modes; A sinusoidal scanning image is obtained, the sinusoidal scanning image is subjected to image processing and analysis, a distortion correction parameter value is generated, and a target scanning distortion mode is generated according to the distortion correction parameter value; The distortion correction mapping relationship is queried, a target correction reference value corresponding to a sampling point at different positions is obtained according to the target scanning distortion mode, and each sampling point in the sinusoidal scanning image is subjected to distortion correction mapping based on the target correction reference value, and a corrected image is generated.

8. A data processing platform for a confocal microendoscope, characterized by An FPGA controller for executing the method of any one of claims 1-7 is included, and the FPGA controller includes a BRAM memory, a control module, a distortion correction module, and a data packaging module, The BRAM memory is used to partition to save at least one control value of a preset parameter, and the control value includes an initial control value and a real-time control value; The control module is used to call the initial control value of each preset parameter from the corresponding preset address of the BRAM memory and sequentially control the corresponding peripheral device, generate an original image of the tissue to be detected, and store the real-time control value of at least one target preset parameter in the control process to the BRAM memory; The distortion correction module is used to correct the distortion of the original image to generate a corrected image; The data packaging module is used to cache and package the corrected image according to the image data type, and send the packaged image data to the host computer through the ARM processor.

9. The data processing platform for a confocal microendoscope according to claim 8, wherein, Further comprising an ARM processor connected with the FPGA controller, and the ARM processor includes a storage module, a state updating thread, and an image transmission thread, The storage module is used to store at least one control value of a preset parameter in the process of using the confocal microscopic endoscope; The state updating thread is used to send the initial control value of the preset parameter to the BRAM memory and / or read the real-time control value of at least one target preset parameter from the BRAM memory in the process of using the confocal microscopic endoscope; The image transmission thread is used to receive the packaged image data and send it to the host computer.

10. The data processing platform for a confocal microendoscope according to claim 9, wherein, The state updating thread sends the initial control value to the BRAM memory through the AXI-Lite bus; and the data packaging module transmits the packaged image data to the ARM processor in a DMA mode through an internal high-speed interface.

Images