Offset calculation method for eliminating in real time image distortion caused by jitter of galvanometer, and related device

Abstract

The present application relates to the field of medical devices. Disclosed are an offset calculation method for eliminating in real time image distortion caused by a jitter of a galvanometer, and a related device. The method comprises: acquiring the latest received data of a target row; acquiring the number of optimal offset searches which corresponds to the target row; on the basis of the number of optimal offset searches, determining an offset search range; and calculating an alignment cost within the offset search range, so as to use, as a distortion offset, a corresponding offset having the minimum alignment cost.

Description

Method for calculating offset caused by image distortion of galvanometer jitter in real time and related equipment

[0001] The present application claims priority to the Chinese patent application No. 202410519932.0 filed on April 28, 2024, and titled "Method for calculating offset caused by image distortion of galvanometer jitter in real time and related equipment", the entire contents of which are incorporated herein by reference. The disclosure of all related applications that are cited herein and that are in the possession of the applicant(s) are hereby incorporated by reference in their entirety. TECHNICAL FIELD

[0002] The present application relates to the field of image processing, and more particularly, to a method for calculating offset caused by image distortion of galvanometer jitter in real time and related equipment. BACKGROUND

[0003] A confocal endoscope is a precision imaging device, which can achieve a resolution of nanometer level. The device can image in vivo in real time, and can present a cell morphology highly consistent with that of biopsy pathology, so that a doctor can accurately determine cancer, precancerous lesions or health status in situ under the premise of causing minimal discomfort to a patient.

[0004] A galvanometer is a core device of the confocal endoscope, and a slight jitter of a resonant mirror (fast mirror) in the galvanometer can cause a slight change in the spatial position of a sampling starting point, which can continuously cause distortion and disorder of a confocal image obtained by scanning, and can cause the confocal endoscope image to fail to achieve an expected resolution and to provide a good image basis for medical diagnosis. Therefore, how to accurately determine the offset caused by image distortion of galvanometer jitter in real time has become a problem to be solved for improving imaging quality. SUMMARY

[0005] A series of simplified concepts are introduced in the summary section, which will be further described in detail in the detailed description section. The summary section of the present application does not mean to attempt to limit the key features and essential technical features of the claimed technical solutions, and does not mean to attempt to determine the protection scope of the claimed technical solutions.

[0006] In a first aspect, the present application provides a method for calculating offset caused by image distortion of galvanometer jitter in real time, which comprises:

[0007] obtaining latest received data of a target row;

[0008] obtaining a number of times of searching for the best offset corresponding to the target row;

[0009] determining an offset search range according to the number of times of searching for the best offset.

[0010] The alignment cost is calculated in the search range of the offset, and a corresponding offset with the minimum alignment cost is taken as the distortion offset.

[0011] In an embodiment, the search range of the offset is determined according to the number of times of searching for the optimal offset, including:

[0012] In a case where the number of times of searching for the optimal offset is one, the search range of the offset is determined according to a half number of elements in the target row;

[0013] In a case where the number of times of searching for the optimal offset is more than one, the search range of the offset is determined according to a last distortion offset and a preset empirical threshold.

[0014] In an embodiment, the alignment cost is calculated in the search range of the offset, including:

[0015] In a case where the target row is a reference calculation row, the alignment cost is calculated according to a product of a reciprocal of a coincidence data amount in a current offset state and a first row reference cumulative difference value corresponding to the latest received data, wherein the first row reference cumulative difference value is an accumulated value of a difference between the latest received data and corresponding row reference data in a range corresponding to the coincidence data amount.

[0016] In an embodiment, the alignment cost is calculated in the search range of the offset, including:

[0017] In a case where the target row is a non-reference calculation row, the alignment cost is calculated according to a product of a reciprocal of a coincidence data amount in a current offset state and a second row reference cumulative difference value corresponding to the latest received data, wherein the second row reference cumulative difference value is an accumulated value of a sum of a first difference value and a second difference value in a range corresponding to the coincidence data amount, the first difference value is a difference between the latest received data and data of a first adjacent reference calculation row, and the second difference value is a difference between the latest received data and data of a second adjacent reference calculation row.

[0018] In an embodiment, the method further includes:

[0019] A brightness maximum value of the received data is determined according to the latest received data.

[0020] In a case where the brightness maximum value of the received data is less than a brightness threshold, a last distortion offset of the target row calculated last time is taken as a distortion offset of the current data.

[0021] In an embodiment, the method further includes:

[0022] updating a data cache pool in real time, wherein the data cache pool comprises H rows, each row of data comprises N pixels, and a capacity of each row of the cache pool is K;

[0023] performing denoising processing on K pieces of data in each row of the data cache pool to obtain denoised data;

[0024] calculating a median or mean of the denoised data to obtain processed data;

[0025] determining a minimum luminance value of the processed data;

[0026] obtaining a maximum luminance value corresponding to a hardware feature;

[0027] determining a luminance threshold value according to the minimum luminance value, the maximum luminance value, and a weighting coefficient.

[0028] In a possible implementation, the method further comprises:

[0029] selecting an odd row or an even row in the data cache pool as a reference calculation row;

[0030] determining a maximum luminance value of data in K pieces of data in each reference calculation row;

[0031] adding the maximum luminance value of the data that is greater than or equal to the luminance threshold value to a candidate data set;

[0032] calculating a median or mean of data in each row in the candidate data set to determine row reference data corresponding to each reference calculation row.

[0033] In a second aspect, an offset calculation device for eliminating image distortion caused by galvanometer jitter is provided, and the device comprises:

[0034] a first obtaining unit configured to obtain latest received data of a target row;

[0035] a second obtaining unit configured to obtain a number of times of searching for an optimal offset corresponding to the target row;

[0036] a determining unit configured to determine an offset search range according to the number of times of searching for the optimal offset;

[0037] a calculating unit configured to calculate an alignment cost in the offset search range, and to take a corresponding offset with a minimum alignment cost as a distortion offset.

[0038] In a third aspect, an electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor is configured to implement the steps of the method for calculating the offset amount of the image distortion caused by the mirror shaking in real time according to any one of the first aspect.

[0039] In a fourth aspect, the present application further provides a computer readable storage medium, which stores a computer program, and the computer program is executable on a processor to implement the method for calculating the offset amount of the image distortion caused by the mirror shaking in real time according to any one of the first aspect.

[0040] In summary, the method for calculating the offset amount of the image distortion caused by the mirror shaking in real time according to the embodiments of the present application includes: obtaining the latest received data of a target row; obtaining the number of times of searching for the best offset corresponding to the target row; determining the search range of the offset amount according to the number of times of searching for the best offset; and calculating the alignment cost in the search range of the offset amount, and taking the offset amount corresponding to the minimum alignment cost as the distortion offset. The method for calculating the offset amount of the image distortion caused by the mirror shaking in real time according to the embodiments of the present application can search and calculate the alignment cost in a specific search range by determining the search range of the offset amount according to the number of times of searching for the best offset, eliminate the influence of the change of the spatial position of the sampling starting point of the mirror in each row on the image, reduce the search range and the number of times, reduce unnecessary calculation, speed up the image processing, make the calculation efficiency of the distortion offset higher, and make the correction process of the image more rapid.

[0041] The method for calculating the offset amount of the image distortion caused by the mirror shaking in real time according to the embodiments of the present application, other advantages, objects and features of the present application will be embodied in part by the following description, and will be understood by those skilled in the art through research and practice of the present application. BRIEF DESCRIPTION OF DRAWINGS

[0042] Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The accompanying drawings are included to provide a description of preferred embodiments, and are not meant to limit the present description. Moreover, the same reference numerals are used throughout the accompanying drawings to represent same or similar components. In the drawings:

[0043] FIG. 1 is a flowchart of a method for calculating the offset amount of the image distortion caused by the mirror shaking in real time according to an embodiment of the present application;

[0044] FIG. 2 is a schematic diagram of the principle of alignment cost calculation when calculating the target behavior reference according to an embodiment of the present application;

[0045] Figure 3 is a schematic diagram illustrating the principle of alignment cost calculation when the target behavior is not a baseline calculation row, according to an embodiment of this application.

[0046] Figure 4 is a structural schematic diagram of an offset calculation device for eliminating image distortion caused by galvanometer jitter provided in an embodiment of this application;

[0047] Figure 5 is a schematic diagram of an electronic device for calculating offset to eliminate image distortion caused by galvanometer jitter, provided in an embodiment of this application. Detailed Implementation

[0048] The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus. The technical solutions of the embodiments of this application will now be clearly and completely described in conjunction with the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them.

[0049] Please refer to Figure 1, which is a flowchart illustrating a method for calculating offsets to eliminate image distortion caused by galvanometer jitter in real time, according to an embodiment of this application. Specifically, it may include:

[0050] S110, Obtain the latest received data for the target line;

[0051] For example, image data from endoscopic scans are collected in real time. The target row refers to the current scan row, while the latest received data refers to the image data obtained from the most recent scan of this row.

[0052] S120. Obtain the number of times to find the optimal offset for the target row mentioned above;

[0053] For example, to correct an image, it is necessary to determine the offset. The optimal offset count is the number of times the current row has previously undergone the operation of finding the optimal offset.

[0054] S130. Determine the offset search range based on the above-mentioned number of times to find the optimal offset;

[0055] For example, the corresponding offset search range is selected according to the number of times of searching for the optimal offset, so that the offset and its alignment cost calculation operation are performed within the offset search range.

[0056] In S140, the alignment cost is calculated within the above-mentioned offset search range, and the corresponding offset with the minimum alignment cost is taken as the distortion offset.

[0057] For example, the optimal offset is determined by calculating the alignment cost corresponding to each possible offset. The alignment cost refers to the degree of improvement in image quality after correcting the image with a certain offset. The offset that minimizes the alignment cost is selected as the final correction offset, thereby ensuring the accuracy and efficiency of image correction.

[0058] In summary, the offset calculation method for eliminating the image distortion caused by the mirror jitter in real time proposed in the embodiments of the present application determines the offset search range according to different numbers of searching for the optimal offset, eliminates the influence of the change in the spatial position of the sampling starting point of the mirror at each row on the image in real time, and can perform searching and alignment cost calculation within a specific search range, thereby reducing the search range and number, reducing unnecessary calculation, speeding up the image processing, making the distortion offset calculation more efficient, and making the correction process of the image more rapid.

[0059] In some examples, the determination of the offset search range according to the number of times of searching for the optimal offset includes:

[0060] In the case where the number of times of searching for the optimal offset is one, the offset search range is determined according to half the number of elements of the target row;

[0061] In the case where the number of times of searching for the optimal offset is more than one, the offset search range is determined according to the last distortion offset and a preset empirical threshold.

[0062] For example, when performing the first offset search, since there is a lack of previous correction data, the strategy is to determine the offset search range based on the number of elements of the target row (for example, the number of pixels of the image row). Specifically, half the number of elements of the target row is used as the offset search range. This means that if the target row has N pixels, the search range will be set to ±N / 2 pixels (or equivalent units). This wide range is intended to ensure that the correct offset is not missed, because there is not enough information to limit a smaller search range when searching for the first time.

[0063] Once at least one offset search has been performed, that is, there is historical data of one or more offset corrections, the strategy changes to using these historical data to more accurately define the search range. In this case, the search range is determined according to the last distortion offset and a preset empirical threshold.

[0064] Specifically, the value range of the offset search range a is [a min ,a max ]:

[0065] (1) If this line is the first time to find the best offset, take

[0066] (2) If this line is not the first time to find the best offset, take a min = a * - Δ, a max = a * + Δ, a * is the best offset found by this line last time. Δ is an empirically pre-set value, which can be 8;

[0067] In some examples, the alignment cost is calculated in the above-mentioned offset search range, including:

[0068] In the case of the above-mentioned target behavior reference calculation line, the alignment cost is calculated according to the product of the reciprocal of the amount of coincident data under the current offset state and the first line reference cumulative difference value corresponding to the latest received data, wherein the first line reference cumulative difference value is the cumulative value of the difference between the latest received data and the corresponding line reference data within the range corresponding to the amount of coincident data.

[0069] For example, as shown in FIG. 2, the reference calculation line can be an odd line or an even line in the cumulative data. When the reference calculation line is an odd line, and the target line is also an odd line; when the reference calculation line is an even line, and the target line is also an even line, the alignment cost is calculated according to the manner provided in the embodiment:

[0070] The amount of coincident data refers to the number of data points that the target line and its reference line coincide under the current offset state.

[0071] The first line reference cumulative difference value refers to the cumulative difference of the data points of the target line and its reference line in the coincident part under the current offset state. This cumulative difference value reflects the overall difference degree between the two lines of data within the considered coincident range. This value is obtained by summing the difference between the latest received data of the target line and the corresponding reference line data at each coincident point. The product of the reciprocal of the amount of coincident data and the first line reference cumulative difference value is taken as the alignment cost, which comprehensively reflects the matching degree of the target line and its reference line under the current offset state. A smaller product means a higher matching degree, i.e., better alignment quality; on the contrary, a larger product means a lower matching degree, i.e., poorer alignment quality.

[0072] Specifically, the alignment cost cost can be calculated by the following formula:

[0073] wherein sid and eid are the start index and end index of the overlap part, sid = max(a, 0), eid = min(N-1, N-1+a), R n is the latest received data, h * is the line number of the latest received data, is the line reference data corresponding to the latest received data.

[0074] In summary, by considering the product of the reciprocal of the overlap data amount and the first line reference cumulative difference value, the present scheme can accurately evaluate the alignment quality under different offset amounts. This method can reflect the matching degree between the target line and the reference line in detail, making the matching process more accurate.

[0075] In some examples, the above calculating the alignment cost in the above offset search range includes:

[0076] In the case of the target behavior being a non-reference calculation line, the alignment cost is calculated according to the product of the reciprocal of the overlap data amount of the adjacent two reference calculation lines and the second line reference cumulative difference value corresponding to the latest received data under the current offset state, wherein the second line reference cumulative difference value is the cumulative value of the sum of the first difference value and the second difference value within the range corresponding to the overlap data amount, the first difference value is the difference between the latest received data and the first adjacent reference calculation line data, and the second difference value is the difference between the latest received data and the second adjacent reference calculation line data.

[0077] For example, as shown in FIG. 3, the reference calculation line can be an odd line or an even line in the cumulative data. When the reference calculation line is an odd line, the target line is an even line, or when the reference calculation line is an even line, the target line is an odd line, the alignment cost is calculated according to the manner provided by the present embodiment:

[0078] The reciprocal of the overlap data amount is used to reflect the influence of the size of the overlap part on the alignment cost, that is, the smaller the overlap data amount is, the larger the reciprocal is, indicating that the reliability is lower, and thus the alignment cost will be increased accordingly. Then, the difference (first difference) between the latest received data of the target row and the data of the first adjacent reference calculation row, and the difference (second difference) between the latest received data of the target row and the data of the second adjacent reference calculation row are calculated. The second row reference cumulative difference value is obtained by accumulating the first difference and the second difference in the range corresponding to the overlap data amount. This accumulated value reflects the overall difference between the target row and the two reference rows in the considered offset state. Finally, the product of the reciprocal of the overlap data amount and the second row reference cumulative difference value is taken as the alignment cost. This alignment cost comprehensively considers the matching degree of the target row and the two reference rows, and a smaller product means a higher matching degree, that is, a better alignment quality; a larger product means a lower matching degree, that is, a poorer alignment quality.

[0079] Specifically, the alignment cost cost can be calculated by the following formula:

[0080] wherein sid and eid are the start subscript and the end subscript of the overlap part, sid = max (a, 0),

[0081] eid = min (N-1, N-1+a), R n is the latest received data, is the data of the first adjacent reference calculation row is the data of the second adjacent reference calculation row.

[0082] In summary, by considering the difference with the two reference calculation rows, this method can more accurately evaluate and adjust the alignment state of the non-reference row, and improve the alignment precision and quality of the overall image.

[0083] In some examples, the above method further comprises:

[0084] determining the maximum brightness value of the received data according to the above latest received data;

[0085] in the case where the maximum brightness value of the received data is less than the brightness threshold value, taking the distortion offset calculated last time for the target row as the distortion offset of the current data.

[0086] For example, first, the maximum brightness value tR is determined from the latest received image data, wherein tR = max{R R n ​}, n = 0, 1, …, N-1, if tR is less than the luminance threshold Lth, this case is low luminance, to avoid unpredictable results, directly using the last time the best offset group as the distortion offset of this data. If tR is greater than or equal to the luminance threshold Lth, the distortion offset is determined according to the scheme in the above embodiment.

[0087] In some examples, the above method further comprises:

[0088] updating the data cache pool in real time, wherein the data cache pool comprises H rows, each row of data comprises N pixels, and the capacity of each row of the cache pool is K;

[0089] performing denoising processing on the K pieces of data of each row in the data cache pool to obtain denoised data;

[0090] obtaining the median or mean of the denoised data to obtain processed data;

[0091] determining the minimum luminance value of the processed data;

[0092] obtaining the maximum luminance value corresponding to the hardware characteristics;

[0093] determining the luminance threshold according to the minimum luminance value, the maximum luminance value, and a weighting coefficient.

[0094] For example, the collected data has H different row numbers, and each row of data has N pixels. After preprocessing (flipping), a cache pool with a capacity of K is set for each row, and no more than K pieces of the latest original data are cached. K is an empirical value, which can be any positive integer. The larger the value, the more accurate the luminance threshold and the real-time offset calculated based on the cached data, but the larger the calculation amount, and the higher the requirement for processor performance.

[0095] Let the cached data be h represents the row number, n represents the nth pixel in a row, and k represents the kth data in the K pieces of data cached in the hth row, sorted from the latest to the oldest in time. Real-time updating of the data cache pool means that when new data is received, if the number of data in the cache pool is less than K, the new data is put into the cache pool; if the number of data in the cache pool has reached K, the oldest data in the cache pool is removed, and the latest data is put into the cache pool. This operation is continuously performed, and after a period of time, the data in the cache pool is always the latest K pieces of data.

[0096] The K pieces of data of each row, i.e. K values for each pixel position, are obtained by taking the mean of the K values. Let the denoised data be

[0097] It should be noted that the above-mentioned average value taking method can also take the median value.

[0098] The effect of averaging / mid-value taking of multiple data is to eliminate the influence of noise, so that the subsequent calculation of brightness threshold and real-time offset is more accurate.

[0099] Statistics The minimum value of the brightness of all brightness is minV

[0100] The maximum value of the brightness determined by the system hardware characteristics is maxV.

[0101] The brightness threshold Lth can be calculated by the following formula: L th = minV + a * (maxV-minV)

[0102] The weight coefficient a is a preset value according to experience, which can be 1 / 2.

[0103] The system hardware characteristics, such as the hardware of the amplification circuit, the AD sampling chip, etc., determine the value range of the brightness value.

[0104] In summary, by updating the data cache pool in real time, it is ensured that the processed data is always the latest, so as to more accurately reflect the current imaging conditions and environmental changes. By denoising each row of data, the influence of data noise on the final imaging quality is reduced, and the clarity and accuracy of the image are improved. By taking the median or mean value of the denoised data and its minimum value, the influence of extreme values can be effectively eliminated, ensuring the stability and reliability of the brightness threshold calculation. By considering the maximum brightness determined by the hardware characteristics, this parameter is introduced when determining the brightness threshold, ensuring that the threshold setting considers both the physical limitations of the hardware and the brightness characteristics of the real-time data. By adjusting the brightness threshold based on real-time data and hardware characteristics, it provides the ability to dynamically adapt to different imaging conditions, making the image processing process more flexible and more adaptable. The introduction of the weighting coefficient allows the calculation of the brightness threshold to be adjusted according to the empirical preset value, providing flexibility for real-time image processing to adapt to different imaging environments and purposes. Through fine data processing and dynamic adjustment of the brightness threshold, the calculation error caused by insufficient or excessive brightness and noise is reduced, and the accuracy of the offset calculation and image correction is improved.

[0105] In some examples, the above method further comprises:

[0106] Selecting odd-numbered rows or even-numbered rows in the data cache pool as reference calculation rows;

[0107] Determining the maximum value of K data in each of the above-mentioned reference calculation rows;

[0108] The data maximum value greater than or equal to the above luminance threshold is added to the candidate data set;

[0109] The median or mean value operation is performed on the data of each row in the candidate data set to determine the row reference data corresponding to each reference calculation row.

[0110] For example, odd rows or even rows are selected as reference calculation rows in the data cache pool. The following describes an example in which all odd rows are used as reference rows.

[0111] The data used to calculate the row reference is selected from the cached data. For the K data of the hth row cache, find the maximum value of each data

[0112] If The kth data is added to the candidate data set S h The number of data in S h is P:

[0113] The data set S h obtained by filtering the hth row is averaged to obtain a data which is the reference of the hth row.

[0114] It should be noted that the above mean value method can also use the median value.

[0115] In summary, the present application only adds data points with luminance greater than or equal to the preset luminance threshold to the candidate data set, ensuring that the data used to calculate the row reference is obtained under good imaging conditions, thereby reducing noise and interference of data under low brightness conditions. By performing median or mean value operation on the data of each row in the candidate data set, the influence of abnormal values can be further reduced, ensuring the representativeness and stability of the row reference data. Selecting odd rows or even rows as reference calculation rows reduces the amount of data to be processed, while ensuring the representativeness and coverage of the data, improving processing efficiency and reducing computational burden. By dynamically determining the luminance threshold in combination with hardware characteristics and real-time data, the determination of the row reference data is more flexible and adaptive, and can adapt to different imaging conditions and environmental changes. Through a series of refined data selection and processing steps, not only the accuracy of the confocal endoscope image correction and the image quality are improved, but also the processing flow is optimized, the efficiency is improved, which has important practical value for improving the accuracy and reliability of medical image analysis.

[0116] Referring to FIG. 4, the offset calculation device structure for eliminating image distortion caused by galvanometer jitter provided by the present application can include:

[0117] The first obtaining unit is configured to obtain latest received data of a target line.

[0118] The second obtaining unit is configured to obtain a number of times of searching for an optimal offset corresponding to the target line.

[0119] The determining unit is configured to determine an offset search range according to the number of times of searching for the optimal offset.

[0120] The calculating unit is configured to calculate an alignment cost in the offset search range, and take a corresponding offset with the minimum alignment cost as a distortion offset.

[0121] As shown in FIG. 5, the embodiment of the present application further provides an electronic device 300, which comprises a memory 310, a processor 320, and a computer program 311 stored in the memory 310 and capable of running on the processor. When the processor 320 executes the computer program 311, the steps of any method for calculating an offset for eliminating image distortion caused by mirror shaking are implemented.

[0122] Any method for calculating an offset for eliminating image distortion caused by mirror shaking provided by the present application can be used in a conventional confocal endoscope or a disposable confocal endoscope.

[0123] Since the electronic device introduced in the embodiment is a device used for implementing the offset calculation device for eliminating image distortion caused by mirror shaking in the embodiment of the present application, the specific implementation of the electronic device and its various changes can be understood by those skilled in the art based on the method introduced in the embodiment of the present application. Therefore, how the electronic device implements the method in the embodiment of the present application is not described in detail, as long as the device used for implementing the method in the embodiment of the present application is used by those skilled in the art, which belongs to the scope of the present application.

[0124] In the implementation process, the computer program 311 can implement any embodiment of the corresponding embodiment of FIG. 1 when executed by the processor.

[0125] It should be noted that in the above embodiments, the description of each embodiment has its own emphasis, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0126] Those skilled in the art will appreciate that embodiments of the application can be supplied as a method, a system, or a computer program product. Accordingly, the application can be embodied in the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the application can be embodied in the form of a computer program product on one or more computer readable storage media (including, but not limited to, disk memory, CD-ROMs, optical storage media, etc.) having computer usable program code embodied thereon.

[0127] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks or in combination with the flowchart block or blocks.

[0128] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart block or blocks or combination thereof.

[0129] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks or in combination with the flowchart block or blocks.

[0130] Embodiments of the present application also provide a computer program product, which includes computer software instructions, when the computer software instructions are run on a processing device, cause the processing device to execute the flow of eliminating the offset amount of the image distortion caused by the galvanometer dithering in the corresponding embodiments.

[0131] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the processes or functions according to the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that the computer can store or be integrated into a data storage device such as a server, data center, etc. containing one or more available media. The available media can be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state disk (SSD)), etc.

[0132] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described system, device and unit can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

[0133] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interface, device or unit, and can be electrical, mechanical or other forms.

[0134] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or they can be distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiments.

[0135] In addition, each function unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software function unit.

[0136] When the integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the embodiments of the method of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0137] The above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements to some technical features. These modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application. Industrial applicability

[0138] The offset calculation method for eliminating the image distortion caused by the mirror shaking provided by the embodiments of the present application can determine the search range of the offset according to different optimal offset search times, can search and calculate the alignment cost in a specific search range, eliminate the influence of the change of the spatial position of the sampling starting point of the mirror at each line on the image, reduce the search range and the number of times, reduce unnecessary calculation, speed up the image processing speed, make the calculation efficiency of the distorted offset higher, and make the correction process of the image more rapid. Therefore, the offset calculation method for eliminating the image distortion caused by the mirror shaking in real time and the related device provided by the embodiments of the present application have industrial applicability.

Claims

1. A method for calculating an offset amount for eliminating image distortion caused by a galvanometer dither in real time, characterized by, The method comprises the following steps: acquiring the latest received data of a target row; acquiring the number of times of searching for the best offset corresponding to the target row; determining an offset search range according to the number of times of searching for the best offset; calculating an alignment cost in the offset search range to take the corresponding offset with the minimum alignment cost as the distortion offset.

2. The method of claim 1, wherein the method further comprises: The step of determining the offset search range according to the number of times of searching for the best offset comprises the following steps: in the case that the number of times of searching for the best offset is one time, determining the offset search range according to the half number of the elements of the target row; in the case that the number of times of searching for the best offset is more than one time, determining the offset search range according to the last distortion offset and a preset empirical threshold.

3. The method of claim 1, wherein the method further comprises: The step of calculating the alignment cost in the offset search range comprises the following steps: in the case that the target row is a reference calculation row, calculating the alignment cost according to the product of the reciprocal of the amount of coincident data in the current offset state and the first row reference cumulative difference value corresponding to the latest received data, wherein the first row reference cumulative difference value is the cumulative value of the difference between the latest received data and the row reference data corresponding thereto within the range corresponding to the amount of coincident data.

4. The method of claim 1, wherein the method further comprises: The step of calculating the alignment cost in the offset search range comprises the following steps: in the case that the target row is a non-reference calculation row, calculating the alignment cost according to the product of the reciprocal of the amount of coincident data of the adjacent two reference calculation rows in the current offset state and the second row reference cumulative difference value corresponding to the latest received data, wherein the second row reference cumulative difference value is the cumulative value of the sum of the first difference value and the second difference value within the range corresponding to the amount of coincident data, the first difference value being the difference between the latest received data and the data of the first adjacent reference calculation row, and the second difference value being the difference between the latest received data and the data of the second adjacent reference calculation row.

5. The method of claim 1, wherein the method further comprises: The method further comprises the following steps: determining the maximum value of the received data according to the latest received data; in the case that the maximum value of the received data is less than a luminance threshold value, taking the distortion offset calculated last time on the target row as the distortion offset of the current data.

6. The method of claim 1 to 5, wherein the method is characterized by, The method further comprises the following steps: updating the data cache pool in real time, wherein the data cache pool comprises H rows, each row of data comprises N pixels, and the capacity of each row of the cache pool is K; performing denoising processing on the K pieces of data of each row in the data cache pool to obtain denoised data; calculating the median or mean value of the denoised data to obtain processed data; determining the minimum luminance value of the processed data; acquiring the maximum luminance value corresponding to the hardware feature; determining a luminance threshold value according to the minimum luminance value, the maximum luminance value and a weighting coefficient.

7. The method of claim 6, wherein the method further comprises: The method further comprises the following steps: selecting an odd row or an even row in the data cache pool as a reference calculation row; determining the maximum data value in the K pieces of data in each reference calculation row; adding the maximum data value greater than or equal to the luminance threshold value to a candidate data set; calculating the median or mean value of the data of each row in the candidate data set to determine the row reference data corresponding to each reference calculation row.

8. An offset calculation device for eliminating image distortion caused by galvanometer jitter, characterized in that, The method comprises the following steps: a first acquisition unit is configured to acquire the latest received data of a target row; a second acquisition unit is configured to acquire the number of times of searching for the best offset corresponding to the target row; A determining unit is configured to determine a search range of the offset according to the number of times of searching for the optimal offset; A calculating unit is configured to calculate alignment costs in the search range of the offset, and take a corresponding offset with the minimum alignment cost as a distortion offset.

9. An electronic device comprising: A memory and a processor, wherein the processor is configured to implement the steps of the method for calculating the offset for eliminating the image distortion caused by the mirror dithering in real time according to any one of claims 1-7 when executing the computer program stored in the memory.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method for calculating the offset for eliminating the image distortion caused by the mirror dithering in real time according to any one of claims 1-7.

Images