A method and system for compressing massive time series data for a photoelectric detection system

Abstract

This invention discloses a method and system for compressing massive time-series data for photoelectric detection systems, relating to the field of photoelectric time-series data compression. The method includes: dividing the detection phase based on the detection state in the photoelectric calibration sequence, calculating local variations and fitting deviations, marking positions that exceed noise boundaries, dividing the time-series segments along the detection phase boundaries and marked positions, and generating a photoelectric segment sequence; establishing a dynamic channel graph based on the channel consistency and delay relationship of the photoelectric segment sequence, eliminating abnormal channels, merging valid channel sample values ​​according to connection weights to form a common optical response, determining the response delay and response ratio, performing time-series migration and amplitude adjustment on the common optical response, calculating channel residual values, and associating the common optical response and channel residual values ​​according to segment positions to generate a channel association sequence. This invention reduces redundant coding, shortens the compressed code length, and maintains reconstruction accuracy.

Description

Technical Field

[0001] This invention relates to the field of photoelectric time-series data compression, and in particular to a method and system for compressing massive time-series data for photoelectric detection systems. Background Technology

[0002] With the continuous expansion of the application of high-speed photodetectors, array sensors and synchronous acquisition equipment in spectral detection, low light measurement and online monitoring, the photoelectric detection process usually records the sampling values, gain levels, integration time and detection status of multiple detection channels continuously at a high sampling frequency. This results in a large amount of multi-channel data with obvious time-series correlation. Existing compression technologies mostly use predictive coding, piecewise fitting, quantization coding and zero-value run coding, which are processed independently by channel. The quantization interval is adjusted according to the preset error limit to reduce the storage capacity and transmission bandwidth requirements of the original sampling data.

[0003] However, existing methods of independent compression per channel or using fixed channel correlation models typically establish prediction relationships directly within each detection channel. This fails to synchronously handle time offsets, gain differences, and abnormal channel disturbances between channels as the detection stage changes. When the same optical event manifests as similar waveforms with different response delays and amplitude ratios in different channels, common optical changes are easily included in the prediction residuals of each channel. Abnormal channels may also change fixed correlation parameters, expanding the residual distribution range, reducing zero-value continuity, and increasing the coding bit width. Under the condition of ensuring that the segment reconstruction deviation does not exceed the limit, problems such as increased compressed code length and fluctuations in compression results with channel status are likely to occur. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides a method for compressing massive time-series data for photoelectric detection systems to solve the problem of repeated encoding of common responses and uncontrollable reconstruction deviations in multi-channel photoelectric time-series data.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: In a first aspect, the present invention provides a method for compressing massive time-series data for photoelectric detection systems, comprising: acquiring photoelectric sampling values ​​and sampling attributes; generating a photoelectric calibration sequence by correcting the sampling time, subtracting dark levels, and unifying the gain magnitude; dividing the detection phases based on the detection states in the photoelectric calibration sequence, calculating local variations and fitting deviations, marking positions that exceed noise boundaries, dividing time-series segments along the detection phase boundaries and marked positions, and generating a photoelectric segment sequence; establishing a dynamic channel diagram based on the channel consistency and time delay relationship of the photoelectric segment sequence, eliminating abnormal channels, merging effective channel sampling values ​​according to connection weights to form a common optical response, determining the response delay and response ratio, performing time-series migration and amplitude adjustment on the common optical response, and calculating... The channel residual values ​​are used to generate a channel association sequence by associating the common optical response with the channel residual values ​​according to the segment position. Candidate prediction methods are established based on the channel association sequence, and the prediction residual values ​​are calculated one by one. The expected code length of each candidate prediction method is determined according to the residual coding characteristics. Candidate prediction methods with reconstruction deviation exceeding the limit are eliminated. The target prediction method with the shortest expected code length is selected from the remaining candidate prediction methods. The prediction residual values ​​of the target prediction method are subjected to hierarchical quantization, zero-value merging, and variable bit-width coding to generate a candidate compression sequence. The candidate compression sequence is temporally reconstructed and the segment reconstruction deviation is calculated. When the segment reconstruction deviation exceeds the limit, the quantization interval is reduced. If it still exceeds the limit, a sampling anchor point is added. The candidate compression sequences that pass the verification are arranged according to the detection time sequence to generate an optoelectronic compressed record.

[0007] As a preferred embodiment of the massive time-series data compression method for photoelectric detection systems described in this invention, the step of generating a photoelectric calibration sequence by correcting the sampling time, subtracting the dark level, and unifying the gain level includes the following specific steps: The time offset of each detection channel is calculated based on the sampling timestamp, channel identifier and synchronization trigger mark corresponding to the photoelectric sampling value, the sampling time is corrected, and the photoelectric sampling values ​​are rearranged according to the corrected sampling time to form a time correction sequence. The channel identifier, gain level, and integration time are extracted from the sampling attributes of the time-correction sequence. The corresponding dark level and gain calibration coefficient are matched, the dark level in the photoelectric sampling value is subtracted, and then converted to a unified reference gain level to generate the photoelectric calibration sequence.

[0008] As a preferred embodiment of the massive time-series data compression method for photoelectric detection systems described in this invention, the specific steps for generating the photoelectric fragment sequence are as follows: The detection states in the photoelectric calibration sequence are compared along the calibration sampling time. The detection stage boundary is defined at the position where the state changes, and the photoelectric calibration values ​​under the same detection state are merged to generate the detection stage sequence. Within each detection stage, photoelectric calibration values ​​are arranged by channel. The local window half-width is determined by the integration time and the reference sampling interval. The absolute difference between the median of the photoelectric calibration values ​​before and after the window is calculated. Local linear fitting is performed after excluding the center position to generate a local change sequence and a fitting deviation sequence. Initial noise boundaries are established based on the median and median absolute deviation of local changes and fitting deviations, respectively. After removing the initial out-of-bounds values, the corresponding noise boundaries are redefined. The positions that exceed any noise boundary are registered as initial marker positions, and the positions with the largest out-of-bounds intensity are retained within the continuous out-of-bounds range to generate a marker position sequence. The detection stage boundaries and marker positions are collected, duplicate markers adjacent to the detection stage boundaries are deleted, and the photoelectric calibration values ​​are divided sequentially according to the retained positions. The time segments are then arranged according to the calibration sampling time to generate a photoelectric segment sequence.

[0009] As a preferred embodiment of the massive time-series data compression method for photoelectric detection systems described in this invention, the specific steps for forming a common optical response are as follows: The photoelectric calibration values ​​of each channel within the same time segment are extracted according to the segment position. After subtracting the segment median, relative time shift is performed. The channel consistency and initial channel delay are determined by the maximum normalized cross-correlation coefficient. Parabolic interpolation is performed on the relevant peaks to generate channel relationship records. Graph nodes are established using channel identifiers. Connection edges are created for channels whose channel consistency reaches the consistency boundary. Connection edges with excessive delay closure error are deleted. Connection weights are configured based on channel consistency and delay closure error to generate a dynamic channel graph. Summarize the connection weights associated with each graph node in the dynamic channel graph, determine the lower bound of the connection strength based on the median and median absolute deviation of the node connection strength, iteratively delete graph nodes that are below the lower bound of the connection strength, and generate an effective dynamic channel graph. The channel with the highest node connection strength in the effective dynamic channel graph is selected as the reference channel. The cumulative delay of the remaining effective channels is determined along the minimum cost path. The sampled values ​​of each effective channel are migrated to the reference time axis and weighted and merged according to the merging weight corresponding to the node connection strength to generate a common optical response.

[0010] As a preferred embodiment of the massive time-series data compression method for photoelectric detection systems described in this invention, the specific steps of determining the response delay and response ratio, performing time-series migration and amplitude adjustment on the common optical response, calculating the channel residual value, and associating the common optical response and channel residual value according to the segment position to generate a channel association sequence are as follows: The common optical response is moved with the accumulated delay as the search center. The response delay of each effective channel is determined by the position of the maximum normalized cross-correlation. Then, unbiased least squares proportional estimation is performed on the time-aligned common optical response to generate channel response parameter records. Based on the common optical response and the response delay, linear interpolation is performed on non-integer sampling times and the amplitude is adjusted according to the response ratio. The channel residual values ​​of each effective channel relative to the adjusted response are calculated. Then, the common optical response, channel response parameters and channel residual values ​​are associated according to the segment position to generate a channel association sequence.

[0011] As a preferred embodiment of the massive time-series data compression method for photoelectric detection systems described in this invention, the specific steps of establishing candidate prediction methods based on channel correlation sequences and calculating prediction residual values ​​one by one are as follows: The common optical response and residual values ​​of each channel are arranged according to the segment position. The continuous sampling range is truncated at the segment boundary and invalid position, and the starting value of each continuous sampling range is retained as the prediction anchor point to generate the prediction object sequence. Prediction rules are configured along the continuous sampling range of the prediction object sequence. For the common optical response, previous value prediction and variation extrapolation prediction are configured. For the channel residual value, zero value prediction, previous value prediction and variation extrapolation prediction are configured. The prediction anchor points corresponding to each prediction rule are registered, and variation extrapolation predictions with insufficient previous sampling positions are deleted to establish candidate prediction methods. The current predicted value is determined point by point according to the candidate prediction method. The difference between the current actual value and the current predicted value is calculated to form the prediction residual value. The previous sampling position of the corresponding prediction method is updated with the current actual value, and the predicted value of the next sampling position is determined.

[0012] As a preferred embodiment of the massive time-series data compression method for photoelectric detection systems described in this invention, the specific steps of determining the expected code length of each candidate prediction method based on residual coding characteristics, eliminating candidate prediction methods with reconstruction deviation exceeding the limit, and selecting the target prediction method with the shortest expected code length from the remaining candidate prediction methods are as follows: The quantization segment is divided along the sampling order of the predicted residual value. The quantization interval is gradually widened starting from the quantization interval corresponding to the photoelectric sampling resolution. The widening is stopped before the theoretical quantization deviation exceeds the allowable reconstruction deviation. The predicted residual value is converted into a signed integer index to generate a trial quantization residual index sequence. The range of consecutive zero values ​​is identified along the trial quantization residual index sequence. The zero value merging position is determined by comparing the occupied code length of length coding and item-by-item coding. Positive and negative interleaving mapping is performed on the remaining signed integer indices. The boundary of the numerical block is determined according to the coding overhead of expanding the current numerical block and creating a new numerical block. Then, the corresponding coding block is simulated and written to determine the expected code length of each candidate prediction method. The current reconstructed value is formed point by point according to the trial quantization residual index sequence, and the trial reconstructed photoelectric calibration value is restored according to the channel response parameters. Candidate prediction methods that exceed the allowable reconstruction deviation are eliminated. If no candidate prediction methods are retained, the quantization interval is reduced and re-verified. Then, the target prediction method with the shortest expected code length is selected from the retained candidate prediction methods.

[0013] As a preferred embodiment of the massive time-series data compression method for photoelectric detection systems described in this invention, the specific steps for performing hierarchical quantization, zero-value merging, and variable bit-width coding on the prediction residual value of the target prediction method to generate a candidate compression sequence are as follows: recalculate the prediction residual value according to the target prediction method and perform hierarchical quantization; merge consecutive zero values ​​according to the code length comparison results; divide the numerical blocks according to the coding overhead corresponding to the effective bit-width change and write them into the quantization index; arrange the coding results according to the segment position to generate a candidate compression sequence.

[0014] As a preferred embodiment of the massive time-series data compression method for photoelectric detection systems described in this invention, the specific steps for generating photoelectric compressed records are as follows: Expand the candidate compressed sequence and recover the photoelectric calibration value point by point. After comparing it with the original photoelectric calibration value, determine the segment reconstruction deviation and generate a segment reconstruction deviation record. When the segment reconstruction deviation exceeds the allowable reconstruction deviation, the quantization interval of the quantization segment corresponding to the maximum deviation position is reduced and re-encoded and verified; when the quantization interval still exceeds the limit after reaching the photoelectric sampling resolution, the original value is written at the maximum deviation position as the sampling anchor point, and prediction and encoding are re-started from the sampling anchor point until the segment reconstruction deviation passes the verification and an effective compressed sequence is generated. Arrange the effective compressed sequence according to the sampling time of the segment start correction, register the segment compression length at the start position of each segment, and then splice the corresponding coded content in sequence to generate an optoelectronic compressed record.

[0015] Secondly, the present invention provides a massive time-series data compression system for photoelectric detection systems, including: a photoelectric calibration module, used to collect photoelectric sampling values ​​and sampling attributes, and generate a photoelectric calibration sequence by correcting the sampling time, subtracting the dark level and unifying the gain level; The time-series segmentation module is used to divide the detection stage by combining the detection state in the photoelectric calibration sequence, calculate the local change and fitting deviation, mark the position that exceeds the noise boundary, divide the time-series segment along the detection stage boundary and the marked position, and generate the photoelectric segment sequence. The channel association module is used to establish a dynamic channel diagram based on the channel consistency and time delay relationship of the photoelectric segment sequence, eliminate abnormal channels, merge the sampled values ​​of effective channels according to the connection weight, form a common optical response, determine the response time delay and response ratio, perform time migration and amplitude adjustment on the common optical response, calculate the channel residual value, and associate the common optical response and channel residual value according to the segment position to generate a channel association sequence. The prediction compression module is used to establish candidate prediction methods based on the channel association sequence, calculate the prediction residual value one by one, determine the expected code length of each candidate prediction method according to the residual coding characteristics, eliminate candidate prediction methods with reconstruction deviation exceeding the limit, select the target prediction method with the shortest expected code length from the remaining candidate prediction methods, and perform hierarchical quantization, zero-value merging and variable bit-width coding on the prediction residual value of the target prediction method to generate candidate compressed sequences. The reconstruction and verification module is used to perform temporal reconstruction of candidate compressed sequences and calculate fragment reconstruction deviation. When the fragment reconstruction deviation exceeds the limit, the quantization interval is reduced. If it still exceeds the limit, the sampling anchor point is increased. The candidate compressed sequences that pass the verification are arranged according to the detection time sequence to generate photoelectric compressed records.

[0016] The beneficial effects of this invention are as follows: by establishing a dynamic channel graph based on the channel consistency and time delay relationship within a time segment, abnormal channels are iteratively eliminated, the time axis of effective channels is unified along the minimum cost path, and common optical responses are formed by merging them according to connection weights. Then, the common optical responses and channel residuals are separated according to response delay and response ratio, so that cross-channel repetitive waveforms are centrally expressed and local differences are transformed into residuals with small amplitudes and continuous zero values. This provides a stable object for subsequent predictive quantization and variable-width coding, reduces repetitive coding, shortens compressed code length, and maintains reconstruction accuracy. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart of a method for compressing massive time-series data for photoelectric detection systems.

[0019] Figure 2 This is a schematic diagram of a massive time-series data compression system for photoelectric detection systems.

[0020] Figure 3 This is a flowchart for photoelectric calibration and detection segmentation.

[0021] Figure 4 A flowchart for constructing dynamic channel correlation and common optical response. Detailed Implementation

[0022] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0023] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0024] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0025] Reference Figures 1-4 This is one embodiment of the present invention, which provides a method for compressing massive time-series data for photoelectric detection systems, including the following steps: S1. Collect photoelectric sampling values ​​and sampling attributes, and generate a photoelectric calibration sequence by correcting the sampling time, subtracting the dark level and unifying the gain level.

[0026] It should also be noted that the analog electrical signals output by each detection channel of the photodetector after receiving incident light are collected, and then pre-amplified and converted from analog to digital to form the original digital sampled values. At the same time, the sampling timestamp, channel identifier, synchronization trigger mark, gain level, integration time, detection status and sampling resolution are recorded to form photoelectric sampled values ​​and sampling attributes.

[0027] S1.1 Calculate the time offset of each detection channel based on the sampling timestamp, channel identifier and synchronization trigger mark corresponding to the photoelectric sampling value, correct the sampling time, and rearrange the photoelectric sampling values ​​according to the corrected sampling time to form a time correction sequence.

[0028] Furthermore, photoelectric sampling values ​​with the same synchronous trigger mark are grouped into the same trigger batch according to the channel identifier. Within the same trigger batch, the order of sampling timestamps is checked, and photoelectric sampling values ​​with missing synchronous trigger marks, duplicate records in the same channel, and invalid sampling timestamps are excluded. The median time of the remaining valid sampling timestamps is determined as the trigger reference time. The time difference between the sampling timestamp of each detection channel and the trigger reference time is calculated. The time differences corresponding to consecutive trigger batches are merged according to the channel identifier, and the median value of the merged time difference is used to determine the time offset of each detection channel.

[0029] The time offset is subtracted from the original sampling timestamp corresponding to the photoelectric sampling value with valid correction conditions, and the subtraction result is registered as the corrected sampling time. When the synchronization trigger mark is missing, there are duplicate records in the same channel within the same trigger batch, the sampling timestamp is invalid, or the number of valid trigger batches is lower than the lower limit of the number of valid trigger batches, the corresponding original sampling timestamp is retained, the time correction anomaly mark is registered, and the time correction anomaly mark is associated with and retained with the corresponding channel identifier, the original sampling timestamp, and the anomaly type.

[0030] The photoelectric sampled values ​​that have completed time correction are arranged in ascending order of the corrected sampling time, and the same corrected sampling time is arranged according to the channel identifier; the photoelectric sampled values ​​with time correction anomaly markers are arranged in the corresponding time positions according to the original sampling timestamp, retaining the qualification to participate in dark level subtraction and gain magnitude conversion, but not participating in the detection stage boundary determination, local window calculation and channel delay estimation that depend on the channel time correspondence, and the time correction anomaly markers are retained in the photoelectric calibration sequence along with the corresponding photoelectric sampled values ​​to form a time correction sequence.

[0031] It should also be noted that the synchronization trigger mark is obtained by the photoelectric acquisition device writing it into the sampling attribute corresponding to the photoelectric sampling value when the unified trigger signal arrives.

[0032] The lower limit for the number of valid trigger batches is set by combining the detection duration and the frequency of synchronous triggering. For example, the example value is five consecutive valid trigger batches.

[0033] S1.2 Extract the channel identifier, gain level and integration time from the sampling attributes of the time calibration sequence, match the corresponding dark level and gain calibration coefficient, subtract the dark level in the photoelectric sampling value, and then convert it to a unified reference gain level to generate the photoelectric calibration sequence.

[0034] Furthermore, the channel identifier, gain level, and integration time corresponding to the photoelectric sampling values ​​in the time calibration sequence are checked item by item, and the calibration records with consistent three fields are associated with the corresponding photoelectric sampling values. When there are multiple valid calibration records, the calibration record with the closest calibration time and earlier than the corresponding sampling timestamp is selected. When the calibration record is missing, the gain calibration coefficient is invalid, or the three fields cannot be consistent, the corresponding photoelectric sampling value is retained, a calibration missing mark is registered, and the dark level baseline correction and gain scale mapping of the corresponding photoelectric sampling value are stopped.

[0035] Baseline stripping is performed on the photoelectric sampling values ​​based on the dark level in the matching calibration record, and the signed change results on both sides of the baseline are retained to form dark level correction values. A scale mapping relationship is established based on the gain calibration coefficient corresponding to the current gain level and the gain calibration coefficient corresponding to the unified reference gain level. The dark level correction values ​​are converted according to the scale mapping relationship so that the photoelectric sampling values ​​corresponding to different detection channels and different gain levels are at the unified reference gain level, thus forming photoelectric calibration values.

[0036] The photoelectric calibration values ​​are arranged according to the calibration sampling time. The same calibration sampling time is arranged according to the channel identifier and associated with and retained the integration time, detection status, time calibration abnormality mark and calibration missing mark. The photoelectric sampling values ​​with calibration missing mark retain the original value and corresponding sampling attributes, and do not participate in the subsequent calculation of local change, fitting deviation, channel consistency determination and common optical response construction, thus generating the photoelectric calibration sequence.

[0037] It should also be noted that the dark level and gain calibration coefficient were obtained through calibration records generated by each detection channel under shading conditions and known light intensity conditions.

[0038] The uniform reference gain level is determined by the reference gain level selected in the photoelectric acquisition configuration record.

[0039] S2. Divide the detection stage by combining the detection state in the photoelectric calibration sequence, calculate the local change and fitting deviation, mark the position that exceeds the noise boundary, divide the time segment along the detection stage boundary and the marked position, and generate the photoelectric segment sequence.

[0040] S2.1 Compare the detection states in the photoelectric calibration sequence along the calibration sampling time, delineate the detection stage boundary at the state change position, and merge the photoelectric calibration values ​​under the same detection state to generate the detection stage sequence.

[0041] Furthermore, the photoelectric calibration sequence is arranged in ascending order according to the calibration sampling time, and the detection status is checked according to the channel identifier within the same calibration sampling time. When the detection status is consistent within the same calibration sampling time, the corresponding photoelectric calibration value is assigned to the same detection status. If the detection status remains unchanged between adjacent calibration sampling times, the merging continues. If the detection status changes between adjacent calibration sampling times, the changed calibration sampling time is registered as the detection stage boundary, and the photoelectric calibration value corresponding to the detection stage boundary position is assigned to the next detection stage. If there is inconsistency or missing detection status within the same calibration sampling time, the current detection stage is terminated before the abnormal position, the abnormal detection status mark is retained, and the merging restarts from the first calibration sampling time with consistent detection status to generate the detection stage sequence.

[0042] It should also be noted that the detection status is obtained by writing the sampling attributes into the photoelectric detection task during exposure, standby, switching, and calibration.

[0043] S2.2 Arrange photoelectric calibration values ​​by channel in each detection stage, determine the local window half-width by integration time and reference sampling interval, calculate the absolute difference between the median of the photoelectric calibration values ​​before and after the window, and exclude the center position to perform local linear fitting to generate local change sequence and fitting deviation sequence.

[0044] Furthermore, the photoelectric calibration values ​​corresponding to the same detection stage and the same channel identifier in the detection stage sequence are arranged according to the correction sampling time. The integration time is converted into the corresponding number of reference sampling intervals and rounded up. The number of sampling positions that can cover a complete integration time is used as the local window half-width. When the integration time is shorter than the reference sampling interval, the sampling position corresponding to a reference sampling interval is used as the local window half-width.

[0045] Effective photoelectric calibration values ​​covering half the width of a local window are extracted from the current sampling position forward and backward. The absolute difference between the median of the front photoelectric calibration value and the median of the back photoelectric calibration value is taken as the local change at the current sampling position. When the effective photoelectric calibration value is insufficient, the effective photoelectric calibration value is insufficient, or the local window crosses the boundary of the detection stage, an invalid local change mark is registered, and the corresponding sampling position is not included in the arrangement of local changes.

[0046] Using the correction sampling time (excluding the center position) within a local window as the independent variable and the corresponding photoelectric calibration value as the dependent variable, the least squares method is used to form a local linear fitting result. The absolute difference between the photoelectric calibration value at the center position and the corresponding value of the local linear fitting result at the center position is recorded as the fitting deviation. When there are duplicate correction sampling times, missing photoelectric calibration values, or insufficient effective sampling positions within the local window, an invalid fitting deviation is recorded. The effective local changes and effective fitting deviations are arranged according to the correction sampling times to generate a local change sequence and a fitting deviation sequence.

[0047] It should also be noted that the reference sampling interval is determined by the sampling period in the photoelectric acquisition configuration record.

[0048] The local window half-width is determined by the amount of coverage relative to the reference sampling interval during integration time.

[0049] The result of local linear fitting is a linear trend formed by fitting the correction sampling time (excluding the center position) with the photoelectric calibration value within a local window.

[0050] S2.3. Establish initial noise boundaries based on the median and median absolute deviation of local variation and fitting deviation, respectively. After removing the initial out-of-bounds values, redetermine the corresponding noise boundaries. Register the position that exceeds any noise boundary as the initial mark position, and retain the position with the largest out-of-bounds intensity within the continuous out-of-bounds range to generate a mark position sequence.

[0051] Furthermore, the effective local changes in the local change sequence are arranged in order of magnitude, and the sum of the median of the local changes and the median absolute deviation of the local changes is taken as the initial local change noise boundary. For example, the value is the median of the local changes plus three times the median absolute deviation of the local changes. The effective fitting deviations in the fitting deviation sequence are processed in the same way to form the initial fitting deviation noise boundary. When the median absolute deviation of the local changes and the median absolute deviation of the fitting deviation are zero, the minimum resolvable change corresponding to the photoelectric sampling resolution is used to replace the corresponding zero value.

[0052] The local change values ​​above the initial local change noise boundary and the fitting deviation values ​​above the initial fitting deviation noise boundary are temporarily removed from the corresponding arrangement. Then, the local change noise boundary and the fitting deviation noise boundary are re-formed using the remaining local change values ​​and the remaining fitting deviation values, respectively. When the remaining effective values ​​are insufficient to form the median and the median absolute deviation, the missing noise boundary markers are retained, and the marker position screening in the corresponding detection stage is stopped.

[0053] The same sampling position is checked item by item from the local change sequence and the fitting deviation sequence. When the local change exceeds the local change noise boundary and the fitting deviation exceeds the fitting deviation noise boundary, the corresponding sampling position is registered as the initial mark position. When there are no valid sampling positions that have not exceeded the boundary between adjacent initial mark positions and they have not crossed the detection stage boundary, they are classified into the same continuous boundary-crossing segment. The boundary-crossing intensity is characterized by the excess ratio of the local change relative to the local change noise boundary and the excess ratio of the fitting deviation relative to the fitting deviation noise boundary, respectively. The sampling position corresponding to the larger excess ratio is retained. When the excess ratios are the same, the sampling position with the earlier correction sampling time is retained to generate the mark position sequence.

[0054] S2.4. Collect the boundaries and marker positions of the detection stages, delete duplicate markers that are adjacent to the boundaries of the detection stages, divide the photoelectric calibration values ​​in order to retain the positions, and arrange each time segment according to the correction sampling time to generate a photoelectric segment sequence.

[0055] Furthermore, the detection stage boundary and the marker position are merged and arranged according to the correction sampling time, and the local window half-width corresponding to the marker position is used as the proximity distance of the stage boundary. When the number of sampling positions between the marker position and the detection stage boundary is not greater than the proximity distance of the stage boundary, the detection stage boundary is retained and the corresponding marker position is deleted. When duplicate marker positions occur at the same correction sampling time, only one marker position is retained, forming a retained position arrangement.

[0056] The starting position of the detection phase, the boundary of the subsequent detection phase, and the position of the retained marker are used as the starting positions of the time segments. The ending position of the previous time segment is set at the last valid sampling position before the starting position of the next time segment. The photoelectric calibration values ​​corresponding to the detection phase boundary and the position of the retained marker are included in the next time segment. When there are no valid photoelectric calibration values ​​between adjacent starting positions, no empty time segments are established. Then, each time segment is arranged according to the starting correction sampling time of the segment. Within the time segment, the photoelectric calibration values ​​are arranged according to the channel identifier and the correction sampling time to generate a photoelectric segment sequence.

[0057] S3. Establish a dynamic channel diagram based on the channel consistency and time delay relationship of the photoelectric segment sequence, eliminate abnormal channels, merge the sampled values ​​of effective channels according to the connection weight, form a common optical response, determine the response delay and response ratio, perform time migration and amplitude adjustment on the common optical response, calculate the channel residual value, and associate the common optical response and channel residual value according to the segment position to generate a channel association sequence.

[0058] S3.1 Extract the photoelectric calibration values ​​of each channel within the same time segment according to the segment position, perform relative time shift after subtracting the segment median, determine the channel consistency and initial channel delay with the maximum normalized cross-correlation coefficient, and perform parabolic interpolation on the relevant peaks to generate channel relationship records.

[0059] Furthermore, the photoelectric calibration values ​​corresponding to different channel identifiers within the same time segment are paired up into channels. The photoelectric calibration values ​​of each channel are arranged according to the correction sampling time, and the segment median of the corresponding channel is used as the zero reference to convert into a centralized photoelectric calibration value. Within the positive and negative positions covered by the upper limit of the channel delay search, one set of centralized photoelectric calibration values ​​is migrated point by point. The positions with consistent correction sampling times after migration are registered as overlapping sampling positions. The normalized cross-correlation method is used to form the normalized cross-correlation coefficient of each candidate time-shifted position. When there are insufficient overlapping effective sampling positions or when any set of centralized photoelectric calibration values ​​remains constant, relevant invalid markers are registered.

[0060] Candidate time-shift positions corresponding to peak values ​​are selected from the effective normalized cross-correlation coefficients. The peak value of the normalized cross-correlation coefficient is registered as channel consistency, and the candidate time-shift position corresponding to the peak value is registered as an integer time delay position. When there are effective normalized cross-correlation coefficients on both sides of the integer time delay position, parabolic interpolation is performed using the normalized cross-correlation coefficients of the peak position and adjacent positions. The integer time delay position is corrected with the position of the vertex of the parabola to form the initial channel time delay. When the peak value is at the search edge and there are relevant invalid markers in the adjacent positions, the integer time delay position is retained and the interpolation invalid marker is registered to generate a channel relationship record.

[0061] It should also be noted that the upper limit of the channel delay search is determined by the maximum arrival time interval of the same synchronization pulse in different detection channels in the synchronization pulse calibration record.

[0062] S3.2. Establish graph nodes using channel identifiers, create connection edges for channels whose channel consistency reaches the consistency boundary, delete connection edges whose delay closure error exceeds the limit, and configure connection weights based on channel consistency and delay closure error to generate a dynamic channel graph.

[0063] Furthermore, graph nodes are established using the channel identifiers in the channel relationship record. Channels with channel consistency not lower than the channel consistency boundary and valid initial channel delay are paired as connecting edges, and channel consistency and initial channel delay are registered in the connecting edges. If the channel consistency is lower than the channel consistency boundary or the initial channel delay has a related invalid mark, the corresponding connecting edge is not established, and the invalid mark of the channel relationship is retained.

[0064] Three interconnected graph nodes are combined to form a time-delay closed path. The indirect channel time delay is synthesized along the direction of two consecutive connecting edges. The indirect channel time delay is then compared with the direct channel time delay between the first and last graph nodes. The deviation between the two time delay relationships forms the time-delay closure error. When the time-delay closure error exceeds the time-delay closure error boundary, the connecting edge with the lowest channel consistency in the time-delay closed path is deleted, and the time-delay closed path affected by the deletion operation is re-verified until the time-delay closure errors corresponding to the remaining connecting edges do not exceed the limit. When there is no time-delay closed path, the corresponding connecting edge is retained and a closure unverified mark is registered.

[0065] Connection weights are configured based on the extent to which channel consistency exceeds the channel consistency boundary and the degree to which delay closure error approaches the delay closure error boundary. High connection weights are configured when channel consistency is high and delay closure error is low, and low connection weights are configured when channel consistency is close to the channel consistency boundary and delay closure error is close to the delay closure error boundary. The association results are organized according to graph nodes, connection edges, initial channel delay, channel consistency, delay closure error, connection weights, and connection states to generate a dynamic channel graph.

[0066] It should also be noted that the channel consistency boundary is determined by shifting the median of effective channel consistency within the same time segment towards a lower value by one absolute deviation of the channel consistency median.

[0067] The time delay closure error boundary is determined by the larger of the reference sampling interval and the resolvable time delay of the parabolic interpolation.

[0068] S3.3 Summarize the connection weights associated with each graph node in the dynamic channel graph, determine the lower bound of the connection strength based on the median and median absolute deviation of the node connection strength, iteratively delete graph nodes that are below the lower bound of the connection strength, and generate an effective dynamic channel graph.

[0069] Furthermore, the connection weights associated with the same graph node in the dynamic channel graph are summarized, and the summary result is recorded as the node connection strength of the corresponding graph node. Then, the median of the node connection strength and the median absolute deviation of the node connection strength are formed for all valid node connection strengths. The lower bound of the connection strength is formed by shifting the median of the node connection strength to a lower value direction by one median absolute deviation of the node connection strength. When the shift position is lower than zero, the lower bound of the connection strength is determined to be zero.

[0070] Select the graph node with the lowest node connection strength from the graph nodes whose node connection strength is lower than the lower bound of connection strength, delete the corresponding graph node and associated connection edges, and then re-summarize the node connection strength of the remaining graph nodes and update the lower bound of connection strength; stop deleting when no node connection strength is lower than the lower bound of connection strength, and register an insufficient effective channel mark when only a single graph node is retained, and generate an effective dynamic channel graph to determine the time delay path between the baseline channel and the effective channel.

[0071] S3.4 Select the channel with the highest node connection strength in the effective dynamic channel graph as the reference channel, determine the cumulative delay of the remaining effective channels along the minimum cost path, migrate the sampled values ​​of each effective channel to the reference time axis, and perform weighted merging according to the merging weight corresponding to the node connection strength to generate a common optical response.

[0072] Furthermore, the channel with the highest node connection strength in the effective dynamic channel graph is selected as the benchmark channel. When the node connection strength is the same, the channel with the channel identifier listed first is selected as the benchmark channel. The relationship that the higher the connection weight corresponds to the lower the connection cost is written into the connection edge. The path cost of different connection paths is compared from the benchmark channel to the other effective channels. The connection path with the lowest path cost is retained, and the initial channel delay is continuously synthesized along the connection path direction to form the cumulative delay of each effective channel relative to the benchmark channel.

[0073] A reference time axis is established from the calibration sampling time of the reference channel. The photoelectric calibration values ​​of each effective channel are migrated to the reference time axis according to the accumulated time delay. When the migration position is between two adjacent calibration sampling times, the proportion of photoelectric calibration values ​​on both sides is allocated according to the time distance from the migration position to the adjacent calibration sampling time to form the migrated channel sampling value. If there is a missing photoelectric calibration value in the adjacent position, an invalid migration value mark is registered. When there is no connection path to the reference channel in the effective dynamic channel graph, the corresponding graph node is excluded and a path missing mark is registered.

[0074] The sum of the connection strengths of all valid nodes at the same reference sampling time is used as the merging basis. The connection strengths of each valid node are converted into a merging weight of one, and then the channel sampling values ​​after migration are aggregated according to the merging weight. When there is only a single valid channel sampling value at the same reference sampling time, the single valid channel sampling value is retained. When there is no valid channel sampling value, a common response missing mark is registered. The aggregated results are arranged according to the reference time axis to generate a common optical response.

[0075] S3.5. Move the common optical response with the accumulated delay as the search center, determine the response delay of each effective channel by the position of the maximum normalized cross-correlation, and then perform unbiased least squares proportional estimation on the time-aligned common optical response to generate channel response parameter records.

[0076] Furthermore, using the accumulated delay of each effective channel as the search center for response delay, the larger of the delay closure error corresponding to the minimum cost path and the reference sampling interval is used as the search width for response delay. The common optical response is migrated successively within the position covered by the search width for response delay, and the migrated common optical response and the photoelectric calibration value of the corresponding effective channel are combined to form an overlapping position according to the same correction sampling time. The normalized cross-correlation method is used to form the normalized cross-correlation coefficient of each migration position, and the migration position corresponding to the largest effective normalized cross-correlation coefficient is selected as the response delay. When the peaks are parallel, the migration position that is closer to the accumulated delay is retained.

[0077] The common optical response corresponding to the response delay is time-aligned with the photoelectric calibration value of the effective channel. The common optical response is used as the explanatory quantity and the photoelectric calibration value of the effective channel is used as the target quantity. Under the condition of fixed zero intercept, the least squares method is used to fit the amplitude correspondence. The amplitude correspondence ratio formed by the fitting is recorded as the response ratio. When the common optical response remains constant, the effective overlapping position is insufficient, or the response ratio is not greater than zero, an invalid response ratio is recorded. Then, the response delay, response ratio and effective status are recorded according to the segment position and channel identifier to generate the channel response parameter record.

[0078] It should also be noted that the response delay search width is determined by the delay closure error corresponding to the minimum cost path and the baseline sampling interval.

[0079] S3.6. Based on the response delay transfer of the common optical response, perform linear interpolation on the non-integer sampling time and adjust the amplitude according to the response ratio. Calculate the channel residual value of each effective channel relative to the adjusted response. Then, associate the common optical response, channel response parameters and channel residual values ​​according to the segment position to generate a channel association sequence.

[0080] Furthermore, the common optical response is migrated to the corrected sampling time of the corresponding effective channel according to the response delay recorded in the channel response parameters. When the migration time is consistent with the original sampling time of the common optical response, the corresponding value is retained. When the migration time is between two adjacent original sampling times, the proportion of the common optical response on both sides is allocated according to the time distance from the migration time to the original sampling times on both sides, forming a common optical response after linear interpolation. When any adjacent original sampling time lacks a common optical response, an invalid migration response is registered, and the amplitude of the effective migration response is converted according to the response ratio to form an adjusted response.

[0081] The photoelectric calibration values ​​of each effective channel at the same calibration sampling time are correlated with the adjustment response, and the channel residual value of each effective channel relative to the adjustment response is calculated. When the photoelectric calibration value is missing at the same calibration sampling time, the effective response delay is not formed in the channel response parameter record, the response ratio has an invalid mark, or the common optical response cannot be migrated to the corresponding calibration sampling time, the channel residual value at the corresponding sampling position is not calculated, and a channel residual missing mark is registered under the corresponding channel identifier and calibration sampling time. Then, the common optical response, channel response parameters and channel residual values ​​are associated according to the time segment position, and the association results are arranged according to the segment start calibration sampling time, channel identifier and calibration sampling time within the segment to generate a channel association sequence.

[0082] The channel residuals of each effective channel relative to the adjusted response are calculated using the following expression: ; in: It is the first The effective channel is in the first The channel residual value at each correction sampling time; It is the first The effective channel is in the first The photoelectric calibration value at each calibration sampling time; It is the first The response ratio of each effective channel; It is the first One calibration sampling time; It is the first Response latency of each effective channel; This indicates the common optical response shift to time. The resulting linear interpolation; It is the valid channel number; It is the correction sampling time sequence number; when Located at adjacent sampling times and When the interval is between, the linear interpolation result is expressed as: ; ; in: It is a moment Located at adjacent sampling times and The positional ratio between them; It is the common optical response at the adjacent sampling time on the front side. The value; It is the common optical response at the next adjacent sampling time. The value; It is the sequence number of the adjacent sampling time on the front side.

[0083] It should also be noted that, By aligning the common optical response after response delay with the first The photoelectric calibration values ​​of each effective channel are obtained by performing unbiased least squares proportional estimation.

[0084] Through the first The corrected sampling time is determined after correcting the time offset of the sampling timestamps of each photoelectric sample value.

[0085] This is achieved by shifting the common optical response near the accumulated time delay and selecting a position similar to the first optical response. The position with the largest normalized cross-correlation coefficient among the effective channel photoelectric calibration values ​​is obtained.

[0086] S4. Establish candidate prediction methods based on the channel association sequence, calculate the prediction residual value one by one, determine the expected code length of each candidate prediction method according to the residual coding characteristics, eliminate candidate prediction methods with reconstruction deviation exceeding the limit, select the target prediction method with the shortest expected code length from the remaining candidate prediction methods, and perform hierarchical quantization, zero-value merging and variable bit-width coding on the prediction residual value of the target prediction method to generate candidate compressed sequences.

[0087] S4.1 Arrange the common optical response and residual values ​​of each channel according to the segment position, truncate the continuous sampling range at the segment boundary and invalid position, and retain the starting value of each continuous sampling range as the prediction anchor point to generate the prediction object sequence.

[0088] Furthermore, the common optical response and residual values ​​of each channel in the channel-related sequence are treated as independent prediction objects, arranged according to segment position, channel identifier, and correction sampling time. When encountering segment boundaries, missing common response markers, or missing channel residual markers, the current continuous sampling range is terminated before the corresponding position, and a new continuous sampling range is established from the first subsequent valid sampling position. The first valid value of each continuous sampling range is recorded as the prediction anchor point, along with the segment position, prediction object category, channel identifier, start correction sampling time, and end correction sampling time. When the continuous sampling range contains only one valid sampling position, only the prediction anchor point is recorded, and subsequent prediction positions are not established, thus generating a prediction object sequence.

[0089] S4.2. Using the continuous sampling range in the prediction object sequence as the prediction rule configuration position, configure previous value prediction and variation extrapolation prediction for the common optical response respectively. The previous value prediction uses the value of the previous sampling position as the current prediction value, and the variation extrapolation prediction extends the change direction and change amplitude formed by the two previous sampling positions to the current sampling position. Configure zero value prediction, previous value prediction, and variation extrapolation prediction for the channel residual values ​​respectively. The zero value prediction uses zero as the current prediction value, and the previous value prediction and variation extrapolation prediction adopt the same previous position relationship as the common optical response.

[0090] The prediction anchor point is registered at the beginning of the continuous sampling range of the corresponding candidate prediction method. If there are fewer than two preceding sampling positions, the variation extrapolation prediction is deleted. If the continuous sampling range only contains the prediction anchor point, no prediction rule is configured. The common optical response is registered in sequence with the previous value prediction and the variation extrapolation prediction. The channel residual value is registered in sequence with the zero value prediction, the previous value prediction, and the variation extrapolation prediction, forming a candidate prediction method with clear rule identification, prediction anchor point, and applicable sampling position.

[0091] S4.3. Starting from the first sampling position after the prediction anchor point, each candidate prediction method is expanded point by point. Zero value prediction determines zero as the current predicted value. Previous value prediction determines the most recent previous actual value as the current predicted value. Change extrapolation prediction extends to the current sampling position along the change direction and change magnitude formed by the two most recent previous actual values. The signed deviation of the current actual value relative to the current predicted value is calculated, and the signed deviation is recorded as the prediction residual value of the current sampling position.

[0092] Write the current actual value to the preceding sampling position of the corresponding candidate prediction method, and continue to form the current prediction value of the next sampling position; stop updating when the continuous sampling range ends, and start prediction again when the next continuous sampling range is reached. Invalid positions do not form prediction residual values. Arrange the prediction residual values ​​according to the candidate prediction method, segment position and correction sampling time to form the prediction residual value sequence corresponding to each candidate prediction method.

[0093] S4.4. Take the continuous sampling range in each predicted residual value sequence as the initial quantization segment, set the quantization level starting from the quantization interval corresponding to the photoelectric sampling resolution, and keep the subsequent quantization level as an integer multiple of the photoelectric sampling resolution, for example, gradually widening it according to one, two, and four times; half of the quantization interval corresponds to the maximum theoretical quantization deviation of the current quantization level. If the maximum theoretical quantization deviation does not exceed the allowable reconstruction deviation, continue to check the next quantization level. When the maximum theoretical quantization deviation will exceed the allowable reconstruction deviation, stop widening and retain the maximum quantization interval that has not exceeded the limit.

[0094] The ratio of the predicted residual value to the quantization interval is mapped to the nearest integer position. When the mapped position is between two integer positions, the integer position with the smaller absolute value is retained. At the same time, the positive and negative directions of the predicted residual value are retained to form a signed integer index. When the predicted residual value corresponds to an invalid position, the current quantization segment is terminated, and a new quantization segment is established from the first valid predicted residual value. The signed integer indexes are arranged according to the sampling order to generate a trial quantization residual index sequence.

[0095] It should also be noted that the sampling resolution of the photodetector, the range of dark level fluctuation, the gain calibration error and the allowable offset range of the detection results are collected. Then, with the results of detection state judgment, peak position identification and amplitude level classification as constraints, multiple reconstruction tests are performed through historical photodetector calibration sequences. The maximum deviation value that can maintain the consistency of the above judgment results is selected and registered as the allowable reconstruction deviation according to the detection stage and channel type.

[0096] S4.5. Retrieve consecutive positions with zero values ​​from the starting position of the quantization segment of the trial quantization residual index sequence, and register the number of consecutive zero values ​​as the zero value length. The code length occupied by the length encoding is formed by the zero value block identifier and the zero value length field. The code length occupied by the item-by-item encoding is formed by writing consecutive zero values ​​into the corresponding zero value codeword. When the code length occupied by the length encoding is small, register the range of consecutive zero values ​​as the zero value merging position. When the two code lengths are the same, retain the item-by-item encoding.

[0097] Perform a positive-negative alternating mapping on the signed integer indices other than the zero-value merging position. The zero index is mapped to zero, and the positive and negative indices are alternately mapped to non-negative integers in ascending order of absolute value, so that the signed integer indices with absolute values ​​close to zero obtain smaller mapped values. Then, group the adjacent mapped values ​​into a value block, and use the number of binary bits required for the largest mapped value in the value block as the effective bit width.

[0098] The extended coding overhead consists of the effective bit width, the block length field, and the encoded content after expanding the current numerical block. The new coding overhead consists of the block identifier, effective bit width field, block length field, and encoded content required to terminate the current numerical block and create a new numerical block. If the extended coding overhead is not higher than the new coding overhead, the current numerical block continues to expand. If the extended coding overhead is higher than the new coding overhead, a numerical block boundary is established before the current mapped value. The prediction rule identifier, prediction anchor point, quantization interval, zero-value coding block, and numerical coding block are simulated and written into the corresponding positions. The occupied bits are summarized to determine the expected code length of each candidate prediction method.

[0099] It should also be noted that the length of the encoded field is obtained by configuring the encoding format used in the photoelectric compression record.

[0100] S4.6. Restore the trial quantization residual index sequence corresponding to each candidate prediction method to the trial reconstruction prediction residual value according to the quantization interval. Start from the prediction anchor point and form the current prediction value point by point. Then, superimpose the trial reconstruction prediction residual value to the current prediction value to form the current trial reconstruction value. The previous value prediction and change extrapolation prediction use the formed previous trial reconstruction value to update the subsequent prediction state. The zero value prediction continues to use zero as the current prediction value to ensure that the trial reconstruction process is consistent with the subsequent decoding process.

[0101] The common optical response and channel residual values ​​of the trial reconstruction are recovered from the trial reconstruction values ​​respectively. The common optical response of the trial reconstruction is migrated according to the response delay recorded in the channel response parameter record. The amplitude after migration is converted according to the response ratio. The amplitude conversion result is then superimposed with the channel residual values ​​of the trial reconstruction to recover the photoelectric calibration values ​​of each effective channel at the corresponding correction sampling time.

[0102] The absolute deviation between the original photoelectric calibration value and the trial reconstructed photoelectric calibration value at the same segment position, channel identifier, and correction sampling time is used as the reconstruction deviation. If the reconstruction deviation at any valid position exceeds the allowable reconstruction deviation, the corresponding candidate prediction method is eliminated. If no candidate prediction method is retained, the quantization interval of the quantization segment where the maximum reconstruction deviation is located is reduced, a new trial quantization residual index sequence is formed, and reconstruction verification is performed until at least one candidate prediction method that does not exceed the limit is formed. The retained candidate prediction methods are arranged in ascending order of expected code length. If the expected code lengths are the same, the candidate prediction method with the smaller maximum reconstruction deviation is retained. If the maximum reconstruction deviations are still the same, the candidate prediction method with the prediction rule identifier listed first is retained, and the target prediction method is generated.

[0103] S4.7 Starting from the prediction anchor point registered in the target prediction method, the current prediction value is formed point by point according to the prediction rule corresponding to the target prediction method. The prediction residual value of the current actual value relative to the current prediction value is calculated, and the quantization index is formed by the quantization segment and quantization interval verified by the target prediction method. During the encoding period, the previous value prediction and the previous sampling position of the change extrapolation prediction are updated with the already formed previous reconstruction value to keep the prediction state of the encoding end consistent with the prediction state of the decoding end.

[0104] Write the range of consecutive zero values ​​in the quantization index into the zero-value coding block according to the code length comparison result. Perform positive and negative interleaving mapping on the quantization index outside the zero-value merging position. Divide the numerical blocks according to the extended coding overhead and new coding overhead corresponding to the change in effective bit width. Write the prediction rule identifier and prediction anchor point at the beginning position of each consecutive sampling range. Write the quantization segment length and quantization interval at the beginning position of each quantization segment. Write the coding block type, coding block length and effective bit width at the beginning position of each coding block, and then write the corresponding quantization index.

[0105] The encoding results are arranged according to the segment start correction sampling time, prediction object category, channel identifier, and intra-segment correction sampling time to generate candidate compressed sequences.

[0106] S5. Perform time-series reconstruction on the candidate compressed sequences and calculate the segment reconstruction deviation. When the segment reconstruction deviation exceeds the limit, reduce the quantization interval. If it still exceeds the limit, add sampling anchor points. Arrange the candidate compressed sequences that have passed the verification according to the detection time sequence to generate photoelectric compressed records.

[0107] S5.1 Expand the candidate compressed sequence and restore the photoelectric calibration value point by point. After comparing it with the original photoelectric calibration value, determine the segment reconstruction deviation and generate a segment reconstruction deviation record.

[0108] Furthermore, the candidate compressed sequence is expanded according to the segment position, and the prediction rule identifier, prediction anchor point, quantization interval, zero-value coding block, numerical coding block, and effective bit width are identified. The quantization index and prediction residual value are recovered, and then the common optical response and the residual value of the trial reconstruction channel are formed point by point according to the corresponding prediction rule. When the coding field is missing, the coding block length is inconsistent, or the number of quantization indices is inconsistent, the recovery of the corresponding time segment is stopped and an invalid reconstruction mark is registered. The common optical response of the trial reconstruction is recovered to each effective channel according to the response delay and response ratio, and then synthesized with the residual value of the trial reconstruction channel to form the trial reconstruction photoelectric calibration value. The original photoelectric calibration value is compared point by point, and the reconstruction deviation at the maximum sampling position, the position of the maximum deviation, and the segment verification status are registered to generate a segment reconstruction deviation record.

[0109] S5.2. Taking the time series segment in the segment reconstruction deviation record that exceeds the allowable reconstruction deviation as the adjustment object, locate the quantization segment to which the maximum deviation position belongs, adjust the quantization interval of the corresponding quantization segment to the adjacent smaller quantization level, and keep the prediction rule, prediction anchor point and quantization segment boundary unchanged, and re-form the quantization index, zero value encoding block and numerical encoding block of the corresponding quantization segment; after completing the re-encoding, expand the complete time series segment again and form the segment reconstruction deviation. If the segment reconstruction deviation still exceeds the limit, continue to reduce the quantization interval of the quantization segment to which the maximum deviation position belongs.

[0110] When the quantization interval of the quantization segment to which the maximum deviation position belongs reaches the photoelectric sampling resolution and the segment reconstruction deviation still exceeds the limit, the original common optical response value or the original channel residual value corresponding to the maximum deviation position is written into the corresponding prediction object sequence as a new sampling anchor point, and the original continuous sampling range is truncated at the new sampling anchor point. The encoded content that has been restored and verified before the new sampling anchor point remains unchanged. The target prediction method, prediction residual quantization, zero value merging and variable bit width coding are re-executed from the first sampling position after the new sampling anchor point to form an updated candidate compressed sequence.

[0111] The photoelectric calibration value is restored from the updated candidate compressed sequence and the fragment reconstruction deviation is verified. When there are multiple positions with the same maximum deviation, the position with the earlier correction sampling time is processed first. If the fragment reconstruction deviation still exceeds the limit, the quantization interval of the corresponding quantization segment is further reduced. When the quantization interval has reached the photoelectric sampling resolution, the sampling anchor point is further increased until the fragment reconstruction deviation does not exceed the allowable reconstruction deviation, and a valid compressed sequence is generated.

[0112] S5.3 Arrange the effective compressed sequence according to the sampling time of the segment start correction, register the segment compression length at the start position of each segment, and splice the corresponding coded content in sequence to generate photoelectric compressed records.

[0113] Furthermore, the effective compressed sequence is arranged according to the segment start correction sampling time. When the segment start correction sampling times are the same, they are arranged according to the segment position in the photoelectric segment sequence. At the start position of each segment, the segment compression length including prediction rule identifier, prediction anchor point, quantization segment field, zero value coding block and numerical coding block is registered. Then, the corresponding coding content is spliced ​​in sequence to generate photoelectric compressed record.

[0114] This embodiment also provides a massive time-series data compression system for photoelectric detection systems, including: a photoelectric calibration module, used to collect photoelectric sampling values ​​and sampling attributes, and generate a photoelectric calibration sequence by correcting the sampling time, subtracting the dark level and unifying the gain level; The time-series segmentation module is used to divide the detection stage by combining the detection state in the photoelectric calibration sequence, calculate the local change and fitting deviation, mark the position that exceeds the noise boundary, divide the time-series segment along the detection stage boundary and the marked position, and generate the photoelectric segment sequence. The channel association module is used to establish a dynamic channel diagram based on the channel consistency and time delay relationship of the photoelectric segment sequence, eliminate abnormal channels, merge the sampled values ​​of effective channels according to the connection weight, form a common optical response, determine the response time delay and response ratio, perform time migration and amplitude adjustment on the common optical response, calculate the channel residual value, and associate the common optical response and channel residual value according to the segment position to generate a channel association sequence. The prediction compression module is used to establish candidate prediction methods based on the channel association sequence, calculate the prediction residual value one by one, determine the expected code length of each candidate prediction method according to the residual coding characteristics, eliminate candidate prediction methods with reconstruction deviation exceeding the limit, select the target prediction method with the shortest expected code length from the remaining candidate prediction methods, and perform hierarchical quantization, zero-value merging and variable bit-width coding on the prediction residual value of the target prediction method to generate candidate compressed sequences. The reconstruction and verification module is used to perform temporal reconstruction of candidate compressed sequences and calculate fragment reconstruction deviation. When the fragment reconstruction deviation exceeds the limit, the quantization interval is reduced. If it still exceeds the limit, the sampling anchor point is increased. The candidate compressed sequences that pass the verification are arranged according to the detection time sequence to generate photoelectric compressed records.

[0115] In summary, this invention establishes a dynamic channel graph based on the channel consistency and delay relationship within a time segment, iteratively eliminates abnormal channels, unifies the time axis of effective channels along the minimum cost path, and merges them into a common optical response according to connection weights. Then, it separates the common optical response from the channel residuals based on the response delay and response ratio. This allows cross-channel repetitive waveforms to be expressed in a concentrated manner, and local differences to be transformed into residuals with small amplitudes and continuous zero values. This provides a stable object for subsequent predictive quantization and variable-width coding, reduces repetitive coding, shortens the compressed code length, and maintains reconstruction accuracy.

[0116] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for compressing massive time-series data for a photoelectric detection system, characterized in that, include: The photoelectric sampling values ​​and sampling attributes are collected, and the dark level is subtracted and the gain level is unified by correcting the sampling time to generate a photoelectric calibration sequence. The detection stages are divided by combining the detection states in the photoelectric calibration sequence. Local changes and fitting deviations are calculated, and the positions that exceed the noise boundary are marked. The time segments are divided along the detection stage boundary and the marked positions to generate a photoelectric segment sequence. A dynamic channel diagram is established based on the channel consistency and time delay relationship of the photoelectric segment sequence. Abnormal channels are eliminated, and the sampled values ​​of effective channels are merged according to the connection weight to form a common optical response. The response time delay and response ratio are determined, and the common optical response is time-shifted and amplitude adjusted. The channel residual value is calculated, and the common optical response and channel residual value are associated according to the segment position to generate a channel association sequence. Candidate prediction methods are established based on the channel association sequence. The prediction residual value is calculated one by one. The expected code length of each candidate prediction method is determined according to the residual coding characteristics. Candidate prediction methods with reconstruction deviation exceeding the limit are eliminated. The target prediction method with the shortest expected code length is selected from the remaining candidate prediction methods. The prediction residual value of the target prediction method is subjected to hierarchical quantization, zero-value merging and variable bit-width coding to generate candidate compressed sequences. The candidate compressed sequences are reconstructed in time and the segment reconstruction deviation is calculated. When the segment reconstruction deviation exceeds the limit, the quantization interval is reduced. If it still exceeds the limit, the sampling anchor point is increased. The candidate compressed sequences that pass the verification are arranged according to the detection time sequence to generate photoelectric compressed records.

2. The method for massive time series data compression for photodetection system oriented system as claimed in claim 1, wherein, The process of generating a photoelectric calibration sequence by correcting the sampling time, subtracting the dark level, and unifying the gain order is as follows: The time offset of each detection channel is calculated based on the sampling timestamp, channel identifier and synchronization trigger mark corresponding to the photoelectric sampling value, the sampling time is corrected, and the photoelectric sampling values ​​are rearranged according to the corrected sampling time to form a time correction sequence. The channel identifier, gain level, and integration time are extracted from the sampling attributes of the time-correction sequence. The corresponding dark level and gain calibration coefficient are matched, the dark level in the photoelectric sampling value is subtracted, and then converted to a unified reference gain level to generate the photoelectric calibration sequence.

3. The method for compressing massive time-series data for photoelectric detection systems as described in claim 1, characterized in that, The specific steps for generating the photoelectric fragment sequence are as follows: The detection states in the photoelectric calibration sequence are compared along the calibration sampling time. The detection stage boundary is defined at the position where the state changes, and the photoelectric calibration values ​​under the same detection state are merged to generate the detection stage sequence. Within each detection stage, photoelectric calibration values ​​are arranged by channel. The local window half-width is determined by the integration time and the reference sampling interval. The absolute difference between the median of the photoelectric calibration values ​​before and after the window is calculated. Local linear fitting is performed after excluding the center position to generate a local change sequence and a fitting deviation sequence. Initial noise boundaries are established based on the median and median absolute deviation of local changes and fitting deviations, respectively. After removing the initial out-of-bounds values, the corresponding noise boundaries are redefined. The positions that exceed any noise boundary are registered as initial marker positions, and the positions with the largest out-of-bounds intensity are retained within the continuous out-of-bounds range to generate a marker position sequence. The detection stage boundaries and marker positions are collected, duplicate markers adjacent to the detection stage boundaries are deleted, and the photoelectric calibration values ​​are divided sequentially according to the retained positions. The time segments are then arranged according to the calibration sampling time to generate a photoelectric segment sequence.

4. The method for compressing massive time-series data for photoelectric detection systems as described in claim 1, characterized in that, The specific steps for forming a common optical response are as follows: The photoelectric calibration values ​​of each channel within the same time segment are extracted according to the segment position. After subtracting the segment median, relative time shift is performed. The channel consistency and initial channel delay are determined by the maximum normalized cross-correlation coefficient. Parabolic interpolation is performed on the relevant peaks to generate channel relationship records. Graph nodes are established using channel identifiers. Connection edges are created for channels whose channel consistency reaches the consistency boundary. Connection edges with excessive delay closure error are deleted. Connection weights are configured based on channel consistency and delay closure error to generate a dynamic channel graph. Summarize the connection weights associated with each graph node in the dynamic channel graph, determine the lower bound of the connection strength based on the median and median absolute deviation of the node connection strength, iteratively delete graph nodes that are below the lower bound of the connection strength, and generate an effective dynamic channel graph. The channel with the highest node connection strength in the effective dynamic channel graph is selected as the reference channel. The cumulative delay of the remaining effective channels is determined along the minimum cost path. The sampled values ​​of each effective channel are migrated to the reference time axis and weighted and merged according to the merging weight corresponding to the node connection strength to generate a common optical response.

5. The method for compressing massive time-series data for photoelectric detection systems as described in claim 4, characterized in that, The steps for determining the response delay and response ratio, performing time-series shifting and amplitude adjustment on the common optical response, calculating channel residual values, and associating the common optical response and channel residual values ​​according to segment positions to generate a channel association sequence are as follows: The common optical response is moved with the accumulated delay as the search center. The response delay of each effective channel is determined by the position of the maximum normalized cross-correlation. Then, unbiased least squares proportional estimation is performed on the time-aligned common optical response to generate channel response parameter records. Based on the common optical response and the response delay, linear interpolation is performed on non-integer sampling times and the amplitude is adjusted according to the response ratio. The channel residual values ​​of each effective channel relative to the adjusted response are calculated. Then, the common optical response, channel response parameters and channel residual values ​​are associated according to the segment position to generate a channel association sequence.

6. The method for compressing massive time-series data for photoelectric detection systems as described in claim 5, characterized in that, The steps for establishing candidate prediction methods based on channel correlation sequences and calculating prediction residuals one by one are as follows: The common optical response and residual values ​​of each channel are arranged according to the segment position. The continuous sampling range is truncated at the segment boundary and invalid position, and the starting value of each continuous sampling range is retained as the prediction anchor point to generate the prediction object sequence. Prediction rules are configured along the continuous sampling range of the prediction object sequence. For the common optical response, previous value prediction and variation extrapolation prediction are configured. For the channel residual value, zero value prediction, previous value prediction and variation extrapolation prediction are configured. The prediction anchor points corresponding to each prediction rule are registered, and variation extrapolation predictions with insufficient previous sampling positions are deleted to establish candidate prediction methods. The current predicted value is determined point by point according to the candidate prediction method. The difference between the current actual value and the current predicted value is calculated to form the prediction residual value. The previous sampling position of the corresponding prediction method is updated with the current actual value, and the predicted value of the next sampling position is determined.

7. The method for compressing massive time-series data for photoelectric detection systems as described in claim 6, characterized in that, The steps for determining the expected code length of each candidate prediction method based on residual coding characteristics, eliminating candidate prediction methods with reconstruction deviation exceeding the limit, and selecting the target prediction method with the shortest expected code length from the remaining candidate prediction methods are as follows: The quantization segment is divided along the sampling order of the predicted residual value. The quantization interval is gradually widened starting from the quantization interval corresponding to the photoelectric sampling resolution. The widening is stopped before the theoretical quantization deviation exceeds the allowable reconstruction deviation. The predicted residual value is converted into a signed integer index to generate a trial quantization residual index sequence. The range of consecutive zero values ​​is identified along the trial quantization residual index sequence. The zero value merging position is determined by comparing the occupied code length of length coding and item-by-item coding. Positive and negative interleaving mapping is performed on the remaining signed integer indices. The boundary of the numerical block is determined according to the coding overhead of expanding the current numerical block and creating a new numerical block. Then, the corresponding coding block is simulated and written to determine the expected code length of each candidate prediction method. The current reconstructed value is formed point by point according to the trial quantization residual index sequence, and the trial reconstructed photoelectric calibration value is restored according to the channel response parameters. Candidate prediction methods that exceed the allowable reconstruction deviation are eliminated. If no candidate prediction methods are retained, the quantization interval is reduced and re-verified. Then, the target prediction method with the shortest expected code length is selected from the retained candidate prediction methods.

8. The method for compressing massive time-series data for photoelectric detection systems as described in claim 1, characterized in that, The specific steps for performing hierarchical quantization, zero-value merging, and variable bit-width coding on the prediction residual value of the target prediction method to generate a candidate compressed sequence are as follows: recalculate the prediction residual value according to the target prediction method and perform hierarchical quantization; merge consecutive zero values ​​according to the code length comparison results; divide the numerical blocks according to the coding overhead corresponding to the effective bit-width change and write them into the quantization index; arrange the coding results according to the segment position to generate a candidate compressed sequence.

9. The method for compressing massive time-series data for photoelectric detection systems as described in claim 1, characterized in that, The specific steps for generating the photoelectric compressed record are as follows: Expand the candidate compressed sequence and recover the photoelectric calibration value point by point. After comparing it with the original photoelectric calibration value, determine the segment reconstruction deviation and generate a segment reconstruction deviation record. When the segment reconstruction deviation exceeds the allowable reconstruction deviation, the quantization interval of the quantization segment corresponding to the maximum deviation position is reduced and re-encoded and verified; when the quantization interval still exceeds the limit after reaching the photoelectric sampling resolution, the original value is written at the maximum deviation position as the sampling anchor point, and prediction and encoding are re-started from the sampling anchor point until the segment reconstruction deviation passes the verification and an effective compressed sequence is generated. Arrange the effective compressed sequence according to the sampling time of the segment start correction, register the segment compression length at the start position of each segment, and then splice the corresponding coded content in sequence to generate an optoelectronic compressed record.

10. A massive time-series data compression system for photoelectric detection systems, based on the massive time-series data compression method for photoelectric detection systems according to any one of claims 1 to 9, characterized in that, include: The photoelectric calibration module is used to collect photoelectric sampling values ​​and sampling attributes, and generate a photoelectric calibration sequence by correcting the sampling time, subtracting the dark level and unifying the gain level. The time-series segmentation module is used to divide the detection stage by combining the detection state in the photoelectric calibration sequence, calculate the local change and fitting deviation, mark the position that exceeds the noise boundary, divide the time-series segment along the detection stage boundary and the marked position, and generate the photoelectric segment sequence. The channel association module is used to establish a dynamic channel diagram based on the channel consistency and time delay relationship of the photoelectric segment sequence, eliminate abnormal channels, merge the sampled values ​​of effective channels according to the connection weight, form a common optical response, determine the response time delay and response ratio, perform time migration and amplitude adjustment on the common optical response, calculate the channel residual value, and associate the common optical response and channel residual value according to the segment position to generate a channel association sequence. The prediction compression module is used to establish candidate prediction methods based on the channel association sequence, calculate the prediction residual value one by one, determine the expected code length of each candidate prediction method according to the residual coding characteristics, eliminate candidate prediction methods with reconstruction deviation exceeding the limit, select the target prediction method with the shortest expected code length from the remaining candidate prediction methods, and perform hierarchical quantization, zero-value merging and variable bit-width coding on the prediction residual value of the target prediction method to generate candidate compressed sequences. The reconstruction and verification module is used to perform temporal reconstruction of candidate compressed sequences and calculate fragment reconstruction deviation. When the fragment reconstruction deviation exceeds the limit, the quantization interval is reduced. If it still exceeds the limit, the sampling anchor point is increased. The candidate compressed sequences that pass the verification are arranged according to the detection time sequence to generate photoelectric compressed records.

Images