An edge-computing-based data compression method for an electric energy meter detection pipeline

Abstract

The application discloses an electric energy meter detection pipeline data compression method based on edge calculation and relates to the technical field of electric energy meter detection data processing. The method comprises the following steps: S1, forming a pipeline detection sequence; S2, performing time bit correction and assembling into a batch detection matrix; S3, performing low rank-sparse decomposition on the batch detection matrix by using a robust principal component analysis method, forming a batch common representation, and arranging the sparse decomposition result into a difference residual matrix; S4, forming a detection residual segment set; S5, inputting the detection residual segment set into an improved GNG model to form a residual prototype topology and a prototype mapping sequence; S6, forming residual compression data; and S7, encapsulating the batch common representation, the residual prototype topology, the residual compression data and a segment recovery index into an electric energy meter detection pipeline compression data packet. The application can reduce detection data storage and uploading overhead and improve residual compression and recovery capability.

Description

Technical Field

[0001] This invention relates to the field of electricity meter testing data processing technology, and in particular to a data compression method for an electricity meter testing pipeline based on edge computing. Background Technology

[0002] The automated electricity meter testing pipeline continuously generates a large amount of data on detection responses, sampling times, detection stages, and batch affiliations during the detection processes, including basic errors, startup, creeping, voltage effects, frequency effects, clock, communication, and carrier waves. As the number of testing stations and sampling frequency increases, the data size of a single testing batch continues to grow, easily leading to edge buffer occupancy, network upload congestion, and central storage pressure. Existing compression methods mostly employ general lossless compression algorithms, primarily eliminating duplicate content in byte sequences, failing to fully utilize the common response changes exhibited by different electricity meters within the same testing batch at the same testing stage, resulting in limited compression ratios. Some compression methods based on prototype clustering or neural networks use a single prototype vector to represent the detection residual. When residual fragments vary along multiple directions within a local region, more prototype nodes are needed, increasing the space occupied by model parameters and index data. Furthermore, sampling times at different testing stations are offset; if the compressed data lacks recovery information such as stage boundaries, original bit order, and effective fragment length, it is difficult to accurately reconstruct the detection response after decompression.

[0003] Therefore, how to provide a data compression method for an energy meter detection pipeline based on edge computing is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0004] One objective of this invention is to propose a data compression method for an energy meter detection pipeline based on edge computing. This invention extracts batch common representations through time-position correction and low-rank sparse decomposition, compresses detection residual fragments by improving the GNG model, and encodes prototype mapping sequences and fragment deviations to reduce the storage and upload volume of detection data.

[0005] A data compression method for an energy meter detection pipeline based on edge computing according to an embodiment of the present invention includes the following steps: S1. Obtain the test data generated by the electricity meter testing pipeline, arrange it according to the execution order of the test items, and form a pipeline test sequence; S2. Perform time-position correction on the pipeline detection sequence within the same detection batch, and assemble the data in the same detection stage into a batch detection matrix; S3. Use robust principal component analysis to perform low-rank sparse decomposition on the batch detection matrix, perform low-rank factor decomposition on the low-rank decomposition results to form a batch commonality representation, and organize the sparse decomposition results into a difference residual matrix. S4. Back-sort the difference residual matrix according to the original position order of the pipeline detection sequence, perform segment segmentation along the boundary of the detection stage and record the segment recovery index to form a set of detection residual segments; S5. Input the set of detected residual fragments into the improved GNG model. The improved GNG model replaces the prototype nodes with residual subspace nodes, sets residual propagation edges between the residual subspace nodes, and performs residual propagation edge splitting and new node insertion according to the edge residual state of the residual propagation edges to form residual prototype topology and prototype mapping sequence. S6. Calculate the fragment deviation between the detected residual fragment and the corresponding residual subspace node, and perform entropy encoding on the prototype mapping sequence and fragment deviation to form residual compressed data. S7. Encapsulate the batch common representation, residual prototype topology, residual compressed data, and fragment recovery index into a compressed data package for the energy meter detection pipeline.

[0006] Optionally, the detection data includes batch attribution data, detection item data, stage time data, and detection response data; The batch attribution data records the identifier of the tested energy meter and the corresponding testing batch; the testing item data records each testing item and its execution order; the stage time data records the sampling time and testing stage corresponding to the testing data; and the testing response data records the testing values ​​generated by the tested energy meter in each testing stage.

[0007] Optionally, forming the pipeline detection sequence includes: The test data were grouped according to the test batch and the affiliation of the tested electricity meters; Within each group, the test data are arranged according to the execution order of the test items, and the test response values ​​within the same test item are sorted by stage and time position. The detection stage boundaries are written between adjacent detection stages, and the arranged detection data are sequentially connected to form a pipeline detection sequence.

[0008] Optionally, S2 includes: S21. According to the boundary of the detection stage, extract the stage data segments in the detection sequence of each production line within the same detection batch, and correct the starting time position of each stage data segment to the starting position of the corresponding detection stage. S22. Set the correction sampling time points for the same detection stage according to a unified sampling interval, perform linear interpolation on the stage data segments, and form a stage response sequence with consistent time points; S23. Arrange the stage response sequences belonging to the same detection stage row by row, using the correction sampling time position as the matrix column position and the stage response sequences corresponding to different tested energy meters as the matrix row positions, assemble them into a batch detection matrix, and retain the correspondence between the matrix row and column positions and the original position sequence of the pipeline detection sequence.

[0009] Optionally, S3 includes: S31. Use robust principal component analysis to alternately perform singular value shrinkage and soft threshold shrinkage on the batch detection matrix to update the low-rank matrix and sparse matrix. S32. Perform singular value decomposition on the low-rank matrix, rearrange the left singular vectors into common mapping coefficients, combine the singular values ​​and the right singular vectors into common basis vectors, and pair the common mapping coefficients with the common basis vectors to form batch common representations. S33. Keeping the row and column positions of the sparse matrix unchanged, rearrange the sparse matrix into a difference residual matrix.

[0010] Optionally, S4 includes: S41. Call the correspondence between the row and column positions of the batch detection matrix and the original position sequence of the pipeline detection sequence, and restore the arrangement of the difference residuals in each row of the difference residual matrix into the energy meter residual sequence. S42. Map the boundary position of the detection stage in the pipeline detection sequence to the residual sequence of the electricity meter, and cut the residual sequence of the electricity meter at the boundary position of the detection stage to form a stage residual segment. S43. Divide the stage residual segments in order of the same segment length, perform padding on the tail residuals that are not long enough, and record the effective residual length to form the detection residual segments. S44. Record the tested energy meter, testing stage, original starting position and effective residual length corresponding to each detection residual segment. Assemble the recorded content into a segment recovery index and arrange the detection residual segments according to the original starting position to form a set of detection residual segments.

[0011] Optionally, the improved GNG model includes a competitive matching unit, a residual subspace node set, a residual recursive edge set, a node edge update unit, and a splitting mapping unit; The competitive matching unit projects the detected residual fragments onto the set of nodes in the residual subspace, and delineates the first and second matching nodes according to the remaining residuals after projection; The node edge update unit establishes or updates the residual propagation edge between the first matching node and the second matching node. It splits the projection residual corresponding to the first matching node into directional residuals along the node connection direction between the first matching node and each adjacent residual subspace node, and writes the directional residuals into the corresponding residual propagation edge. Each residual propagation edge accumulates the directional residuals and updates the edge residual state. The splitting mapping unit delineates the residual propagation edge to be split according to the edge residual state, generates a new residual subspace node based on the node parameters at both ends of the residual propagation edge to be split and the edge residual state, and connects the new residual subspace node to the node connection relationship. The node edge update unit updates the node parameters of the residual subspace nodes and the edge residual state of the residual propagation edge, and works with the splitting mapping unit to form the residual prototype topology; The competitive matching unit re-defines the residual subspace nodes corresponding to each detected residual segment in the residual prototype topology, and the splitting mapping unit arranges the corresponding node indices according to the original position order of the detected residual segments to form a prototype mapping sequence.

[0012] Optionally, the node edge update unit updates the node parameters of the residual subspace nodes and the edge residual state of the residual propagation edges, and works with the splitting mapping unit to form a residual prototype topology, including: The node edge update unit subtracts the detected residual fragment from the center vector of the first matching node bit by bit, and projects the resulting fragment difference to each residual basis vector of the first matching node; The node edge update unit merges the projection components corresponding to each residual basis vector, superimposes the merged projection components with the center vector in the same position to form a node reconstruction fragment, and deducts the node reconstruction fragment from the detected residual fragment to obtain the projection residual; The node edge update unit establishes or updates the residual propagation edge between the first matching node and the second matching node, and splits the projection residual along the edge direction between the first matching node and each adjacent residual subspace node. The directional residual is written into the corresponding residual propagation edge. Each residual propagation edge accumulates the residual in the same direction and subtracts the residual in the opposite direction, and updates the edge residual vector and the edge residual accumulation. The node edge update unit superimposes the fragment difference to the center vector of the first matching node according to the node update step size, multiplies the projection residual by the projection coefficient of the fragment difference on the corresponding residual basis vector, and superimposes the resulting product to the corresponding residual basis vector, and performs orthogonalization processing on the updated residual basis vector. The splitting mapping unit delineates the residual propagation edge to be split according to the cumulative residual amount. Based on the node parameters at both ends of the residual propagation edge to be split and the edge residual vector, it generates the center vector and residual basis vector of the new residual subspace node and connects the new residual subspace node to the node connection relationship to form the residual prototype topology.

[0013] Optionally, S6 includes: S61. Group the prototype mapping sequence according to the tested energy meter and the testing stage. Arrange the node indexes in each group according to the original starting position of the detected residual segment. Retain the first node index of each group. Combine adjacent node indices in each group into prototype index transfer symbols and count the frequency of occurrence of prototype index transfer symbols. S62. Perform component quantization on the segment deviation between the detected residual segment and the corresponding residual subspace node to form a segment deviation symbol, and count the frequency of occurrence of the segment deviation symbol. S63. Configure the index transfer coding status table and the fragment deviation coding status table according to the frequency of occurrence of the prototype index transfer symbol and the fragment deviation symbol, respectively; S64. The prototype index transfer symbol and segment deviation symbol in each group are encoded using an asymmetric digital system encoding method to form an encoding segment corresponding to the tested energy meter and the testing stage. The first node index, index transfer encoding status table, segment deviation encoding status table and encoding segment of each group are assembled into residual compressed data.

[0014] Optionally, S7 includes: S71. Arrange the batch commonality representations according to the detection batch and detection stage, and assemble the residual prototype topology corresponding to the same detection batch with the batch commonality representation; S72. Bind the corresponding code segment of each tested energy meter at each testing stage to the corresponding segment recovery index, and record the length of the corresponding code segment in the segment recovery index; S73. Write the batch common representation, residual prototype topology, residual compressed data and fragment recovery index into the compressed data segment, and concatenate the compressed data segments along the execution order of the detection items to form a compressed data package for the electricity meter detection pipeline.

[0015] The beneficial effects of this invention are: 1. By performing time-position correction and matrix assembly on the detection responses within the same detection batch, and then using robust principal component analysis to split the batch detection matrix into a low-rank common part and a sparse difference part, the repeated detection response changes between different tested energy meters are centrally stored in the batch common representation, reducing the duplicate recording of common data within the same detection stage and reducing the storage and upload volume of energy meter detection pipeline data.

[0016] 2. By representing the local center position and multiple change directions of the detection residual fragments through residual subspace nodes, and accumulating the directional residuals between adjacent nodes that are not absorbed by the residual subspace through residual propagation edges, new nodes can be inserted at the residual set positions. Compared with the GNG model that uses a single prototype vector, it can cover the local distribution of the detection residual fragments with fewer nodes, reducing the data size of the fragment projection residuals and prototype topology.

[0017] 3. By using the fragment recovery index, the meter attribution, detection stage, original starting position, and effective residual length of the detected residual fragment are preserved. Asymmetric digital system encoding is performed on the prototype index transfer symbol and fragment deviation symbol respectively, so that the compressed data packet can retain the node parameters, time position relationship, and position information required for detection response reconstruction while reducing the space occupied by residual data. This supports edge computing nodes to complete the continuous compression, transmission, and recovery of detection data. Attached Figure Description

[0018] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart of a data compression method for an energy meter detection pipeline based on edge computing proposed in this invention; Figure 2 This is a schematic diagram of an improved GNG model for a data compression method for an energy meter detection pipeline based on edge computing proposed in this invention. Figure 3 This is a flowchart illustrating the compressed data encapsulation and recovery process of a data compression method for an energy meter detection pipeline based on edge computing proposed in this invention. Detailed Implementation

[0019] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0020] refer to Figures 1-3 A data compression method for an energy meter detection pipeline based on edge computing includes the following steps: S1. Obtain the test data generated by the electricity meter testing pipeline, arrange it according to the execution order of the test items, and form a pipeline test sequence; S2. Perform time-position correction on the pipeline detection sequence within the same detection batch, and assemble the data in the same detection stage into a batch detection matrix; S3. Use robust principal component analysis to perform low-rank sparse decomposition on the batch detection matrix, perform low-rank factor decomposition on the low-rank decomposition results to form a batch commonality representation, and organize the sparse decomposition results into a difference residual matrix. S4. Back-sort the difference residual matrix according to the original position order of the pipeline detection sequence, perform segment segmentation along the boundary of the detection stage and record the segment recovery index to form a set of detection residual segments; S5. Input the set of detected residual fragments into the improved GNG model. Replace the prototype nodes with residual subspace nodes in the improved GNG model. Set residual propagation edges between the residual subspace nodes. Perform residual propagation edge splitting and new node insertion according to the edge residual state of the residual propagation edges to form residual prototype topology and prototype mapping sequence. S6. Calculate the fragment deviation between the detected residual fragment and the corresponding residual subspace node, and perform entropy encoding on the prototype mapping sequence and fragment deviation to form residual compressed data. S7. Encapsulate the batch common representation, residual prototype topology, residual compressed data, and fragment recovery index into a compressed data package for the energy meter detection pipeline.

[0021] In this embodiment, the detection data includes batch attribution data, detection item data, stage time data, and detection response data; The batch attribution data records the identifier of the tested energy meter and the corresponding testing batch; the testing item data records each testing item and its execution order; the stage time data records the sampling time and testing stage corresponding to the testing data; and the testing response data records the testing values ​​generated by the tested energy meter in each testing stage.

[0022] In the specific implementation process, edge computing nodes are deployed on the electricity meter testing pipeline side. The edge computing nodes receive the testing records generated by each testing station and complete local aggregation before the testing records enter the compression process. Batch attribution data is obtained from the pipeline testing tasks and the flow records of the tested electricity meters, testing item data is obtained from the testing process configuration, stage time data is synchronously written by the testing station when the testing response is generated, and testing response data is obtained from the output records of the corresponding testing equipment.

[0023] Edge computing nodes use the identifier of the tested energy meter and the test batch as the aggregation index to bind and store the test items, stage timestamps, and test responses corresponding to the same test record. The execution order of test items adopts the sequence number in the test process configuration, and the test stage adopts the stage identifier written by the test station during the execution of a test item. The test stage identifier is formed by combining the test item execution sequence number and the internal stage number of the test item. The test stage identifiers corresponding to different test items within the same test batch are unique, and the sampling time adopts the time record carried by the edge computing node when receiving the test response.

[0024] Once the bound test records are temporarily stored in the local cache of the edge computing node, they are arranged according to the test batch and the identifier of the tested energy meter. After the test records meet the aggregation conditions for a test batch, the edge computing node retrieves the corresponding test records from the local cache and enters the pipeline test sequence formation process.

[0025] In this embodiment, forming the pipeline detection sequence includes: The test data were grouped according to the test batch and the affiliation of the tested electricity meters; Within each group, the test data are arranged according to the execution order of the test items, and the test response values ​​within the same test item are sorted by stage and time position. The detection stage boundaries are written between adjacent detection stages, and the arranged detection data are sequentially connected to form a pipeline detection sequence.

[0026] In practice, edge computing nodes retrieve detection records corresponding to the same detection batch from their local cache and combine the detection batch identifier with the identifier of the inspected energy meter to form a grouping index. Detection records with the same grouping index are grouped into the same data group, while data groups corresponding to different inspected energy meters are independent of each other, avoiding overlap of detection responses generated by different energy meters during the arrangement process.

[0027] The data group reads the execution sequence number from the test item data and arranges the test items in ascending order of execution sequence number. When the same test item contains multiple test stages, the test records are first grouped according to the test stage identifier, and then the test responses within the same test stage are arranged from front to back according to the sampling time, so that the order of the test responses is consistent with the actual execution process of the test station.

[0028] When the detection stage identifier corresponding to adjacent detection records changes, the detection stage boundary is written at the connection position of the two detection stages, and the end position of the previous detection stage and the start position of the next detection stage are recorded. After all detection items and detection stages are arranged, the detection records are sequentially connected according to the execution order of the detection items to form a pipeline detection sequence for the corresponding tested energy meter.

[0029] In this embodiment, S2 includes: S21. According to the boundary of the detection stage, extract the stage data segments in the detection sequence of each production line within the same detection batch, and correct the starting time position of each stage data segment to the starting position of the corresponding detection stage. S22. Set the correction sampling time points for the same detection stage according to a unified sampling interval, perform linear interpolation on the stage data segments, and form a stage response sequence with consistent time points; S23. Arrange the stage response sequences belonging to the same detection stage row by row, using the correction sampling time position as the matrix column position and the stage response sequences corresponding to different tested energy meters as the matrix row positions, assemble them into a batch detection matrix, and retain the correspondence between the matrix row and column positions and the original position sequence of the pipeline detection sequence.

[0030] In the specific implementation process, the edge computing node reads the detection stage boundaries of the detection sequence records of each pipeline within the same detection batch, and extracts the continuous detection records corresponding to each detection stage. Each stage data segment uses the sampling time corresponding to the first detection response as the stage start time bit. Subtracting the stage start time bit from each sampling time within the stage data segment yields the relative sampling time bits arranged starting from 0, thereby eliminating the start time offset caused when different tested energy meters enter the same detection stage.

[0031] The uniform duration of the same detection stage is read from the detection process configuration. Using the start position of the detection stage as time 0, the correction sampling time is set according to a uniform sampling interval of 20ms, with the last correction sampling time not exceeding the uniform duration. Each stage data segment entering the batch detection matrix should cover the time range from time 0 to the last correction sampling time. If a stage data segment has not yet covered the last correction sampling time, the corresponding detection record is retained in the local cache of the edge computing node, and linear interpolation is performed only after the detection records for the corresponding detection stage are fully collected.

[0032] For detection positions where the corrected sampling time coincides with the original sampling time, the corresponding detection response value is directly retained. For detection positions where the corrected sampling time is located between two adjacent original sampling time positions, the previous original sampling time position, the next original sampling time position, and the corresponding detection response value are read. The time ratio occupied by the corrected sampling time position between the two original sampling time positions is calculated. The difference between the two detection response values ​​is multiplied by the time ratio and then superimposed onto the previous detection response value to obtain the interpolated response value corresponding to the corrected sampling time position.

[0033] After linear interpolation, the stage response sequences corresponding to each tested energy meter within the same testing phase have the same number of correction sampling time bits. The stage response sequence corresponding to each tested energy meter is loaded into a matrix row position, and the correction sampling time bits are loaded into the matrix column positions in chronological order to form a batch testing matrix.

[0034] For each original sampling time position in the pipeline testing sequence, the preceding and following corrected sampling time positions located on either side of the original sampling time position are found in the corrected sampling time position sequence. The identifier of the tested energy meter, the testing stage identifier, the original position sequence, the column position of the preceding and following corrected matrix, and the time proportion occupied by the original sampling time position between the two corrected sampling time positions are recorded. When an original sampling time position coincides with a corrected sampling time position, both the preceding and following corrected matrix columns are recorded as their corresponding matrix columns, and the time proportion is recorded as 0. These records form the original position sequence backorder relationship and are saved together with the batch testing matrix.

[0035] In this embodiment, S3 includes: S31. Use robust principal component analysis to alternately perform singular value shrinkage and soft threshold shrinkage on the batch detection matrix to update the low-rank matrix and sparse matrix. S32. Perform singular value decomposition on the low-rank matrix, rearrange the left singular vectors into common mapping coefficients, combine the singular values ​​and the right singular vectors into common basis vectors, and pair the common mapping coefficients with the common basis vectors to form batch common representations. S33. Keeping the row and column positions of the sparse matrix unchanged, rearrange the sparse matrix into a difference residual matrix.

[0036] In the specific implementation process, the number of rows in the batch detection matrix corresponds to the number of energy meters under inspection in the same detection stage, and the number of columns in the batch detection matrix corresponds to the number of correction sampling times. Before entering the low-rank sparse decomposition, the average value of the detection response value of each column of the batch detection matrix is ​​calculated. The average value of the corresponding matrix column is subtracted from each detection response value to form a centralized batch detection matrix, and the average value of each matrix column is written into the batch commonality representation.

[0037] Both the low-rank matrix and the sparse matrix are initialized to zero matrices with the same dimensions as the centralized batch detection matrix. The sparsity contraction coefficient is the reciprocal of the square root of the larger of the number of rows and columns in the matrix. The spectral norm of the centralized batch detection matrix is ​​calculated, and the infinity norm of the centralized batch detection matrix is ​​divided by the sparsity contraction coefficient; the larger of the two values ​​is taken as the normalization factor. When the normalization factor is greater than 0, the centralized batch detection matrix is ​​divided element-wise by the normalization factor to form the initial values ​​of the Lagrange multiplier matrix, and 1.25 is divided by the largest singular value of the centralized batch detection matrix as the initial penalty parameter. The penalty parameter is multiplied by 1.5 after each iteration, and the upper limit of the penalty parameter is set to ten million times the initial penalty parameter. When the normalization factor is equal to 0, the low-rank matrix and the sparse matrix are kept as zero matrices, and the low-rank-sparse decomposition of the current detection stage ends.

[0038] In each iteration, the current sparse matrix is ​​first subtracted from the centralized batch detection matrix, and then the correction amount obtained by dividing the Lagrange multiplier matrix by the current penalty parameter is added. Singular value decomposition is performed on the calculation result. Each singular value is subtracted from the reciprocal of the current penalty parameter. Results less than zero are set to zero, and non-zero contracted singular values ​​and their corresponding left and right singular vectors are retained. The low-rank matrix is ​​reconstructed using the retained results. After the low-rank matrix is ​​updated, the updated low-rank matrix is ​​subtracted from the centralized batch detection matrix, and the correction amount obtained by dividing the Lagrange multiplier matrix by the current penalty parameter is added. Soft thresholding is performed on the calculation result element by element. When the absolute value of an element is not greater than the product of the sparse contraction coefficient and the reciprocal of the current penalty parameter, the corresponding element is set to zero. When the absolute value of an element is greater than this product, the product is subtracted from the absolute value of the element while retaining the original sign, resulting in the updated sparse matrix.

[0039] After the low-rank matrix and sparse matrix are updated, they are subtracted from the centralized batch detection matrix to form the current decomposition difference. This current decomposition difference is multiplied by the current penalty parameter and then added to the Lagrange multiplier matrix. The square root of the sum of squares of each element in the current decomposition difference and the square root of the sum of squares of each element in the centralized batch detection matrix are calculated separately. Iteration stops when the ratio of the two is not greater than 0.000001, and the current decomposition process ends when the number of iterations reaches 200. After iteration, the low-rank matrix retains the response changes common to multiple tested energy meters in the same detection stage, and the sparse matrix maintains the original row and column positions of the batch detection matrix.

[0040] Calculate the square root of the sum of squares of all elements in the low-rank matrix. If the result is no greater than 0.00000001, the number of singular components retained is recorded as 0, and the common mapping coefficients and common basis vectors are both set to empty. Batch commonality means retaining the column mean obtained by the centering process. When restoring the low-rank part, the column mean is repeatedly arranged according to the number of rows in the matrix to form the common response of the corresponding batch detection matrix.

[0041] When the calculation result is greater than 0.00000001, singular value decomposition is performed on the low-rank matrix to obtain a left singular vector matrix, a singular value diagonal matrix, and a right singular vector matrix. The singular components are arranged in descending order of singular value, and the squares of the singular values ​​are accumulated starting from the largest singular value. When the accumulated squares of the singular values ​​reach 95% of the sum of all singular value squares, the singular components within the current accumulation range are retained. The columns in the left singular vector matrix corresponding to the retained singular components are retained, and the elements in each retained column of the matrix row containing each tested energy meter are arranged sequentially as the common mapping coefficients for the corresponding tested energy meters. Each retained singular value is multiplied by the corresponding right singular vector to form the corresponding common basis vector.

[0042] The batch commonality representation is based on the assembly of commonality mapping coefficients, commonality basis vectors, and the column mean retained after centering, according to the testing batch and testing stage. The row positions of the commonality mapping coefficients correspond to the identifier of the tested energy meter, and the column positions of the commonality basis vectors correspond to the positions of the correction sampling time, enabling the batch commonality representation to reconstruct the low-rank part according to the row and column positions of the original matrix. The sparse matrix is ​​no longer rearranged in rows and columns; it is directly organized into a difference residual matrix according to the row and column positions of the batch testing matrix, and carries the original positional correspondence stored in the batch testing matrix into the difference residual rearrangement process.

[0043] In this embodiment, S4 includes: S41. Call the correspondence between the row and column positions of the batch detection matrix and the original position sequence of the pipeline detection sequence, and restore the arrangement of the difference residuals in each row of the difference residual matrix into the energy meter residual sequence. S42. Map the boundary position of the detection stage in the pipeline detection sequence to the residual sequence of the electricity meter, and cut the residual sequence of the electricity meter at the boundary position of the detection stage to form a stage residual segment. S43. Divide the stage residual segments in order of the same segment length, perform padding on the tail residuals that are not long enough, and record the effective residual length to form the detection residual segments. S44. Record the tested energy meter, testing stage, original starting position and effective residual length corresponding to each detection residual segment. Assemble the recorded content into a segment recovery index and arrange the detection residual segments according to the original starting position to form a set of detection residual segments.

[0044] In the specific implementation process, the difference residual matrix maintains the row and column positions of the batch detection matrix. The matrix row positions correspond to the identifier of the tested energy meter, and the matrix column positions correspond to the correction sampling time positions. During the reflow, the corresponding matrix row in the difference residual matrix is ​​read according to the identifier of the tested energy meter, and the original sampling time positions of the same detection stage in the pipeline detection sequence are read one by one.

[0045] When the original sampling time position coincides with the corrected sampling time position, the difference residual of the corresponding matrix column is directly read and written into the original position corresponding to the original sampling time position. When the original sampling time position is between two adjacent corrected sampling time positions, the difference residuals corresponding to the previous and subsequent corrected sampling time positions are read, the time proportion occupied by the original sampling time position between the two corrected sampling time positions is calculated, the difference between the two difference residuals is multiplied by the time proportion and then added to the difference residual corresponding to the previous corrected sampling time position to obtain the residual value corresponding to the original sampling time position.

[0046] After completing the reordering of the difference residuals from all the original sampling positions of the same tested energy meter, the residual values ​​are arranged from front to back according to the original position order to form the energy meter residual sequence.

[0047] The detection stage boundaries recorded in the pipeline detection sequence also carry the original positional sequence on both sides of the boundary. The original positional sequence on both sides of the detection stage boundary is written to the same position in the energy meter residual sequence. The energy meter residual sequence is then cut at locations where changes occur during the detection stage, ensuring that each stage residual segment contains only consecutive residual values ​​from within a single detection stage. The detection stage boundaries are not included in the segmentation as residual values; only the detection stage identifier and the starting position of the stage residual segment in the energy meter residual sequence are retained.

[0048] The detection residual segment length is set to 32 residual values. 32 consecutive residual values ​​are truncated from the start position of each stage residual segment, with the end position of the previous truncation serving as the start position for the next truncation, until the entire stage residual segment is segmented. If the number of remaining residuals at the end of a stage residual segment is less than 32, zeros are padded after the remaining residuals until the segment length reaches 32, and the actual number of residuals before padding is recorded as the effective residual length. If the stage residual segment length is less than 32, the entire stage residual segment forms a single detection residual segment, and zeros are padded to the remaining positions.

[0049] Each detection residual segment corresponds to the recorded identifier of the tested energy meter, the detection stage identifier, the original starting position, and the effective residual length. The original starting position is the position of the first effective residual in the pipeline detection sequence within the detection residual segment; the effective residual length of a complete segment is recorded as 32, and the effective residual length of the tail segment is recorded as the actual number of residuals before padding. These records are assembled into a segment recovery index according to the same arrangement order as the detection residual segments. When recovering the detection data, each recovered segment is grouped according to the tested energy meter identifier and the detection stage identifier, the padding position is deleted according to the effective residual length, and then the effective residual is rewritten back to the corresponding pipeline detection sequence position according to the original starting position.

[0050] After completing the fragment recovery index assembly, the detection residual fragments are first grouped together according to the identifier of the tested energy meter, and then the detection residual fragments corresponding to the same tested energy meter are arranged in sequence according to the detection stage and the original starting position to form a set of detection residual fragments.

[0051] In this embodiment, the improved GNG model includes a competitive matching unit, a residual subspace node set, a residual recursive edge set, a node edge update unit, and a splitting mapping unit. The competitive matching unit projects the detected residual fragments onto the set of nodes in the residual subspace, and delineates the first and second matching nodes according to the remaining residuals after projection; The node edge update unit establishes or updates the residual propagation edge between the first matching node and the second matching node. It splits the projection residual corresponding to the first matching node into directional residuals along the node connection direction between the first matching node and each adjacent residual subspace node, and writes the directional residuals into the corresponding residual propagation edge. Each residual propagation edge accumulates the directional residuals and updates the edge residual state. The splitting mapping unit delineates the residual propagation edge to be split according to the edge residual state, generates a new residual subspace node based on the node parameters at both ends of the residual propagation edge to be split and the edge residual state, and connects the new residual subspace node to the node connection relationship. The node edge update unit updates the node parameters of the residual subspace nodes and the edge residual state of the residual propagation edge, and works with the splitting mapping unit to form the residual prototype topology; The competitive matching unit re-defines the residual subspace nodes corresponding to each detected residual segment in the residual prototype topology, and the splitting mapping unit arranges the corresponding node indices according to the original position order of the detected residual segments to form a prototype mapping sequence.

[0052] In this embodiment, the node edge update unit updates the node parameters of the residual subspace nodes and the edge residual state of the residual propagation edges, and works with the splitting mapping unit to form a residual prototype topology, including: The node edge update unit subtracts the detected residual fragment from the center vector of the first matching node bit by bit, and projects the resulting fragment difference to each residual basis vector of the first matching node; The node edge update unit merges the projection components corresponding to each residual basis vector, superimposes the merged projection components with the center vector in the same position to form a node reconstruction fragment, and deducts the node reconstruction fragment from the detected residual fragment to obtain the projection residual; The node edge update unit establishes or updates the residual propagation edge between the first matching node and the second matching node, and splits the projection residual along the edge direction between the first matching node and each adjacent residual subspace node. The directional residual is written into the corresponding residual propagation edge. Each residual propagation edge accumulates the residual in the same direction and subtracts the residual in the opposite direction, and updates the edge residual vector and the edge residual accumulation. The node edge update unit superimposes the fragment difference to the center vector of the first matching node according to the node update step size, multiplies the projection residual by the projection coefficient of the fragment difference on the corresponding residual basis vector, and superimposes the resulting product to the corresponding residual basis vector, and performs orthogonalization processing on the updated residual basis vector. The splitting mapping unit delineates the residual propagation edge to be split according to the cumulative residual amount. Based on the node parameters at both ends of the residual propagation edge to be split and the edge residual vector, it generates the center vector and residual basis vector of the new residual subspace node and connects the new residual subspace node to the node connection relationship to form the residual prototype topology.

[0053] In its implementation, the improved GNG model employs an unsupervised iterative training method using a set of detection residual segments. Each detection residual segment contains 32 residual values ​​arranged in their original positional order. The original GNG model only had one prototype vector with the same dimension as the detection residual segment in each node, and the connecting edges between nodes only recorded the adjacency relationship between two nodes. The position of newly added nodes was mainly determined by the cumulative error of the nodes. The improved GNG model retains the competitive matching and topology growth processes, replaces the prototype nodes with residual subspace nodes, and sets residual propagation edges between adjacent residual subspace nodes. Each residual subspace node includes a 32-dimensional center vector and four 32-dimensional residual basis vectors, which are unit-length and mutually orthogonal. The residual propagation edge records the two connected nodes, the edge residual vector, and the cumulative residual value. The edge residual vector contains 32 components, and the cumulative residual value is the square root of the sum of the squares of the components of the edge residual vector.

[0054] At the start of training, the first detection residual is read from the set of detection residuals, and the 32 residual values ​​of the first detection residual are copied as the center vector of the first residual subspace node.

[0055] When the detection residual fragment set contains only one detection residual fragment, the center vector of the first residual subspace node is copied, and the thirty-second component of the copy result is increased by 0.000001 to form the center vector of the second residual subspace node.

[0056] When the set of detection residual segments contains more than two detection residual segments, the bitwise differences and sum of squared differences between each of the remaining detection residual segments and the center vector of the first residual subspace node are calculated segment by segment. If the maximum sum of squared differences is greater than 0.00000001, the detection residual segment corresponding to the maximum sum of squared differences is taken as the center vector of the second residual subspace node; if the maximum sum of squared differences is not greater than 0.00000001, the center vector of the first residual subspace node is copied, and the thirty-second component of the copied result is increased by 0.000001 to form the center vector of the second residual subspace node.

[0057] The four residual basis vectors of the two initial nodes are initialized as 32-dimensional unit vectors with the first, second, third, and fourth bits set to 1 and the remaining bits set to 0. A first residual propagation edge is established between the two initial nodes, with all 32 components of the edge residual vector set to 0 and the cumulative residual value set to 0.

[0058] The competitive matching unit sequentially reads the detection residual segments and calculates the projected residual of each detection residual segment relative to each residual subspace node. When calculating a residual subspace node, first, each residual value of the detection residual segment is subtracted from the value at the same position in the center vector to form a segment difference. Then, the segment difference is multiplied bit-by-bit by the four residual basis vectors and accumulated to obtain four projection coefficients. Each projection coefficient is multiplied by the corresponding residual basis vector, and the products are added at the same position and then superimposed with the center vector to form a node reconstruction segment. The node reconstruction segment is subtracted bit-by-bit from the detection residual segment to obtain the projected residual of the corresponding node.

[0059] The competing matching unit calculates the sum of squares of the residuals for each projection. The residual subspace node with the smallest sum of squares is designated as the first matching node, and the residual subspace node with the second smallest sum of squares is designated as the second matching node. If there is no residual propagation edge between the first matching node and the second matching node, a residual propagation edge is directly established between the two nodes; if there is already a residual propagation edge, the current connection relationship is preserved, and the edge residual state is updated.

[0060] The node edge update unit reads all adjacent residual subspace nodes of the first matching node. Each residual propagation edge is set with a uniform edge direction according to the ascending direction of the two endpoint node indices. For the current residual propagation edge, the center vectors of the node with the smaller index and the node with the larger index are read, and the center vector of the node with the smaller index is subtracted bit by bit from the center vector of the node with the larger index to form the edge center difference value.

[0061] When the square root of the sum of the squares of the edge center differences is greater than 0.00000001, the edge center differences are divided by the square root of the sum of the squares of the edge center differences to obtain a unified edge direction; when the square root of the sum of the squares of the edge center differences is not greater than 0.00000001, the direction residual of the current residual is skipped and passed on to the edge.

[0062] The node edge update unit multiplies the projection residual corresponding to the first matching node bit by bit with the unified edge direction and accumulates them to obtain a directional projection value with a positive or negative sign. When the directional projection value is greater than or equal to 0, the directional projection value is multiplied by the unified edge direction to form a same-direction directional residual, and the same-direction directional residual is added bit by bit to the edge residual vector; when the directional projection value is less than 0, the absolute value of the directional projection value is multiplied by the unified edge direction to form a reverse-direction residual, and the reverse-direction residual is subtracted bit by bit from the edge residual vector.

[0063] After completing the same-direction accumulation or reverse-direction deduction, calculate the square root of the sum of squares of each component of the updated edge residual vector, and write the calculation result into the edge residual cumulative quantity.

[0064] After completing the residual recursive edge update, the node edge update unit adjusts the node parameters of the first matching node. The center vector update step size is 0.05. The segment difference between the detected residual segment and the center vector is multiplied by 0.05 bit by bit, and then superimposed on the corresponding position of the center vector. The residual basis vector update step size is 0.01. The projected residual is multiplied by the projection coefficient of the segment difference on the corresponding residual basis vector, then multiplied by 0.01, and the resulting 32-dimensional vector is superimposed on the corresponding residual basis vector.

[0065] After the four residual basis vectors are updated, orthogonalization is performed in the order of the first to the fourth residual basis vectors. For the current residual basis vector, the projection components of the current residual basis vector in the directions of each orthogonally transformed residual basis vector are subtracted in turn, and then the square root of the sum of squares of the subtracted components is calculated.

[0066] If the length of the subtraction result is greater than 0.00000001, the subtraction result is divided by the corresponding length to form a normalized residual basis vector. If the length of the subtraction result is no greater than 0.00000001, a standard unit vector is selected from the 32 standard unit vectors that has the smallest sum of the absolute values ​​of the projections of the selected standard unit vector onto the existing residual basis vectors. The projection components of the selected standard unit vector onto the existing residual basis vectors are then subtracted, and normalization is performed again to form the current residual basis vector. After processing the four residual basis vectors, the four residual basis vectors maintain a unit length and are mutually orthogonal.

[0067] Every 256 detected residual fragments have been matched and updated, or a round of detected residual fragment traversal has been completed, the splitting mapping unit performs a residual recursion edge splitting check. The splitting mapping unit reads the cumulative residual value of all residual recursion edges, calculates the average of all cumulative residual values, and identifies the residual recursion edge with the largest cumulative residual value. When the largest cumulative residual value is greater than 0.00000001, not less than 1.5 times the average value, and the number of nodes in the residual subspace is less than 128, the corresponding residual recursion edge is split; otherwise, splitting is not performed.

[0068] When performing residual recursive edge splitting, the center vectors of the residual subspace nodes at both ends of the residual recursive edge are read. The corresponding components of the two center vectors are averaged to form the central intermediate vector. The edge residual vector is divided by the cumulative edge residual to obtain the edge residual direction. The distance between the center vectors of the two endpoints is calculated, and one-quarter of the distance is multiplied by the edge residual direction. This value is then added bit by bit to the central intermediate vector to form the center vector of the new residual subspace node.

[0069] The first residual basis vector of the new residual subspace node adopts the edge residual direction. The second to fourth residual basis vectors respectively read the residual basis vectors in the same order from the nodes at both ends of the residual propagation edge, average the corresponding components of the two residual basis vectors, and then successively subtract the projection components of the average result onto the already generated residual basis vector direction. When the length of the residual basis vector after subtracting the projection components is greater than 0.00000001, the residual basis vector is normalized; when the length of the residual basis vector is not greater than 0.00000001, the standard unit vector with the smallest sum of the absolute values ​​of the projections of the standard unit vectors onto the already generated residual basis vectors is selected from 32 standard unit vectors, the projection components of the standard unit vector onto the already generated residual basis vector direction are subtracted, and then normalization is performed. The new center vector and the four residual basis vectors together form the new residual subspace node.

[0070] The splitting mapping unit deletes the original residual propagation edge that was split, and establishes two new residual propagation edges between the nodes in the new residual subspace and the nodes at both ends of the original residual propagation edge. The edge residual vectors of the two new residual propagation edges are both half of the original edge residual vectors, and the cumulative edge residuals are both half of the cumulative cumulative residuals of the original edge. The new residual subspace nodes and the two new residual propagation edges are then connected to the original node connections, completing one topology growth.

[0071] One training round is defined as processing the entire set of detected residual fragments in their original order. The training can be performed up to 20 rounds. After each round, the average of the sum of squares of the projected residuals of all detected residual fragments is calculated.

[0072] If the average value of the previous round is greater than 0.00000001, divide the absolute value of the difference between the average value of the current round and the average value of the previous round by the average value of the previous round to obtain the average residual change ratio; if the average value of the previous round is not greater than 0.00000001 and the average value of the current round is also not greater than 0.00000001, record the average residual change ratio as 0; if the average value of the previous round is not greater than 0.00000001 but the average value of the current round is greater than 0.00000001, the current round is determined to be non-converged.

[0073] Training ends when no new residual subspace nodes are inserted for three consecutive rounds and the average residual change rate is no greater than 0.0001. Training also ends when the number of residual subspace nodes reaches 128 or when 20 rounds of training are completed. After training, the connections between residual subspace nodes and residual propagation edges form the residual prototype topology.

[0074] After the residual prototype topology is formed, competitive matching is performed again segment by segment according to the original position order of the detected residual segments. For each detected residual segment, the residual subspace node with the smallest sum of squared projected residuals is selected, the corresponding node index is recorded, and the node indices are concatenated according to the order of the detected residual segments to form a prototype mapping sequence.

[0075] For each detected residual segment, the center vector and four residual basis vectors of the corresponding residual subspace node are read. The detected residual segment is subtracted digit by digit from the center vector, and then multiplied digit by digit by the four residual basis vectors and summed to obtain four projection coefficients. The four projection coefficients are arranged first, followed by the 32 projected residual components, forming a segment bias containing 36 components. The four projection coefficients record the position of the detected residual segment in the residual subspace, and the 32 projected residual components record the remaining differences that the residual subspace cannot represent.

[0076] The original GNG model represented the local residual distribution using a single prototype vector. Detection residual segments varying in different directions within the same local region were easily assigned to multiple prototype nodes. The residual subspace node, on the other hand, represents the local residual distribution using a center vector and four residual basis vectors. A single node can handle detection residual segments varying in multiple directions near the center location, reducing the number of nodes corresponding to similar residual segments.

[0077] The original connection edges only recorded adjacency relationships and could not retain the unexplained residuals that continuously appeared between adjacent nodes. The residual propagation edges accumulate the projected residuals along the node connection direction to form the edge residual state. The splitting position is determined by the accumulated edge residual, and new residual subspace nodes are generated along the edge residual direction, ensuring that newly added nodes fall into residual regions not covered by the original residual subspace. After processing, the projected residual of the detected residual fragments is reduced, the node index changes in the prototype mapping sequence are more concentrated, and the data volume of the residual prototype topology and prototype mapping sequence is compressed.

[0078] In this embodiment, S6 includes: S61. Group the prototype mapping sequence according to the tested energy meter and the testing stage. Arrange the node indexes in each group according to the original starting position of the detected residual segment. Retain the first node index of each group. Combine adjacent node indices in each group into prototype index transfer symbols and count the frequency of occurrence of prototype index transfer symbols. S62. Perform component quantization on the segment deviation between the detected residual segment and the corresponding residual subspace node to form a segment deviation symbol, and count the frequency of occurrence of the segment deviation symbol. S63. Configure the index transfer coding status table and the fragment deviation coding status table according to the frequency of occurrence of the prototype index transfer symbol and the fragment deviation symbol, respectively; S64. The prototype index transfer symbol and segment deviation symbol in each group are encoded using an asymmetric digital system encoding method to form an encoding segment corresponding to the tested energy meter and the testing stage. The first node index, index transfer encoding status table, segment deviation encoding status table and encoding segment of each group are assembled into residual compressed data.

[0079] In the specific implementation process, edge computing nodes read the prototype mapping sequence and detect the segment deviation between the residual segments and the corresponding residual subspace nodes, and first group them according to the identifier of the tested energy meter and the identifier of the testing stage. The node indices in the same group are arranged in ascending order according to the original starting position of the segment recovery index record, and the number of node indices is consistent with the number of detected residual segments. Each group performs symbol sorting and encoding separately to avoid combining node indices that do not have a continuous relationship between different tested energy meters or different testing stages into transfer symbols.

[0080] Each group retains the first node index in the sequence. Starting from the second node index, the previous node index and the current node index are read sequentially. The previous node index is multiplied by the total number of nodes in the residual prototype topology, and then added to the current node index to obtain the prototype index transfer symbol. All adjacent node indices in the group are processed in the same way to form a sequence of prototype index transfer symbols. This symbol also retains the combination relationship between the previous node and the current node. During recovery, the prototype index transfer symbol is divided by the previous node index and the total number of nodes, and the remainder is the current node index.

[0081] After completing the organization of prototype index transfer symbols, the occurrence frequency of each prototype index transfer symbol within the same detection batch is counted. Prototype index transfer symbols are counted jointly by multiple groups, ensuring that frequently occurring node jumps under the same residual prototype topology share a single index transfer coding status table. If different detection batches form separate residual prototype topologies, the occurrence frequency of prototype index transfer symbols is counted separately for each detection batch, and a corresponding coding status table is configured.

[0082] Each segment bias contains four projection coefficients and 32 projection residual components, for a total of 36 components. Edge computing nodes read all segment biases group by group and calculate the maximum absolute value of each segment bias component in the current group. If the maximum value is greater than 0, it is divided by 2047 to obtain the quantization step size for the current group; if the maximum value is equal to 0, the quantization step size for the current group is set to 1, and all segment bias components are quantized to 0. Each segment bias component is then divided by the quantization step size of the current group, and the result is rounded to the nearest integer, limiting the quantization result to between -2047 and +2047 to obtain a signed quantized value.

[0083] When the signed quantization value is greater than or equal to 0, the signed quantization value is multiplied by 2 to form the segment bias symbol; when the signed quantization value is less than 0, the absolute value of the signed quantization value is multiplied by 2 and then subtracted by 1 to form the segment bias symbol. The detected residual segments are arranged according to their original starting position. Then, the segment bias symbols are arranged according to the order of the first to fourth projection coefficients, and the first to thirty-second projection residual components in each segment bias, forming a segment bias symbol sequence. The quantization step size of the current group is written into the code segment header of the corresponding coded code segment.

[0084] The occurrence counts of prototype index transfer symbols and fragment deviation symbols are counted separately, with each type of symbol using 16384 coding state positions. For any type of symbol, the number of valid symbols that have already appeared is first counted, and a coding state position is assigned to each valid symbol. The number of valid symbols is then subtracted from 16384 to obtain the number of remaining coding state positions.

[0085] Divide the occurrence count of each valid symbol by the total number of symbols in that class, multiply by the remaining number of coded state positions, round the result down, and use the integer as the number of appended state positions for the corresponding symbol. Add a base state position to the number of appended state positions to obtain the initial allocation quantity for the corresponding symbol.

[0086] When the sum of the initial allocations of all symbols is less than 16384, an additional coding state position is added to the corresponding symbol one by one, in descending order of the decimal part of the result of the appended state position calculation, until the total number of allocations equals 16384.

[0087] When the total number of prototype index transfer symbols is 0, the number of valid symbols in the index transfer encoding state table is recorded as 0, the length of the index transfer sub-code stream is recorded as 0, and the final state of the index transfer encoding maintains the initial encoding state; the corresponding group recovers the prototype mapping sequence only through the first node index.

[0088] The symbols are arranged in ascending order of their numerical values. The cumulative starting position of the first symbol is set to 0, and the cumulative starting position of the next symbol is the sum of the cumulative starting position of the previous symbol and the quantity allocated to the previous symbol.

[0089] The header of the index transfer coding status table records the number of valid index transfer symbols, and the table itself records the prototype index transfer symbols, the number of symbols allocated, and the cumulative starting position. The header of the fragment offset coding status table records the number of valid fragment offset symbols, and the table itself records the fragment offset symbols, the number of symbols allocated, and the cumulative starting position. The quantization step size for each group is only written to the header of the corresponding coded code segment and not to the fragment offset coding status table.

[0090] An asymmetric digital system encoding method with byte-normalization is adopted. The encoding precision is set to 14 bits, corresponding to 16384 encoding state positions; the encoding state bound is set to 8388608, and the initial encoding state is 8388608. The prototype index transfer symbol sequence and the fragment deviation symbol sequence are encoded from the last symbol to the first symbol, respectively, so that the decoding end can recover the symbols in the original order.

[0091] When encoding a symbol, the symbol allocation quantity and cumulative starting position are read from the corresponding encoding state table. When the current encoding state is greater than or equal to the product of the symbol allocation quantity and 131072, the lowest 8 bits of the current encoding state are written to a temporary byte area, and then the current encoding state is divided by 256, retaining the integer quotient. This process is repeated until the current encoding state is less than the product of the symbol allocation quantity and 131072. After state normalization, the integer quotient obtained by dividing the current encoding state by the symbol allocation quantity is multiplied by 16384, and then the remainder obtained by dividing the current encoding state by the symbol allocation quantity and the cumulative starting position are added to obtain the updated encoding state.

[0092] After all prototype index transfer symbols are encoded, the final state of the index transfer encoding is recorded, and the bytes in the temporary byte area are reversed according to the writing order to form the index transfer sub-codestream. The fragment deviation symbols are processed in the same way, recording the final state of the fragment deviation encoding and forming the fragment deviation sub-codestream.

[0093] The coded segment corresponding to each group is written sequentially with the identifier of the tested energy meter, the test stage identifier, the number of test residual segments, the index of the first node, the quantization step size of the current group, the final state of the index transfer encoding, the final state of the segment deviation encoding, the length of the index transfer sub-code stream, the length of the segment deviation sub-code stream, the index transfer sub-code stream, and the segment deviation sub-code stream.

[0094] The index transfer coding status table and the fragment deviation coding status table are each stored separately within the same testing batch. Each coded segment is arranged sequentially according to the original starting position of the tested energy meter identifier, the testing stage identifier, and the fragment recovery index record, and together with the two coding status tables, forms the residual compressed data. The fragment recovery index is bound to the corresponding coded segment during the compressed data packet encapsulation stage.

[0095] The residual subspace nodes are numbered consecutively from 0 to the total number of nodes minus 1. When recovering the prototype mapping sequence, the first node index and the number of detected residual segments are read from the encoded code segment. The number of detected residual segments is then subtracted by 1 to obtain the number of prototype index transfer symbols that need to be decoded.

[0096] When decoding a prototype indexed transfer symbol, first calculate the remainder when the current encoding state is divided by 16384, and use this remainder as the current state position. Search the indexed transfer encoding state table for prototype indexed transfer symbols whose cumulative starting position is not greater than the current state position, and whose current state position is less than the sum of the cumulative starting position and the symbol allocation quantity. Read the symbol allocation quantity and cumulative starting position corresponding to the found prototype indexed transfer symbol. Multiply the symbol allocation quantity by the integer quotient obtained by dividing the current encoding state by 16384, and then add the difference between the current state position and the cumulative starting position to form the updated encoding state.

[0097] When the updated encoded state is less than 8388608, read one byte according to the order of the index transfer sub-stream. Multiply the encoded state by 256 and then add the bytes to the read list. Repeat this byte reading process until the encoded state is not less than 8388608. Divide the decoded prototype index transfer symbol by the total number of nodes, take the remainder as the current node index, and append the current node index to the previous node index. Repeat the above process until all node indices of the current group are recovered.

[0098] When recovering fragment biases, the number of fragment bias symbols is determined by multiplying the number of detected residual fragments by 36. The same state position lookup, encoding state update, and byte padding process is used to recover the fragment bias symbols. When the number of fragment bias symbols is even, the fragment bias symbol is divided by 2 to obtain a non-negative signed quantization value; when the number of fragment bias symbols is odd, the fragment bias symbol is added by 1, divided by 2, and the negative value is obtained to obtain a negative signed quantization value. The signed quantization value is multiplied by the quantization step size recorded in the encoded code segment, and a set of four projection coefficients and 32 projected residual components are recovered for every 36 fragment bias components.

[0099] By constructing index transfer coding state tables and fragment bias coding state tables respectively, detection residual fragments continuously mapped to the same or adjacent nodes can form high-frequency prototype index transfer symbols, while the quantized fragment biases are concentrated in the range of 0 and its neighboring symbols. After these two types of concentrated distributions are encoded into the asymmetric digital system, the number of bits occupied by the prototype mapping sequence and fragment biases can be reduced, while retaining the first node index, coding state table, coding final state, and quantization step size. This allows the residual compressed data to be decoded and recovered in the order of the original fragments.

[0100] In this embodiment, S7 includes: S71. Arrange the batch commonality representations according to the detection batch and detection stage, and assemble the residual prototype topology corresponding to the same detection batch with the batch commonality representation; S72. Bind the corresponding code segment of each tested energy meter at each testing stage to the corresponding segment recovery index, and record the length of the corresponding code segment in the segment recovery index; S73. Write the batch common representation, residual prototype topology, residual compressed data and fragment recovery index into the compressed data segment, and concatenate the compressed data segments along the execution order of the detection items to form a compressed data package for the electricity meter detection pipeline.

[0101] In the specific implementation process, edge computing nodes aggregate batch common representations, residual prototype topologies, residual compressed data, and fragment recovery indexes according to the detection batches. The batch common representations are first arranged according to the execution order of the detection items, and then arranged according to the detection stage within the same detection item.

[0102] For each testing stage, the batch commonality representation segment is first written with the testing item identifier, testing stage identifier, number of rows in the batch testing matrix, number of columns in the batch testing matrix, number of singular components to be retained, and correction sampling time position sequence. Then, the commonality mapping coefficients are arranged according to the identifier of the tested energy meter, and the commonality basis vectors and column mean are arranged according to the correction sampling time position.

[0103] In addition to recording the identifier of the tested energy meter, the testing stage identifier, the original starting position, and the effective residual length, the fragment recovery index section also sequentially writes the original position, the column position of the previous correction matrix, the column position of the next correction matrix, and the time ratio according to each original sampling position. The recovery end reads the corresponding low-rank response or difference residual according to the column positions of the previous and next correction matrices, and then performs linear interpolation according to the time ratio to restore the low-rank response and difference residual to the same original position.

[0104] The residual prototype topology formed in the same detection batch is saved once in the compressed data packet. The starting position of the residual prototype topology segment is written with the number of residual subspace nodes and the number of residual propagation edges.

[0105] All residual subspace nodes are arranged in ascending order of node index, and the node index, center vector, and four residual basis vectors are written sequentially. All residual recursive edges are arranged in ascending order of their endpoint node indices, and only the endpoint node indices of each residual recursive edge are written. The edge residual vector and edge residual cumulative amount are not written to the compressed data package after model training. The node indexes remain consistent with those in the prototype mapping sequence, enabling the recovery end to read the center vector and residual basis vector of the corresponding residual subspace node according to the node index.

[0106] The residual compressed data is grouped according to the identifier of the tested energy meter and the testing stage. Within the same tested energy meter, the testing stages are first arranged according to the execution order of the testing items, and then the coded segments are arranged according to the original starting position in the segment recovery index. Each coded segment contains the first node index of the group, the index transfer sub-code stream, the segment deviation sub-code stream, and the corresponding final coded state; the index transfer coding state table and the segment deviation coding state table are saved according to the testing batch and correspond to the coded segments using the same coding state table.

[0107] The fragment recovery index is arranged in the same order as the coded segments. The edge computing node reads all the detection residual segments corresponding to a coded segment, binds the identified energy meter, detection stage, original start bit sequence, and effective residual length to the coded segment, and writes the byte length of the coded segment into the corresponding fragment recovery index. When a coded segment corresponds to multiple detection residual segments, the fragment recovery index sequentially records the original start bit sequence and effective residual length of each detection residual segment, and writes the coded segment length into the first record. After reading the coded segment length, the recovery end can accurately extract the current coded segment and perform symbol decoding according to the number of detection residual segments and the segment length.

[0108] Compressed data segments are concatenated according to the execution order of the test items. For each test item, the compressed data segment first writes the batch common representation corresponding to that test item, then writes the coded segment and fragment recovery index for the corresponding test stage. The residual prototype topology is written to the first compressed data segment of the same test batch, and its position is recorded in other compressed data segments of the same batch to avoid repeatedly writing the same topology data. The segment length in bytes is written at the beginning of each compressed data segment. Within each segment, the batch common representation, residual compressed data, and fragment recovery index are written consecutively. After concatenating all test item segments, a compressed data packet for the energy meter testing pipeline is formed.

[0109] When recovering the energy meter detection data, first read the residual prototype topology, batch common representation, residual compressed data and fragment recovery index from the compressed data packet, and then extract each coded segment in sequence according to the coded segment length recorded in the fragment recovery index.

[0110] The prototype mapping sequence is recovered using an index-transfer sub-codestream, and the four projection coefficients and 32 projected residual components corresponding to each detected residual segment are recovered using a segment deviation sub-codestream. The center vector and four residual basis vectors of the corresponding residual subspace node are read according to the node index in the prototype mapping sequence. The four projection coefficients are multiplied by their corresponding residual basis vectors, and the four products are then added bit-by-bit to the center vector to reconstruct the node reconstruction segment. The node reconstruction segment is then added bit-by-bit to the 32 projected residual components to recover the detected residual segment. The padding positions in the detected residual segment are deleted according to the effective residual length, and the effective residual is written back to the corresponding energy meter residual sequence according to the original starting bit order.

[0111] The low-rank matrix under the correction sampling position is reconstructed using the common mapping coefficients, common basis vectors, and column mean values ​​in the batch common representation. When the number of singular components is 0, the column mean values ​​are repeatedly arranged according to the number of rows in the batch detection matrix to form a low-rank matrix; when the number of singular components is greater than 0, the common mapping coefficients corresponding to each tested energy meter are multiplied by the corresponding common basis vectors, and the column mean values ​​are superimposed after accumulating each product bit by bit to form the low-rank response of the corresponding matrix row.

[0112] According to the original positional order recorded in the fragment recovery index segment, the column positions of the previous correction matrix, the next correction matrix, and the time ratio are read sequentially. When the column positions of the previous and next correction matrices are the same, the low-rank response of the corresponding matrix column position is read directly; when the column positions of the previous and next correction matrices are different, the low-rank responses corresponding to the two matrix column positions are read, the difference between the two low-rank responses is multiplied by the time ratio, and then superimposed on the low-rank response corresponding to the column position of the previous correction matrix to obtain the common response corresponding to the original positional order.

[0113] Arrange the common responses according to their original position sequence to form a common response sequence of the electricity meter. Then, superimpose the common response sequence of the electricity meter with the residual sequence of the electricity meter under test, the same test stage, and the same original position sequence, position by position, to recover the test response data.

[0114] Example 1: To verify the feasibility of this invention in practice, it was applied to an automated testing line for three-phase smart meters at a meter manufacturing company. This testing line has 96 parallel testing stations, testing approximately 2400 three-phase smart meters per workday. The testing process includes basic error detection, startup detection, voltage influence detection, clock detection, communication detection, and carrier detection, divided into 30 testing stages. The initial sampling intervals at each testing station range from 10ms to 40ms, and there is a certain initial timing offset when different meters enter the same testing stage. The original processing method packages and uploads the testing records separately according to the station, failing to utilize the common responses between different meters within the same testing batch. The average data volume generated by a single batch of 96 meters is 12.86MB, which easily leads to edge cache backlog when production tasks are concentrated.

[0115] Edge computing nodes were deployed on the testing pipeline side, each equipped with an 8-core processor, 16GB of RAM, and 256GB of SSD storage. Ten testing batches generated during continuous operation were selected as test data, each batch containing 96 tested electricity meters, for a total of 960 tested electricity meters. The total uncompressed data volume was 128.60MB.

[0116] Edge computing nodes first aggregate test records according to the test batch and the identifier of the tested energy meter, and then arrange them according to the execution order of the test items, the test stage, and the sampling time. The stage boundary between adjacent test stages is recorded, with the start position of the test stage as the 0 time position. The correction sampling time position is set according to the 20ms sampling interval, and linear interpolation is used to organize the test responses of each tested energy meter into an equal-length stage response sequence, which is then assembled to form a batch test matrix.

[0117] Robust principal component analysis was performed on the batch detection matrix. The iteration stopping condition was set to the ratio of the decomposition difference to the norm of the centered batch detection matrix being no greater than 0.000001, and the maximum number of iterations was set to 200 rounds. In the experiment, each batch detection matrix reached the stopping condition after an average of 37 iterations. Singular value decomposition was performed on the low-rank matrix, retaining singular components whose cumulative squared singular values ​​reached 95%. An average of 6.8 singular components were retained in each detection stage, and the batch commonality representation was composed of commonality mapping coefficients, commonality basis vectors, and column means.

[0118] The difference residuals are rearranged according to their original position and then segmented along the boundary of the detection stage. Every 32 residual values ​​are divided into a detection residual segment, with zeros padded at the end where there are fewer than 32 residual values, and the effective residual length is recorded. A total of 324,860 detection residual segments are generated from 10 detection batches.

[0119] Each residual subspace node in the improved GNG model comprises a 32-dimensional center vector and four 32-dimensional residual basis vectors. The update step size for the center vector is set to 0.05, and the update step size for the residual basis vectors is set to 0.01. A residual propagation edge splitting check is performed every 256 detected residual fragments or after completing one round of traversal. When the cumulative residual of the largest edge reaches 1.5 times the average, a new node is inserted between the nodes at both ends of the corresponding residual propagation edge. The maximum number of residual subspace nodes is 128, and the maximum number of training rounds is 20. In the experiments, the average training time was 13.4 rounds, and the average number of residual prototype topologies per batch was 74.3 residual subspace nodes.

[0120] After model training, the corresponding node indices are recorded according to the original positional order of the detected residual segments, and four projection coefficients and 32 projected residual components are recorded for each detected residual segment. The prototype index transfer symbol and segment deviation symbol are encoded using a 14-bit asymmetric digital system to form residual compressed data. Batch common representation, residual prototype topology, coded code segments, and segment recovery index are jointly encapsulated into a compressed data package for the energy meter detection pipeline.

[0121] The test results are shown in Table 1. The original DEFLATE lossless compression method, the robust principal component analysis combined with the ordinary GNG model and Huffman coding method were compared with the method of the present invention.

[0122] Table 1. Comparison of Data Compression Effects in Electricity Meter Testing Line

[0123] In Table 1, the compression factor is the ratio of the total amount of uncompressed data to the total amount of compressed data, the upload data reduction rate represents the proportion of data reduced after compression, and the root mean square error of reconstruction represents the proportion of the deviation between the recovered detection response and the original detection response to the detection range.

[0124] Test results show that the DEFLATE lossless compression method achieves a compression ratio of only 1.87 times; while the ordinary GNG model increases the compression ratio to 5.00 times, it requires an average of 119.6 prototype nodes to represent the detection residual fragments. The method of this invention compresses a total data volume of 16.80 MB, achieving a compression ratio of 7.65 times, reducing uploaded data by 86.94%, decreasing the average number of topology nodes to 74.3, and achieving a root mean square error of 0.0146%FS.

[0125] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A data compression method for an energy meter detection pipeline based on edge computing, characterized in that, Includes the following steps: S1. Obtain the test data generated by the electricity meter testing pipeline, arrange it according to the execution order of the test items, and form a pipeline test sequence; S2. Perform time-position correction on the pipeline detection sequence within the same detection batch, and assemble the data in the same detection stage into a batch detection matrix; S3. Use robust principal component analysis to perform low-rank sparse decomposition on the batch detection matrix, perform low-rank factor decomposition on the low-rank decomposition results to form a batch commonality representation, and organize the sparse decomposition results into a difference residual matrix. S4. Back-sort the difference residual matrix according to the original position order of the pipeline detection sequence, perform segment segmentation along the boundary of the detection stage and record the segment recovery index to form a set of detection residual segments; S5. Input the set of detected residual fragments into the improved GNG model. The improved GNG model replaces the prototype nodes with residual subspace nodes, sets residual propagation edges between the residual subspace nodes, and performs residual propagation edge splitting and new node insertion according to the edge residual state of the residual propagation edges to form residual prototype topology and prototype mapping sequence. S6. Calculate the fragment deviation between the detected residual fragment and the corresponding residual subspace node, and perform entropy encoding on the prototype mapping sequence and fragment deviation to form residual compressed data. S7. Encapsulate the batch common representation, residual prototype topology, residual compressed data, and fragment recovery index into a compressed data package for the energy meter detection pipeline.

2. The data compression method for an energy meter detection pipeline based on edge computing according to claim 1, characterized in that, The detection data includes batch attribution data, detection item data, stage time data, and detection response data; The batch attribution data records the identifier of the tested energy meter and the corresponding testing batch; the testing item data records each testing item and its execution order; the stage time data records the sampling time and testing stage corresponding to the testing data; and the testing response data records the testing values ​​generated by the tested energy meter in each testing stage.

3. The data compression method for an energy meter detection pipeline based on edge computing according to claim 1, characterized in that, The formation of the automated detection sequence includes: The test data were grouped according to the test batch and the affiliation of the tested electricity meters; Within each group, the test data are arranged according to the execution order of the test items, and the test response values ​​within the same test item are sorted by stage and time position. The detection stage boundaries are written between adjacent detection stages, and the arranged detection data are sequentially connected to form a pipeline detection sequence.

4. The data compression method for an energy meter detection pipeline based on edge computing according to claim 1, characterized in that, S2 includes: S21. According to the boundary of the detection stage, extract the stage data segments in the detection sequence of each production line within the same detection batch, and correct the starting time position of each stage data segment to the starting position of the corresponding detection stage. S22. Set the correction sampling time points for the same detection stage according to a unified sampling interval, perform linear interpolation on the stage data segments, and form a stage response sequence with consistent time points; S23. Arrange the stage response sequences belonging to the same detection stage row by row, using the correction sampling time position as the matrix column position and the stage response sequences corresponding to different tested energy meters as the matrix row positions, assemble them into a batch detection matrix, and retain the correspondence between the matrix row and column positions and the original position sequence of the pipeline detection sequence.

5. The data compression method for an energy meter detection pipeline based on edge computing according to claim 1, characterized in that, S3 includes: S31. Use robust principal component analysis to alternately perform singular value shrinkage and soft threshold shrinkage on the batch detection matrix to update the low-rank matrix and sparse matrix. S32. Perform singular value decomposition on the low-rank matrix, rearrange the left singular vectors into common mapping coefficients, combine the singular values ​​and the right singular vectors into common basis vectors, and pair the common mapping coefficients with the common basis vectors to form batch common representations. S33. Keeping the row and column positions of the sparse matrix unchanged, rearrange the sparse matrix into a difference residual matrix.

6. The method for data compression in an energy meter detection pipeline based on edge computing according to claim 1, characterized in that, S4 includes: S41. Call the correspondence between the row and column positions of the batch detection matrix and the original position sequence of the pipeline detection sequence, and restore the arrangement of the difference residuals in each row of the difference residual matrix into the energy meter residual sequence. S42. Map the boundary position of the detection stage in the pipeline detection sequence to the residual sequence of the electricity meter, and cut the residual sequence of the electricity meter at the boundary position of the detection stage to form a stage residual segment. S43. Divide the stage residual segments in order of the same segment length, perform padding on the tail residuals that are not long enough, and record the effective residual length to form the detection residual segments. S44. Record the tested energy meter, testing stage, original starting position and effective residual length corresponding to each detection residual segment. Assemble the recorded content into a segment recovery index and arrange the detection residual segments according to the original starting position to form a set of detection residual segments.

7. The data compression method for an energy meter detection pipeline based on edge computing according to claim 1, characterized in that, The improved GNG model includes a competitive matching unit, a residual subspace node set, a residual recursive edge set, a node edge update unit, and a splitting mapping unit. The competitive matching unit projects the detected residual fragments onto the set of nodes in the residual subspace, and delineates the first and second matching nodes according to the remaining residuals after projection; The node edge update unit establishes or updates the residual propagation edge between the first matching node and the second matching node. It splits the projection residual corresponding to the first matching node into directional residuals along the node connection direction between the first matching node and each adjacent residual subspace node, and writes the directional residuals into the corresponding residual propagation edge. Each residual propagation edge accumulates the directional residuals and updates the edge residual state. The splitting mapping unit delineates the residual propagation edge to be split according to the edge residual state, generates a new residual subspace node based on the node parameters at both ends of the residual propagation edge to be split and the edge residual state, and connects the new residual subspace node to the node connection relationship. The node edge update unit updates the node parameters of the residual subspace nodes and the edge residual state of the residual propagation edge, and works with the splitting mapping unit to form the residual prototype topology; The competitive matching unit re-defines the residual subspace nodes corresponding to each detected residual segment in the residual prototype topology, and the splitting mapping unit arranges the corresponding node indices according to the original position order of the detected residual segments to form a prototype mapping sequence.

8. The data compression method for an energy meter detection pipeline based on edge computing according to claim 7, characterized in that, The node edge update unit updates the node parameters of the residual subspace nodes and the edge residual state of the residual propagation edges, and works with the splitting mapping unit to form a residual prototype topology, including: The node edge update unit subtracts the detected residual fragment from the center vector of the first matching node bit by bit, and projects the resulting fragment difference to each residual basis vector of the first matching node; The node edge update unit merges the projection components corresponding to each residual basis vector, superimposes the merged projection components with the center vector in the same position to form a node reconstruction fragment, and deducts the node reconstruction fragment from the detected residual fragment to obtain the projection residual; The node edge update unit establishes or updates the residual propagation edge between the first matching node and the second matching node, and splits the projection residual along the edge direction between the first matching node and each adjacent residual subspace node. The directional residual is written into the corresponding residual propagation edge. Each residual propagation edge accumulates the residual in the same direction and subtracts the residual in the opposite direction, and updates the edge residual vector and the edge residual accumulation. The node edge update unit superimposes the fragment difference to the center vector of the first matching node according to the node update step size, multiplies the projection residual by the projection coefficient of the fragment difference on the corresponding residual basis vector, and superimposes the resulting product to the corresponding residual basis vector, and performs orthogonalization processing on the updated residual basis vector. The splitting mapping unit delineates the residual propagation edge to be split according to the cumulative residual amount. Based on the node parameters at both ends of the residual propagation edge to be split and the edge residual vector, it generates the center vector and residual basis vector of the new residual subspace node and connects the new residual subspace node to the node connection relationship to form the residual prototype topology.

9. The method for data compression in an energy meter detection pipeline based on edge computing according to claim 1, characterized in that, S6 includes: S61. Group the prototype mapping sequence according to the tested energy meter and the testing stage. Arrange the node indexes in each group according to the original starting position of the detected residual segment. Retain the first node index of each group. Combine adjacent node indices in each group into prototype index transfer symbols and count the frequency of occurrence of prototype index transfer symbols. S62. Perform component quantization on the segment deviation between the detected residual segment and the corresponding residual subspace node to form a segment deviation symbol, and count the frequency of occurrence of the segment deviation symbol. S63. Configure the index transfer coding status table and the fragment deviation coding status table according to the frequency of occurrence of the prototype index transfer symbol and the fragment deviation symbol, respectively; S64. The prototype index transfer symbol and segment deviation symbol in each group are encoded using an asymmetric digital system encoding method to form an encoding segment corresponding to the tested energy meter and the testing stage. The first node index, index transfer encoding status table, segment deviation encoding status table and encoding segment of each group are assembled into residual compressed data.

10. A data compression method for an energy meter detection pipeline based on edge computing according to claim 1, characterized in that, S7 includes: S71. Arrange the batch commonality representations according to the detection batch and detection stage, and assemble the residual prototype topology corresponding to the same detection batch with the batch commonality representation; S72. Bind the corresponding code segment of each tested energy meter at each testing stage to the corresponding segment recovery index, and record the length of the corresponding code segment in the segment recovery index; S73. Write the batch common representation, residual prototype topology, residual compressed data and fragment recovery index into the compressed data segment, and concatenate the compressed data segments along the execution order of the detection items to form a compressed data package for the electricity meter detection pipeline.

Images