Weak network mobile energy station log edge temporary storage desensitization upload method, system, device and medium

Abstract

A weak network mobile energy station log edge temporary storage desensitization uploading method, system, device and medium. The method comprises the following steps: deploying an edge gateway locally in the mobile energy station, and collecting original logs in real time; according to a preset desensitization rule, performing fuzzy or replacement processing on sensitive fields to generate desensitized logs; temporarily storing the desensitized logs in a non-volatile storage medium to form a log queue; monitoring the current network state, and when the network quality meets the uploading condition, sequentially taking the desensitized logs from the queue, and uploading the desensitized logs to the cloud platform by using block compression; and after successful uploading, clearing the locally uploaded log blocks. The application effectively solves the problem of log loss in a weak network environment by using the edge temporary storage and conditional uploading mechanism, realizes offline uploading and batch retransmission, performs desensitization processing in advance at the edge end, ensures that sensitive information is anonymized before leaving the energy station, guarantees data security and user privacy, and avoids invalid retransmission, thereby reducing energy and communication overhead.

Description

Technical Field

[0001] This application relates to the field of data communication and processing technology, specifically to a method, system, device, and medium for edge storage, desensitization, and uploading of mobile energy station logs in a weak network environment. Background Technology

[0002] With the rapid development of distributed energy and mobile energy storage technologies, mobile energy stations have been widely used in scenarios such as emergency power supply, field operations, and temporary events. These energy stations are typically equipped with complex management systems that continuously generate a large amount of operational logs, status data, and alarm information, which are crucial for analyzing equipment health status and optimizing operation and maintenance strategies.

[0003] In existing technologies, mobile energy stations typically operate online in real-time, transmitting log data to a central cloud platform via cellular networks (such as 4G / 5G). To ensure data integrity, some solutions employ breakpoint resume or data compression techniques. For example, a prior art Chinese patent discloses a vehicle-mounted data upload method that adjusts the upload rate by detecting network signal strength.

[0004] However, mobile power stations are often deployed in remote mountainous areas, basements, tunnel construction sites, and other areas with weak network signals, and existing technologies have significant shortcomings in practical applications. First, data transmission in weak network environments is extremely unstable, and real-time uploads are prone to failure, leading to the loss of log data. Second, directly uploading unprocessed raw logs may contain sensitive information such as device serial numbers, precise geographical locations, and user power consumption habits, posing risks to data security and privacy. Finally, frequent retransmission attempts not only exacerbate network congestion but also significantly consume the already limited battery power and computing resources of the power station. Summary of the Invention

[0005] This application provides a method, system, device, and medium for edge-based temporary storage and desensitized uploading of logs from mobile energy stations in weak network environments, which can effectively solve the problems of easy log loss, data security risks, and high resource consumption in the background technology under weak network environments.

[0006] To achieve the above objectives, this application provides the following technical solution: a method for edge-stored, anonymized, and uploaded logs from a mobile energy station in a weak network, comprising: deploying an edge gateway locally at the mobile energy station to collect the station's raw logs in real time; the edge gateway, according to preset anonymization rules, blurring or replacing sensitive fields in the raw logs to generate anonymized logs; temporarily storing the anonymized logs in the edge gateway's non-volatile storage medium to form a log queue; the edge gateway monitoring the current network status, and when the network quality meets preset upload conditions, retrieving the anonymized logs sequentially from the log queue, compressing them in blocks, and uploading them to the cloud platform; after successful upload, clearing the locally uploaded log blocks. This method, by completing local temporary storage, anonymization, and condition-triggered upload of logs at the network edge, effectively avoids data loss, ensures the security of sensitive information, and reduces resource overhead caused by invalid transmissions.

[0007] Furthermore, the preset de-identification rules include: identifying the device's unique identifier, precise GPS coordinates, and user identity information fields in the original logs, and replacing them with hash values, gridded coordinate area codes, or generalized identifiers, respectively. The effect of this technology is that while retaining the logs' value for macro-statistical and fault analysis, it eliminates personally identifiable information at the source, thus meeting data compliance requirements.

[0008] Furthermore, the edge gateway monitors the current network status, specifically including: periodically measuring network round-trip latency, packet loss rate, and signal strength between the gateway and the cloud platform, and calculating a network health score based on a preset weighted algorithm; the upload condition is configured such that the network health score is higher than a preset threshold. The effect of this technology is to avoid frequently initiating invalid connections and uploads during periods of signal fluctuation or extremely poor signal strength, thus saving power and bandwidth resources at the energy station.

[0009] Furthermore, the log queue employs a circular buffer or linked storage structure, assigning different upload priorities based on log level and generation time. After a system crash or power outage recovery, processing resumes from the point of interruption based on log integrity verification. This technology improves the reliability and flexibility of local storage, ensuring that critical alarm logs are uploaded with priority.

[0010] Furthermore, the chunked compression upload includes: dividing a single or batch of de-identified logs into fixed-size data blocks, each data block having an independent checksum and global sequence number; the cloud platform reassembles and verifies the integrity of the received data blocks based on the checksum and global sequence number, and if missing data is found, requests retransmission of the specific data block from the edge gateway. This technology achieves breakpoint resumption and selective retransmission, greatly improving the transmission success rate in weak network environments.

[0011] Furthermore, when the edge gateway detects a network disconnection or continuous degradation, it automatically switches to a pure local storage mode and records upload failure events. Once the network recovers, it batch-uploads the temporarily stored logs according to a first-in-first-out (FIFO) or high-priority-first-out (HFO) strategy. This technology ensures zero log loss during network outages and automatically completes data synchronization when the network becomes available.

[0012] Furthermore, the system also includes a data receiving and restoration module deployed on a cloud platform. This module receives uploaded data blocks, reassembles them according to global sequence numbers, and reversibly converts gridded coordinates or hash values ​​into the original information based on the corresponding anonymization rule mapping table, making it available for authorized operation and maintenance users to query. The effect of this technology is that it ensures data anonymization during transmission and storage without affecting authorized personnel's ability to audit and trace the original, authentic data.

[0013] Furthermore, the mobile energy station includes a battery array, an inverter, and an energy management system; the edge gateway is integrated into the energy management system or connected to the energy station's communication bus as a standalone hardware device. The advantage of this technology is that it provides a specific deployment method, enhancing the feasibility of the solution.

[0014] Furthermore, the non-volatile storage medium is eMMC, UFS, or an industrial-grade SD card, and its capacity is dynamically configured based on the energy station log generation rate and the expected longest network outage duration. This technology achieves a balance between cost and reliability, ensuring sufficient capacity to handle common network outage cycles.

[0015] A log edge temporary storage and desensitization uploading system for mobile energy stations in weak networks includes: a data acquisition module, a desensitization processing module, a local temporary storage management module, a network status monitoring module, and a conditional upload module. Each module implements the steps in any of the above methods.

[0016] A device for edge-based temporary storage and desensitization of logs from a mobile energy station in a weak network includes: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the method described above.

[0017] A computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the method described in any of the preceding claims.

[0018] Compared with the prior art, the beneficial effects of this application are: 1. This application solves the problem of easy loss of logs during real-time upload under weak network conditions by using edge storage and conditional upload mechanism, realizes network interruption resume and batch re-upload, and greatly improves the integrity and reliability of log data.

[0019] 2. This application anonymizes sensitive information before it leaves the energy station by performing pre-processing at the edge. The entire transmission and cloud storage process uses anonymized data, which effectively protects user privacy and device security and complies with increasingly stringent data security regulations.

[0020] 3. This application avoids invalid retransmission attempts by using intelligent network status monitoring and block compression upload, reduces the computing and communication power consumption of the energy station, and extends the effective working time of the mobile energy station, making it particularly suitable for outdoor scenarios that rely on battery power. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of the structure of the edge-storage and desensitized upload system for weak network mobile energy station logs in this embodiment of the application.

[0022] In the diagram: 1. Mobile Energy Station; 2. Edge Gateway; 3. Cloud Platform; 4. Raw Log Stream; 5. De-identification Processing Module; 6. Local Temporary Storage Area; 7. Network Status Monitor; 8. Conditional Upload Module; 9. De-identified Log Stream; 10. Data Block; 11. Hash Calculation Unit; 12. Log Queue; 13. Network Link. Detailed Implementation

[0023] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0024] In the description of this application, any descriptions of orientation, such as up, down, front, back, left, right, etc., indicating orientation or positional relationships based on the orientation or positional relationships shown in the accompanying drawings, are merely for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application. When a feature is referred to as being set, fixed, or connected to another feature, it can be directly set, fixed, or connected to the other feature, or it can be indirectly set, fixed, or connected to the other feature.

[0025] Please see Figure 1 This application provides the following technical solution embodiments: a log edge temporary storage and desensitization uploading system for mobile energy stations in weak networks.

[0026] The system includes a mobile energy station 1, an edge gateway 2 deployed locally at the mobile energy station 1, and a remote cloud platform 3. The mobile energy station 1 contains a battery array, inverter, and energy management system, and its generated raw log stream 4 is transmitted to the edge gateway 2 in real time. The edge gateway 2 integrates a de-identification module 5, which performs de-identification processing on sensitive fields in the raw log stream 4 according to preset de-identification rules, generating de-identified logs and storing them in a log queue 12 in a local temporary storage area 6. Specifically, the de-identification rules replace the device ID with a hash value (completed by a hash calculation unit 11), convert precise GPS coordinates into a gridded area code with a side length of 1 kilometer, and replace the user's meter number with a generalized identifier. A network status monitor 7 periodically measures the round-trip latency, packet loss rate, and signal strength of the network link 13 between the edge gateway 2 and the cloud platform 3, and calculates a health score characterizing network quality based on these indicators. The conditional upload module 8 retrieves de-identified logs from the log queue 12 according to a high-priority first-out strategy only when the health score is higher than a preset threshold (e.g., packet loss rate below 5% and signal strength above -85 dB / mW). It then compresses and divides these logs into multiple fixed-size data blocks 10, each with an independent checksum and global sequence number. These blocks are then sent to the cloud platform 3 via the conditional upload module 8. Upon successful upload, the conditional upload module 8 notifies the local temporary storage area 6 to delete the corresponding data block 10. If the upload fails or the network is interrupted, the data block 10 remains in the log queue 12 and will continue to be uploaded according to the global sequence number once the network is restored.

[0027] During operation: After the mobile energy station 1 is activated, the edge gateway 2 automatically begins collecting raw log stream 4. The de-identification processing module 5 processes each log entry in real time to ensure that no sensitive information in plaintext leaves the edge gateway 2. The de-identified logs are immediately appended to the tail of the log queue 12, which consists of non-volatile storage media (e.g., a 32GB industrial-grade SD card). Simultaneously, the network status monitor 7 performs a network probe every 30 seconds. When the network is detected to be available and of acceptable quality, an upload task is triggered. The upload task reads data from the head of the log queue 12, compresses and segments it, and streams it using the HTTP / 2 protocol. After receiving the data block 10, the cloud platform 3 sorts it according to its global sequence number, verifies its integrity through a checksum, and sends an acknowledgment message. If the edge gateway 2 does not receive an acknowledgment message within the timeout period, or receives a retransmission request for a specific block, it will reinsert the block into the head of the upload queue. The entire process achieves a robust data path with pre-storage, de-identification protection, and breakpoint resumption even in weak network environments.

[0028] It is worth noting that the core chip of the edge gateway 2 disclosed in this embodiment can be an industrial-grade ARM Cortex-A series processor, such as the NXP i.MX8MMini model, and runs an embedded Linux operating system. The comprehensive method for judging network health is to simultaneously examine signal strength, packet loss rate, and round-trip time. Only when the signal strength is higher than a set threshold, the packet loss rate is lower than the maximum allowable value, and the round-trip time does not exceed a preset upper limit, is the network quality considered usable. The capacity of the local log queue is recommended to be configured to maintain at least 7 days of data and generate an average of 1GB of logs per day. The hash algorithm in the de-identification module can use SHA-256. The selection and parameters of the above components are only examples; they can be adjusted according to specific needs in actual applications. The control program is written in C or Python, commonly used in existing technologies, and implements the above logic by calling the system network API, file input / output interface, and encryption library.

[0029] Although embodiments of this application have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for edge-based temporary storage and de-identification uploading of logs from a mobile power station in a weak network, characterized in that: include: Deploy an edge gateway locally at the mobile energy station to collect the station's raw logs in real time; The edge gateway, based on preset desensitization rules, blurs or replaces sensitive fields in the original logs to generate desensitized logs; The de-identified logs are temporarily stored in the non-volatile storage medium of the edge gateway to form a log queue; The edge gateway monitors the current network status. When the network quality meets the preset upload conditions, it retrieves the de-identified logs from the log queue in sequence and uploads them to the cloud platform using block compression. After successful upload, clear the uploaded log blocks from the local machine.

2. The method for edge-based temporary storage and de-identification uploading of logs from a mobile energy station in a weak network according to claim 1, characterized in that: The preset desensitization rules include: identifying the device unique identifier, precise GPS coordinates, and user identity information fields in the original logs, and replacing them with hash values, gridded coordinate area codes, or generalized identifiers, respectively.

3. The method for edge-based temporary storage and de-identification uploading of logs from a mobile energy station in a weak network according to claim 1, characterized in that: The edge gateway monitors the current network status, specifically including: periodically measuring the network round-trip latency, packet loss rate, and signal strength between the gateway and the cloud platform, and comprehensively calculating the network health score; the upload condition is configured such that the network health score is higher than a preset threshold.

4. The method for edge-based temporary storage and de-identification uploading of logs from a mobile energy station in a weak network according to claim 1, characterized in that: The log queue adopts a circular buffer or chained storage structure, and assigns different upload priorities according to log level and generation time; after system crash or power failure recovery, processing continues from the breakpoint based on log integrity verification.

5. The method for edge-based temporary storage and de-identification uploading of logs from a mobile energy station in a weak network according to claim 1, characterized in that: The segmented compression upload includes: dividing a single or batch of de-identified logs into data blocks of fixed size, with each data block having an independent checksum and global sequence number; the cloud platform reassembles and verifies the integrity of the received data blocks based on the checksum and global sequence number, and if a missing data block is found, it requests the edge gateway to retransmit the specific data block.

6. The method for edge-based temporary storage and de-identification uploading of logs from a mobile energy station in a weak network according to claim 1, characterized in that: When the edge gateway detects a network disconnection or continuous degradation, it automatically switches to a pure local storage mode and records upload failure events. When the network is restored, it re-uploads the temporary logs in batches according to the first-in-first-out or high-priority first-out strategy.

7. A log edge temporary storage and desensitization uploading system for mobile energy stations in weak network conditions, characterized in that: include: The data acquisition module is used to collect raw logs from the mobile energy station in real time. The desensitization module is used to blur or replace sensitive fields in the original logs according to preset desensitization rules, and generate desensitized logs. The local temporary storage management module is used to temporarily store de-identified logs in non-volatile storage media to form a log queue; The network status monitoring module is used to monitor whether the current network status meets the preset upload conditions; The conditional upload module is used to retrieve de-identified logs from the log queue in sequence when the network quality meets the upload conditions, upload them to the cloud platform using block compression, and clear the locally uploaded log blocks after successful upload.

8. The edge-storage and de-identified upload system for mobile energy station logs in a weak network as described in claim 7, characterized in that: The system also includes a data receiving and restoration module deployed on a cloud platform. This module is used to receive uploaded data blocks, reassemble them according to global sequence numbers, and reversibly convert gridded coordinates or hash values ​​into original information based on the corresponding desensitization rule mapping table for authorized operation and maintenance users to query.

9. A device for edge-based temporary storage and desensitization of logs from a mobile power station in a weak network, characterized in that: include: A processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the method of any one of claims 1 to 6.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by the processor, it implements the method of any one of claims 1 to 6.

Images