Data transmission method, device, medium, equipment and product based on edge computing

Abstract

A data transmission method, apparatus, medium, device, and product based on edge computing, relating to the field of edge computing, includes the following steps: generating a first file based on target data collected from an edge device and storing the first file in a storage device in an edge data center; transmitting the target data from the first file to a central data center of a cloud platform, and generating a second file for recording file offsets based on the transmission status of the target data during the transmission process; deleting both the first and second files if the file offset in the second file includes the file offset of the last line in the first file. This improves data integrity and data transmission reliability. Furthermore, deleting both the first and second files when all the target data in the first file has been transmitted to the central data center improves storage space utilization and reduces storage resource waste.

Description

Technical Field

[0001] This disclosure relates to the field of edge computing technology, and more specifically, to a data transmission method, apparatus, medium, device, and product based on edge computing. Background Technology

[0002] With the explosive growth of IoT devices and the surge in demand for real-time data processing, edge computing, as an important extension of cloud computing, is becoming one of the core technologies for digital transformation. However, in related technologies, due to the instability of the network between the cloud platform and edge nodes, as well as the instability of the edge devices, data loss and transmission interruptions occur during data transmission between the cloud platform and edge nodes. Summary of the Invention

[0003] This content section is provided to briefly introduce the concepts, which will be described in detail in the subsequent detailed description section. This content section is not intended to identify key or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0004] In a first aspect, this disclosure provides a data transmission method based on edge computing, the data transmission method based on edge computing comprising:

[0005] A first file is generated based on the target data collected from the edge device, and the first file is stored in the storage device of the edge data center;

[0006] The target data in the first file is transmitted to the central computer room of the cloud platform. During the transmission of the target data, a second file is generated according to the transmission status of the target data. The second file is used to record the file offset. The file offset is used to characterize the offset of the storage position of the target file line in the first file relative to the starting storage position of the first file. The target file line is the file line in the first file where the data has been transmitted to the central computer room.

[0007] If the file offset in the second file includes the file offset of the last line in the first file, delete both the first file and the second file.

[0008] Secondly, this disclosure provides a data transmission device based on edge computing, the data transmission device based on edge computing comprising:

[0009] The first processing module is used to generate a first file based on the target data collected from the edge device, and store the first file in the storage device of the edge data center;

[0010] The second processing module is used to transmit the target data in the first file to the central computer room of the cloud platform, and generate a second file according to the transmission status of the target data during the transmission process. The second file is used to record the file offset, which is used to characterize the offset of the storage position of the target file line in the first file relative to the starting storage position of the first file. The target file line is the file line in the first file where the data has been transmitted to the central computer room.

[0011] The third processing module is used to delete the first file and the second file if the file offset in the second file includes the file offset of the last line in the first file.

[0012] Thirdly, this disclosure provides a computer-readable medium having a computer program stored thereon, which, when executed by a processing device, implements the steps of the method described in the first aspect.

[0013] Fourthly, this disclosure provides an electronic device, comprising:

[0014] A storage device on which computer programs are stored;

[0015] A processing device for executing the computer program in the storage device to implement the steps of the method in the first aspect.

[0016] Fifthly, this disclosure provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in the first aspect.

[0017] The above technical solution allows for the installation of a storage device in the edge data center. Upon receiving target data reported by the edge device, the target data can be stored in the storage device first, and then transmitted to the central data center. Since the target data is already stored in the edge data center's storage device, if data integrity is compromised during data transmission, the corresponding data can be retrieved from the storage device for transmission, effectively improving data integrity and reliability. Alternatively, if data is lost due to transmission link failure or network instability, the corresponding data can be retrieved from the storage device again after the transmission link or network is restored, further improving data integrity and transmission reliability. Furthermore, a second file recording file offsets is generated during data transmission. Both the first and second files can be deleted if the file offset includes the offset of the last line in the first file. This allows for timely deletion of both files after all target data in the first file has been transmitted to the central data center, improving storage space utilization and reducing storage resource waste.

[0018] Other features and advantages of this disclosure will be described in detail in the following detailed description section. Attached Figure Description

[0019] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale. In the drawings:

[0020] Figure 1 This is a flowchart illustrating a data transmission method based on edge computing according to an exemplary embodiment of the present disclosure;

[0021] Figure 2 This is a flowchart illustrating another edge computing-based data transmission method according to an exemplary embodiment of the present disclosure;

[0022] Figure 3 This is a timing diagram illustrating a data transmission method based on edge computing according to an exemplary embodiment of the present disclosure;

[0023] Figure 4 This is a structural block diagram of a data transmission device based on edge computing, according to an exemplary embodiment of the present disclosure;

[0024] Figure 5 This is a schematic diagram of the structure of an electronic device according to an exemplary embodiment of the present disclosure. Detailed Implementation

[0025] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0026] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0027] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0028] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0029] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0030] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0031] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0032] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this disclosed technical solution, based on the prompt message.

[0033] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0034] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0035] Meanwhile, it is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and related provisions.

[0036] It should be understood that with the rapid development of information technology, edge computing, as an emerging computing paradigm, is gradually being widely applied in many fields due to its significant advantages such as low latency and low bandwidth costs. For example, in the field of industrial automation, edge computing can process large amounts of data in the production process in real time, achieving precise control and improving production efficiency and product quality; in intelligent driving scenarios, it can quickly respond to vehicle sensor data, ensuring driving safety; in telemedicine scenarios, edge computing can enable the instant processing and transmission of patient data, improving the timeliness and accuracy of medical services; in the Internet of Things (IoT) environment, edge computing effectively reduces the data processing pressure on the cloud, making communication between devices more efficient. However, the large-scale application of edge computing also faces many technical challenges that urgently need to be addressed.

[0037] First, network connectivity reliability is a major challenge. Edge devices are often located in vast areas far from cloud servers, where the network environment is complex and variable, susceptible to interference from geographical and climatic factors. This leads to frequent network latency, packet loss, and even disconnections, significantly compromising data transmission stability and increasing the risk of data loss or transmission errors. Second, bandwidth resources are limited and prone to bottlenecks. The network bandwidth between edge devices and cloud servers is typically low. When facing massive data transmissions or high-frequency data interactions, limited bandwidth is insufficient, resulting in a sharp increase in transmission latency and low data transmission efficiency, severely impacting the real-time performance and effectiveness of edge computing systems. Third, the stability of the edge devices themselves cannot be ignored. These devices are exposed to complex environments for extended periods, making them susceptible to temperature, humidity, and power supply fluctuations, leading to equipment failures, data transmission interruptions, and compromised data integrity. Furthermore, cloud-edge data transmission involves various transmission protocols and communication standards. Differences in protocols between different devices, platforms, and applications can cause transmission interruptions, data parsing errors, or data loss, greatly hindering the smoothness of data transmission. Finally, the lack of fault recovery and fault tolerance mechanisms is a critical weakness in edge computing scenarios. In the face of unreliable network connections and volatile edge device conditions, the lack of effective fault recovery and fault tolerance mechanisms means that data loss or transmission interruption is inevitable once a failure occurs, which seriously affects the reliability and stability of the edge computing system.

[0038] In view of this, the present disclosure provides a data transmission method, apparatus, medium, device and product based on edge computing to solve the above-mentioned technical problems.

[0039] The embodiments of this disclosure will be further explained below with reference to the accompanying drawings.

[0040] Figure 1 This is a flowchart illustrating a data transmission method based on edge computing according to an exemplary embodiment of the present disclosure, with reference to... Figure 1 This edge computing-based data transmission method may include the following steps:

[0041] S101: Generate a first file based on the target data collected from the edge device, and store the first file in the storage device of the edge data center.

[0042] It should be understood that edge devices refer to devices deployed at the network edge in an edge computing architecture, close to the data source or user end, and possessing functions such as data collection and processing. Edge devices differ in different application scenarios, and the target data collected from them also varies. For example, in the content delivery field, an edge device could be a short video application, which can be configured with a tracking SDK (Software Development Kit). With user authorization, the tracking SDK can collect user behavior data and generate different files (the first file) based on the behavior data collected within different time periods. For instance, one file could be generated based on behavior data collected between 10:00 and 11:00, and another file could be generated based on behavior data collected between 11:00 and 12:00, and so on.

[0043] It should also be understood that the storage devices in the edge data center can be any device with storage capabilities, such as optical discs or solid-state drives, and the specific device can be determined according to the actual situation. This disclosure does not impose any restrictions on this. Given that disk data recovery technology is relatively mature and data storage is relatively stable, that is, data will not be lost in the event of power failure or long-term idleness, disks can be used as the storage devices in the edge data center whenever possible.

[0044] Furthermore, it should be understood that during storage device use, there may be device malfunctions or the storage space becoming full, preventing the first file from being stored on the storage device. Therefore, to reduce the risk of data loss, after generating the first file, it can be determined whether the storage device in the edge data center is functioning properly. If the storage device is functioning properly, the first file is stored on the storage device; if it is not functioning properly, the first file is directly transferred to the central data center of the cloud platform. In other words, among possible methods, edge computing-based data transmission methods may also include:

[0045] Determine the storage status of the storage device, which indicates whether the storage device is available;

[0046] Accordingly, the storage device for storing the first file in the edge data center may include:

[0047] If the storage status indicates that the storage device is available, store the first file to the storage device;

[0048] Accordingly, edge computing-based data transmission methods may also include:

[0049] If the storage status indicates that the storage device is unavailable, the first file will be transferred to the central data center of the cloud platform.

[0050] It should be understood that a cloud platform typically has multiple central data centers, each associated with a data center access domain name. Therefore, when transferring the first file to the cloud platform's central data center, the access domain name of one central data center can be randomly resolved to obtain the corresponding IP (Internet Protocol) address. Then, network connectivity is checked based on this IP address. If the network is connected, the first file can be forwarded to the cloud platform's central data center, for example, via a socket file. If the network is not connected, the access domain name of another central data center can be selected for resolution, and the above process is repeated until the first file is forwarded to the cloud platform's central data center.

[0051] S102: Transfer the target data from the first file to the central data room of the cloud platform. During the transfer of the target data, generate a second file based on the transfer status of the target data. The second file is used to record the file offset. The file offset is used to represent the offset of the storage position of the target file line in the first file relative to the starting storage position of the first file. The target file line is the file line in the first file whose data has been transferred to the central data room.

[0052] For example, if the first file contains ten lines of target data, during the transmission of the target data, the target data and file offset of the first line can be determined first, and then the target data of the first line can be transmitted to the central data center. After the target data of the first line is successfully transmitted to the central data center, the file offset of the first line is recorded in the second file. Next, the target data and file offset of the second line can be determined, and then the target data of the second line can be transmitted to the central data center. After the target data of the second line is successfully transmitted to the central data center, the file offset of the second line is recorded in the second file. The above process is repeated until the target data of the tenth line is successfully transmitted to the central data center, and the file offset of the tenth line is recorded in the second file.

[0053] Given the inefficiency of determining, transmitting, and recording file offsets line by line as described above, to improve data processing efficiency, a possible approach is to set up a disk queue in the edge data center to temporarily store the target data and file offsets for each file line. In this way, the target data and its corresponding file offset for each line of the first file can be written to the disk queue first, and then data can be read from the disk queue sequentially for processing. This allows the determination, transmission, and writing of the target data to occur simultaneously, reducing the frequent interactions and waiting time caused by line-by-line processing, thereby significantly improving data processing efficiency. In other words, in a possible approach, the target data from the first file can be transmitted to the central data center of the cloud platform, and a second file can be generated based on the transmission status of the target data during the transmission process. This could include:

[0054] For each line in the first file, determine the file offset and the target data corresponding to the line, and write the file offset and the target data as a write quantity to the first disk queue in the edge data center; create a blank second file; for each write quantity in the first disk queue, transfer the target data in the write quantity to the central data center of the cloud platform; after transferring the target data in the write quantity to the central data center, record the file offset in the write quantity in the second file, and delete the write quantity from the first disk queue.

[0055] S103: If the file offset in the second file includes the file offset of the last line in the first file, delete the first file and the second file.

[0056] It should be understood that if the file offset in the second file includes the file offset of the last line in the first file, it indicates that all target data in the first file has been successfully transmitted to the central computer room. Therefore, in order to reduce the invalid use of storage space, the first and second files can be deleted, thereby improving the space utilization of storage space.

[0057] The above technical solution allows for the installation of a storage device in the edge data center. Upon receiving target data reported by the edge device, the target data can be stored in the storage device first, and then transmitted to the central data center. Since the target data is already stored in the edge data center's storage device, if data integrity is compromised during data transmission, the corresponding data can be retrieved from the storage device for transmission, effectively improving data integrity and reliability. Alternatively, if data is lost due to transmission link failure or network instability, the corresponding data can be retrieved from the storage device again after the transmission link or network is restored, further improving data integrity and transmission reliability. Furthermore, a second file recording file offsets is generated during data transmission. Both the first and second files can be deleted if the file offset includes the offset of the last line in the first file. This allows for timely deletion of both files after all target data in the first file has been transmitted to the central data center, improving storage space utilization and reducing storage resource waste.

[0058] To facilitate understanding of the edge computing-based data transmission method provided in this disclosure, the following describes the possible implementations of this disclosure.

[0059] In some possible ways, the central data center may include a data proxy service, which includes multiple domain names associated with weight values. The weight values ​​characterize the probability that a domain name will be selected for data transmission. Accordingly, transmitting the target data from the first file to the central data center of the cloud platform may include:

[0060] Based on the weight value associated with each domain name, the target domain name for data transmission is determined from multiple domain names; the target data in the first file is transmitted to the data proxy service through the target domain name.

[0061] It should be understood that the weight value of domain association can be determined based on factors such as domain name resolution speed, packet loss rate, and / or network quality. Therefore, different domains can have the same or different weight values. When different domains have different weight values, the highest weight value can be determined from among multiple weight values, and the domain corresponding to the highest weight value can be used as the target domain. When different domains have the same weight value, one of the domains can be randomly selected as the target domain.

[0062] In one possible approach, determining the target domain for data transmission from among multiple domains based on the weight value associated with each domain can include:

[0063] Based on the weight value associated with each domain, select the first domain with the highest weight value from multiple domains, and repeat the following process:

[0064] The target data is transmitted to the data proxy service through the first domain name, and the transmission status of the target data is determined. The transmission status is used to indicate whether the target data has been transmitted to the data proxy service. If the transmission status indicates that the target data has not been transmitted to the data proxy service, the weight value of the first domain name is reduced, and according to the weight value associated with each domain name, the second domain name corresponding to the largest weight value is selected from multiple domain names, and the second domain name is used as the new first domain name, until the transmission status indicates that the target data has been transmitted to the data proxy service.

[0065] For example, a data proxy service may include a primary domain a, a primary domain b with dynamic acceleration capabilities, a backup domain a, and a backup domain b with dynamic acceleration capabilities. The primary domain a has a weight of 0.8, the primary domain b has a weight of 0.79, the backup domain a has a weight of 0.68, and the backup domain b has a weight of 0.76. Since the primary domain a has the highest weight, when determining the target domain, the primary domain a can be considered as the target domain, and the target data can be attempted to be transmitted to the data proxy service through the primary domain a. If the target data can be transmitted to the data proxy service through the primary domain a, then the primary domain a can be determined as the target domain. If the target data cannot be transmitted to the data proxy service through the primary domain a, the weight of the primary domain a can be reduced. For example, a weight adjustment value can be preset, and then the weight of the primary domain a can be adjusted based on the weight adjustment value. For example, the weight adjustment value can be set to 0.05, thereby adjusting the weight of the primary domain a to 0.75. After adjusting the weight value of the main domain, since the main domain b has the highest weight value, we can try to use the main domain b as the target domain and repeat the above process until the target data can be transmitted to the data proxy service.

[0066] In some possible ways, to reduce the probability of transmission failures via the first domain name due to occasional malfunctions or network fluctuations, if the transmission status indicates that the target data has not been successfully transmitted to the data proxy service, the transmission status of the target data can be re-attempted to transmit the target data to the data proxy service via the first domain name, and the transmission status of the target data can be re-determined. If the transmission still fails after multiple attempts, the weight value of the first domain name should be reduced.

[0067] By using the above method, the domain name used to transmit target data to the data proxy service can be selected based on the weight value associated with each domain name, and the domain name used to transmit target data to the data proxy service can be changed when the transmission fails, thereby further improving the reliability and stability of the overall data transmission.

[0068] Among the possible approaches, edge computing-based data transmission methods may also include:

[0069] After determining the target domain name for data transmission from multiple domain names, the corresponding data transmission method and data backup method are determined based on the target domain name. The data transmission method includes compressed transmission and uncompressed transmission, and the data backup method includes compressed backup and uncompressed backup.

[0070] Accordingly, transferring the target data from the first file to the data proxy service via the target domain name may include:

[0071] The target data in the first file is transmitted to the data proxy service according to the data transmission method via the target domain name;

[0072] Accordingly, edge computing-based data transmission methods may also include:

[0073] After all target data in the first file is transmitted to the data broker service, the first file is transmitted to the cloud platform through the data backup link between the edge data center and the cloud platform in accordance with the data backup method.

[0074] For example, a configuration file can be pre-set, which can be used to represent the correspondence between domain name, data transmission method and data backup method. Thus, after determining the target domain name, the corresponding data transmission method and data backup method can be determined based on the target domain name and the configuration file.

[0075] For example, the configuration file can be set as follows:

[0076] Main domain 'a': Data transmission method is uncompressed transmission, and data backup method is uncompressed backup;

[0077] Main domain name b: Data transmission method is compressed transmission, and the compression method is either method 1 or method 3; Data backup method is compressed backup, and the compression method is method 2;

[0078] Alternate domain name a: Data transmission method is compressed transmission, and the compression method is mode 1; Data backup method is uncompressed backup;

[0079] Alternate domain name b: Data transmission method is compressed transmission, and the compression method is method 4; data backup method is compressed backup, and the compression method is method 1.

[0080] If the target domain is the primary domain 'a', then the data transmission method and data backup method can be determined as uncompressed transmission and uncompressed backup, respectively. Therefore, when transmitting the target data from the first file to the data proxy service via the target domain, the target data in the first file can be transmitted to the data proxy service in an uncompressed manner via the target domain. Furthermore, after all the target data in the first file has been transmitted to the data proxy service, the first file can be transmitted to the cloud platform in an uncompressed backup manner via the data backup link between the edge data center and the cloud platform.

[0081] By employing the methods described above, data can be transmitted according to the data transmission method corresponding to the target domain name. This improves data transmission speed and quality, reduces latency and packet loss, and further enhances the reliability and stability of data transmission. Furthermore, after data transmission is complete, data backup can be performed according to the data backup method corresponding to the target domain name. This allows for rapid data recovery in the event of data loss or system failure, reducing the risks and losses caused by data issues and further improving the efficiency and accuracy of data recovery and fault tolerance in edge computing scenarios.

[0082] In some possible configurations, the central computer room may also include a second disk queue and data consumption devices; correspondingly, the data transmission method based on edge computing may also include:

[0083] When the transmission status indicates that the target data is being transmitted to the data broker service, the target data is written to the second disk queue through the data broker service; the target data in the second disk queue is consumed asynchronously by the data consumption device, and the target data is transmitted to the distributed message queue system.

[0084] For example, a second disk queue can be built using Network Attached Storage (NAS) as the foundation in the central computer room. After the data broker service receives the target data, it can first write the target data to the second disk queue, and then asynchronously read the target data from the second disk queue through the data consumption device and write it to a distributed message queue system, such as a Kafka cluster.

[0085] It should be understood that the central data center in related technologies generally does not include data proxy services, disk queues, and data consumption devices. Therefore, when edge data centers transmit data to the central data center, they typically transmit the data directly to the central data center's storage devices or write it directly to the central data center's database system. This approach may function normally when the data volume is small and network conditions are good, but when the data volume is large or network fluctuations occur, it can easily lead to data transmission failures or data loss. In this embodiment, by setting up data proxy services, disk queues, and data consumption devices in the central data center, the edge data center can first transmit the target data to the data proxy service when transmitting data to the central data center. After receiving the target data, the data proxy service writes the target data to the disk queue, and then the data consumption device asynchronously reads the target data from the disk queue and writes it to the distributed message queue system. Since disk queues can act as data buffers, they can ensure that data is not lost when the network fluctuates or the data consuming device malfunctions, and can continue processing after the network is restored or the data consuming device returns to normal. Distributed message queue systems can further improve the efficiency of data storage and management, support high-concurrency data read and write operations, and significantly improve the performance and reliability of the entire system, thereby achieving efficient and stable data transmission and storage, and further improving data integrity and the reliability of data transmission.

[0086] In one possible manner, the first file is stored in a preset directory on the storage device, and correspondingly, the target data in the first file is transferred to the central data center of the cloud platform, which may include:

[0087] Periodically retrieve files from a preset directory; determine the file's naming information, which indicates whether the file's naming rules are consistent with the preset naming rules; if the naming information indicates that the file's naming rules are consistent with the preset naming rules, designate the file as the first file and transfer the target data in the first file to the cloud platform; if the naming information indicates that the file's naming rules are inconsistent with the preset naming rules, return to the steps of periodically retrieving files from the preset directory and determining the file's naming information.

[0088] It should be understood that the preset naming rules can be determined according to the actual situation, and this disclosure does not impose any restrictions on them. For example, the preset naming rules can be: file content description + file type + collection time + storage directory, etc.

[0089] It should also be understood that due to user errors or automatic system backups, the files in the preset directory may include other files besides the primary file. Since different files generally have different naming rules, this embodiment, to reduce the probability of unnecessary data being transmitted to the central data center, pre-sets a file naming rule and determines whether a file is the primary file based on this rule. This achieves effective file filtering and accurate data transmission, improving data accuracy, reducing unnecessary data transmission and storage costs, and also reducing security risks caused by erroneous file transmission, further enhancing data transmission reliability.

[0090] In some possible embodiments, the cloud platform may also include a backup data center for backing up stored files on storage devices. Correspondingly, the data transmission method based on edge computing may also include:

[0091] Determine the network status between the edge data center and the central data center. The network status is used to indicate whether the network between the edge data center and the central data center is abnormal. If the network status indicates that the network between the edge data center and the central data center is abnormal, back up the first file in the storage device to obtain a backup file for the first file. Then, transfer the backup file to the backup central data center through the data transmission link between the edge data center and the backup central data center.

[0092] It should be understood that in edge computing, network anomalies, such as network interruptions, may occur between the edge data center and the central data center due to data center cutovers, shutdowns, physical machine failures, etc. This can prevent data from being transferred from the disk queue to the central data center. Therefore, to further improve data integrity and reliability and reduce the risk of data loss or corruption, a data transmission link between the edge data center and a backup central data center can be enabled to back up the target data in the first file to the backup data center. Furthermore, to further improve the system's fault tolerance and reliability and reduce the risk of business interruption due to network anomalies, after the central data center recovers, the backup data from the backup central data center can be completely migrated to the central data center through a data synchronization mechanism, thereby improving data consistency.

[0093] It should be understood that a backup data center can also include a data proxy service, and this service can include multiple domains associated with weighted values. Therefore, when transferring backup files to the backup data center via the data transmission link between the edge data center and the backup data center, the target domain can be determined from among multiple domains based on the weighted values. After determining the target domain, the data transmission link between the edge data center and the backup data center is established. Once the data transmission link is established, the network between the edge data center and the backup data center can be monitored in real time. If the network is unavailable, no action is taken, and the network is checked periodically until it is restored. If the network is available, the backup files are transferred to the backup data center via the data transmission link between the edge data center and the backup data center.

[0094] It should be understood that the central data center and the backup central data center are physically isolated and have disaster recovery capabilities. Therefore, if the target data cannot be transmitted to the central data center due to a failure of the central data center or network failure, the backup central data center can still operate normally and receive data independently, effectively avoiding system paralysis caused by a single point of failure and further enhancing the fault tolerance of the edge computing system.

[0095] In some possible embodiments, the storage device is a disk, and correspondingly, the data transfer method based on edge computing may also include:

[0096] Determine the network status between the edge data center and the central data center. The network status is used to indicate whether the network between the edge data center and the central data center is abnormal. If the network status indicates that the network between the edge data center and the central data center is abnormal, compress the stored files on the disk.

[0097] It should be understood that during operation, edge data centers continuously collect target data from edge devices. However, disk storage space is limited. If a network anomaly occurs between the edge data center and the central data center, the initial file stored on the disk cannot be transferred to the central data center for deletion, leading to reduced disk space and impacting subsequent file storage. Therefore, to mitigate the impact of insufficient disk space, files stored on the disk can be periodically compressed to save disk space, thereby increasing the storage capacity of the initial file and reducing the probability of data loss due to insufficient disk storage space, thus improving data integrity.

[0098] To further understand the edge computing-based data transmission method provided in the embodiments of this disclosure, the following is in conjunction with the appendix. Figure 2 and attached Figure 3 Further explanation of the plan:

[0099] First, the edge computing system in this embodiment will be described. The edge computing system in this embodiment may include an edge cluster and a central cluster. The central cluster may include a backup data center and multiple central data centers. Each backup data center and each central data center may be configured with a data proxy service, a NAS disk queue, and a data consumer. The data proxy service is used to transmit data back to the central data center interface, enabling elastic load scaling and cross-data center disaster recovery strategies. The NAS disk queue can be used to temporarily store data transmitted from the edge cluster, acting as a buffer. When the data transmission speed is fast or the central data center's data processing capacity is temporarily insufficient, the NAS disk queue can store this data to prevent data loss, waiting for subsequent data consumers to process it, thereby ensuring the continuity and stability of data transmission. The data consumer can be used to asynchronously consume data in the NAS disk queue and transmit it to a distributed message queue system, etc.

[0100] An edge cluster can include multiple edge data centers and edge devices communicating with each data center. These edge devices can be configured with a data tracking SDK for customized tracking protocols, writing data to specified areas and files. Each edge data center can include disks, file collectors, and disk queues. Disk queues can be used for local persistent storage of data in network outage or weak network scenarios. File collectors can be used for various status maintenance of local files, such as file discovery, file scanning, cursor collection (i.e., collecting file offsets), data recovery, data backup, and data cleanup.

[0101] Next, the data transmission process of this disclosure will be described, such as... Figure 2 and Figure 3 As shown:

[0102] Step 1: Collect data from edge devices using the point-tracking SDK and generate point-tracking files that need to be sent back to the central data center;

[0103] Step 2: Determine the disk status to see if the log file can be written to the disk. If the log file cannot be written to the disk, forward it directly to the data broker service via the sock file and proceed to Step 6; if the log file can be written to the disk, write it to the file in the specified time segment.

[0104] Step 3: The file collector periodically scans a specified directory area to obtain files. It then determines whether the obtained files are dotted files based on preset file naming rules. If a file is a dotted file, it reads the file offset and corresponding data from the dotted file and writes it to the disk queue. If the obtained file is not a dotted file, Step 3 is repeated.

[0105] Step 4: Determine the central computer room for receiving data and the target domain name for data transmission in the data proxy service, and determine the data transmission method and data backup method based on the target domain name;

[0106] Step 5: Read the data in the disk queue and determine if the data is valid. If the data is invalid, delete it from the disk queue and read the next data. If the data is valid, transmit it to the data proxy service via the target domain name and the corresponding data transmission method, and determine if the data transmission was successful. If the data transmission fails, periodic retries can be performed until the data transmission is successful or the maximum number of transmissions is reached. If the data transmission is successful, record the file offset in the second file and delete the data from the disk queue. In addition, it can be determined whether the file offset in the second file includes the file offset of the last file line in the log file. If the file offset in the second file includes the file offset of the last file line in the log file, the log file can be backed up to the backup central data center via the TOS (Type of Service) link. After the log file is successfully backed up to the backup central data center, the first and second files can be deleted.

[0107] Step 6: After receiving the data, the data proxy service writes the data to the NAS disk queue. If the write is successful, it can return a first message to the edge data center indicating that the data has been successfully transmitted to the central data center. If the write fails, it can return a second message to the edge data center indicating that the data has not been successfully transmitted to the central data center, so as to inform the edge data center to retry.

[0108] Step 7: The asynchronous data consumer in the central computer room consumes the disk queue and replays the disk queue into a distributed message queue system such as Kafka for use by the business.

[0109] In addition, the network status between the edge data center and the central data center can be monitored in real time. If the network status between the edge data center and each central data center is abnormal, the data logging file can be backed up. The data can be transmitted to the data proxy service of the backup central data center through the backup link between the edge data center and the backup central data center. The data proxy service transmits the backup file to the data consumer through the TOS (Type of Service) link.

[0110] Based on the same concept, embodiments of this disclosure also provide a data transmission device based on edge computing, such as... Figure 4 As shown, the edge computing-based data transmission device 400 may include:

[0111] The first processing module 401 is used to generate a first file based on the target data collected from the edge device and store the first file in the storage device of the edge data center;

[0112] The second processing module 402 is used to transmit the target data in the first file to the central computer room of the cloud platform, and generate a second file according to the transmission status of the target data during the transmission process. The second file is used to record the file offset. The file offset is used to represent the offset of the storage position of the target file line in the first file relative to the starting storage position of the first file. The target file line is the file line in the first file whose data has been transmitted to the central computer room.

[0113] The third processing module 403 is used to delete the first file and the second file if the file offset in the second file includes the file offset of the last file line in the first file.

[0114] The aforementioned edge computing-based data transmission device 400 allows for the installation of a storage device in the edge data center. Upon receiving target data reported by the edge device, the target data can be stored in the storage device first, and then transmitted to the central data center. Since the target data is already stored in the edge data center's storage device, if data integrity is compromised during data transmission, the corresponding data can be retrieved from the storage device for transmission, effectively improving data integrity and reliability. Alternatively, if data is lost due to transmission link failure or network instability, the corresponding data can be retrieved from the storage device again after the transmission link or network is restored, further improving data integrity and reliability. Furthermore, a second file recording file offsets is generated during data transmission. Both the first and second files can be deleted if the file offset includes the offset of the last line in the first file. This allows for timely deletion of both files after all target data in the first file has been transmitted to the central data center, improving storage space utilization and reducing storage resource waste.

[0115] In one possible manner, the second processing module 402 may include:

[0116] The first processing submodule is used to determine the file offset and the target data corresponding to each file line in the first file, and write the file offset and the file offset as a write amount to the first disk queue in the edge data center.

[0117] Create a submodule to create a blank second file;

[0118] The second processing submodule is used to transfer the target data of each write volume in the first disk queue to the central data room of the cloud platform. After transferring the target data of the write volume to the central data room, the file offset of the write volume is recorded in the second file, and the write volume in the first disk queue is deleted.

[0119] In one possible approach, the central computer room may include a data proxy service, which includes multiple domain names associated with weight values, whereby the weight values ​​characterize the probability that a domain name is selected for data transmission. Accordingly, the second processing module 402 may include:

[0120] The first determination submodule is used to determine the target domain name for data transmission from multiple domain names based on the weight value associated with each domain name;

[0121] The first transmission submodule is used to transmit the target data in the first file to the data proxy service via the target domain name.

[0122] In one possible approach, the first determining submodule can be used to: filter out the first domain corresponding to the highest weight value from multiple domains based on the weight value associated with each domain, and then iteratively execute the following process:

[0123] The target data is transmitted to the data proxy service through the first domain name, and the transmission status of the target data is determined. The transmission status is used to indicate whether the target data has been transmitted to the data proxy service. If the transmission status indicates that the target data has not been transmitted to the data proxy service, the weight value of the first domain name is reduced, and according to the weight value associated with each domain name, the second domain name corresponding to the largest weight value is selected from multiple domain names, and the second domain name is used as the new first domain name, until the transmission status indicates that the target data has been transmitted to the data proxy service.

[0124] In some possible embodiments, the second processing module 402 may also include:

[0125] The second determination submodule is used to determine the target domain name for data transmission from multiple domain names, and then determine the corresponding data transmission method and data backup method based on the target domain name. The data transmission method includes compressed transmission and uncompressed transmission, and the data backup method includes compressed backup and uncompressed backup.

[0126] Accordingly, the first transmission submodule can be used to: transmit the target data in the first file to the data proxy service according to the data transmission method via the target domain name;

[0127] Accordingly, the second processing module 402 may further include:

[0128] The second transmission submodule is used to transmit the first file to the cloud platform through the data backup link between the edge data center and the cloud platform after all the target data in the first file has been transmitted to the data proxy service, in accordance with the data backup method.

[0129] In some possible configurations, the central computer room may also include a second disk queue and data consumption devices; correspondingly, data transmission based on edge computing may also include:

[0130] The fourth processing module is used to write the target data to the second disk queue through the data proxy service when the transmission status indicates that the target data has been transmitted to the data proxy service.

[0131] The fifth processing module is used to asynchronously consume the target data in the second disk queue through the data consumption device and transmit the target data to the distributed message queue system.

[0132] In one possible manner, the first file is stored in a preset directory on the storage device, and correspondingly, the second processing module 402 may include:

[0133] The `get` submodule is used to periodically retrieve files from a preset directory.

[0134] The third determination submodule is used to determine the naming information of the file. The naming information is used to indicate whether the naming rules of the file are consistent with the preset naming rules.

[0135] The third processing submodule is used to identify the file as the first file when the naming rules of the file indicated by the naming information are consistent with the preset naming rules, and to transfer the target data in the first file to the cloud platform.

[0136] The fourth processing submodule is used to periodically obtain files in a preset directory through the acquisition submodule and determine the file naming information through the third determination submodule when the naming rules of the file indicated by the naming information are inconsistent with the preset naming rules.

[0137] In some possible configurations, the cloud platform may also include a backup data center for backing up stored files on storage devices. Correspondingly, data transmission based on edge computing may also include:

[0138] The first determination module is used to determine the network status between the edge data center and the central data center. The network status is used to indicate whether the network between the edge data center and the central data center is abnormal.

[0139] The backup module is used to back up the first file in the storage device when the network status indicates a network anomaly between the edge data center and the central data center, to obtain a backup file for the first file, and then transmit the backup file to the backup central data center through the data transmission link between the edge data center and the backup central data center.

[0140] In some possible embodiments, the storage device is a disk, and correspondingly, the data transfer method based on edge computing may also include:

[0141] The second determination module is used to determine the network status between the edge data center and the central data center. The network status is used to indicate whether the network between the edge data center and the central data center is abnormal.

[0142] The compression module is used to compress stored files on the disk when the network status indicates a network anomaly between the edge data center and the central data center.

[0143] Among the possible approaches, edge computing-based data transmission methods may also include:

[0144] The third determination module is used to determine the storage status of the storage device, which indicates whether the storage device is available.

[0145] Accordingly, the first processing module 401 can be used to: store the first file to the storage device when the storage status indicates that the storage device is available;

[0146] Accordingly, edge computing-based data transmission methods may also include:

[0147] The transmission module is used to transmit the first file to the central computer room of the cloud platform when the storage status indicates that the storage device is unavailable.

[0148] Based on the same concept, embodiments of this disclosure also provide a computer-readable medium having a computer program stored thereon, which, when executed by a processing device, implements the steps of any of the above-described edge computing-based data transmission methods.

[0149] Based on the same concept, this disclosure also provides an electronic device that may include:

[0150] A storage device on which computer programs are stored;

[0151] A processing device for executing a computer program stored in a storage device to implement the steps of any of the above-described edge computing-based data transmission methods.

[0152] Based on the same concept, embodiments of this disclosure also provide a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the above-described edge computing-based data transmission methods.

[0153] The following is for reference. Figure 5 This diagram illustrates a structural schematic of an electronic device 500 suitable for implementing embodiments of the present disclosure. The terminal devices in these embodiments may include, but are not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 5 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0154] like Figure 5 As shown, electronic device 500 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 501, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 502 or a program loaded from storage device 508 into random access memory (RAM) 503. RAM 503 also stores various programs and data required for the operation of electronic device 500. Processing unit 501, ROM 502, and RAM 503 are interconnected via bus 504. Input / output (I / O) interface 505 is also connected to bus 504.

[0155] Typically, the following devices can be connected to I / O interface 505: input devices 506 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 507 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 508 including, for example, magnetic tapes, hard disks, etc.; and communication devices 509. Communication device 509 allows electronic device 500 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 5 An electronic device 500 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0156] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 509, or installed from a storage device 508, or installed from a ROM 502. When the computer program is executed by the processing device 501, it performs the functions defined in the methods of embodiments of this disclosure.

[0157] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0158] In some implementations, communication can be conducted using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol), and can be interconnected with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and end-to-end networks (e.g., ad hoc end-to-end networks), as well as any currently known or future-developed networks.

[0159] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0160] The aforementioned computer-readable medium carries one or more programs. When the electronic device executes one or more of these programs, the electronic device causes the following actions: to generate a first file based on target data collected from an edge device, and to store the first file in a storage device in an edge data center; to transmit the target data from the first file to the central data center of the cloud platform, and during the transmission of the target data, to generate a second file based on the transmission status of the target data, wherein the second file records file offsets, the file offsets representing the offset of the target file line in the first file relative to the starting storage position of the first file, and the target file line being the file line in the first file whose data has been transmitted to the central data center; and, if the file offset in the second file includes the file offset of the last file line in the first file, to delete both the first and second files.

[0161] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0162] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0163] The modules described in the embodiments of this disclosure can be implemented in software or hardware. The names of the modules are not, in some cases, intended to limit the functionality of the module itself.

[0164] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0165] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0166] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0167] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0168] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative forms of implementing the claims. Regarding the apparatus in the above embodiments, the specific manner in which the various modules perform their operations has been described in detail in the embodiments relating to the method, and will not be elaborated upon here.

Claims

1. A data transmission method based on edge computing, characterized in that, The edge computing-based data transmission method includes: A first file is generated based on the target data collected from the edge device, and the first file is stored in the storage device of the edge data center; The target data in the first file is transmitted to the central computer room of the cloud platform. During the transmission of the target data, a second file is generated according to the transmission status of the target data. The second file is used to record the file offset. The file offset is used to characterize the offset of the storage position of the target file line in the first file relative to the starting storage position of the first file. The target file line is the file line in the first file where the data has been transmitted to the central computer room. If the file offset in the second file includes the file offset of the last line in the first file, delete both the first file and the second file. The step of transmitting the target data from the first file to the central data room of the cloud platform, and generating the second file based on the transmission status of the target data during the transmission process, includes: The file is traversed according to the storage location order of the file lines in the first file. For each file line traversed, the file offset corresponding to the file line and the target data corresponding to the file line are determined, and the file offset and the target data are written as a write amount to the first disk queue of the edge data center. Create a blank second file; For each write operation in the first disk queue, the target data in the write operation is transferred to the central data center of the cloud platform. After the target data in the write operation is transferred to the central data center, the file offset in the write operation is recorded in the second file, and the write operation in the first disk queue is deleted.

2. The data transmission method based on edge computing according to claim 1, characterized in that, The central data center includes a data proxy service, which includes multiple domain names associated with weight values. The weight values ​​represent the probability that a domain name will be selected for data transmission. The step of transmitting the target data from the first file to the central data center of the cloud platform includes: Based on the weight value associated with each of the domain names, a target domain name for data transmission is determined from the plurality of domain names; The target data in the first file is transmitted to the data proxy service via the target domain name.

3. The data transmission method based on edge computing according to claim 2, characterized in that, The step of determining the target domain name for data transmission from the plurality of domain names based on the weight value associated with each domain name includes: Based on the weight value associated with each domain name, the first domain name corresponding to the highest weight value is selected from the plurality of domain names, and the following process is executed cyclically: The target data is transmitted to the data proxy service through the first domain name, and the transmission status of the target data is determined. The transmission status is used to indicate whether the target data has been transmitted to the data proxy service. If the transmission status indicates that the target data has not been transmitted to the data proxy service, the weight value of the first domain name is reduced, and according to the weight value associated with each domain name, the second domain name corresponding to the largest weight value is selected from the multiple domain names, and the second domain name is used as the new first domain name, until the transmission status indicates that the target data has been transmitted to the data proxy service.

4. The data transmission method based on edge computing according to claim 2, characterized in that, The edge computing-based data transmission method further includes: After determining the target domain name for data transmission from the plurality of domain names, the corresponding data transmission method and data backup method are determined according to the target domain name. The data transmission method includes compressed transmission and uncompressed transmission, and the data backup method includes compressed backup and uncompressed backup. The step of transmitting the target data from the first file to the data proxy service via the target domain name includes: The target data in the first file is transmitted to the data proxy service via the target domain name according to the data transmission method. The edge computing-based data transmission method further includes: After all target data in the first file is transmitted to the data proxy service, the first file is transmitted to the cloud platform through the data backup link between the edge data center and the cloud platform in accordance with the data backup method.

5. The data transmission method based on edge computing according to claim 3, characterized in that, The central computer room also includes a second disk queue and data consumption devices, and the edge computing-based data transmission method further includes: If the transmission status indicates that the target data is to be transmitted to the data proxy service, the target data is written to the second disk queue through the data proxy service; The data consumption device asynchronously consumes the target data in the second disk queue and transmits the target data to the distributed message queue system.

6. The data transmission method based on edge computing according to claim 1, characterized in that, The first file is stored in a preset directory on the storage device, and the step of transferring the target data in the first file to the central computer room of the cloud platform includes: Periodically retrieve files from the preset directory; Determine the naming information of the file, wherein the naming information is used to indicate whether the naming rule of the file is consistent with the preset naming rule; If the naming information indicates that the naming rule of the file is consistent with the preset naming rule, the file is identified as the first file, and the target data in the first file is transmitted to the central data center of the cloud platform. If the naming information indicates that the naming rules of the file are inconsistent with the preset naming rules, return to the step of periodically obtaining the files in the preset directory and determining the naming information of the files.

7. The data transmission method based on edge computing according to claim 1, characterized in that, The cloud platform also includes a backup data center, which is used to back up the stored files in the storage device. The edge computing-based data transmission method further includes: Determine the network status between the edge data center and the central data center, the network status being used to indicate whether the network between the edge data center and the central data center is abnormal; If the network status indicates a network anomaly between the edge data center and the central data center, the first file in the storage device is backed up to obtain a backup file for the first file, and the backup file is transmitted to the backup data center through the data transmission link between the edge data center and the backup central data center.

8. The data transmission method based on edge computing according to claim 1, characterized in that, The storage device is a disk, and the edge computing-based data transmission method further includes: Determine the network status between the edge data center and the central data center, the network status being used to indicate whether the network between the edge data center and the central data center is abnormal; If the network status indicates a network anomaly between the edge data center and the central data center, the stored files on the disk are compressed.

9. The data transmission method based on edge computing according to claim 1, characterized in that, The edge computing-based data transmission method further includes: Determine the storage status of the storage device, the storage status being used to indicate whether the storage device is available; The storage device for storing the first file in the edge data center includes: If the storage status indicates that the storage device is available, the first file is stored in the storage device; The edge computing-based data transmission method further includes: If the storage status indicates that the storage device is unavailable, the first file is transferred to the central computer room of the cloud platform.

10. A data transmission device based on edge computing, characterized in that, The edge computing-based data transmission device includes: The first processing module is used to generate a first file based on the target data collected from the edge device, and store the first file in the storage device of the edge data center; The second processing module is used to transmit the target data in the first file to the central computer room of the cloud platform, and generate a second file according to the transmission status of the target data during the transmission process. The second file is used to record the file offset, which is used to characterize the offset of the storage position of the target file line in the first file relative to the starting storage position of the first file. The target file line is the file line in the first file where the data has been transmitted to the central computer room. The third processing module is used to delete the first file and the second file if the file offset in the second file includes the file offset of the last file line in the first file. The second processing module includes: The first processing submodule is used to traverse the file lines in the first file according to their storage location order. For each traversed file line, the module determines the file offset and the target data corresponding to the file line, and writes the file offset and the target data as a write amount to the first disk queue of the edge data center. Create a submodule to create a blank second file; The second processing submodule is used to transfer the target data of each write volume in the first disk queue to the central data room of the cloud platform. After transferring the target data of the write volume to the central data room, the file offset of the write volume is recorded in the second file, and the write volume in the first disk queue is deleted.

11. A computer-readable medium having a computer program stored thereon, characterized in that, When executed by a processing device, the computer program performs the steps of the method according to any one of claims 1-9.

12. An electronic device, characterized in that, include: A storage device on which computer programs are stored; A processing device for executing the computer program in the storage device to implement the steps of the method according to any one of claims 1-9.

13. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1-9.

Images