Data transmission method and device based on edge calculation, medium, equipment and product

Abstract

The invention discloses a data transmission method and device based on edge computing, a medium, equipment and a product, and relates to the field of edge computing, and the method comprises the steps: generating a first file according to target data collected from edge equipment, and storing the first file in storage equipment of an edge machine room; transmitting the target data in the first file to a central machine room of the cloud platform, and generating a second file for recording the file offset according to the transmission condition of the target data in the transmission process of the target data; and under the condition that the file offset in the second file comprises the file offset of the last file line in the first file, deleting the first file and the second file. Therefore, the data integrity and the data transmission reliability can be improved. Besides, under the condition that all the target data in the first file are transmitted to the central machine room, the first file and the second file can be deleted, so that the space utilization rate of the storage space can be improved, and the waste of storage resources is reduced.

Description

Technical Field

[0001] The present disclosure relates to the field of edge computing technology, and in particular, to a data transmission method, apparatus, medium, device, and product based on edge computing. Background Art

[0002] With the explosive growth of Internet of Things devices and the surging demand for real-time data processing, edge computing, as an important extension of cloud computing, is becoming one of the core technologies for digital transformation. However, in related technologies, due to the instability of the network between the cloud platform and edge nodes and the status of edge devices, problems such as data loss and transmission interruption exist during the data transmission process between the cloud platform and edge nodes. Summary of the Invention

[0003] This content part is provided to briefly introduce concepts, which will be described in detail in the subsequent detailed implementation part. This content part is not intended to identify the key features or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0004] In a first aspect, the present disclosure provides a data transmission method based on edge computing, and the data transmission method based on edge computing includes: Generating a first file according to target data collected from an edge device, and storing the first file in a storage device of an edge computer room; Transmitting the target data in the first file to a central computer room of a cloud platform, and generating a second file according to the transmission situation of the target data during the transmission of the target data, where the second file is used to record a file offset, and the file offset is used to represent the offset of the storage position of a target file line in the first file relative to the starting storage position of the first file, and the target file line is the file line in the first file whose data has been transmitted to the central computer room; Deleting the first file and the second file when the file offset in the second file includes the file offset of the last file line in the first file.

[0005] In a second aspect, the present disclosure provides a data transmission apparatus based on edge computing, and the data transmission apparatus based on edge computing includes: A first processing module, configured to generate a first file according to target data collected from an edge device, and store the first file in a storage device of an edge computer room; A second processing module, configured to transmit the target data in the first file to the central computer room of the cloud platform, and generate a second file according to the transmission condition of the target data during the transmission of the target data, where the second file is used to record a file offset, and the file offset is used to characterize the offset of the storage position of the target file line in the first file relative to the starting storage position of the first file in the first file, and the target file line is the file line in the first file whose data has been transmitted to the central computer room; A third processing module, configured to delete the first file and the second file when the file offset in the second file includes the file offset of the last file line in the first file.

[0006] In a third aspect, the present disclosure provides a computer-readable medium, on which a computer program is stored, and when the computer program is executed by a processing device, the steps of the method described in the first aspect are implemented.

[0007] In a fourth aspect, the present disclosure provides an electronic device, including: A storage device, on which a computer program is stored; A processing device, configured to execute the computer program in the storage device to implement the steps of the method described in the first aspect.

[0008] In a fifth aspect, the present disclosure provides a computer program product, including a computer program, and when the computer program is executed by a processor, the steps of the method described in the first aspect are implemented.

[0009] Through the above technical solutions, a storage device can be set in the edge computer room. Thus, after the edge computer room receives the target data reported by the edge device, the target data can be first stored in the storage device, and then the target data in the storage device can be transmitted to the central computer room. Since the target data has been stored in the storage device in the edge computer room, if problems such as damaged data integrity occur during the data transmission process, the corresponding data can be obtained again from the storage device for data transmission, thereby effectively improving the integrity and reliability of the data transmission. Or, if data is lost during the data transmission process due to a transmission link failure or network instability, etc., after the transmission link or network is restored, the corresponding data can be obtained again from the storage device for data transmission, thereby improving the data integrity and the reliability of the data transmission. In addition, since a second file for recording the file offset is also generated during the data transmission process, and the first file and the second file can be deleted when the file offset includes the file offset of the last file line in the first file, the first file and the second file can be deleted in a timely manner when all the target data in the first file has been transmitted to the central computer room, thereby improving the space utilization rate of the storage space and reducing the waste of storage resources.

[0010] Other features and advantages of the present disclosure will be described in detail in the following detailed description section. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In combination with the accompanying drawings and with reference to the following detailed description, the above and other features, advantages, and aspects of the embodiments of the present disclosure will become more apparent. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic and that the original elements and elements are not necessarily drawn to scale. In the drawings: Figure 1 is a flowchart of a data transmission method based on edge computing shown according to an exemplary embodiment of the present disclosure; Figure 2 is a flowchart of another data transmission method based on edge computing shown according to an exemplary embodiment of the present disclosure; Figure 3 is a timing diagram of a data transmission method based on edge computing shown according to an exemplary embodiment of the present disclosure; Figure 4 is a structural block diagram of a data transmission device based on edge computing shown according to an exemplary embodiment of the present disclosure; Figure 5 is a schematic structural diagram of an electronic device shown according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0012] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although some embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Instead, these embodiments are provided to more thoroughly and completely understand the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not used to limit the scope of protection of the present disclosure.

[0013] It should be understood that the various steps recited in the method embodiments of the present disclosure can be executed in a different order and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this regard.

[0014] As used herein, the term "including" and its variations are open-ended, i.e., "including but not limited to". The term "based on" is "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0015] It should be noted that the concepts such as "first" and "second" mentioned in this disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0016] It should be noted that the modification of "one" and "multiple" mentioned in this disclosure is illustrative rather than restrictive. Those skilled in the art should understand that unless otherwise clearly specified in the context, it should be understood as "one or more".

[0017] The names of the messages or information exchanged between multiple devices in the embodiments of this disclosure are only for illustrative purposes and are not used to limit the scope of these messages or information.

[0018] It can be understood that before using the technical solutions disclosed in the embodiments of this disclosure, the types, usage scopes, usage scenarios, etc. of the personal information involved in this disclosure should be informed to users and the authorization of users should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0019] For example, when responding to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested by it will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, an application program, a server or a storage medium that executes the operations of the technical solutions of this disclosure according to the prompt message.

[0020] As an optional but non-limiting implementation manner, the way of sending a prompt message to the user in response to receiving an active request from the user can be, for example, in the form of a pop-up window, and the prompt message can be presented in text in the pop-up window. In addition, the pop-up window can also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0021] It can be understood that the above process of notifying and obtaining user authorization is only illustrative and does not constitute a limitation on the implementation manner of this disclosure. Other ways that meet relevant laws and regulations can also be applied to the implementation manner of this disclosure.

[0022] At the same time, it can be understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) should comply with the requirements of corresponding laws, regulations and related provisions.

[0023] It should be understood that with the rapid development of information technology, edge computing, as an emerging computing paradigm, is gradually being widely applied in many fields by virtue of its significant advantages such as low latency and low bandwidth cost. For example, in the field of industrial automation, edge computing can process a large amount of data in the production process in real time, achieve precise control, and improve production efficiency and product quality; in the scenario of intelligent driving, it can quickly respond to vehicle sensor data to ensure driving safety; in the scenario of remote medical treatment, with the help of edge computing, instant processing and transmission of patient data can be realized, improving the timeliness and accuracy of medical services; in the Internet of Things environment, edge computing effectively reduces the data processing pressure on the cloud and makes device - to - device communication more efficient. However, the large - scale application of edge computing also faces many technical pain points that need to be solved urgently.

[0024] First of all, the reliability of network connection is a major challenge. Since edge devices are often distributed in a vast area far from the cloud server, the network environment is complex and changeable, and is easily interfered by factors such as geography and climate, resulting in frequent phenomena such as network latency, packet loss, and even disconnection. The stability of data transmission is greatly reduced, and the risk of data loss or transmission error increases significantly. Secondly, the bandwidth resources are limited and prone to bottlenecks. The network bandwidth between edge devices and the cloud server is usually low. When facing massive data transmission or high - frequency data interaction, the limited bandwidth is difficult to meet the requirements, the transmission latency rises sharply, and the data transmission efficiency is low, seriously affecting the real - time performance and effectiveness of the edge - computing system. Moreover, the state stability of edge devices themselves cannot be ignored. These devices are long - term exposed to complex environments and are easily affected by conditions such as temperature, humidity, and power supply, resulting in device failures, and further causing problems such as data transmission interruption and data integrity damage. In addition, cloud - edge data transmission involves a variety of different transmission protocols and communication standards. The differences in protocols between different devices, platforms, and application programs may lead to transmission interruption, data parsing errors, or data loss, posing a great obstacle to the smoothness of data transmission. Finally, the lack of fault - recovery and fault - tolerance mechanisms is a key shortcoming in the edge - computing scenario. In an unreliable network connection and a changeable state of edge devices, without an effective fault - recovery and fault - tolerance mechanism, once a fault occurs, data loss or transmission interruption is inevitable, seriously affecting the reliability and stability of the edge - computing system.

[0025] In view of this, the present disclosure provides a data - transmission method, device, medium, equipment, and product based on edge computing to solve the above - mentioned technical problems.

[0026] The following further explains the embodiments of the present disclosure with reference to the accompanying drawings.

[0027] Figure 1 is a flowchart of a data - transmission method based on edge computing shown according to an exemplary embodiment of the present disclosure. Refer to Figure 1, the edge-computing-based data transmission method may include the following steps: S101: Generate a first file according to the target data collected from the edge device, and store the first file in the storage device of the edge computer room.

[0028] It should be understood that an edge device refers to a device deployed at the network edge in the edge computing architecture, which is close to the data source or the client side and has functions such as data collection and data processing. In different application scenarios, the edge devices are different, and the target data collected from the edge devices is also different. By way of example, in the field of content delivery, the edge device may be a short video application program, and a tagging SDK (Software Development Kit) may be configured in the short video application program. Thus, with the user's authorization, the behavior data of the user can be collected through the tagging SDK, and different files, that is, the first files, can be generated based on the behavior data collected in different time periods. For example, a file can be generated based on the behavior data collected from 10:00 to 11:00, and another file can be generated based on the behavior data collected from 11:00 to 12:00, etc.

[0029] It should also be understood that the storage device in the edge computer room may be any device with a storage function, such as an optical disc or a solid-state drive, etc., which can be specifically determined according to the actual situation, and the embodiments of the present disclosure do not impose any restrictions thereon. Given that the data recovery technology of disks is relatively mature and the data storage is relatively stable, that is, the data will not be lost in the case of power failure or long-term idleness, thus in a possible manner, the storage device in the edge computer room can be set as a disk.

[0030] In addition, it should also be understood that during the use of the storage device, there may be situations such as device failure or the storage space being full, resulting in the first file being unable to be stored in the storage device. Thus, in order to reduce the risk of data loss, after generating the first file, it can be first determined whether the storage device in the edge computer room can be used normally. If the storage device can be used normally, the first file is stored in the storage device; if it cannot be used normally, the first file is directly transmitted to the central computer room of the cloud platform. That is to say, in a possible manner, the edge-computing-based data transmission method may further include: Determine the storage state of the storage device, where the storage state is used to indicate whether the storage device is available; Correspondingly, storing the first file in the storage device of the edge computer room may include: When the storage state indicates that the storage device is available, store the first file in the storage device; Correspondingly, the edge-computing-based data transmission method may further include: When the storage status indicates that the storage device is unavailable, transfer the first file to the central computer room of the cloud platform.

[0031] It should be understood that the central computer room of the cloud platform generally includes multiple ones, and each central computer room is associated with a computer room access domain name. Therefore, when transferring the first file to the central computer room of the cloud platform, the access domain name of a central computer room can be randomly resolved first to obtain the IP (Internet Protocol) address corresponding to the central computer room, and then the network connectivity can be checked based on this IP address. If the network is connected, the first file can be forwarded to the central computer room of the cloud platform. For example, the first file can be forwarded to the central computer room of the cloud platform through a sock (socket) file; if the network is not connected, the access domain name of another central computer room can be selected for resolution, and the above process can be repeated until the first file is forwarded to the central computer room of the cloud platform.

[0032] S102: Transfer the target data in the first file to the central computer room of the cloud platform, and during the transfer process of the target data, generate a second file according to the transfer situation of the target data. The second file is used to record the file offset, and the file offset is used to represent the offset of the storage position of the target file line in the first file relative to the starting storage position of the first file. The target file line is the file line in the first file whose data has been transferred to the central computer room.

[0033] Exemplarily, if the first file includes ten lines of target data, during the transfer process of the target data, the target data and file offset of the first line can be determined first, and then the target data of the first line can be transferred to the central computer room. After the target data of the first line is successfully transferred to the central computer room, record the file offset of the first line in the second file. Then, the target data and file offset of the second line can be determined, and then the target data of the second line can be transferred to the central computer room. After the target data of the second line is successfully transferred to the central computer room, record the file offset of the second line in the second file. Repeat the above process until the target data of the tenth line is successfully transferred to the central computer room and record the file offset of the tenth line in the second file.

[0034] In view of the inefficiency of the above-mentioned method of determining, transmitting, and recording file offsets line by line, in order to improve data processing efficiency, among possible methods, a disk queue for temporarily storing the target data and file offsets of each file line can be set up in the edge computer room. In this way, the target data of each line in the first file and its corresponding file offset can be written into the disk queue first, and then the data can be read from the disk queue in sequence for processing. In this way, the determination of the target data, the transmission, and the writing of the file offset can be carried out synchronously, reducing the frequent interactions and waiting times caused by line-by-line processing, thereby significantly improving data processing efficiency. That is to say, among possible methods, transmitting the target data in the first file to the central computer room of the cloud platform and generating a second file according to the transmission situation of the target data during the transmission of the target data may include: For each file line in the first file, determine the file offset corresponding to the file line and the target data corresponding to the file line, and write the file offset and the file offset as a write amount into the first disk queue in the edge computer room; create a blank second file; for each write amount in the first disk queue, transmit the target data in the write amount to the central computer room of the cloud platform. After transmitting the target data in the write amount to the central computer room, record the file offset in the write amount in the second file, and delete the write amount in the first disk queue.

[0035] S103: When the file offset in the second file includes the file offset of the last file line in the first file, delete the first file and the second file.

[0036] It should be understood that when the file offset in the second file includes the file offset of the last file line in the first file, it indicates that all the target data in the first file has been successfully transmitted to the central computer room. Therefore, in order to reduce the ineffective occupation of storage space, the first file and the second file can be deleted, thereby improving the space utilization rate of the storage space.

[0037] Through the above technical solution, a storage device can be set up in the edge computer room. After the edge computer room receives the target data reported by the edge device, the target data can be first stored in the storage device and then the target data in the storage device can be transmitted to the central computer room. Since the target data has been stored in the storage device in the edge computer room, if problems such as damaged data integrity occur during the data transmission process, the corresponding data can be retrieved from the storage device again for data transmission, thereby effectively improving the integrity and reliability of data transmission. Or, if data is lost during the data transmission process due to a transmission link failure or network instability, etc., after the transmission link or network is restored, the corresponding data can be retrieved from the storage device again for data transmission, thereby improving the integrity of the data and the reliability of data transmission. In addition, since a second file for recording the file offset is generated during the data transmission process and the first file and the second file can be deleted when the file offset includes the file offset of the last file line in the first file, the first file and the second file can be deleted in a timely manner when all the target data in the first file is transmitted to the central computer room, thereby improving the space utilization rate of the storage space and reducing the waste of storage resources.

[0038] To facilitate understanding of the edge computing-based data transmission method provided by the present disclosure, the possible implementation manners in the present disclosure are described below.

[0039] In a possible manner, the central computer room may include a data proxy service, and the data proxy service includes multiple domain names associated with weight values. The weight value is used to represent the probability that the domain name is selected for data transmission. Correspondingly, transmitting the target data in the first file to the central computer room of the cloud platform may include: Determine a target domain name for data transmission among multiple domain names according to the weight value associated with each domain name; transmit the target data in the first file to the data proxy service through the target domain name.

[0040] It should be understood that the weight value associated with the domain name can be determined based on the domain name resolution speed, packet loss rate, and / or network quality, etc. Therefore, the weight values associated with different domain names can be the same or different. When the weight values associated with different domain names are different, the maximum weight value can be determined among multiple weight values, and then the domain name corresponding to the maximum weight value can be used as the target domain name. When the weight values associated with different domain names are the same, one of the domain names can be randomly selected as the target domain name.

[0041] In a possible manner, determining a target domain name for data transmission among multiple domain names according to the weight value associated with each domain name may include: According to the weight value associated with each domain name, screen out the first domain name corresponding to the maximum weight value among multiple domain names, and loop to execute the following process: Transmit the target data to the data proxy service through the first domain name, and determine the transmission status of the target data, where the transmission status is used to indicate whether the target data is transmitted to the data proxy service; in the case where the transmission status indicates that the target data is not transmitted to the data proxy service, reduce the weight value of the first domain name, and based on the weight value associated with each domain name, screen out the second domain name corresponding to the maximum weight value among multiple domain names, and use the second domain name as the new first domain name until the transmission status indicates that the target data is transmitted to the data proxy service.

[0042] Exemplarily, the data proxy service may include the primary domain name a, the primary domain name b with dynamic acceleration ability, the standby domain name a, and the standby domain name b with dynamic acceleration ability, and the weight value of the primary domain name a is 0.8, the weight value of the primary domain name b is 0.79, the weight value of the standby domain name a is 0.68, and the weight value of the standby domain name b is 0.76. Since the weight value of the primary domain name a is the largest, when determining the target domain name, the primary domain name a can be tried as the target domain name, and the target data can be tried to be transmitted to the data proxy service through the primary domain name a. If the target data can be transmitted to the data proxy service through the primary domain name a, the primary domain name a can be determined as the target domain name. If the target data cannot be transmitted to the data proxy service through the primary domain name a, the weight value of the primary domain name a can be reduced. For example, a weight adjustment value can be preset in advance, and then the weight value of the primary domain name a can be adjusted based on the weight adjustment value. For instance, the weight adjustment value can be set to 0.05, and thus, the weight value of the primary domain name can be adjusted to 0.75. After adjusting the weight value of the primary domain name, since the weight value of the primary domain name b is the largest, the primary domain name b can be tried as the target domain name, and the above process can be repeated until the target data can be transmitted to the data proxy service.

[0043] In a possible way, in order to reduce the probability of transmission failure through the first domain name due to factors such as occasional failures or network fluctuations, in the case where the transmission status indicates that the target data is not successfully transmitted to the data proxy service, the target data can be tried to be transmitted to the data proxy service through the first domain name again, and the transmission status of the target data can be re-determined. If the transmission still fails after multiple attempts, the weight value of the first domain name is reduced.

[0044] Through the above method, the domain name used to transmit the target data to the data proxy service can be selected according to the weight value associated with each domain name, and the domain name used to transmit the target data to the data proxy service can be replaced when the transmission fails, thereby further improving the reliability and stability of the overall data transmission.

[0045] In a possible way, the data transmission method based on edge computing may further include: After determining the target domain name for data transmission among multiple domain names, determine the corresponding data transmission method and data backup method according to the target domain name. The data transmission method includes compressed transmission and non-compressed transmission, and the data backup method includes compressed backup and non-compressed backup; Correspondingly, transmitting the target data in the first file to the data proxy service through the target domain name may include: Transmitting the target data in the first file to the data proxy service through the target domain name according to the data transmission method; Correspondingly, the data transmission method based on edge computing may further include: After all the target data in the first file are transmitted to the data proxy service, transmit the first file to the cloud platform through the data backup link between the edge computer room and the cloud platform according to the data backup method.

[0046] Exemplarily, a configuration file may be preset in advance. The configuration file may be used to represent the corresponding relationship among the domain name, the data transmission method, and the data backup method. Thus, after determining the target domain name, the corresponding data transmission method and data backup method can be determined based on the target domain name and the configuration file.

[0047] For example, the configuration file may be set as follows: Main domain name a: The data transmission method is non-compressed transmission, and the data backup method is non-compressed backup; Main domain name b: The data transmission method is compressed transmission, and the compression method is method 1 or method 3; the data backup method is compressed backup, and the compression method is method 2; Standby domain name a: The data transmission method is compressed transmission, and the compression method is method 1; the data backup method is non-compressed backup; Standby domain name b: The data transmission method is compressed transmission, and the compression method is method 4; the data backup method is compressed backup, and the compression method is method 1.

[0048] If the target domain name is the main domain name a, the data transmission method can be determined as non-compressed transmission, and the data backup method can be determined as non-compressed backup. Thus, when transmitting the target data in the first file to the data proxy service through the target domain name, the target data in the first file can be transmitted to the data proxy service through the target domain name in the non-compressed transmission manner, and after all the target data in the first file are transmitted to the data proxy service, the first file can be transmitted to the cloud platform through the data backup link between the edge computer room and the cloud platform in the non-compressed backup manner.

[0049] In the above manner, data transmission can be carried out according to the data transmission method corresponding to the target domain name, thereby improving the data transmission speed and quality, reducing problems such as latency and packet loss, and further enhancing the reliability and stability of data transmission. Additionally, after the data transmission is completed, data backup can be performed according to the data backup method corresponding to the target domain name. Thus, in the event of unexpected situations such as data loss or system failures, data can be quickly restored through the data backup, reducing the risks and losses brought by data problems, and further improving the efficiency and accuracy of data recovery and fault tolerance in the edge computing scenario.

[0050] Among possible ways, the central computer room may further include a second disk queue and a data consumption device. Correspondingly, the data transmission method based on edge computing may further include: In the case where the transmission status indicates that the target data is transmitted to the data proxy service, the target data is written into the second disk queue through the data proxy service; the data consumption device asynchronously consumes the target data in the second disk queue and transmits the target data to the distributed message queue system.

[0051] Exemplarily, Network Attached Storage (NAS) can be used as a basis in the central computer room to construct the second disk queue. After the data proxy service receives the target data, the target data can be first written into the second disk queue, and then the data consumption device asynchronously reads the target data from the second disk queue and writes it into the distributed message queue system, for example, writing it into a Kafka cluster.

[0052] It should be understood that the central computer room in the related art generally does not include a data proxy service, a disk queue, and a data consumption device. Therefore, when the edge computer room transmits data to the central computer room, it generally directly transmits the data to the storage device of the central computer room or directly writes it into the database system of the central computer room. This method may still operate normally when the data volume is small and the network condition is good, but when the data volume is large or the network fluctuates, it is very easy to cause data transmission failure or data loss. In the embodiments of the present application, by setting up a data proxy service, a disk queue, and a data consumption device in the central computer room, when the edge computer room transmits data to the central computer room, it can first transmit the target data to the data proxy service. After receiving the target data, the data proxy service then writes the target data into the disk queue, and then the data consumption device asynchronously reads the target data from the disk queue and writes it into the distributed message queue system. Since the disk queue can be used as a data buffer, when the network fluctuates or the data consumption device fails, it can ensure that the data will not be lost and wait until the network recovers or the data consumption device returns to normal before continuing to process; the distributed message queue system can further improve the storage and management efficiency of data, support high-concurrency data read and write operations, significantly improve the performance and reliability of the entire system, and thus achieve efficient and stable transmission and storage of data, and further improve the integrity of data and the reliability of data transmission.

[0053] In a possible way, the first file is stored in a preset directory of the storage device. Correspondingly, transmitting the target data in the first file to the central computer room of the cloud platform may include: Periodically obtain the files in the preset directory; determine the naming information of the files, where the naming information is used to indicate whether the naming rule of the file is consistent with the preset naming rule; when the naming information indicates that the naming rule of the file is consistent with the preset naming rule, determine the file as the first file and transmit the target data in the first file to the cloud platform; when the naming information indicates that the naming rule of the file is inconsistent with the preset naming rule, return to execute the steps of periodically obtaining the files in the preset directory and determining the naming information of the files.

[0054] It should be understood that the preset naming rule can be determined according to the actual situation, and the embodiments of the present disclosure do not impose any restrictions on this. Exemplarily, the preset naming rule can be: file content description + file type + collection time + storage directory, etc.

[0055] It should be further understood that due to reasons such as user's incorrect operation or system automatic backup, the files in the preset directory may include other files in addition to the first file. Since the naming rules of different files are generally different, in order to reduce the probability of transmitting unnecessary data to the central computer room, in this embodiment, a file naming rule is preset, and whether a file is the first file is determined according to the file naming rule and the naming rule of the file, so as to realize effective screening of files and accurate transmission of data. On the one hand, the accuracy of data is improved, and unnecessary data transmission and storage costs are reduced. On the other hand, the security risk caused by incorrect file transmission can be reduced, and the reliability of data transmission is further improved.

[0056] In a possible manner, the cloud platform may further include a standby central computer room, and the standby central computer room is used to back up the stored files in the storage device. Correspondingly, the data transmission method based on edge computing may further include: Determine the network status between the edge computer room and the central computer room, where the network status is used to indicate whether the network between the edge computer room and the central computer room is abnormal; in the case where the network status indicates that the network between the edge computer room and the central computer room is abnormal, back up the first file in the storage device to obtain a backup file for the first file, and transmit the backup file to the standby central computer room through the data transmission link between the edge computer room and the standby central computer room.

[0057] It should be understood that in edge computing, due to reasons such as computer room cutover, withdrawal, and physical machine failure, the network between the edge computer room and the central computer room may be abnormal, such as network interruption. As a result, it is impossible to transmit the data in the disk queue to the central computer room. Therefore, in order to further improve the integrity and reliability of data and reduce the risk of data loss or damage, the data transmission link between the edge computer room and the standby central computer room can be enabled to back up the target data in the first file to the standby computer room. In addition, in order to further improve the fault tolerance and reliability of the system and reduce the risk of service interruption caused by network abnormalities, after the central computer room is restored, the backup data in the standby central computer room can also be completely migrated to the central computer room through the data synchronization mechanism, so as to improve data consistency.

[0058] It should be understood that the standby central computer room may also include a data proxy service, and the data proxy service may also include multiple domain names associated with weight values. Thus, when transmitting the backup file to the standby central computer room through the data transmission link between the edge computer room and the standby central computer room, the target domain name can also be determined among the multiple domain names based on the magnitude of the weight value. After determining the target domain name, the data transmission link between the edge computer room and the standby central computer room is determined based on the target domain name. After obtaining the data transmission link between the edge computer room and the standby central computer room, the network between the edge computer room and the standby central computer room can be detected in real time. If the network is unavailable, no operation is performed, and the network is periodically checked until the network is restored. If the network is available, the backup file is transmitted to the standby central computer room through the data transmission link between the edge computer room and the standby central computer room.

[0059] It should be understood that the central computer room and the standby central computer room are isolated and disaster-tolerant at the physical level. Thus, in the case where the target data cannot be transmitted to the central computer room due to a central computer room failure or a network failure, the backup central computer room can also operate independently and receive data normally, effectively avoiding system paralysis caused by a single failure point and further enhancing the fault tolerance of the edge computing system.

[0060] In a possible manner, the storage device is a disk. Correspondingly, the data transmission method based on edge computing may further include: Determine the network status between the edge computer room and the central computer room, where the network status is used to indicate whether the network between the edge computer room and the central computer room is abnormal. In the case where the network status indicates that the network between the edge computer room and the central computer room is abnormal, compress the stored files on the disk.

[0061] It should be understood that during the operation of the edge computer room, target data is continuously collected from edge devices. However, the storage space of the disk is limited. If the network between the edge computer room and the central computer room is abnormal, the first file stored on the disk cannot be transmitted to the central computer room and deleted, resulting in a reduction in disk space and affecting subsequent file storage. Thus, in order to reduce the impact caused by insufficient disk space, the stored files on the disk can be periodically compressed to save disk space, thereby increasing the storage capacity of the first file, further reducing the probability of data loss caused by insufficient disk storage space, and enhancing data integrity.

[0062] To further understand the data transmission method based on edge computing provided by the embodiments of the present disclosure, the following combines the attached Figure 2 and the attached Figure 3 to further illustrate the solution: First, the edge computing system in the embodiment of the present disclosure is described. The edge computing system in this embodiment may include an edge cluster and a central cluster, wherein the central cluster may include a spare computer room and multiple central computer rooms, and the spare computer room and each central computer room may be configured with a data proxy service, a NAS disk queue, and a data consumer, etc. The data proxy service is used to transmit data back to the central computer room interface to implement elastic strategies such as load elastic scaling and cross-computer room disaster recovery. The NAS disk queue can be used to temporarily store data transmitted back from the edge cluster to act as a buffer. When the data transmission speed is fast or the data processing capacity of the central computer room is temporarily insufficient, the NAS disk queue can store the data to avoid data loss and wait for subsequent data consumers to process, thereby ensuring the continuity and stability of data transmission; the data consumer can be used to asynchronously consume data in the NAS disk queue and transmit it to a distributed message queue system, etc.

[0063] The edge cluster can include multiple edge computer rooms and edge devices that communicate with each edge computer room, and the edge devices can be configured with a dotting SDK for customizing the dotting protocol to write data to a specified area and a specified file. Each edge computer room can include a disk, a file collector, and a disk queue. The disk queue can be used for local persistent storage of data in disconnected and weak network scenarios, and the file collector can be used for various status maintenance of local files, such as file discovery, file scanning, collection cursors, that is, collection file offsets, data recovery, data backup, and data cleaning.

[0064] Next, the data transmission process of the present disclosure is described. Figure 2 and Figure 3 As shown: Step 1: Collect data from edge devices through the Dot SDK and generate dot files that need to be sent back to the central computer room; Step 2: Determine the disk status to determine whether the dot file can be written to the disk. If the dot file cannot be written to the disk, the dot file is directly forwarded to the data proxy service through the sock file and step 6 is executed; if the dot file can be written to the disk, the dot file is written to the file of the specified time slice; Step 3: The file collector obtains files by scanning the specified area directory at regular intervals, and determines whether the obtained file is a dot file through the preset file naming rules. If the obtained file is a dot file, the file offset and corresponding data in the dot file are read and written to the disk queue; if the obtained file is not a dot file, step 3 is repeated: Step 4: Determine the central computer room for receiving data and the target domain name for data transmission in the data proxy service, and determine the data transmission method and data backup method based on the target domain name; Step 5: Read the data in the disk queue and determine whether the data is valid. If the data is invalid, delete the data from the disk queue and read the next data. If the data is valid, transfer the data to the data proxy service according to the corresponding data transfer method by the target domain name, and determine whether the data transfer is successful. If the data transfer fails, periodic retry can be performed until the data transfer is successful or the maximum number of transfers is reached. If the data transfer is successful, record the file offset in the second file and delete the data from the disk queue. In addition, it can also be determined whether the file offset in the second file includes the file offset of the last file line in the dot file. If the file offset in the second file includes the file offset of the last file line in the dot file, the dot file can be backed up to the standby central computer room through the TOS (Type of Service) link, and after the dot file is successfully backed up to the standby central computer room, the first file and the second file can be deleted. Step 6: After receiving the data, the data proxy service writes the data into the NAS disk queue. If the write is successful, the first message indicating that the data has been successfully transferred to the central computer room can be returned to the edge computer room. If the write fails, the second message indicating that the data has not been successfully transferred to the central computer room can be returned to the edge computer room to inform the edge computer room to retry. Step 7: The asynchronous data consumer in the central computer room consumes the disk queue and replays the disk queue into a distributed message queue system such as kafka for use by the business side.

[0065] In addition, the network status between the edge computer room and the central computer room can be detected in real time. If the network status between the edge computer room and each central computer room is abnormal, the dot file can be backed up, and the data can be transferred to the data proxy service in the standby central computer room through the standby link between the edge computer room and the standby central computer room. The data proxy service transfers the backup file to the data consumer through the TOS (Type of Service) link.

[0066] Based on the same concept, the embodiments of the present disclosure also provide a data transfer device based on edge computing, as Figure 4 shown, the data transfer device 400 based on edge computing may include: A first processing module 401, configured to generate a first file according to the target data collected from the edge device and store the first file in the storage device of the edge computer room. The second processing module 402 is configured to transfer the target data in the first file to the central computer room of the cloud platform, and generate a second file according to the transmission situation of the target data during the transmission of the target data. The second file is used to record the file offset, and the file offset is used to represent the offset of the storage position of the target file line in the first file relative to the starting storage position of the first file. The target file line is the file line in the first file whose data has been transferred to the central computer room; The third processing module 403 is configured to delete the first file and the second file when the file offset in the second file includes the file offset of the last file line in the first file.

[0067] Through the above-mentioned edge-computing-based data transmission device 400, a storage device can be set up in the edge computer room. Thus, after the edge computer room receives the target data reported by the edge device, the target data can be first stored in the storage device, and then the target data in the storage device can be transferred to the central computer room. Since the target data has been stored in the storage device in the edge computer room, if problems such as damaged data integrity occur during the data transmission process, the corresponding data can be obtained again from the storage device for data transmission, thereby effectively improving the integrity and reliability of the data transmission. Or, if data is lost during the data transmission process due to a transmission link failure or network instability, etc., after the transmission link or network is restored, the corresponding data can be obtained again from the storage device for data transmission, thereby improving the data integrity and the reliability of the data transmission. In addition, since a second file for recording the file offset will also be generated during the data transmission process, and the first file and the second file can be deleted when the file offset includes the file offset of the last file line in the first file, the first file and the second file can be deleted in a timely manner when all the target data in the first file has been transferred to the central computer room, thereby improving the space utilization rate of the storage space and reducing the waste of storage resources.

[0068] In a possible manner, the second processing module 402 may include: The first processing sub-module is configured to, for each file line in the first file, determine the file offset corresponding to the file line and the target data corresponding to the file line, and write the file offset and the file offset as a write amount into the first disk queue in the edge computer room; The creation sub-module is configured to create a blank second file; The second processing sub-module is configured to, for each write amount in the first disk queue, transfer the target data in the write amount to the central computer room of the cloud platform. After the target data in the write amount is transferred to the central computer room, record the file offset in the write amount in the second file, and delete the write amount in the first disk queue.

[0069] In a possible manner, the central computer room may include a data proxy service, and the data proxy service includes multiple domain names associated with weight values. The weight value is used to represent the probability that a domain name is selected for data transmission. Correspondingly, the second processing module 402 may include: A first determination sub-module, configured to determine a target domain name for data transmission among multiple domain names according to the weight value associated with each domain name; A first transmission sub-module, configured to transmit the target data in the first file to the data proxy service through the target domain name.

[0070] In a possible manner, the first determination sub-module may be configured to: screen out the first domain name corresponding to the maximum weight value among multiple domain names according to the weight value associated with each domain name, and loop to execute the following process: Transmit the target data to the data proxy service through the first domain name, and determine the transmission status of the target data. The transmission status is used to indicate whether the target data is transmitted to the data proxy service; in the case where the transmission status indicates that the target data is not transmitted to the data proxy service, reduce the weight value of the first domain name, and screen out the second domain name corresponding to the maximum weight value among multiple domain names according to the weight value associated with each domain name, and use the second domain name as the new first domain name until the transmission status indicates that the target data is transmitted to the data proxy service.

[0071] In a possible manner, the second processing module 402 may further include: A second determination sub-module, configured to determine the corresponding data transmission method and data backup method according to the target domain name after determining the target domain name for data transmission among multiple domain names. The data transmission method includes compressed transmission and non-compressed transmission, and the data backup method includes compressed backup and non-compressed backup; Correspondingly, the first transmission sub-module may be configured to: transmit the target data in the first file to the data proxy service through the target domain name according to the data transmission method; Correspondingly, the second processing module 402 may further include: A second transmission sub-module, configured to transmit the first file to the cloud platform through the data backup link between the edge computer room and the cloud platform according to the data backup method after all the target data in the first file are transmitted to the data proxy service.

[0072] In a possible manner, the central computer room may further include a second disk queue and a data consumption device. Correspondingly, the data transmission based on edge computing may further include: A fourth processing module, configured to write the target data to the second disk queue through the data proxy service in the case where the transmission status indicates that the target data is transmitted to the data proxy service; The fifth processing module is used to asynchronously consume the target data in the second disk queue through a data consumption device and transmit the target data to the distributed message queue system.

[0073] In a possible way, the first file is stored in a preset directory of the storage device. Correspondingly, the second processing module 402 may include: An acquisition sub-module, used to periodically acquire files in the preset directory; A third determination sub-module, used to determine the naming information of the file, and the naming information is used to indicate whether the naming rule of the file is consistent with the preset naming rule; A third processing sub-module, used to determine the file as the first file when the naming information indicates that the naming rule of the file is consistent with the preset naming rule, and transmit the target data in the first file to the cloud platform; A fourth processing sub-module, used to periodically acquire files in the preset directory through the acquisition sub-module and determine the naming information of the file through the third determination sub-module when the naming information indicates that the naming rule of the file is inconsistent with the preset naming rule.

[0074] In a possible way, the cloud platform may further include a standby central computer room, and the standby central computer room is used to back up the stored files in the storage device. Correspondingly, the data transmission based on edge computing may further include: A first determination module, used to determine the network status between the edge computer room and the central computer room, and the network status is used to indicate whether the network between the edge computer room and the central computer room is abnormal; A backup module, used to back up the first file in the storage device to obtain a backup file for the first file and transmit the backup file to the standby central computer room through the data transmission link between the edge computer room and the standby central computer room when the network status indicates that the network between the edge computer room and the central computer room is abnormal.

[0075] In a possible way, the storage device is a disk. Correspondingly, the data transmission method based on edge computing may further include: A second determination module, used to determine the network status between the edge computer room and the central computer room, and the network status is used to indicate whether the network between the edge computer room and the central computer room is abnormal; A compression module, used to perform compression processing on the stored files in the disk when the network status indicates that the network between the edge computer room and the central computer room is abnormal.

[0076] In a possible way, the data transmission method based on edge computing may further include: A third determination module, used to determine the storage status of the storage device, and the storage status is used to indicate whether the storage device is available; Accordingly, the first processing module 401 can be used to: when the storage status indicates that the storage device is available, store the first file in the storage device; Accordingly, the data transmission method based on edge computing may further include: A transmission module, configured to transmit the first file to the central computer room of the cloud platform when the storage status indicates that the storage device is unavailable.

[0077] Based on the same concept, an embodiment of the present disclosure further provides a computer-readable medium, on which a computer program is stored, and when the program is executed by a processing device, the steps of any of the above-mentioned data transmission methods based on edge computing are implemented.

[0078] Based on the same concept, an embodiment of the present disclosure further provides an electronic device, which may include: A storage device, on which a computer program is stored; A processing device, configured to execute the computer program in the storage device to implement the steps of any of the above-mentioned data transmission methods based on edge computing.

[0079] Based on the same concept, an embodiment of the present disclosure further provides a computer program product, including a computer program, and when the computer program is executed by a processor, the steps of any of the above-mentioned data transmission methods based on edge computing are implemented.

[0080] Next, refer to Figure 5 , which shows a schematic structural diagram of an electronic device 500 suitable for implementing the embodiments of the present disclosure. The terminal device in the embodiments of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Tablet Computers), PMPs (Portable Multimedia Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 5 The electronic device shown is only an example, and should not impose any limitation on the functions and usage scope of the embodiments of the present disclosure.

[0081] As Figure 5 shown, the electronic device 500 may include a processing device (such as a central processing unit, a graphics processing unit, etc.) 501, which may perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 502 or the program loaded from the storage device 508 into the random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the electronic device 500 are also stored. The processing device 501, the ROM 502, and the RAM 503 are connected to each other through a bus 504. The input / output (I / O) interface 505 is also connected to the bus 504.

[0082] Typically, the following devices can be connected to the I / O interface 505: input devices 506 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 507 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 508 including, for example, magnetic tapes, hard disks, etc.; and a communication device 509. The communication device 509 can allow the electronic device 500 to communicate with other devices wirelessly or wiredly to exchange data. Although Figure 5 the electronic device 500 with various devices is shown, it should be understood that it is not required to implement or have all the shown devices. Instead, more or fewer devices can be implemented or had.

[0083] In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, an embodiment of the present disclosure includes a computer program product that includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes program codes for executing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from a network through the communication device 509, or installed from the storage device 508, or installed from the ROM 502. When the computer program is executed by the processing device 501, the above functions defined in the methods of the embodiments of the present disclosure are executed.

[0084] It should be noted that the above-mentioned computer-readable medium in the present disclosure can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, the computer-readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, apparatus, or device. In the present disclosure, the computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, in which computer-readable program code is carried. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium can also be any computer-readable medium other than the computer-readable storage medium, and this computer-readable signal medium can send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0085] In some embodiments, communication can be carried out using any currently known or future-developed network protocol such as HTTP (HyperText Transfer Protocol), and can be interconnected with digital data communication in any form or medium (for example, a communication network). Examples of communication networks include local area networks ("LAN"), wide area networks ("WAN"), the Internet (for example, the Internet), and end-to-end networks (for example, ad hoc end-to-end networks), as well as any currently known or future-developed networks.

[0086] The above-mentioned computer-readable medium can be included in the above-mentioned electronic device; it can also exist separately and not be assembled into the electronic device.

[0087] The above computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to: generate a first file based on target data collected from an edge device and store the first file in a storage device of an edge computer room; transmit the target data in the first file to a central computer room of a cloud platform, and during the transmission of the target data, generate a second file according to the transmission situation of the target data, where the second file is used to record a file offset, and the file offset is used to represent the offset of the storage position of a target file line in the first file relative to the starting storage position of the first file, and the target file line is the file line in the first file whose data has been transmitted to the central computer room; delete the first file and the second file when the file offset in the second file includes the file offset of the last file line in the first file.

[0088] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages or combinations thereof. The programming languages include, but are not limited to, object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (for example, by using an Internet service provider to connect through the Internet).

[0089] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code that contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, may be implemented by a dedicated hardware-based system for performing the specified functions or operations, or may be implemented by a combination of dedicated hardware and computer instructions.

[0090] The modules involved in the embodiments of the present disclosure can be implemented in software or in hardware. In some cases, the name of a module does not constitute a limitation on the module itself.

[0091] The functions described above herein can be performed, at least in part, by one or more hardware logic components. By way of example, and without limitation, the types of hardware logic components that can be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system on a chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0092] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0093] The above description is only a preferred embodiment of the present disclosure and an illustration of the technical principles applied. Those skilled in the art should understand that the scope of the disclosure involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above technical features, and should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosure concept. For example, the technical solutions formed by mutually replacing the above features with the technical features (but not limited to) having similar functions disclosed in the present disclosure.

[0094] In addition, although the operations are depicted in a particular order, this should not be construed as requiring that the operations be performed in the particular order shown or in sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, although several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of the present disclosure. Certain features described in the context of separate embodiments can also be implemented combinatorially in a single embodiment. Conversely, the various features described in the context of a single embodiment can also be implemented separately or in any suitable sub-combination in multiple embodiments.

[0095] Although the subject matter has been described in language specific to structural features and / or methodological logical acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. On the contrary, the specific features and acts described above are merely example forms for implementing the claims. Regarding the apparatus in the above embodiments, the specific manner in which each module performs operations has been described in detail in the embodiments related to the method, and will not be elaborated herein.

Claims

1. A data transmission method based on edge computing, characterized in that, The data transmission method based on edge computing includes: Generating a first file according to the target data collected from the edge device, and storing the first file in the storage device of the edge computer room; Transmitting the target data in the first file to the central computer room of the cloud platform, and generating a second file according to the transmission condition of the target data during the transmission of the target data. The second file is used to record the file offset, and the file offset is used to represent the offset of the storage position of the target file line in the first file relative to the starting storage position of the first file. The target file line is the file line in the first file whose data has been transmitted to the central computer room; Deleting the first file and the second file when the file offset in the second file includes the file offset of the last file line in the first file.

2. The data transmission method based on edge computing according to claim 1, wherein The step of transmitting the target data in the first file to the central computer room of the cloud platform and generating the second file according to the transmission condition of the target data during the transmission of the target data includes: For each file line in the first file, determining the file offset corresponding to the file line and the target data corresponding to the file line, and writing the file offset and the file offset as a write amount into the first disk queue of the edge computer room; Creating a blank second file; For each write amount in the first disk queue, transmitting the target data in the write amount to the central computer room of the cloud platform. After transmitting the target data in the write amount to the central computer room, recording the file offset in the write amount in the second file, and deleting the write amount in the first disk queue.

3. The data transmission method based on edge computing according to claim 1 or 2, characterized in that The central computer room includes a data proxy service, and the data proxy service includes multiple domain names associated with weight values. The weight value is used to represent the probability that the domain name is selected for data transmission. The step of transmitting the target data in the first file to the central computer room of the cloud platform includes: Determining a target domain name for data transmission from the multiple domain names according to the weight value associated with each domain name; Transmitting the target data in the first file to the data proxy service through the target domain name.

4. The data transmission method based on edge computing according to claim 3, wherein The step of determining a target domain name for data transmission from the multiple domain names according to the weight value associated with each domain name includes: Filtering out the first domain name corresponding to the maximum weight value from the multiple domain names according to the weight value associated with each domain name, and looping to execute the following process: Transmitting the target data to the data proxy service through the first domain name, and determining the transmission status of the target data. The transmission status is used to indicate whether the target data has been transmitted to the data proxy service; When the transmission status indicates that the target data has not been transmitted to the data proxy service, reduce the weight value of the first domain name, and based on the weight values associated with each domain name, screen out the second domain name corresponding to the maximum weight value among the multiple domain names, and use the second domain name as the new first domain name until the transmission status indicates that the target data has been transmitted to the data proxy service.

5. The data transmission method based on edge computing according to claim 3, characterized in that, The edge computing-based data transmission method further includes: After determining the target domain name for data transmission among the multiple domain names, determine the corresponding data transmission method and data backup method according to the target domain name. The data transmission method includes compressed transmission and non-compressed transmission, and the data backup method includes compressed backup and non-compressed backup; The step of transmitting the target data in the first file to the data proxy service through the target domain name includes: Transmitting the target data in the first file to the data proxy service through the target domain name according to the data transmission method; The edge computing-based data transmission method further includes: After all the target data in the first file has been transmitted to the data proxy service, transmit the first file to the cloud platform through the data backup link between the edge computer room and the cloud platform according to the data backup method.

6. The data transmission method based on edge computing according to claim 4, wherein The central computer room further includes a second disk queue and a data consumption device. The edge computing-based data transmission method further includes: When the transmission status indicates that the target data has been transmitted to the data proxy service, write the target data into the second disk queue through the data proxy service; Asynchronously consume the target data in the second disk queue through the data consumption device and transmit the target data to the distributed message queue system.

7. The data transmission method based on edge computing according to claim 1 or 2, characterized in that, The first file is stored in a preset directory of the storage device. The step of transmitting the target data in the first file to the central computer room of the cloud platform includes: Periodically obtain the files in the preset directory; Determine the naming information of the file, where the naming information is used to indicate whether the naming rule of the file is consistent with the preset naming rule; When the naming information indicates that the naming rule of the file is consistent with the preset naming rule, determine the file as the first file and transmit the target data in the first file to the central computer room of the cloud platform; When the naming information indicates that the naming rule of the file is inconsistent with the preset naming rule, return to execute the steps of periodically obtaining the files in the preset directory and determining the naming information of the file.

8. The data transmission method based on edge computing according to claim 1 or 2, characterized in that, The cloud platform further includes a standby central computer room, which is used to back up the stored files in the storage device. The edge computing-based data transmission method further includes: Determine the network status between the edge computer room and the central computer room, where the network status is used to indicate whether the network between the edge computer room and the central computer room is abnormal; When the network status indicates a network anomaly between the edge computer room and the central computer room, back up the first file in the storage device to obtain a backup file for the first file, and transmit the backup file to the standby central computer room through the data transmission link between the edge computer room and the standby central computer room.

9. The data transmission method based on edge computing according to claim 1 or 2, characterized in that The storage device is a disk, and the edge-computing-based data transmission method further includes: Determine the network status between the edge computer room and the central computer room, where the network status is used to indicate whether there is a network anomaly between the edge computer room and the central computer room; When the network status indicates a network anomaly between the edge computer room and the central computer room, perform compression processing on the stored files in the disk.

10. The data transmission method based on edge computing according to claim 1 or 2, characterized in that, The edge-computing-based data transmission method further includes: Determine the storage status of the storage device, where the storage status is used to indicate whether the storage device is available; The storing of the first file into the storage device of the edge computer room includes: When the storage status indicates that the storage device is available, store the first file into the storage device; The edge-computing-based data transmission method further includes: When the storage status indicates that the storage device is unavailable, transmit the first file to the central computer room of the cloud platform.

11. A data transmission device based on edge computing, characterized in that, The edge-computing-based data transmission device includes: A first processing module, configured to generate a first file according to target data collected from an edge device and store the first file in a storage device of an edge computer room; A second processing module, configured to transmit the target data in the first file to the central computer room of the cloud platform, and generate a second file according to the transmission condition of the target data during the transmission of the target data, where the second file is used to record a file offset, and the file offset is used to characterize the offset of the storage position of the target file line in the first file relative to the starting storage position of the first file in the first file, and the target file line is the file line in the first file whose data has been transmitted to the central computer room; A third processing module, configured to delete the first file and the second file when the file offset in the second file includes the file offset of the last file line in the first file.

12. A computer-readable medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processing device, it implements the steps of the method according to any one of claims 1-10.

13. An electronic device, characterized in that, Including: A storage device, on which a computer program is stored; A processing device, configured to execute the computer program in the storage device to implement the steps of the method according to any one of claims 1-10.

14. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1-10.

Images