Data processing method and device, equipment and storage medium
By combining central gateway devices and edge gateway devices, data is first processed on the edge gateway devices and then merged on the central gateway devices. This solves the problem of low efficiency in processing massive amounts of data in traditional central cloud systems, and achieves efficient data fusion and real-time processing.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHINA UNITED NETWORK COMM GRP CO LTD
- Filing Date
- 2023-06-12
- Publication Date
- 2026-05-12
AI Technical Summary
Traditional centralized cloud systems struggle to efficiently handle massive amounts of data, and storage and computing resources alone are no longer sufficient to meet the demands.
The system adopts a combination of a central gateway device and multiple edge gateway devices. The edge gateway devices first process the collected data and extract data features, and then the central gateway device performs data fusion to achieve efficient fusion and processing of data from multiple acquisition devices.
It improved data processing efficiency, ensured efficient data integration and real-time performance, distributed computational pressure, and met the high real-time requirements of data flow.
Smart Images
Figure CN116743662B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of communication technology, and in particular to a data processing method, apparatus, device and storage medium. Background Technology
[0002] The edge computing model involves deploying small, computing-capable micro-servers / processors on the data side, application side, and device side to offload the business processing load of the central cloud. This allows business applications to process the required data or business on the edge servers themselves, thereby reducing information transmission latency and improving data processing efficiency.
[0003] In actual production scenarios, systems built using information technology and intelligent methods often rely on massive amounts of data. Traditional central cloud systems process data from data sources in a unified manner. However, as the scale of data increases exponentially, it is difficult to efficiently guarantee data processing by relying solely on the storage and computing resources of the central cloud. Summary of the Invention
[0004] This application proposes a data processing method, apparatus, device, and storage medium for improving the efficiency of data processing.
[0005] To achieve the above objectives, this application adopts the following technical solution:
[0006] Firstly, a data processing method is provided, applied to a central gateway device in a data processing system. The data processing system also includes multiple edge gateway devices and multiple acquisition devices, with each edge gateway device corresponding to one acquisition device. The method includes: the central gateway device receiving first data information sent by a first edge gateway device, the data information representing data acquired by the acquisition devices, the first edge gateway device being any one of the multiple edge gateway devices; and determining a waiting delay corresponding to the first data information, the waiting delay being less than the delay requirement of the first acquisition device, the first acquisition device being an acquisition device connected to the first edge gateway device. Further, the central gateway device receives second data information sent by a second edge gateway device, the second edge gateway device being any one of the multiple edge gateway devices other than the first edge gateway device, the second data information being the data information received by the central gateway device within the waiting delay; and performing fusion processing on the first data information and the second data information to obtain target fused data, the target fused data representing the data characteristics of the first data information and the second data information.
[0007] In the data processing method provided in this application, when the acquisition device in the data processing system acquires data and reports it to the server through the edge gateway device, the edge gateway device first processes the acquired data to obtain the corresponding data information. Then, the central gateway device merges the data acquired by multiple acquisition devices and reports it to the server, thereby improving the efficiency of data processing and achieving efficient data fusion within the latency requirements.
[0008] In one possible design, the central gateway device receives second data information sent by the second edge gateway device, including: the central gateway device receiving third data information sent by the third edge gateway device during a waiting delay; and, if the third edge gateway device is different from both the first and second edge gateway devices, identifying the third edge gateway device as the second edge gateway device and identifying the third data information as the second data information. This design provides a specific implementation method for receiving second data information sent by the second edge gateway device.
[0009] In one possible design, the central gateway device receives second data information sent by the second edge gateway device. During a waiting delay, the central gateway device receives third data information sent by the third edge gateway device. If the third edge gateway device is either the first or second edge gateway device, it is identified as the first edge gateway device, and the third data information is identified as the first edge information. This design ensures that if data information from the same edge device is received during the waiting delay, the waiting period ends, and the previously identified first and second data information are merged.
[0010] In one possible design, the central gateway device determines the waiting delay corresponding to the first data information by: the central gateway device determining the difference between the delay required by the first acquisition device and a preset processing delay and a preset fusion delay as the waiting delay corresponding to the first data information; the preset processing delay is the delay of the first edge gateway device processing the message sent by the first acquisition device, and the preset fusion delay is the delay of the central gateway device fusing and processing the first data information. This design provides a specific implementation method for the central gateway device to determine the waiting delay corresponding to the first data information.
[0011] In one possible design, the data processing method further includes: if the waiting delay corresponding to the second data information is less than the waiting delay corresponding to the first data information, the central gateway device updates the waiting delay corresponding to the first data information to the waiting delay corresponding to the second data information. This design determines the waiting delay by identifying the minimum delay corresponding to the multiple data information pieces awaiting fusion, thus ensuring that all data information meets the required latency.
[0012] Secondly, a data processing device is provided as a central gateway device deployed in a data processing system. The data processing system also includes multiple edge gateway devices and multiple acquisition devices, with each edge gateway device corresponding to one acquisition device. The data processing device includes a receiving unit, a determining unit, and a processing unit. The receiving unit receives first data information sent by a first edge gateway device, which represents data acquired by the acquisition devices. The first edge gateway device is any one of the multiple edge gateway devices. The determining unit determines a waiting delay corresponding to the first data information, which is less than the delay requirement of the first acquisition device, which is an acquisition device connected to the first edge gateway device. The receiving unit also receives second data information sent by a second edge gateway device, which is an edge gateway device other than the first edge gateway device. The second data information is the data information received by the central gateway device within the waiting delay. The processing unit performs fusion processing on the first data information and the second data information to obtain target fused data, which represents the data characteristics of the first and second data information.
[0013] In one possible design, the receiving unit is specifically configured to receive third data information sent by the third edge gateway device during a waiting delay. The determining unit is further configured to determine the third edge gateway device as the second edge gateway device and the third data information as the second data information when the third edge gateway device is different from the first edge gateway device and the second edge gateway device.
[0014] In one possible design, the receiving unit is specifically configured to receive third data information sent by the third edge gateway device during a waiting delay. The determining unit is further configured to, if the third edge gateway device is one of the first edge gateway device and the second edge gateway device, determine the third edge gateway device as the first edge gateway device and the third data information as the first edge information.
[0015] In one possible design, the determining unit is specifically used to determine the difference between the latency required by the first acquisition device and the preset processing latency and the preset fusion latency as the waiting latency corresponding to the first data information; the preset processing latency is the latency of the first edge gateway device processing the message sent by the first acquisition device, and the preset fusion latency is the latency of the central gateway device fusing and processing the first data information.
[0016] In one possible design, the processing unit is further configured to update the waiting delay corresponding to the first data information to the waiting delay corresponding to the second data information if the waiting delay corresponding to the second data information is less than the waiting delay corresponding to the first data information.
[0017] Thirdly, a network device is provided, the network device including a memory and a processor; the memory and the processor are coupled, the memory being used to store computer program code including computer instructions, and when the processor executes the computer instructions, the network device performs a data processing method as provided in the first aspect or any possible design thereof.
[0018] Fourthly, a computer-readable storage medium is provided, in which instructions are stored, which, when executed on a network device, cause the network device to perform a data processing method as provided in the first aspect or any possible implementation thereof.
[0019] Fifthly, a computer program product is provided, comprising computer instructions that, when executed on a network device, enable the network device to perform a data processing method as provided in the first aspect or any possible implementation thereof.
[0020] In the data processing method provided in this application, for scenarios involving real-time fusion of multi-source data, a central + edge multi-gateway device model is used to distribute the computational pressure, ensuring both efficient data discretization processing and the high real-time requirements of data flow. Furthermore, by adopting an asynchronous processing and synchronous fusion forwarding model, edge gateway devices extract data features, and central gateway devices coordinate scheduling and multi-task collaborative operation using preset strategies, thus guaranteeing both real-time data transmission and improved data processing efficiency. Attached Figure Description
[0021] Figure 1 A schematic diagram of a data processing system structure provided for an embodiment of this application;
[0022] Figure 2 A schematic flowchart of a data processing method provided for embodiments of this application. Figure 1 ;
[0023] Figure 3 A schematic flowchart of a data processing method provided for embodiments of this application. Figure 2 ;
[0024] Figure 4 A schematic flowchart of a data processing method provided for embodiments of this application. Figure 3 ;
[0025] Figure 5 A schematic diagram of a data processing device structure provided for an embodiment of this application;
[0026] Figure 6 A schematic diagram of a network device structure provided for embodiments of this application. Figure 1 ;
[0027] Figure 7 A schematic diagram of a network device structure provided for embodiments of this application. Figure 2 . Detailed Implementation
[0028] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.
[0029] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.
[0030] In the description of this application, unless otherwise stated, " / " means "or". For example, A / B can mean A or B. "And / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. Furthermore, "at least one" and "multiple" refer to two or more. The terms "first," "second," etc., do not limit the quantity or order of execution, and "first," "second," etc., do not necessarily imply differences.
[0031] Currently, in actual production scenarios, systems built using information technology and intelligent methods often rely on massive amounts of data for support. Traditional central cloud systems process data from data sources in a unified manner. However, as the scale of data increases exponentially, it is difficult to efficiently guarantee data processing by relying solely on the storage and computing resources of the central cloud.
[0032] To address the aforementioned problems, this application proposes a data processing method, apparatus, device, and storage medium applied to a central gateway device in a data processing system. The data processing system also includes multiple edge gateway devices and multiple acquisition devices, with each edge gateway device corresponding to one acquisition device. The method includes: the central gateway device receiving first data information sent by a first edge gateway device, the data information representing data acquired by the acquisition devices, where the first edge gateway device is any one of the multiple edge gateway devices; and determining a waiting delay corresponding to the first data information, the waiting delay being less than the delay requirement of the first acquisition device, where the first acquisition device is an acquisition device connected to the first edge gateway device. Further, the central gateway device receives second data information sent by a second edge gateway device, the second edge gateway device being any one of the multiple edge gateway devices other than the first edge gateway device, the second data information being the data information received by the central gateway device within the waiting delay; and performing fusion processing on the first data information and the second data information to obtain target fused data, the target fused data representing the data characteristics of the first data information and the second data information.
[0033] In this way, in the data processing method provided by this application, when the acquisition device in the data processing system acquires data and reports it to the server through the edge gateway device, the edge gateway device will first process the acquired data to obtain the corresponding data information, and then the central gateway device will realize the fusion of data acquired by multiple acquisition devices before reporting it to the server, thereby improving the efficiency of data processing and achieving efficient data fusion within the latency requirements.
[0034] Figure 1 This application illustrates a data processing system, and the data processing method provided in this embodiment can be applied to, for example, […]. Figure 1 The data processing system shown is used to improve the efficiency of data processing. For example... Figure 1 As shown, the data processing system 10 includes a central gateway device 11, multiple edge gateway devices 12, multiple acquisition devices 13, and a server 14.
[0035] The following needs to be explained: Figure 1The example illustration shows three acquisition devices 13 and three edge gateway devices 12, which does not constitute a limitation on the number of acquisition devices and edge gateway devices. It is only used to illustrate that the data processing method provided in this application embodiment can be applied to a data processing system including multiple acquisition devices and multiple edge gateway devices. Furthermore, among the multiple acquisition devices 13, the acquisition devices can be of the same type and model, or they can be of different types and models. Similarly, among the multiple edge gateway devices 12, the edge gateway devices can be of the same type and model, or they can be of different types and models. This application embodiment does not make specific limitations in this regard.
[0036] The central gateway device 11 is connected to multiple edge gateway devices 12 and a server 14. Each of the multiple edge gateway devices 12 is connected to one of the multiple acquisition devices 13. The connection can be wired or wireless, and this embodiment does not limit the connection.
[0037] It should be noted that the central gateway device 11 includes a task distribution module 111 and a fusion module 112; each of the multiple edge gateway devices 12 includes a configuration module 121 and an execution module 122.
[0038] The acquisition device 13 is the data source for the data processing system 10. After acquiring the data, the acquisition device 13 can transmit the acquired data to the edge gateway device 12 connected to it in real time.
[0039] It should be noted that the data acquisition device can be a sensor, camera, or other device used for data acquisition. For example, the data acquisition device can be a smoke sensor, a distance sensor, a surveillance camera, etc. This application embodiment does not specifically limit this.
[0040] The edge gateway device 12 is connected to a data acquisition device 13 and can be used to perform data processing tasks, extract data features, and upload the data information obtained after data processing after receiving data sent by the data acquisition device 13.
[0041] The configuration module 121 can be used to receive data processing tasks issued by the task distribution module 111 of the central gateway device 11, that is, the data processing strategy sent to the acquisition device 13.
[0042] Configuration module 121 can also be used to extract data features from the data collected by acquisition device 13 and send them to central gateway device 11.
[0043] The execution module 122 can be used to perform data feature extraction and data processing according to the data processing strategy configured by the configuration module 121, and upload the data information obtained after processing to the fusion module 112 of the central gateway device 11 so that the fusion module 112 can perform data fusion processing.
[0044] The central gateway device 11 can be used to formulate data processing strategies for each acquisition device 13 based on the data characteristics sent by the configuration module 121 of the edge gateway device 12 and in combination with business requirements, and then distribute these strategies to the configuration module 121 of the edge gateway device 12. Additionally, the central gateway device 11 can also perform fusion processing on the data information sent by each edge gateway device 12, for use by the server 14.
[0045] In some embodiments, before accessing the data processing system, the acquisition device 13 sends test data similar to the acquired data to the edge gateway device 12. The edge gateway device 12 extracts data features based on the test data and sends the extracted data features to the central gateway device 11. The central gateway device 11 formulates an edge configuration policy based on the data features and business requirements and sends it to the edge gateway device 12, so that after the acquisition device 13 completes its access, the edge gateway device 12 processes the acquired data of the acquisition device 13 according to the edge configuration policy.
[0046] It should be noted that data characteristics can be data standards uniformly defined by the central gateway device. For example, data characteristics include data collection frequency (the time period during which collected data reaches the edge gateway device), main data fields (key fields for business requirements), and data structure. Among these, data collection frequency considers the time dimension standard of the data, while main data fields and data structure consider the spatial dimension standard of the data.
[0047] For example, when the data collected by the acquisition device 13 is unstructured data (such as images), the edge configuration strategy can be to decode the images and extract key fields; when the data collected by the acquisition device 13 is structured data, the edge configuration strategy can be to extract key fields.
[0048] The task distribution module 111 can be used to formulate the edge configuration policy of the edge gateway device 12 based on the data characteristics sent by the configuration module 121 in the edge gateway device 12 and in combination with business requirements, and send the edge configuration policy to the configuration module 121 of the edge gateway device 12.
[0049] The fusion module 112 can be used to perform fusion processing on the data information sent by the execution module 122 in the edge gateway device 12, and send the fused data information to the server 14.
[0050] Server 14 can be used to store data information sent by the central gateway device 11 for use by external devices.
[0051] The central gateway device 11 can also be used to receive first data information sent by the first edge gateway device.
[0052] The data information is used to characterize the data collected by the acquisition device 13, and the first edge gateway device is any one of the multiple edge gateway devices 12.
[0053] The central gateway device 11 can also be used to determine the waiting delay corresponding to the first data information.
[0054] Wherein, the waiting latency is less than the latency requirement of the first acquisition device, and the first acquisition device is the acquisition device connected to the first edge gateway device.
[0055] The central gateway device 11 can also be used to receive second data information sent by the second edge gateway device.
[0056] The second edge gateway device is an edge gateway device other than the first edge gateway device among the multiple edge gateway devices 12, and the second data information is the data information received by the central gateway device during the waiting delay.
[0057] The central gateway device 11 can also be used to fuse the first data information and the second data information to obtain the target fused data.
[0058] Among them, the target fusion data is used to characterize the data features of the first data information and the second data information.
[0059] Figure 2 This is a flowchart illustrating a data processing method according to some exemplary embodiments. In some embodiments, the above-described data processing method can be applied to, for example... Figure 1 The data processing system 10 shown includes a central gateway device 11. Hereinafter, this application will describe the data processing method using an example of the data processing method applied to the central gateway device 11.
[0060] like Figure 2 As shown, the data processing method provided in this application embodiment includes the following S201-S204.
[0061] S201, The central gateway device receives the first data information sent by the first edge gateway device.
[0062] Among them, the data information is used to characterize the data collected by the acquisition device, and the first edge gateway device is any one of multiple edge gateway devices.
[0063] It should be noted that the edge gateway device processes the data sent by the acquisition device, extracts data information, and sends it to the central gateway device based on the edge configuration policy pre-configured within the device.
[0064] As one possible implementation, after receiving data sent by the acquisition device connected to it, the first edge gateway device processes the data based on a pre-configured edge configuration strategy, extracts the first data information, and sends it to the central gateway device.
[0065] Correspondingly, the central gateway device receives the first data information sent by the edge gateway device.
[0066] It should be noted that the data information refers to the key fields of the data collected by the acquisition device. For example, if the acquisition device is a smoke sensor, the key field is the field indicating the smoke concentration in the smoke concentration data collected by the smoke sensor. This field can be pre-set in the edge gateway device by the operation and maintenance personnel of the data processing system. This application embodiment does not make specific limitations on this.
[0067] S202, The central gateway device determines the waiting delay corresponding to the first data information.
[0068] Wherein, the waiting latency is less than the latency requirement of the first acquisition device, and the first acquisition device is the acquisition device connected to the first edge gateway device.
[0069] As one possible implementation, after receiving the first data information, the central gateway device queries the server for the latency requirement of the data from the first acquisition device based on the identifier of the first acquisition device, and determines the waiting latency corresponding to the first data information based on the latency requirement and the preset value.
[0070] For example, when the preset value is a fixed duration, the waiting delay corresponding to the first data information is determined to be the difference between the required delay and the preset value; when the preset value is a percentage, the waiting delay corresponding to the first data information is determined to be the product of the required delay and the preset value.
[0071] For example, if the latency requirement of the first acquisition device is 1000 milliseconds, and the preset value is 150 milliseconds, the waiting latency corresponding to the first data information is 1000-150=850 milliseconds; and the preset value is 90%, the waiting latency corresponding to the first data information is 1000*90%=900 milliseconds.
[0072] It should be noted that the latency requirement of the first acquisition device can be preset in the central gateway device by the operation and maintenance personnel of the data processing system according to the data requirements of the first acquisition device. This application embodiment does not make specific limitations on this.
[0073] In some embodiments, if a preset processing delay and a preset fusion delay are set in the central gateway device, wherein the preset processing delay is the delay for the first edge gateway device to process the message sent by the first acquisition device, and the preset fusion delay is the delay for the central gateway device to fuse and process the first data information, then the central gateway device determines the waiting delay corresponding to the first data information by: the central gateway device determining the difference between the delay corresponding to the delay requirement of the first acquisition device and the preset processing delay and the preset fusion delay as the waiting delay corresponding to the first data information.
[0074] For example, if the required latency of the first acquisition device is t, the preset processing latency is t1, and the preset fusion latency is t2, then the central gateway device determines the waiting latency corresponding to the first data information to be t-t1-t2.
[0075] S203, The central gateway device receives the second data information sent by the second edge gateway device.
[0076] The second edge gateway device is an edge gateway device other than the first edge gateway device among multiple edge gateway devices, and the second data information is the data information received by the central gateway device during the waiting delay.
[0077] As one possible implementation, after the central gateway device determines the waiting delay corresponding to the first data information based on the above step S202, it starts timing, determines the data information received within the waiting delay as the second data information, determines the edge gateway corresponding to the second data information as the second edge gateway device, and realizes the reception of the second data information sent by the second edge gateway device.
[0078] In some embodiments, the central gateway device receiving the second data information sent by the second edge gateway device may further include: the central gateway device receiving the third data information sent by the third edge gateway device during the waiting delay; and if the third edge gateway device is different from the first edge gateway device and the second edge gateway device, the central gateway device may identify the third edge gateway device as the second edge gateway device and the third data information as the second data information.
[0079] In other embodiments, to avoid repeatedly receiving data information sent by the same edge gateway device, the central gateway device may receive the second data information sent by the second edge gateway device in the following ways: the central gateway device receives the third data information sent by the third edge gateway device during the waiting delay; and if the third edge gateway device is one of the first edge gateway device and the second edge gateway device, the third edge gateway device is identified as the first edge gateway device, and the third data information is identified as the first edge information.
[0080] Furthermore, the central gateway device will update the waiting latency to 0, or the central gateway device will end the waiting for the first data information and the second data information, and control the fusion module to perform fusion processing on the first data information and the second data information.
[0081] In one design, to prevent the waiting time for the second data information from exceeding the required latency of the second acquisition device, the central gateway device updates the waiting latency corresponding to the first data information to the waiting latency corresponding to the second data information if the waiting latency corresponding to the second data information is less than the waiting latency corresponding to the first data information.
[0082] The second acquisition device is an acquisition device connected to the second edge gateway.
[0083] S204. The central gateway device performs fusion processing on the first data information and the second data information to obtain the target fused data.
[0084] Among them, the target fusion data is used to characterize the data features of the first data information and the second data information.
[0085] As one possible implementation, after receiving the first data information and the second data information, the central gateway device determines whether there is a logical relationship between them. If there is no logical relationship, the central gateway device concatenates and merges the first and second data information; if there is a logical relationship, the central gateway device merges the first and second data information according to the existing logical relationship to obtain the target fused data.
[0086] For example, if the acquisition devices corresponding to the first data information and the second data information are both smoke sensors and are deployed in a space, and the server requires the average concentration of the space, then the central gateway device determines the logical relationship between the first data information and the second data information as determining the average value of multiple data information.
[0087] In some embodiments, after determining the target fusion data, the central gateway device sends the target fusion data to the server so that external devices can retrieve the corresponding data through the server.
[0088] Understandably, in the data processing method provided in this application, when the acquisition device in the data processing system acquires data and reports it to the server through the edge gateway device, the edge gateway device first processes the acquired data to obtain the corresponding data information, and then the central gateway device integrates the data acquired by multiple acquisition devices before reporting it to the server, thereby improving the efficiency of data processing and realizing efficient data integration.
[0089] In one design, if the acquisition device is connected to the data processing system for the first time, in order to subsequently achieve the fusion of the data acquired by the acquisition device, the data processing method provided in this application embodiment, such as... Figure 3 As shown, it also includes S301-S309.
[0090] S301. The data acquisition device sends test data to the edge gateway device.
[0091] S302, The configuration module of the edge gateway device extracts the data characteristics of the test data.
[0092] S303, The edge gateway device sends data characteristics to the task distribution module of the central gateway device.
[0093] S304 The task distribution module of the central gateway device formulates edge configuration strategies and data fusion strategies based on data characteristics and business requirements.
[0094] S305. The task distribution module sends edge configuration policies to the edge gateway device.
[0095] S306. The edge gateway device extracts data information from the data collected by the acquisition device based on the edge configuration policy.
[0096] S307. The edge gateway device sends the extracted data information to the central gateway device.
[0097] S308. The central gateway device performs data fusion processing on the data information sent by the edge gateway device based on the data fusion strategy to obtain the target fused data.
[0098] S309. The central gateway device sends the target fusion data to the server for external devices to retrieve.
[0099] It should be noted that the specific execution of steps S301-S309 by each device can be found in the description in the above embodiments of this application, and will not be repeated here.
[0100] In one design, after receiving data from the edge gateway device, the central gateway device executes a data processing method, as exemplified by the following flow: Figure 4 As shown, it includes S401-S407.
[0101] S401, The central gateway device receives data information sent by the acquisition device.
[0102] S402. The central gateway device determines whether there is data information to be merged within the central gateway device.
[0103] It should be noted that if it is determined that there is no data to be merged inside the central gateway device, the central gateway device executes step S403; if it is determined that there is data to be merged inside the central gateway device, the central gateway device executes step S406.
[0104] S403, the central gateway device identifies the data information as the data information to be merged.
[0105] S404. The central gateway device determines whether the waiting time for the data information to be merged exceeds the waiting delay.
[0106] It should be noted that if the waiting time for the data to be merged exceeds the waiting delay, the central gateway device executes step S405; if the waiting time for the data to be merged does not exceed the waiting delay, the central gateway device repeats step S404.
[0107] S405. The central gateway device determines whether the number of data information to be merged is greater than 1.
[0108] It should be noted that if the number of data information to be merged is greater than 1, the central gateway device executes step S406; if the number of data information to be merged is equal to 1, the central gateway device executes step S407.
[0109] S406. The central gateway device merges multiple data information to be merged to obtain the target merged data.
[0110] S407. The central gateway device reports the target fusion data or data to be fused to the server.
[0111] It should be noted that the specific execution of steps S401-S407 by the central gateway device can be found in the description in the above embodiments of this application, and will not be repeated here.
[0112] Understandably, the data processing method provided in the above embodiments of this application, for scenarios involving real-time fusion of multi-source data, distributes the computational pressure through a central + edge multi-gateway device model. This ensures both efficient data discretization processing and the high real-time requirements of data flow. Furthermore, by adopting an asynchronous processing and synchronous fusion forwarding model, edge gateway devices extract data features, and central gateway devices coordinate scheduling and multi-task collaborative operation using preset strategies. This not only guarantees the real-time nature of data transmission but also improves data processing efficiency.
[0113] The foregoing mainly describes the solutions provided by the embodiments of this application from a methodological perspective. To achieve the above functions, it includes corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should readily recognize that, in conjunction with the units and algorithm steps of the various examples described in the embodiments disclosed herein, the embodiments of this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0114] This application embodiment can divide the user equipment into functional modules according to the above method example. For example, each function can be divided into a separate functional module, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware or as a software functional module. Optionally, the module division in this application embodiment is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.
[0115] Figure 5 This is a schematic diagram of a data processing apparatus provided in an embodiment of this application. The data processing apparatus is used to execute the data processing method described above. Figure 5 As shown, the data processing device 50 includes a receiving unit 501, a determining unit 502, and a processing unit 503.
[0116] The receiving unit 501 is used to receive first data information sent by the first edge gateway device. The data information represents the data collected by the acquisition device. The first edge gateway device can be any one of multiple edge gateway devices. For example, ... Figure 2 As shown, the receiving unit 501 can be used to execute S201.
[0117] The determining unit 502 is used to determine the waiting delay corresponding to the first data information. The waiting delay is less than the delay requirement of the first acquisition device, which is an acquisition device connected to the first edge gateway device. For example, ... Figure 2 As shown, the determining unit 502 can be used to execute S202.
[0118] The receiving unit 501 is further configured to receive second data information sent by a second edge gateway device, wherein the second edge gateway device is an edge gateway device other than the first edge gateway device among a plurality of edge gateway devices, and the second data information is data information received by the central gateway device during the waiting delay. For example, such as Figure 2 As shown, the receiving unit 501 can be used to execute S203.
[0119] Processing unit 503 is used to fuse the first data information and the second data information to obtain target fused data, which is used to characterize the data features of the first data information and the second data information. For example, Figure 2 As shown, the processing unit 503 can be used to execute S204.
[0120] Optional, such as Figure 5 As shown, in the data processing apparatus 50 provided in this embodiment, the receiving unit 501 is specifically used to receive third data information sent by the third edge gateway device during the waiting delay. The determining unit 502 is further used to determine the third edge gateway device as the second edge gateway device and the third data information as the second data information when the third edge gateway device is different from the first edge gateway device and the second edge gateway device.
[0121] Optional, such as Figure 5 As shown, in the data processing apparatus 50 provided in this embodiment, the receiving unit 501 is specifically used to receive third data information sent by the third edge gateway device during the waiting delay. The determining unit 502 is further used to determine the third edge gateway device as the first edge gateway device and the third data information as first edge information when the third edge gateway device is one of the first edge gateway device and the second edge gateway device.
[0122] Optional, such as Figure 5 As shown, in the data processing apparatus 50 provided in this application embodiment, the determining unit 502 is specifically used to determine the difference between the latency required by the first acquisition device and the preset processing latency and the preset fusion latency as the waiting latency corresponding to the first data information; the preset processing latency is the latency of the first edge gateway device processing the message sent by the first acquisition device, and the preset fusion latency is the latency of the central gateway device fusing and processing the first data information.
[0123] Optional, such as Figure 5As shown, in the data processing apparatus 50 provided in this application embodiment, the processing unit 503 is further configured to update the waiting delay corresponding to the first data information to the waiting delay corresponding to the second data information when the waiting delay corresponding to the second data information is less than the waiting delay corresponding to the first data information.
[0124] In implementing the functions of the integrated modules described above using hardware, this application provides a possible structural diagram of a network device. This network device is used to execute the data processing method performed by the data processing device in the above embodiments. Figure 6 As shown, the network device 60 includes a processor 601, a memory 602, and a bus 603. The processor 601 and the memory 602 can be connected via the bus 603.
[0125] Processor 601 is the control center of the network device. It can be a single processor or a collective term for multiple processing elements. For example, processor 601 can be a general-purpose central processing unit (CPU) or other general-purpose processors. The general-purpose processor can be a microprocessor or any conventional processor.
[0126] As one embodiment, processor 601 may include one or more CPUs, for example Figure 6 CPU 0 and CPU 1 are shown in the diagram.
[0127] The memory 602 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto.
[0128] In one possible implementation, the memory 602 can exist independently of the processor 601. The memory 602 can be connected to the processor 601 via a bus 603 and is used to store instructions or program code. When the processor 601 calls and executes the instructions or program code stored in the memory 602, it can implement the data processing method provided in the embodiments of this application.
[0129] In another possible implementation, the memory 602 can also be integrated with the processor 601.
[0130] Bus 603 can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 6 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.
[0131] It should be pointed out that, Figure 6 The structure shown does not constitute a limitation on the network device 60. Except... Figure 6 In addition to the components shown, the network device 60 may include more than Figure 6 It can show more or fewer parts, or combine certain parts, or arrange different parts.
[0132] As an example, combined Figure 5 The functions implemented by the receiving unit 501, the determining unit 502, and the processing unit 503 in the data processing device 50 are the same as those of the receiving unit 501, the determining unit 502, and the processing unit 503. Figure 6 The processor 601 in it has the same function.
[0133] Optional, such as Figure 6 As shown, the network device provided in this application embodiment may further include a communication interface 604.
[0134] Communication interface 604 is used to connect with other devices via a communication network. This communication network can be Ethernet, a wireless access network, a wireless local area network (WLAN), etc. Communication interface 604 may include an acquisition unit for receiving data and a transmission unit for sending data.
[0135] In one design, the communication interface of the network device provided in this application embodiment can also be integrated into the processor.
[0136] Figure 7 Another hardware structure of the network device in an embodiment of this application is shown. For example... Figure 7 As shown, network device 70 may include processor 701 and communication interface 702. Processor 701 is coupled to communication interface 702.
[0137] The functions of processor 701 can be referred to in the description of processor 601 above. In addition, processor 701 also has storage functions, which can be referred to in the description of memory 602 above.
[0138] The communication interface 702 is used to provide data to the processor 701. The communication interface 702 can be an internal interface of the network device or an external interface of the network device (equivalent to communication interface 604).
[0139] It should be pointed out that, Figure 7 The structure shown does not constitute a limitation on network devices, except Figure 7 In addition to the components shown, the network device 70 may include more or fewer components than illustrated, or combine certain components, or have different component arrangements.
[0140] Through the above description of the embodiments, those skilled in the art will clearly understand that, for the sake of convenience and brevity, only the division of the above functional units is used as an example. In practical applications, the above functions can be assigned to different functional units as needed, that is, the internal structure of the device can be divided into different functional units to complete all or part of the functions described above. The specific working process of the system, device, and unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.
[0141] This application also provides a computer-readable storage medium storing instructions. When a computer executes these instructions, the computer performs each step of the method flow shown in the above-described method embodiments.
[0142] Embodiments of this application provide a computer program product containing instructions that, when executed on a computer, cause the computer to perform the data processing method described in the above method embodiments.
[0143] The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), registers, hard disks, optical fibers, compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing, or any other form of computer-readable storage medium in the art. An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium may also be a component of the processor. The processor and the storage medium may reside in an application-specific integrated circuit (ASIC). In the embodiments of this application, the computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.
[0144] Since the apparatus, device, computer-readable storage medium, and computer program product in the embodiments of this application can be applied to the above methods, the technical effects that can be obtained can also be referred to the above method embodiments. The embodiments of this application will not be repeated here.
[0145] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. A data processing method, characterized by, A central gateway device is used in a data processing system, which also includes multiple edge gateway devices and multiple acquisition devices, wherein the multiple edge gateway devices correspond one-to-one with the multiple acquisition devices. The method includes: The device receives first data information sent by a first edge gateway device, the data information being used to characterize the data collected by the acquisition device, wherein the first edge gateway device is any one of the plurality of edge gateway devices; The difference between the latency requirement of the first acquisition device and the preset processing latency and preset fusion latency is determined as the waiting latency corresponding to the first data information. The waiting latency is less than the latency requirement of the first acquisition device. The first acquisition device is an acquisition device connected to the first edge gateway device. The preset processing latency is the latency of the first edge gateway device processing the message sent by the first acquisition device, and the preset fusion latency is the latency of the central gateway device fusing and processing the first data information. The central gateway device receives second data information sent by a second edge gateway device, wherein the second edge gateway device is an edge gateway device other than the first edge gateway device among the plurality of edge gateway devices, and the second data information is data information received by the central gateway device during the waiting delay. If the waiting delay corresponding to the second data information is less than the waiting delay corresponding to the first data information, the waiting delay corresponding to the first data information is updated to the waiting delay corresponding to the second data information. The first data information and the second data information are fused to obtain target fused data, which is used to characterize the data features of the first data information and the second data information.
2. The data processing method according to claim 1, characterized in that, The second data information received from the second edge gateway device includes: During the waiting delay, receive third data information sent by the third edge gateway device; If the third edge gateway device is different from the first edge gateway device and the second edge gateway device, the third edge gateway device is identified as the second edge gateway device, and the third data information is identified as the second data information.
3. The data processing method according to claim 1, characterized in that, The second data information received from the second edge gateway device includes: During the waiting delay, receive third data information sent by the third edge gateway device; In the case where the third edge gateway device is one of the first edge gateway device and the second edge gateway device, the third edge gateway device is identified as the first edge gateway device, and the third data information is identified as the first data information.
4. A data processing apparatus, characterized in that, The data processing system includes a central gateway device deployed in a data processing system, which also includes multiple edge gateway devices and multiple acquisition devices. The multiple edge gateway devices correspond one-to-one with the multiple acquisition devices. The data processing device includes a receiving unit, a determining unit, and a processing unit. The receiving unit is used to receive first data information sent by the first edge gateway device. The data information is used to characterize the data collected by the acquisition device. The first edge gateway device is any one of the plurality of edge gateway devices. The determining unit is used to determine the difference between the latency required by the first acquisition device and the preset processing latency and preset fusion latency as the waiting latency corresponding to the first data information, wherein the waiting latency is less than the latency requirement of the first acquisition device, and the first acquisition device is an acquisition device connected to the first edge gateway device; wherein the preset processing latency is the latency of the first edge gateway device processing the message sent by the first acquisition device, and the preset fusion latency is the latency of the central gateway device fusing and processing the first data information; The receiving unit is further configured to receive second data information sent by the second edge gateway device, wherein the second edge gateway device is an edge gateway device other than the first edge gateway device among the plurality of edge gateway devices, and the second data information is data information received by the central gateway device within the waiting delay; The processing unit is further configured to update the waiting delay corresponding to the first data information to the waiting delay corresponding to the second data information when the waiting delay corresponding to the second data information is less than the waiting delay corresponding to the first data information. The processing unit is used to perform fusion processing on the first data information and the second data information to obtain target fused data, which is used to characterize the data features of the first data information and the second data information.
5. The data processing apparatus according to claim 4, characterized in that, The receiving unit is specifically used to receive third data information sent by the third edge gateway device during the waiting delay. The determining unit is further configured to, when the third edge gateway device is different from the first edge gateway device and the second edge gateway device, determine the third data information as the second data information.
6. The data processing apparatus according to claim 4, characterized in that, The receiving unit is specifically used to receive third data information sent by the third edge gateway device during the waiting delay. The determining unit is further configured to, when the third edge gateway device is one of the first edge gateway device and the second edge gateway device, determine the third edge gateway device as the first edge gateway device and the third data information as the first data information.
7. A network device, characterized in that, Including memory and processor; The memory and the processor are coupled; The memory is used to store computer program code, which includes computer instructions; When the processor executes the computer instructions, the network device performs the data processing method as described in any one of claims 1-3.
8. A computer-readable storage medium storing instructions, characterized in that, When the instruction is executed on the network device, the network device performs the data processing method as described in any one of claims 1-3.