A 5G IoT communication data processing system based on smart cities

Abstract

This invention discloses a 5G IoT communication data processing system based on smart cities, relating to the field of data processing technology. It includes a communication center, which is communicatively connected to a data acquisition module, a data processing module, a data analysis module, and a communication loading module. The data acquisition module optimizes communication data acquisition through data source selection, sampling rate optimization, data stream processing, and network transmission protocol optimization. The data processing module performs computational optimization scheduling, encryption operations, and node verification. The data analysis module includes a data reprocessing unit, a data modeling unit, and a data visualization unit. The communication loading module has a loading queue and a queuing queue; by increasing or decreasing the number of loading queues, communication data can be loaded on demand, thereby achieving the purpose of processing the communication data.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, specifically a 5G Internet of Things (IoT) communication data processing system based on smart cities. Background Technology

[0002] A communication data processing system refers to a system that connects distributed data terminal equipment to a computer system via data circuits to achieve data transmission, exchange, storage, and processing.

[0003] Traditional communication data processing systems cannot adjust data sampling according to changes in data traffic, and they perform calculations locally, resulting in an excessively large amount of data to be processed. In addition, the security of communication data is the most pressing concern, as data leaks have occurred frequently in previous communication data processing systems. All of these issues are problems we need to consider. Summary of the Invention

[0004] To address the aforementioned problems, the present invention aims to provide a 5G Internet of Things (IoT) communication data processing system based on smart cities.

[0005] The objective of this invention can be achieved through the following technical solution: a 5G Internet of Things communication data processing system based on smart cities, comprising a communication center, wherein the communication center is communicatively connected to a data acquisition module, a data processing module, a data analysis module and a communication loading module;

[0006] The data acquisition module is used to collect communication data, which is a summary of communication data from several city blocks. The acquisition of communication data is optimized through data source selection, sampling rate optimization, data stream processing, and network transmission protocol optimization.

[0007] The data processing module is used to perform computational optimization scheduling, encryption operations, and node verification on communication data. It optimizes and schedules communication data through intelligent edge computing, selects whether to perform local or remote computation on communication data through intelligent edge computing, performs dynamic encryption operations on communication data of several city blocks through blockchain technology, and verifies the nodes of the blocks where the communication data is stored to integrate them into the blockchain network.

[0008] The data analysis module is equipped with a data reprocessing unit, a data modeling unit, and a data visualization unit. The communication data is processed by the data reprocessing unit to generate dirty data and modeling data. After the data modeling unit obtains the modeling data, it generates different communication data models. The data visualization unit associates each communication data model with the communication data models and generates different image presentation methods.

[0009] The communication loading module is equipped with a loading queue and a queuing queue. The on-demand loading of communication data is achieved by increasing or decreasing the number of loading queues.

[0010] Furthermore, the process of selecting the data source and optimizing the sampling rate includes:

[0011] The data source selection includes both public and private communication data.

[0012] When the selected data source is public communication data, the communication data is collected through the set collection stack; when the selected data source is private communication data, the communication data is collected through the set encrypted channel.

[0013] When the communication data is collected in the acquisition stack or encrypted channel, different sampling rates are set, and sampling rate optimization is performed by using different sampling rates. The sampling rates include low-pass sampling rate, medium-pass sampling rate and high-pass sampling rate, and different sampling rates are set with different frequencies and rates of change.

[0014] Furthermore, the data stream processing and network transmission protocol optimization process includes:

[0015] When communication data is transmitted in the acquisition stack or encrypted channel, it is converted into a data stream for transmission. The data stream is configured with a standard data stream interception range. Data streams within the standard data stream interception range are intercepted for data filtering and data cleaning, while data streams outside the standard data stream interception range are directly filtered out.

[0016] The network transmission protocols include UDP and TCP. UDP is used for real-time transmission of communication data, while TCP is used for integrity transmission of communication data. A timeout retransmission mechanism is associated with the network transmission protocols. The timeout retransmission mechanism sets a data retransmission time, and the communication data is retransmitted every time the data retransmission time has elapsed.

[0017] Furthermore, the data processing module is used to perform calculation optimization scheduling, encryption operations, and node verification on the communication data.

[0018] Furthermore, the computational optimization scheduling process includes:

[0019] The computational optimization scheduling is performed through intelligent edge computing. After acquiring communication data in real time, the intelligent edge computing uploads the communication data to the set edge device. The edge device is equipped with a data capacity analysis unit. After analyzing the communication data, the data capacity analysis unit generates high-capacity data and low-capacity data. The high-capacity data is uploaded to the cloud for off-site computing, while the low-capacity data is computed locally in the computing module built into the edge device.

[0020] Furthermore, the encryption operation process includes:

[0021] The encryption operation is dynamic encryption, which converts city blocks into block nodes, selects one as a base node and forms a diagonal, the diagonal divides the block node into four block parts, and applies asymmetric encryption, equal-length encoding encryption, digital signature encryption and hybrid encryption to the four block parts, sets a dynamic encryption time period, and swaps the blocks after each dynamic encryption time period.

[0022] Furthermore, the node verification process includes:

[0023] A number of block nodes that have undergone dynamic encryption operations are obtained. Each block node includes a block header, transaction information, and proof-of-work. The block header is checked. The block header includes a timestamp and a target difficulty. If the timestamp is within the allowed access timestamps and meets the set target difficulty, the transaction information is checked. The transaction information includes legal information and illegal information. Legal information is included and illegal information is removed. The proof-of-work includes the maximum hash value allowed to be incorporated into the blockchain network. If the hash value is within the maximum hash value, the block node is incorporated into the blockchain network. Otherwise, a new block node that meets the maximum hash value is generated and incorporated into the blockchain network.

[0024] Furthermore, the data analysis module includes a data reprocessing unit, a data modeling unit, and a data visualization unit;

[0025] The data reprocessing unit is used to reprocess communication data to generate dirty data and modeling data;

[0026] The data modeling unit generates different communication data models after acquiring the modeling data.

[0027] The data visualization unit is used to associate communication data models and present them as visual images.

[0028] Furthermore, the on-demand loading process includes:

[0029] The communication loading module is equipped with a loading queue and a queuing queue. It obtains the total capacity value of the communication data and the storage capacity limit of the loading queue. The loading queue is set with two states: "loaded" and "pending loading".

[0030] Get the number of loading queues in the "loaded" and "pending" states, and perform state transitions for the loading queues. Each time a state transition occurs, set an upper limit on the number of loading queue transitions. If loading queues exceeding this upper limit are loaded simultaneously, the communication data in the excess loading queues will be added to the queuing queue.

[0031] Compared with existing technologies, the beneficial effects of this invention are as follows: The sampling period or interval is dynamically adjusted according to changes in data traffic, making the sampled data more targeted; the timeout retransmission mechanism reduces packet loss, increases transmission throughput, and reduces data transmission latency; communication data is divided into high-capacity and low-capacity data through computational optimization scheduling, with high-capacity data uploaded to the cloud for off-site computation, reducing the computational load on local computing; asymmetric encryption, equal-length encoding encryption, digital signature encryption, and hybrid encryption are applied to the block nodes storing communication data, enhancing the storage security of communication data; the dynamic encryption time period further reduces the possibility of external data decryption; different communication data models generated based on modeling data meet the communication needs of city blocks with different spans; only two loading queues are allowed to store communication data at the same time, with any exceeding two being transferred to a queuing queue, avoiding congestion caused by too many loading queues, thus realizing the processing of communication data for smart cities. Attached Figure Description

[0032] Figure 1 This is a schematic diagram of the present invention. Detailed Implementation

[0033] like Figure 1 As shown, a 5G Internet of Things (IoT) communication data processing system based on a smart city includes a communication center, which is communicatively connected to a data acquisition module, a data processing module, a data analysis module, and a communication loading module.

[0034] The data acquisition module is used to collect communication data, which is a summary of communication data from several city blocks. The acquisition of communication data is optimized through data source selection, sampling rate optimization, data stream processing, and network transmission protocol optimization.

[0035] The data processing module is used to perform computational optimization scheduling, encryption operations, and node verification on communication data. It optimizes and schedules communication data through intelligent edge computing, selects whether to perform local or remote computation on communication data through intelligent edge computing, performs dynamic encryption operations on communication data of several city blocks through blockchain technology, and verifies the nodes of the blocks where the communication data is stored to integrate them into the blockchain network.

[0036] The data analysis module is equipped with a data reprocessing unit, a data modeling unit, and a data visualization unit. The communication data is processed by the data reprocessing unit to generate dirty data and modeling data. After the data modeling unit obtains the modeling data, it generates different communication data models. The data visualization unit associates each communication data model with the communication data models and generates different image presentation methods.

[0037] The communication loading module is equipped with a loading queue and a queuing queue. The on-demand loading of communication data is achieved by increasing or decreasing the number of loading queues.

[0038] The data acquisition module is used to collect communication data, which is generated by aggregating communication data from several city blocks.

[0039] Obtain city blocks and assign them numbers, denoted as i, i = 1, 2, 3, ..., n, where n is a natural number. Each city block consists of several block storage points, which are assigned numbers, denoted as j, j = 1, 2, 3, ..., m, where m is a natural number.

[0040] Obtain the city block number i and the block storage point number j, construct a two-dimensional array, denoted as Arr, where Arr = a[i][j], a is a common symbol for the array, and each a[i][j] corresponds to a part of the stored communication data;

[0041] Specifically, the process of selecting the data source and optimizing the sampling rate includes:

[0042] The data sources included in the communication data are: communication data of ordinary citizens, communication data of government agencies, communication data of school administrations, and communication data of high-tech enterprises; among them, communication data of ordinary citizens and communication data of school administrations are public communication data, while communication data of high-tech enterprises and communication data of government agencies are private communication data.

[0043] When the selected data source is public type communication data, the communication data corresponding to the two-dimensional array Arr will be collected through the set acquisition stack;

[0044] When the selected data source is private communication data, the communication data corresponding to the two-dimensional array Arr will be collected through the set encrypted channel;

[0045] When the communication data is collected in the acquisition stack or encrypted channel, different sampling rates are set, and the sampling rate is optimized by using different sampling rates;

[0046] The frequency range of the communication data is 300-2500 Hz, and the rate of change ranges from 1000 to 4600 bits / s. The frequency range is divided into segments: communication data in the 300-1000 Hz range is low frequency, communication data in the 1000-1800 Hz range is medium frequency, and communication data in the 1800-2500 Hz range is high frequency. The rate of change range is also divided into segments: 1000-1900 bits / s is low rate of change, 1900-3300 bits / s is medium rate of change, and 3300-4500 bits / s is high rate of change.

[0047] The sampling rate is set to low-pass sampling rate, medium-pass sampling rate and high-pass sampling rate;

[0048] The relationship between sampling rate, frequency, and rate of change is as follows:

[0049] Low frequency—low rate of change—low pass sampling rate;

[0050] Medium frequency — medium rate of change — medium pass sampling rate;

[0051] High frequency—high rate of change—high-pass sampling rate;

[0052] It should be noted that the sampling rate can be dynamically adjusted. For example, the sampling period or interval can be dynamically adjusted according to changes in data traffic in order to collect as much adaptive, high-value data as possible.

[0053] Specifically, the data stream processing and network transmission protocol optimization process includes:

[0054] When communication data is transmitted in the acquisition stack or encrypted channel, it is transmitted in a way that is converted into a data stream, and the data stream is configured with a standard data stream interception range.

[0055] Data streams within the standard data stream interception range are intercepted for data filtering and cleaning, while data streams outside the standard data stream interception range are directly screened out and removed.

[0056] The network transmission protocol includes UDP and TCP protocols. UDP is used for real-time transmission of communication data, and TCP is used for integrity transmission of communication data. Real-time transmission means setting a real-time period unit, and the transmission is updated whenever the real-time period unit is updated. Integrity transmission means acquiring and transmitting communication data from all city blocks.

[0057] The network transmission protocol is associated with a timeout retransmission mechanism, which is set with a data retransmission time, denoted as T. The communication data is retransmitted every T interval.

[0058] It should be noted that the timeout retransmission mechanism reduces the packet loss rate of communication data, increases the throughput of communication data transmission, and reduces the latency of data transmission.

[0059] The data processing module is used to perform calculation, optimization, scheduling, encryption, and node verification on communication data.

[0060] Optimize and schedule communication data computation through intelligent edge computing;

[0061] Encrypting communication data across several city blocks using blockchain technology;

[0062] The nodes storing communication data are verified to integrate them into the blockchain network.

[0063] Specifically, the computational optimization scheduling process includes:

[0064] The computational optimization scheduling is performed through intelligent edge computing. After acquiring communication data in real time, the intelligent edge computing uploads the communication data to edge devices, which include edge routers, edge gateways, and edge servers.

[0065] The edge device is equipped with a data capacity analysis unit. After analyzing the communication data, the data capacity analysis unit generates high-capacity data and low-capacity data. The high-capacity data is uploaded to the cloud for off-site computing, while the low-capacity data is processed locally in the computing module built into the edge device.

[0066] Specifically, the encryption operation includes:

[0067] The encryption operation is dynamic encryption. Several city blocks are converted into block nodes through blockchain technology. A two-dimensional array Arr of city blocks corresponding to the block nodes is obtained. The block nodes in Arr = a[i][j] where i and j are equal are taken as the base nodes.

[0068] Connect the reference nodes sequentially to form a diagonal. Arrange the block nodes into a rectangle. The diagonal divides the rectangle into two blocks, labeled A and B respectively. Divide the blocks represented by A and B into four parts with equal areas: A1, A2, B1, and B2.

[0069] Asymmetric encryption is used for the block nodes included in A1, equal-length encoding encryption is used for the block nodes included in A2, digital signature encryption is used for the block nodes included in B1, and a hybrid encryption of symmetric and asymmetric encryption is used for the block nodes included in B2.

[0070] Set the dynamic encryption time period, denoted as T. 动 T 动 = [t1, t2], after each dynamic encryption time period, A1 and A2, B1 and B2 are swapped;

[0071] Specifically, the node verification process includes:

[0072] Obtain several block nodes that have undergone dynamic encryption operations. The block nodes include block headers, transaction information, and proof of work. The block nodes are generated from old blocks and new blocks.

[0073] First, check the block header of the new block. The block header is set to a fixed length header, which includes the block header of the previous old block, the timestamp, and the target difficulty. If the previous block header belongs to a trusted block node, the timestamp is within the allowed access timestamp and meets the set target difficulty, then the transaction information is checked. The transaction information includes legal information and illegal information. After verification, legal information is included and illegal information is removed.

[0074] After the transaction information is verified, the proof-of-work verification continues to obtain the field data string of the new block. The field data string has a corresponding hash value. The proof-of-work includes the maximum hash value allowed to be merged into the blockchain network. When the hash value corresponding to the field data string is within the maximum hash value, the block node is merged into the blockchain network. Otherwise, a new block node that can satisfy the hash value of the field data string is generated until the newly generated block node can be merged into the blockchain network.

[0075] It should be noted that by optimizing the scheduling through computation, communication data is divided into high-capacity data and low-capacity data. High-capacity data is uploaded to the cloud for off-site computation, reducing the computational load on local computing. Asymmetric encryption, equal-length encoding encryption, digital signature encryption, and hybrid encryption are applied to the block nodes storing communication data, enhancing the storage security of communication data. The dynamic encryption time period setting further reduces the possibility of external data decryption. The node verification operation requires the block nodes to be reviewed before they can be incorporated into the blockchain network. Block nodes that fail the review are processed to allow them to be incorporated into the blockchain network.

[0076] The data analysis module is equipped with a data reprocessing unit, a data modeling unit, and a data visualization unit;

[0077] The data reprocessing unit is used to reprocess communication data to generate dirty data and modeling data;

[0078] The data modeling unit generates different communication data models after acquiring the modeling data.

[0079] The data visualization unit is used to associate the communication data model and present it as a visual image;

[0080] The communication data is first filtered into preliminary screening data by the data reprocessing unit. Then, the preliminary screening data is passed through the set dirty data calibration range and modeling data calibration range to generate dirty data and modeling data respectively. The dirty data includes redundant data, incomplete data and external attack data.

[0081] Acquire modeling data and generate a communication data model based on the modeling data. The communication data model includes a long-distance communication model, a medium-to-long-distance communication model, and a short-distance communication model.

[0082] The long-distance communication model spans 8-10 city blocks, the medium-to-long-distance communication model spans 4-6 city blocks, while the short-distance communication model can only conduct cross-border communication between adjacent city blocks.

[0083] After the data visualization unit accesses different types of communication data models, it establishes a one-to-one association and generates a visualization image for each data communication model.

[0084] The visualization images include histograms, line graphs, tree diagrams, and pie charts. After the visualization images are generated, they are received by data terminals in different city blocks. Users of communication data within a city block can obtain the visualization images by communicating with the data terminal of that city block through portable mobile devices.

[0085] It should be noted that the data reprocessing unit filters and generates initial screening data, and generates dirty data and modeling data respectively according to the set dirty data calibration range and modeling data calibration range. Dirty data is directly removed, reducing the impact of dirty data. Different communication data models generated based on the modeling data meet the communication needs of urban blocks with different spans. The visualization images generated by the data visualization unit can be received and acquired by portable mobile terminals, allowing communication data to be obtained intuitively in a visual way.

[0086] The communication loading module is equipped with a loading queue and a queuing queue. Communication data is loaded on demand by increasing or decreasing the number of loading queues.

[0087] Specifically, the on-demand loading process includes:

[0088] The communication loading module is equipped with a loading queue and a queuing queue. After acquiring communication data, it calculates the total capacity of the communication data, obtains the number of loading queues, denoted as Num, obtains the upper limit of the storage capacity of a single loading queue, denoted as R, and records the total capacity value of the communication data as Sum. The loading queue is set with two states: "loaded" and "pending loading". The number of loading queues in the "loaded" state is obtained, denoted as num1, and the number of loading queues in the "pending loading" state is obtained, denoted as num2.

[0089] Based on the total communication data capacity Sum, the number of loading queues in the "loaded" state num1, and the storage capacity limit R of a single loading queue, the number of loading queues in the "pending loading" state num2 is increased or decreased.

[0090] If Sum > num1 × R, the loading queue in the "pending loading" state is changed to the "loaded" state, i.e., num2++, until sum = num1 × R is satisfied;

[0091] If Sum = num1 × R, it means that the number of loading queues in the "loaded" state is just enough to store the total capacity of the communication data, and there is no need to convert the loading queues in the "pending loading" state.

[0092] If Sum < num1 × R, the loading queue in the "loaded" state is changed to the "pending loading" state, i.e., num1 --, until sum = num1 × R is satisfied;

[0093] During the process of increasing or decreasing the number of loading queues, only two loading queues are allowed to switch between the "loaded" and "pending" states at the same time. If more than two loading queues are loaded at the same time, the communication data stored in the more than two loading queues will be added to the queuing queue.

[0094] It should be noted that communication data is loaded on demand by setting up loading queues and queuing queues. The loading queue has two states: "loaded" and "pending loading". The switching between the two states prevents the loading queue from having too many data tables at the same time. Only two data tables are allowed to be stored in the loading queue at the same time. Any data exceeding two are transferred to the queuing queue, thus avoiding loading congestion caused by too many loading queues.

[0095] The above embodiments are only used to illustrate the technical methods of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical methods of the present invention without departing from the spirit and scope of the present invention.

Claims

1. A 5G Internet of Things (IoT) communication data processing system based on smart cities, comprising a communication center, characterized in that, The communication center is connected to a data acquisition module, a data processing module, a data analysis module, and a communication loading module. The data acquisition module is used to collect communication data, which is a summary of communication data from several city blocks. The acquisition of communication data is optimized through data source selection, sampling rate optimization, data stream processing, and network transmission protocol optimization. The data processing module is used to perform computational optimization scheduling, encryption operations, and node verification on communication data. It optimizes and schedules communication data through intelligent edge computing, selects whether to perform local or remote computation on communication data through intelligent edge computing, performs dynamic encryption operations on communication data of several city blocks through blockchain technology, and verifies the nodes of the blocks where the communication data is stored to integrate them into the blockchain network. The data analysis module is equipped with a data reprocessing unit, a data modeling unit, and a data visualization unit. The communication data is processed by the data reprocessing unit to generate dirty data and modeling data. After the data modeling unit obtains the modeling data, it generates different communication data models. The data visualization unit associates each communication data model with the communication data models and generates different image presentation methods. The communication loading module is equipped with a loading queue and a queuing queue. The on-demand loading of communication data is achieved by increasing or decreasing the number of loading queues. The on-demand loading process includes: The communication loading module is equipped with a loading queue and a queuing queue. It obtains the total capacity value of communication data and the storage capacity limit of the loading queue. The loading queue has two states: "loaded" and "pending loading". It obtains the number of loading queues in the "loaded" and "pending loading" states and performs state transitions. Each time a state transition occurs, an upper limit is set for the number of loading queue transitions. If loading queues exceeding this upper limit are loaded simultaneously, the communication data in the excess loading queues are added to the queuing queue.

2. The 5G IoT communication data processing system based on smart cities according to claim 1, characterized in that, The process of selecting the data source and optimizing the sampling rate includes: The data source selection includes both public and private communication data. When the selected data source is public communication data, the communication data is collected through the set collection stack; when the selected data source is private communication data, the communication data is collected through the set encrypted channel. When the communication data is collected in the acquisition stack or encrypted channel, different sampling rates are set, and sampling rate optimization is performed by using different sampling rates. The sampling rates include low-pass sampling rate, medium-pass sampling rate and high-pass sampling rate, and different sampling rates are set with different frequencies and rates of change.

3. The 5G IoT communication data processing system based on smart cities according to claim 2, characterized in that, The data stream processing and network transmission protocol optimization process includes: When communication data is transmitted in the acquisition stack or encrypted channel, it is converted into a data stream for transmission. The data stream is configured with a standard data stream interception range. Data streams within the standard data stream interception range are intercepted for data filtering and data cleaning, while data streams outside the standard data stream interception range are directly filtered out. The network transmission protocols include UDP and TCP. UDP is used for real-time transmission of communication data, while TCP is used for integrity transmission of communication data. A timeout retransmission mechanism is associated with the network transmission protocols. The timeout retransmission mechanism sets a data retransmission time, and the communication data is retransmitted every time the data retransmission time has elapsed.

4. A 5G IoT communication data processing system based on a smart city according to claim 1, characterized in that, The data processing module is used to perform calculations, optimization scheduling, encryption operations, and node verification on communication data.

5. A 5G IoT communication data processing system based on a smart city according to claim 4, characterized in that, The computational optimization scheduling process includes: The computational optimization scheduling is performed through intelligent edge computing. After acquiring communication data in real time, the intelligent edge computing uploads the communication data to the set edge device. The edge device is equipped with a data capacity analysis unit. After analyzing the communication data, the data capacity analysis unit generates high-capacity data and low-capacity data. The high-capacity data is uploaded to the cloud for off-site computing, while the low-capacity data is computed locally in the computing module built into the edge device.

6. A 5G IoT communication data processing system based on a smart city according to claim 4, characterized in that, The encryption process includes: The encryption operation is dynamic encryption, which converts city blocks into block nodes, selects one as a base node and forms a diagonal, the diagonal divides the block node into four block parts, and applies asymmetric encryption, equal-length encoding encryption, digital signature encryption and hybrid encryption to the four block parts, sets a dynamic encryption time period, and swaps the blocks after each dynamic encryption time period.

7. A 5G IoT communication data processing system based on a smart city according to claim 6, characterized in that, The node verification process includes: A number of block nodes that have undergone dynamic encryption operations are obtained. Each block node includes a block header, transaction information, and proof-of-work. The block header is checked. The block header includes a timestamp and a target difficulty. If the timestamp is within the allowed access timestamps and meets the set target difficulty, the transaction information is checked. The transaction information includes legal information and illegal information. Legal information is included and illegal information is removed. The proof-of-work includes the maximum hash value allowed to be incorporated into the blockchain network. If the hash value is within the maximum hash value, the block node is incorporated into the blockchain network. Otherwise, a new block node that meets the maximum hash value is generated and incorporated into the blockchain network.

8. A 5G IoT communication data processing system based on a smart city according to claim 1, characterized in that, The data analysis module is equipped with a data reprocessing unit, a data modeling unit, and a data visualization unit; The data reprocessing unit is used to reprocess the communication data to generate dirty data and modeling data. The data modeling unit generates different communication data models after acquiring the modeling data. The data visualization unit is used to associate the communication data models and present them with visual images.

Images