Sensor network equipment data return dimension reduction compression method, device and equipment
By compressing and diminishing the sensor network equipment data based on the preset strategy of the data type application in the computing gateway, the high load problem brought by the data returned by the data in the sensor collection network, realizing the reduction of the data volume and the realization Optimization of network load.
Patent Information
- Application Number
- CN202411430652.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-14
- Publication Date
- 2025-01-28
- Estimated Expiration
- 2044-10-14
AI Technical Summary
In the sensing acquisition network, sensor nodes collect data on terminal devices through fixed frequency and high frequency and directly pass the original data back, resulting in a large amount of data bandwidth and processing load, and even the overall failure of the network.
In Ziwang convergence of the computing gateway, the data to receive the processing data sent by the sensor collection node, according to the data type (health state data or statistical filtering data) applied to the preset compression strategy or counter statistical strategy, compress and reduce the dimension reduction of the data Store or delete data to reduce loads.
Through data compression and dimension reduction, the amount of data storage and uploaded to the cloud server can be effectively reduced, the load of data bandwidth occupation and data processing load is reduced, and the overall network failure is avoided.
Smart Images

Figure CN118945238B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of network data compression, and in particular to a method, device and equipment for dimensionality reduction compression of sensor network device data return. Background Art
[0002] At present, the self-propagation sniffing technology based on sensor processes has been applied to network expansion scenarios. The sensor collection network built based on the above technology includes multiple sub-networks, each of which includes multiple sensor nodes (each sensor node is deployed on a terminal device), and each sensor node is used to collect multiple types of device data (such as device working status data, etc.) from the terminal device where it is deployed.
[0003] All sensors collect data of various types of devices from the deployed terminal devices at a fixed frequency and high frequency. The amount of raw data collected is huge. If all the raw data collected by all sensors is directly sent back to the data analysis center (which can also be regarded as a backend server, cloud server, etc.) for storage and processing, it will bring a lot of data bandwidth and data processing load. Moreover, if the data storage and computing load of the data analysis center cannot be borne, the overall sensor collection network will fail. Summary of the invention
[0004] The embodiments of the present invention provide a sensor network device data feedback dimensionality reduction compression method, device and equipment, aiming to solve the problem in the prior art that when all sensors in the sensor acquisition network collect multi-type device data from the deployed terminal devices at a fixed frequency and high frequency and directly return the original data in full to the data analysis center for storage and processing, it will bring a large amount of data bandwidth and data processing load, and even cause the overall failure of the sensor acquisition network.
[0005] In a first aspect, an embodiment of the present invention provides a method for reducing the dimension of sensor network device data transmission, which is applied to a subnet convergence computing gateway, wherein the subnet convergence computing gateway is connected to multiple sensor acquisition nodes in a communication manner, and the subnet convergence computing gateway is also connected to a cloud server in a communication manner; the method for reducing the dimension of sensor network device data transmission includes:
[0006] Receive the data to be processed sent by the sensor acquisition node, and obtain the data type of the data to be processed;
[0007] If it is determined that the data type is health status data, and it is determined that the health status data corresponding to the data to be processed does not exceed the preset node survival threshold, the data to be processed is compressed and stored based on a preset health status data compression strategy; wherein the health status data compression strategy is used to store the health status data of the data to be processed in the form of a status identification value when the health status data of the data to be processed does not exceed the preset node survival threshold;
[0008] If it is determined that the data type is statistical filtering type data, counting statistics are performed on the data to be processed based on a preset counter statistical strategy, and the data to be processed is deleted after the counting statistics are completed.
[0009] In a second aspect, an embodiment of the present invention further provides a sensor network device data return dimension reduction compression device, which is configured in a subnet convergence computing gateway, the subnet convergence computing gateway is communicatively connected to a plurality of sensor acquisition nodes, and the subnet convergence computing gateway is also communicatively connected to a cloud server; the sensor network device data return dimension reduction compression device comprises:
[0010] A data type determination unit, used to receive the data to be processed sent by the sensor acquisition node, and obtain the data type of the data to be processed;
[0011] A first dimensionality reduction compression unit is used to compress and store the data to be processed based on a preset health status data compression strategy if it is determined that the data type is health status data and it is determined that the health status data corresponding to the data to be processed does not exceed a preset node survival threshold; wherein the health status data compression strategy is used to store the health status data of the data to be processed in the form of a status identification value when the health status data of the data to be processed does not exceed a preset node survival threshold;
[0012] The second dimensionality reduction compression unit is used to perform counting statistics on the data to be processed based on a preset counter statistical strategy if it is determined that the data type is statistical filtering type data, and delete the data to be processed after completing the counting statistics.
[0013] In a third aspect, an embodiment of the present invention further provides a computer device, which includes a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the method described in the first aspect is implemented.
[0014] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the method described in the first aspect above can be implemented.
[0015] The embodiment of the present invention provides a sensor network device data backhaul dimensionality reduction compression method, device and equipment, which are applied to a subnet convergence computing gateway, the subnet convergence computing gateway is connected to multiple sensor acquisition nodes, and the subnet convergence computing gateway is also connected to a cloud server; the method includes: receiving the data to be processed sent by the sensor acquisition node, and obtaining the data type of the data to be processed; if the data type is determined to be health status data, and it is determined that the health status data corresponding to the data to be processed does not exceed the preset node survival threshold, then compressing and storing the data to be processed based on a preset health status data compression strategy; wherein the health status data compression strategy is used to store the data in the form of a state identification value when the health status data of the data to be processed does not exceed the preset node survival threshold; if the data type is determined to be statistical filtering data, then counting and statistics are performed on the data to be processed based on a preset counter statistics strategy, and the data to be processed is deleted after the counting and statistics are completed. The embodiment of the present invention can compress and reduce the dimensions of multiple types of data to be processed collected by multiple sensor acquisition networks based on the health status data compression strategy or the counter statistics strategy, effectively reducing the amount of local data storage and the amount of data uploaded to the cloud server later, and reducing the data bandwidth occupancy and the load of data processing load. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.
[0017] Figure 1 A schematic diagram of an application scenario of the sensor network device data return dimension reduction compression method provided by an embodiment of the present invention;
[0018] Figure 2 A schematic diagram of a flow chart of a method for reducing dimension and compressing sensor network device data backhaul provided by an embodiment of the present invention;
[0019] Figure 3 A schematic diagram of a sub-process of a method for reducing the dimension of sensor network device data transmission provided by an embodiment of the present invention;
[0020] Figure 4 A schematic diagram of the data structure of compressed sequence data in the sensor network device data return dimension reduction compression method provided by an embodiment of the present invention;
[0021] Figure 5 A schematic diagram of a sub-process of a method for reducing the dimension of sensor network device data transmission provided by an embodiment of the present invention;
[0022] Figure 6A schematic diagram of a sub-process of a method for reducing the dimension of sensor network device data transmission provided by an embodiment of the present invention;
[0023] Figure 7 A schematic diagram of a process of performing counting statistics on a piece of to-be-processed data of a statistical filtering type in a sensor network device data return dimension reduction compression method provided by an embodiment of the present invention;
[0024] Figure 8 A schematic block diagram of a sensor network device data return dimension reduction and compression device provided by an embodiment of the present invention;
[0025] Fig. 9 A schematic block diagram of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0026] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0027] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.
[0028] It should also be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.
[0029] It should be further understood that the term "and / or" used in the present description and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.
[0030] Please also refer to Figure 1 and Figure 2 ,in Figure 1 Schematic diagram of a scenario of a method for reducing dimension and compressing sensor network device data backhaul according to an embodiment of the present invention. Figure 2 FIG. 1 is a flow chart of a method for reducing the dimension of sensor network device data transmission provided by an embodiment of the present invention. Figure 1As shown, the sensor network device data backhaul dimensionality reduction and compression method provided in an embodiment of the present invention is applied to a subnet convergence computing gateway 10 (which can be understood as an edge server, edge gateway, etc.), and the subnet convergence computing gateway 10 is communicatively connected with a plurality of sensor acquisition nodes 20 (that is, it includes N sensor acquisition nodes 21, N is a positive integer greater than or equal to 1), and the subnet convergence computing gateway 10 is also communicatively connected with a cloud server 30 (the cloud server can also be a background server or other server that can receive data).
[0031] like Figure 2 As shown, the method includes the following steps S110-S130.
[0032] S110, receiving the data to be processed sent by the sensor acquisition node, and obtaining the data type of the data to be processed.
[0033] In this embodiment, the technical solution is described with the subnet aggregation computing gateway as the execution subject. Figure 1 As shown, the subnet convergence computing gateway 10 and multiple sensor acquisition nodes 20 are in the same sensor acquisition subnet (and the complete sensor acquisition network includes multiple sensor acquisition subnets). When the subnet convergence computing gateway 10 is used as the core device for processing multiple sensor acquisition nodes 20, it is mainly used to perform data dimension reduction compression on the original full data uploaded by multiple sensor acquisition nodes 20, and then upload it to the cloud server 30, so as to converge the data transmission and processing pressure of the sensor acquisition subnet to the subnet convergence computing gateway 10, and then the data volume uploaded by the subnet convergence computing gateway 10 to the cloud server 30 is greatly reduced, which effectively reduces the bandwidth occupied by the cloud server in receiving data and the load of data processing.
[0034] For example, in the embodiment of the present application, any one of the sensor collection nodes 21 of the multiple sensor collection nodes 20 uploads a piece of data to be processed to the subnet convergence computing gateway 10 for dimensionality reduction and compression processing. When the subnet convergence computing gateway 10 receives a piece of data to be processed sent by the sensor collection node, it is necessary to first determine its specific data type. After that, in combination with the data type of the data to be processed, the corresponding data dimensionality reduction and compression strategy is called for processing, thereby performing dimensionality reduction and compression on the data to be processed.
[0035] S120. If it is determined that the data type is health status data, and it is determined that the health status data corresponding to the data to be processed does not exceed a preset node survival threshold, the data to be processed is compressed and stored based on a preset health status data compression strategy.
[0036] The health status data compression strategy is used to store the health status data of the data to be processed in the form of a status identification value when the health status data of the data to be processed does not exceed a preset node survival threshold.
[0037] In this embodiment, if the data to be processed is one of the two data types, namely health status data and statistical filtering data, it is the object that needs to be selected and processed for dimensionality reduction in this application. Specifically, if it is determined that the data type is health status data, and it is determined that the health status data corresponding to the data to be processed does not exceed the preset node survival threshold, it means that the data to be processed is a type of data that can be stored by the subnet aggregation computing gateway. Of course, if the original data of the data to be processed is directly stored in the subnet aggregation computing gateway, the effect of data dimensionality reduction and compression cannot be achieved. At this time, it is necessary to compress and store the data to be processed based on the preset health status data compression strategy, so as to realize the storage of the data to be processed in the specified storage space in the subnet aggregation computing gateway.
[0038] In one embodiment, if Figure 3 As shown, step S120 includes:
[0039] S121, determining target compressed sequence data corresponding to the sensor acquisition node from a plurality of local compressed sequence data based on the health status data compression strategy; wherein the target compressed sequence data has at least one free storage bit;
[0040] S122: If it is determined that the health status data does not exceed the node survival threshold and corresponds to normal status data, the to-be-processed data is converted into a first preset status identification value based on the health status data compression strategy, and stored in the target compression sequence data, and the target compression sequence data is updated;
[0041] S123. If it is determined that the health status data does not exceed the node survival threshold and corresponds to abnormal status data, the data to be processed will be converted into a second preset status identification value based on the health status data compression strategy, stored in the target compression sequence data, and the target compression sequence data will be updated.
[0042] In this embodiment, if the sensor acquisition node in step S110 uploads a piece of data to be processed every interval T1 (such as T1=5S, 10S, 15S, etc., of course, it is not limited to the above example in the specific implementation, and can be flexibly set according to the actual needs of the user) as an example, more specifically, taking T1=5S as an example, the sensor acquisition node collects 1 piece of health status data (which can also be understood as survival health status data) of the sensor acquisition node every 5 seconds, so that the sensor acquisition node can collect 12 pieces of health status data in 1 minute. Among them, each health status data includes at least indicators such as CPU load rate (i.e., central processing unit load rate), communication delay, etc.
[0043] After 12 pieces of health status data are acquired in the subnet aggregation computing gateway in one minute, at each time point when the health status data is acquired, it is necessary to compress and reduce the dimension of the data to be processed based on the health status data compression strategy. For example, the data to be processed including multi-dimensional information is compressed into a one-dimensional state identification value and then stored. Of course, in order to more accurately obtain the storage location of the data to be processed after it is compressed into a one-dimensional state identification value, the target compressed sequence data corresponding to the sensor acquisition node is first determined from the multiple local compressed sequence data. For example, the subnet aggregation computing gateway includes multiple compressed sequence data, each of which includes a 12-bit state identification value, and is used to store the reduced-dimensional compressed state identification value corresponding to the 12 pieces of health status data uploaded by the same sensor acquisition node within 1 minute according to the acquisition cycle of 5S, such as Figure 4 It is illustrated in the figure that a compressed sequence data can store multiple state identification values. When each sensor acquisition node uploads the data to be processed, the subnet convergence computing gateway can quickly determine the target compressed sequence data corresponding to the sensor acquisition node. Moreover, each compressed sequence data is not limited to storing only 12 state identification values as in the above example, and can be determined according to the data acquisition interval T1 of the sensor acquisition node, specifically the rounding result of 60 / T1, 90 / T1, 120 / T1, etc. (rounding up or rounding down is possible). The data acquisition interval T1 of the sensor acquisition node determines that the maximum number of digits of the state identification value in each compressed sequence data can be adjusted according to the actual situation to achieve compression of the storage results.
[0044] Afterwards, if it is determined that the health status data does not exceed the node survival threshold and corresponds to normal status data, the data to be processed will be converted into a first preset state identification value (for example, the first preset state identification value is 1) based on the health status data compression strategy, and the first preset state identification value will be stored in the target compression sequence data, and the target compression sequence data will be updated. Afterwards, the data type of the data to be processed collected and uploaded by the sensor acquisition node at the next data acquisition time point is continuously obtained, and when it is determined that the data type is health status data and the health status data corresponding to the data to be processed does not exceed the preset node survival threshold, the data to be processed is also compressed and stored with reference to the above-mentioned dimensionality reduction compression process.
[0045] If it is determined that the health status data does not exceed the node survival threshold and corresponds to abnormal status data, it will mean that although the health status data is abnormal, it can be determined that the sensor node is in a survival state. At this time, it is still necessary to convert the data to be processed into a second preset state identification value (for example, the second preset state identification value is 0) based on the health status data compression strategy, and store the second preset state identification value in the target compressed sequence data, and update the target compressed sequence data. It can be seen that through the above method, the subnet convergence computing gateway realizes dimensionality reduction compression for the data to be processed uploaded by each sensor acquisition node, the data type of which is health status data and the corresponding health status data does not exceed the preset node survival threshold, and converts it into a one-dimensional first preset state identification value or a second preset state identification value for storage.
[0046] In one embodiment, if Figure 5 As shown, step S121 includes:
[0047] S1211, obtaining the current sensor node ID number corresponding to the data to be processed;
[0048] S1212, determining compressed sequence data having the ID number of the current sensor node from a plurality of local compressed sequence data to form a candidate compressed sequence data set;
[0049] S1213. Based on the health status data compression strategy, filter and obtain the compressed sequence data with at least one free storage bit in the candidate compressed sequence data set and written as the last data to be processed corresponding to the current sensor node ID number, as the target compressed sequence data.
[0050] In this embodiment, the subnet aggregation computing gateway can accurately parse and obtain the current sensor node ID number corresponding to each piece of data to be processed uploaded by each sensor acquisition node, and the multiple compressed sequence data local to the subnet aggregation computing gateway also correspond to a sensor node ID number for each compressed sequence data. Therefore, the current sensor node ID number can be used as a screening condition to determine the compressed sequence data with the current sensor node ID number from the multiple local compressed sequence data to form a candidate compressed sequence data set.
[0051] Generally, one compressed sequence data of the same sensor acquisition node's to-be-processed data in the subnet aggregation computing gateway cannot fully meet the actual storage requirements. It is necessary to continuously create compressed sequence data corresponding to the sensor acquisition node in the subnet aggregation computing gateway, and convert multiple to-be-processed data that meet the conditions (that is, the data type is health status data and the corresponding health status data is determined to not exceed the preset node survival threshold) into the first preset state identification value or the second preset state identification value for storage in chronological order. This requires that the compressed sequence data created for the same sensor acquisition node must have the same sensor node ID number.
[0052] Moreover, when the data to be processed uploaded by the same sensor collection node at the previous data collection time point has been stored in one of the bits of a compressed sequence data in chronological order, such as Figure 4 In the example of a compressed sequence data that can store multiple state identification values, the subnet aggregation computing gateway uploads the last data to be processed at the last data collection time point for the sensor collection node AA1 and performs dimension reduction compression processing and stores it in Figure 4 The corresponding compressed sequence data is the second bit. Since the second bit in the compressed sequence data has not yet reached the end position corresponding to the 12th bit in the compressed sequence data, the sensor acquisition node AA1 uploads the data to be processed at this time and performs dimension reduction compression processing and stores it in the following manner: Figure 4 The third position in the corresponding compressed sequence data. It can be seen that based on the above method, the target compressed sequence data corresponding to the data to be processed can be quickly and accurately determined in the subnet convergence computing gateway as the compressed data storage location.
[0053] In one embodiment, if Figure 5 As shown, after step S1212, the following steps are also included:
[0054] S1214: If it is determined that each compressed sequence data in the candidate compressed sequence data set is in a fully stored state, then newly created compressed sequence data is created based on the current sensor node ID number, and the newly created compressed sequence data is used as the target compressed sequence data.
[0055] In this embodiment, if it is determined that each compressed sequence data in the candidate compressed sequence data set is in a full storage state, it means that the sensor acquisition node uploaded the last data to be processed at the last data acquisition time point and stored the last bit in a compressed sequence data after dimensionality reduction compression processing, and the compressed sequence data can no longer store more digits of the first preset state identification value or the second preset state identification value. At this time, a new compressed sequence data can be created based on the current sensor node ID number, and the new compressed sequence data is used as the target compressed sequence data. It can be seen that based on the above method, the target compressed sequence data corresponding to the data to be processed can also be quickly and accurately determined in the subnet aggregation computing gateway as the compressed data storage location.
[0056] S130: If it is determined that the data type is statistical filtering data, count and calculate the data to be processed based on a preset counter statistical strategy, and delete the data to be processed after completing the count and calculation.
[0057] In this embodiment, if it is determined that the data type is statistical filtering data, it means that the data to be processed is a type of data that can be processed by the subnet convergence computing gateway. Since the greatest use of statistical filtering data is to be counted by the subnet convergence computing gateway rather than storing raw data, the data to be processed can be counted based on a preset counter statistics strategy, and the data to be processed can be deleted after the counting statistics are completed.
[0058] In one embodiment, the counter statistical strategy includes a preset cumulative counter and a feature counter, and the feature counter stores preset data feature screening conditions; Figure 6 As shown, step S130 includes:
[0059] S131, obtaining the current data cumulative count number of the cumulative counter, adding the data number corresponding to the data to be processed to the current data cumulative count number, and updating the current data cumulative count number;
[0060] S132, extracting feature data from the data to be processed based on the feature counter to obtain feature data to be processed;
[0061] S133. If it is determined that the feature data to be processed meets the preset data feature screening condition, the current cumulative number of feature data of the feature counter is obtained, the number of data corresponding to the data to be processed is accumulated to the current cumulative number of feature data, and the current cumulative number of feature data is updated.
[0062] In this embodiment, when the sensor acquisition node aggregates the pending data of the statistical filtering type data to the subnet aggregation computing gateway at a certain frequency, the subnet aggregation computing gateway does not directly transmit the pending data of the statistical filtering type data back to the data analysis center (i.e., the cloud server), but performs data filtering and counting locally. The pending data that needs to be filtered and counted is divided into two data types. One type is a cumulative counter without any filtering, which is used to record the number of all received data within the preset data upload cycle (i.e., the cumulative count number of current data). This type of data is used to analyze the proportion of volatility problems. The other type is a feature counter for risk monitoring feature quantities (which can also be understood as a filter counter), for example, recording the number of pending data that exceeds a certain data feature screening condition. Whenever the subnet aggregation computing gateway receives new data to be processed, the processing method of statistical filtering data and the processing method of health status data in the subnet aggregation computing gateway are different in that the cumulative counter as the first type of counter does not perform any feature extraction processing on the data to be processed, but instead accumulates the number of data corresponding to the data to be processed (generally 1, that is, multiple sub-data in one data to be processed are regarded as a complete data set, rather than multiple scattered sub-data) to the current data cumulative count number in the cumulative counter (specifically, +1 processing is performed), thereby updating the current data cumulative count number.
[0063] Afterwards, feature data extraction can be further performed on the data to be processed based on the feature counter to obtain feature data to be processed. For example, if the feature data to be processed is the number of times the communication delay exceeds 200ms and the CPU load exceeds 80% in the data to be processed, and it is determined that the feature data to be processed meets the preset data feature screening condition, it means that the data to be processed is a risk data that needs to be counted by the feature counter. At this time, the number of data corresponding to the data to be processed (generally 1) is accumulated and added to the current cumulative number of feature data of the feature counter (specifically, +1 processing is performed), thereby updating the current cumulative number of feature data. Among them, if it is determined that the feature data to be processed does not meet the preset data feature screening condition, there is no need for the feature counter to perform counting processing on the data to be processed, and it can be directly deleted. For example, when the cumulative counter and the feature counter perform counting statistics on a piece of data to be processed of a statistical filtering type, the specific processing process diagram is as follows. Figure 7As shown, if a piece of data to be processed does not meet the preset data feature screening condition, the cumulative count of the current data in the cumulative counter is processed by +1; if a piece of data to be processed meets the preset data feature screening condition, the cumulative count of the current feature data in the feature counter is processed by +1. It can be seen that after the counting processing of the above two counters, the original data to be processed is no longer saved, but can be directly deleted, which effectively reduces the amount of original data storage in the subnet convergence computing gateway.
[0064] In one embodiment, if Figure 6 As shown, after step S133, the following steps are also included:
[0065] S134, if it is determined that the time interval between the current system time and the last counter data sending time is equal to the preset data upload period, the current data cumulative count number corresponding to the cumulative counter and the current feature data cumulative number corresponding to the feature counter are sent to the cloud server;
[0066] S135. Reset the current data cumulative count and the current feature data cumulative count to 0.
[0067] In this embodiment, the preset cumulative counter and feature counter included in the subnet convergence computing gateway do not always perform unlimited cumulative statistics on data, but after a preset data upload cycle corresponding to the counter statistics strategy in the subnet convergence computing gateway ends (that is, the time interval between the current system time and the previous counter data sending time is determined to be equal to the preset data upload cycle), the current data cumulative count number corresponding to the cumulative counter and the current feature data cumulative number corresponding to the feature counter are sent to the cloud server, so that the performance analysis of the sensor acquisition node can be performed in the cloud server according to the current data cumulative count number and the current feature data cumulative count number obtained by the sensor acquisition node in a preset data upload cycle. The subnet convergence computing gateway uploads only the current data cumulative count number and the current feature data cumulative count number to the cloud server, which is a very small amount of data transmission, effectively reducing the network bandwidth occupied by the data transmission process. Moreover, after completing the upload process, the current data cumulative count number and the current feature data cumulative count number need to be reset to 0, so that the cumulative counter and the feature counter perform data counting processing for the next preset data upload cycle.
[0068] In one embodiment, after step S110, the method further includes:
[0069] If it is determined that the data type is health status data, and it is determined that the health status data corresponding to the data to be processed exceeds a preset node survival threshold, monitoring alarm information corresponding to the data to be processed is generated and sent to the cloud server.
[0070] In this embodiment, if Figure 3 and Figure 4 As shown, when the to-be-processed data of the sensor acquisition node is continuously compressed and stored based on the compressed sequence data, if the duration of a compressed sequence data to completely write the state identification value of all bits is used as a health status data acquisition cycle, when the to-be-processed data whose data type is health status data and the corresponding health status data is determined to exceed the preset node survival threshold is detected within a health status data acquisition cycle, the to-be-processed data is no longer compressed and converted into a state identification value and then written into the compressed sequence data, but the monitoring alarm information corresponding to the to-be-processed data is directly generated (the monitoring alarm information may include the original to-be-processed data and the preset alarm prompt information) and sent to the cloud server. In the above process, the cloud server does not need to disassemble and analyze the compressed health status data again, but the cloud server directly receives the monitoring alarm information and can directly perform subsequent linkage processing after the alarm, so the purpose of reducing the computational load of the state data restoration of the collected data is achieved.
[0071] It can be seen that the implementation embodiment of the method can compress and reduce the dimension of various types of processed data collected by multiple sensor acquisition networks based on the health status data compression strategy or the counter statistics strategy, effectively reducing the amount of local data storage and the amount of data subsequently uploaded to the cloud server, reducing data bandwidth occupancy and data processing load.
[0072] Figure 8 1 is a schematic block diagram of a sensor network device data return dimension reduction compression device provided by an embodiment of the present invention. Figure 8 As shown, corresponding to the above sensor network device data return dimension reduction compression method, the present invention also provides a sensor network device data return dimension reduction compression device 100. Figure 1 and Figure 8 As shown, the sensor network device data return dimension reduction compression device 100 is configured in a subnet convergence computing gateway 10, and the subnet convergence computing gateway 10 is connected to multiple sensor acquisition nodes 20 (i.e., including N sensor acquisition nodes 21, N is a positive integer greater than or equal to 1), and the subnet convergence computing gateway 10 is also connected to a cloud server 30 (the cloud server can also be a background server or other server that can receive data). The sensor network device data return dimension reduction compression device 100 includes: a data type determination unit 110, a first dimension reduction compression unit 120, and a second dimension reduction compression unit 130.
[0073] The data type determination unit 110 is used to receive the data to be processed sent by the sensor acquisition node and obtain the data type of the data to be processed.
[0074] In this embodiment, the technical solution is described with the subnet aggregation computing gateway as the execution subject. Figure 1 As shown, the subnet convergence computing gateway 10 and multiple sensor acquisition nodes 20 are in the same sensor acquisition subnet (and the complete sensor acquisition network includes multiple sensor acquisition subnets). When the subnet convergence computing gateway 10 is used as the core device for processing multiple sensor acquisition nodes 20, it is mainly used to perform data dimension reduction compression on the original full data uploaded by multiple sensor acquisition nodes 20, and then upload it to the cloud server 30, so as to converge the data transmission and processing pressure of the sensor acquisition subnet to the subnet convergence computing gateway 10, and then the data volume uploaded by the subnet convergence computing gateway 10 to the cloud server 30 is greatly reduced, which effectively reduces the bandwidth occupied by the cloud server in receiving data and the load of data processing.
[0075] For example, in the embodiment of the present application, any one of the sensor collection nodes 21 of the multiple sensor collection nodes 20 uploads a piece of data to be processed to the subnet convergence computing gateway 10 for dimensionality reduction and compression processing. When the subnet convergence computing gateway 10 receives a piece of data to be processed sent by the sensor collection node, it is necessary to first determine its specific data type. After that, in combination with the data type of the data to be processed, the corresponding data dimensionality reduction and compression strategy is called for processing, thereby performing dimensionality reduction and compression on the data to be processed.
[0076] The first dimensionality reduction compression unit 120 is used to compress and store the data to be processed based on a preset health status data compression strategy if it is determined that the data type is health status data and that the health status data corresponding to the data to be processed does not exceed a preset node survival threshold.
[0077] The health status data compression strategy is used to store the health status data of the data to be processed in the form of a status identification value when the health status data of the data to be processed does not exceed a preset node survival threshold.
[0078] In this embodiment, if the data to be processed is one of the two data types, namely health status data and statistical filtering data, it is the object that needs to be selected and processed for dimensionality reduction in this application. Specifically, if it is determined that the data type is health status data, and it is determined that the health status data corresponding to the data to be processed does not exceed the preset node survival threshold, it means that the data to be processed is a type of data that can be stored by the subnet aggregation computing gateway. Of course, if the original data of the data to be processed is directly stored in the subnet aggregation computing gateway, the effect of data dimensionality reduction and compression cannot be achieved. At this time, it is necessary to compress and store the data to be processed based on the preset health status data compression strategy, so as to realize the storage of the data to be processed in the specified storage space in the subnet aggregation computing gateway.
[0079] In one embodiment, the first dimension reduction compression unit 120 is specifically used for:
[0080] Determining target compressed sequence data corresponding to the sensor acquisition node from a plurality of local compressed sequence data based on the health status data compression strategy; wherein the target compressed sequence data has at least one free storage bit;
[0081] If it is determined that the health status data does not exceed the node survival threshold and corresponds to normal status data, the to-be-processed data is converted into a first preset status identification value based on the health status data compression strategy, and stored in the target compression sequence data, and the target compression sequence data is updated;
[0082] If it is determined that the health status data does not exceed the node survival threshold and corresponds to abnormal status data, the data to be processed will be converted into a second preset status identification value based on the health status data compression strategy, stored in the target compression sequence data, and the target compression sequence data will be updated.
[0083] In this embodiment, if the sensor acquisition node uploads a piece of data to be processed every interval T1 (such as T1=5S, 10S, 15S, etc., of course, it is not limited to the above example in the specific implementation, and can be flexibly set according to the actual needs of the user), more specifically, taking T1=5S as an example, the sensor acquisition node collects 1 piece of health status data (which can also be understood as survival health status data) of the sensor acquisition node every 5 seconds, so that the sensor acquisition node can collect 12 pieces of health status data in 1 minute. Among them, each health status data includes at least indicators such as CPU load rate (i.e., central processing unit load rate), communication delay, etc.
[0084] After 12 pieces of health status data are acquired in the subnet aggregation computing gateway in one minute, at each time point when the health status data is acquired, it is necessary to compress and reduce the dimension of the data to be processed based on the health status data compression strategy. For example, the data to be processed including multi-dimensional information is compressed into a one-dimensional state identification value and then stored. Of course, in order to more accurately obtain the storage location of the data to be processed after it is compressed into a one-dimensional state identification value, the target compressed sequence data corresponding to the sensor acquisition node is first determined from the multiple local compressed sequence data. For example, the subnet aggregation computing gateway includes multiple compressed sequence data, each of which includes a 12-bit state identification value, and is used to store the reduced-dimensional compressed state identification value corresponding to the 12 pieces of health status data uploaded by the same sensor acquisition node within 1 minute according to the acquisition cycle of 5S, such as Figure 4 It is illustrated in the figure that a compressed sequence data can store multiple state identification values. When each sensor acquisition node uploads the data to be processed, the subnet convergence computing gateway can quickly determine the target compressed sequence data corresponding to the sensor acquisition node. Moreover, each compressed sequence data is not limited to storing only 12 state identification values as in the above example, and can be determined according to the data acquisition interval T1 of the sensor acquisition node, specifically the rounding result of 60 / T1, 90 / T1, 120 / T1, etc. (rounding up or rounding down is possible). The data acquisition interval T1 of the sensor acquisition node determines that the maximum number of digits of the state identification value in each compressed sequence data can be adjusted according to the actual situation to achieve compression of the storage results.
[0085] Afterwards, if it is determined that the health status data does not exceed the node survival threshold and corresponds to normal status data, the data to be processed will be converted into a first preset state identification value (for example, the first preset state identification value is 1) based on the health status data compression strategy, and the first preset state identification value will be stored in the target compression sequence data, and the target compression sequence data will be updated. Afterwards, the data type of the data to be processed collected and uploaded by the sensor acquisition node at the next data acquisition time point is continuously obtained, and when it is determined that the data type is health status data and the health status data corresponding to the data to be processed does not exceed the preset node survival threshold, the data to be processed is also compressed and stored with reference to the above-mentioned dimensionality reduction compression process.
[0086] If it is determined that the health status data does not exceed the node survival threshold and corresponds to abnormal status data, it will mean that although the health status data is abnormal, it can be determined that the sensor node is in a survival state. At this time, it is still necessary to convert the data to be processed into a second preset state identification value (for example, the second preset state identification value is 0) based on the health status data compression strategy, and store the second preset state identification value in the target compressed sequence data, and update the target compressed sequence data. It can be seen that through the above method, the subnet convergence computing gateway realizes dimensionality reduction compression for the data to be processed uploaded by each sensor acquisition node, the data type of which is health status data and the corresponding health status data does not exceed the preset node survival threshold, and converts it into a one-dimensional first preset state identification value or a second preset state identification value for storage.
[0087] In one embodiment, determining the target compressed sequence data corresponding to the sensor acquisition node from a plurality of local compressed sequence data based on the health status data compression strategy includes:
[0088] Obtaining the current sensor node ID number corresponding to the data to be processed;
[0089] Determine the compressed sequence data with the ID number of the current sensor node from a plurality of local compressed sequence data to form a candidate compressed sequence data set;
[0090] Based on the health status data compression strategy, the candidate compression sequence data set is screened to obtain the compression sequence data that has at least one free storage bit and is written as the last data to be processed corresponding to the current sensor node ID number, as the target compression sequence data.
[0091] In this embodiment, the subnet aggregation computing gateway can accurately parse and obtain the current sensor node ID number corresponding to each piece of data to be processed uploaded by each sensor acquisition node, and the multiple compressed sequence data local to the subnet aggregation computing gateway also correspond to a sensor node ID number for each compressed sequence data. Therefore, the current sensor node ID number can be used as a screening condition to determine the compressed sequence data with the current sensor node ID number from the multiple local compressed sequence data to form a candidate compressed sequence data set.
[0092] Generally, one compressed sequence data of the same sensor acquisition node's to-be-processed data in the subnet aggregation computing gateway cannot fully meet the actual storage requirements. It is necessary to continuously create compressed sequence data corresponding to the sensor acquisition node in the subnet aggregation computing gateway, and convert multiple to-be-processed data that meet the conditions (that is, the data type is health status data and the corresponding health status data is determined to not exceed the preset node survival threshold) into the first preset state identification value or the second preset state identification value for storage in chronological order. This requires that the compressed sequence data created for the same sensor acquisition node must have the same sensor node ID number.
[0093] Moreover, when the data to be processed uploaded by the same sensor collection node at the previous data collection time point has been stored in one of the bits of a compressed sequence data in chronological order, such as Figure 4 In the example of a compressed sequence data that can store multiple state identification values, the subnet aggregation computing gateway uploads the last data to be processed at the last data collection time point for the sensor collection node AA1 and performs dimension reduction compression processing and stores it in Figure 4 The corresponding compressed sequence data is the second bit. Since the second bit in the compressed sequence data has not yet reached the end position corresponding to the 12th bit in the compressed sequence data, the sensor acquisition node AA1 uploads the data to be processed at this time and performs dimension reduction compression processing and stores it in the following manner: Figure 4 The third position in the corresponding compressed sequence data. It can be seen that based on the above method, the target compressed sequence data corresponding to the data to be processed can be quickly and accurately determined in the subnet convergence computing gateway as the compressed data storage location.
[0094] In one embodiment, after determining the compressed sequence data having the ID number of the current sensor node from the plurality of compressed sequence data locally to form a candidate compressed sequence data set, the method further comprises:
[0095] If it is determined that each compressed sequence data in the candidate compressed sequence data set is in a fully stored state, newly created compressed sequence data is created based on the current sensor node ID number, and the newly created compressed sequence data is used as the target compressed sequence data.
[0096] In this embodiment, if it is determined that each compressed sequence data in the candidate compressed sequence data set is in a full storage state, it means that the sensor acquisition node uploaded the last data to be processed at the last data acquisition time point and stored the last bit in a compressed sequence data after dimensionality reduction compression processing, and the compressed sequence data can no longer store more digits of the first preset state identification value or the second preset state identification value. At this time, a new compressed sequence data can be created based on the current sensor node ID number, and the new compressed sequence data is used as the target compressed sequence data. It can be seen that based on the above method, the target compressed sequence data corresponding to the data to be processed can also be quickly and accurately determined in the subnet aggregation computing gateway as the compressed data storage location.
[0097] The second dimensionality reduction compression unit 130 is used to perform counting statistics on the data to be processed based on a preset counter statistical strategy if it is determined that the data type is statistical filtering type data, and delete the data to be processed after completing the counting statistics.
[0098] In this embodiment, if it is determined that the data type is statistical filtering data, it means that the data to be processed is a type of data that can be processed by the subnet convergence computing gateway. Since the greatest use of statistical filtering data is to be counted by the subnet convergence computing gateway rather than storing raw data, the data to be processed can be counted based on a preset counter statistics strategy, and the data to be processed can be deleted after the counting statistics are completed.
[0099] In one embodiment, the counter statistical strategy includes a preset cumulative counter and a feature counter, and the feature counter stores a preset data feature screening condition; the second dimensionality reduction compression unit 130 is specifically used for:
[0100] Obtaining the current data cumulative count number of the cumulative counter, adding the data number corresponding to the data to be processed to the current data cumulative count number, and updating the current data cumulative count number;
[0101] Extracting feature data from the data to be processed based on the feature counter to obtain feature data to be processed;
[0102] If it is determined that the feature data to be processed meets the preset data feature screening condition, the current cumulative number of feature data of the feature counter is obtained, the number of data corresponding to the data to be processed is accumulated to the current cumulative number of feature data, and the current cumulative number of feature data is updated.
[0103] In this embodiment, when the sensor acquisition node aggregates the pending data of the statistical filtering type data to the subnet aggregation computing gateway at a certain frequency, the subnet aggregation computing gateway does not directly transmit the pending data of the statistical filtering type data back to the data analysis center (i.e., the cloud server), but performs data filtering and counting locally. The pending data that needs to be filtered and counted is divided into two data types. One type is a cumulative counter without any filtering, which is used to record the number of all received data within the preset data upload cycle (i.e., the cumulative count number of current data). This type of data is used to analyze the proportion of volatility problems. The other type is a feature counter for risk monitoring feature quantities (which can also be understood as a filter counter), for example, recording the number of pending data that exceeds a certain data feature screening condition. Whenever the subnet aggregation computing gateway receives new data to be processed, the processing method of statistical filtering data and the processing method of health status data in the subnet aggregation computing gateway are different in that the cumulative counter as the first type of counter does not perform any feature extraction processing on the data to be processed, but instead accumulates the number of data corresponding to the data to be processed (generally 1, that is, multiple sub-data in one data to be processed are regarded as a complete data set, rather than multiple scattered sub-data) to the current data cumulative count number in the cumulative counter (specifically, +1 processing is performed), thereby updating the current data cumulative count number.
[0104] Afterwards, feature data extraction can be further performed on the data to be processed based on the feature counter to obtain feature data to be processed. For example, if the feature data to be processed is the number of times the communication delay exceeds 200ms and the CPU load exceeds 80% in the data to be processed, and it is determined that the feature data to be processed meets the preset data feature screening condition, it means that the data to be processed is a risk data that needs to be counted by the feature counter. At this time, the number of data corresponding to the data to be processed (generally 1) is accumulated and added to the current cumulative number of feature data of the feature counter (specifically, +1 processing is performed), thereby updating the current cumulative number of feature data. Among them, if it is determined that the feature data to be processed does not meet the preset data feature screening condition, there is no need for the feature counter to perform counting processing on the data to be processed, and it can be directly deleted. For example, when the cumulative counter and the feature counter perform counting statistics on a piece of data to be processed of a statistical filtering type, the specific processing process diagram is as follows. Figure 7As shown, if a piece of data to be processed does not meet the preset data feature screening condition, the cumulative count of the current data in the cumulative counter is processed by +1; if a piece of data to be processed meets the preset data feature screening condition, the cumulative count of the current feature data in the feature counter is processed by +1. It can be seen that after the counting processing of the above two counters, the original data to be processed is no longer saved, but can be directly deleted, which effectively reduces the amount of original data storage in the subnet convergence computing gateway.
[0105] In one embodiment, the second dimensionality reduction and compression unit 130 is further specifically configured to:
[0106] If it is determined that the time interval between the current system time and the last counter data sending time is equal to the preset data upload period, the current data cumulative count number corresponding to the cumulative counter and the current feature data cumulative number corresponding to the feature counter are sent to the cloud server;
[0107] The current data cumulative count and the current feature data cumulative count are reset to 0.
[0108] In this embodiment, the preset cumulative counter and feature counter included in the subnet convergence computing gateway do not always perform unlimited cumulative statistics on data, but after a preset data upload cycle corresponding to the counter statistics strategy in the subnet convergence computing gateway ends (that is, the time interval between the current system time and the previous counter data sending time is determined to be equal to the preset data upload cycle), the current data cumulative count number corresponding to the cumulative counter and the current feature data cumulative number corresponding to the feature counter are sent to the cloud server, so that the performance analysis of the sensor acquisition node can be performed in the cloud server according to the current data cumulative count number and the current feature data cumulative count number obtained by the sensor acquisition node in a preset data upload cycle. The subnet convergence computing gateway uploads only the current data cumulative count number and the current feature data cumulative count number to the cloud server, which is a very small amount of data transmission, effectively reducing the network bandwidth occupied by the data transmission process. Moreover, after completing the upload process, the current data cumulative count number and the current feature data cumulative count number need to be reset to 0, so that the cumulative counter and the feature counter perform data counting processing for the next preset data upload cycle.
[0109] In one embodiment, the sensor network device data return dimension reduction compression device 100 further includes:
[0110] The monitoring alarm information generating unit is used to generate monitoring alarm information corresponding to the data to be processed and send it to the cloud server if it is determined that the data type is health status data and the health status data corresponding to the data to be processed exceeds a preset node survival threshold.
[0111] In this embodiment, if Figure 3 and Figure 4 As shown, when the to-be-processed data of the sensor acquisition node is continuously compressed and stored based on the compressed sequence data, if the duration of a compressed sequence data to completely write the state identification value of all bits is used as a health status data acquisition cycle, when the to-be-processed data whose data type is health status data and the corresponding health status data is determined to exceed the preset node survival threshold is detected within a health status data acquisition cycle, the to-be-processed data is no longer compressed and converted into a state identification value and then written into the compressed sequence data, but the monitoring alarm information corresponding to the to-be-processed data is directly generated (the monitoring alarm information may include the original to-be-processed data and the preset alarm prompt information) and sent to the cloud server. In the above process, the cloud server does not need to disassemble and analyze the compressed health status data again, but the cloud server directly receives the monitoring alarm information and can directly perform subsequent linkage processing after the alarm, so the purpose of reducing the computational load of the state data restoration of the collected data is achieved.
[0112] It can be seen that the implementation embodiment of the device can compress and reduce the dimension of various types of processed data collected by multiple sensor acquisition networks based on the health status data compression strategy or the counter statistics strategy, effectively reducing the amount of local data storage and the amount of data subsequently uploaded to the cloud server, and reducing data bandwidth occupancy and data processing load.
[0113] The sensor network device data return dimension reduction and compression device can be implemented in the form of a computer program. The computer program can be used in Fig. 9 Runs on the computer device shown.
[0114] See also Fig. 9 , Fig. 9 The present invention is a schematic block diagram of a computer device provided by an embodiment of the present invention. The computer device integrates any sensor network device data return dimension reduction compression device provided by an embodiment of the present invention.
[0115] See also Fig. 9 The computer device 400 includes a processor 402 , a memory and a network interface 405 connected via a system bus 401 , wherein the memory may include a storage medium 403 and an internal memory 404 .
[0116] The storage medium 403 can store an operating system 4031 and a computer program 4032. The computer program 4032 includes program instructions, and when the program instructions are executed, the processor 402 can execute the above-mentioned sensor network device data return dimension reduction compression method.
[0117] The processor 402 is used to provide computing and control capabilities to support the operation of the entire computer device.
[0118] The internal memory 404 provides an environment for the operation of the computer program 4032 in the storage medium 403. When the computer program 4032 is executed by the processor 402, the processor 402 can execute the above-mentioned sensor network device data return dimension reduction compression method.
[0119] The network interface 405 is used to communicate with other devices over the network. Fig. 9 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present invention, and does not constitute a limitation on the computer device to which the solution of the present invention is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0120] The processor 402 is used to run the computer program 4032 stored in the memory to implement the above-mentioned sensor network device data return dimension reduction compression method.
[0121] It should be understood that in the embodiment of the present invention, the processor 402 may be a central processing unit (CPU), and the processor 402 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.
[0122] It can be understood by those skilled in the art that all or part of the processes in the method for implementing the above embodiment can be completed by instructing the relevant hardware through a computer program. The computer program includes program instructions, and the computer program can be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in the computer device to implement the process steps of the embodiment of the above method.
[0123] Therefore, the present invention also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions. When the program instructions are executed by a processor, the processor executes the above-mentioned sensor network device data return dimension reduction compression method.
[0124] The computer-readable storage medium may be a USB flash drive, a mobile hard disk, a read-only memory (ROM), a magnetic disk, or an optical disk, etc., which are computer-readable storage media that can store program codes.
[0125] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.
[0126] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of each unit is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed.
[0127] The steps in the method of the embodiment of the present invention can be adjusted in order, combined and deleted according to actual needs. The units in the device of the embodiment of the present invention can be combined, divided and deleted according to actual needs. In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.
[0128] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, terminal, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention.
[0129] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.
Claims
1. A method for reducing the dimension of sensor network device data transmission, applied to a subnet aggregation computing gateway, characterized in that: The subnet convergence computing gateway is connected to multiple sensor acquisition nodes in communication, and the subnet convergence computing gateway is also connected to a cloud server in communication; the sensor network device data return dimension reduction compression method includes: Receive the data to be processed sent by the sensor acquisition node, and obtain the data type of the data to be processed; If it is determined that the data type is health status data, and it is determined that the health status data corresponding to the data to be processed does not exceed the preset node survival threshold, the data to be processed is compressed and stored based on a preset health status data compression strategy; wherein the health status data compression strategy is used to store the health status data of the data to be processed in the form of a status identification value when the health status data of the data to be processed does not exceed the preset node survival threshold; If it is determined that the data type is statistical filtering data, counting and counting the data to be processed is performed based on a preset counter statistical strategy, and the data to be processed is deleted after the counting and statistics are completed; The compressing and storing the data to be processed based on a preset health status data compression strategy includes: Determining target compressed sequence data corresponding to the sensor acquisition node from a plurality of local compressed sequence data based on the health status data compression strategy; wherein the target compressed sequence data has at least one free storage bit; If it is determined that the health status data does not exceed the node survival threshold and corresponds to normal status data, the to-be-processed data is converted into a first preset status identification value based on the health status data compression strategy, and stored in the target compression sequence data, and the target compression sequence data is updated; If it is determined that the health status data does not exceed the node survival threshold and corresponds to abnormal status data, the to-be-processed data is converted into a second preset status identification value based on the health status data compression strategy, and stored in the target compression sequence data, and the target compression sequence data is updated; The counter statistical strategy includes a preset cumulative counter and a feature counter, and the feature counter stores preset data feature screening conditions; The counting and statistics of the data to be processed based on a preset counter statistics strategy, and deleting the data to be processed after the counting and statistics are completed, includes: Obtaining the current data cumulative count number of the cumulative counter, adding the data number corresponding to the data to be processed to the current data cumulative count number, and updating the current data cumulative count number; Extracting feature data from the data to be processed based on the feature counter to obtain feature data to be processed; If it is determined that the feature data to be processed meets the preset data feature screening condition, the current cumulative number of feature data of the feature counter is obtained, the number of data corresponding to the data to be processed is accumulated to the current cumulative number of feature data, and the current cumulative number of feature data is updated.
2. The method according to claim 1, characterized in that After the step of receiving the data to be processed sent by the sensor acquisition node and obtaining the data type of the data to be processed, the method further includes: If it is determined that the data type is health status data, and it is determined that the health status data corresponding to the data to be processed exceeds a preset node survival threshold, monitoring alarm information corresponding to the data to be processed is generated and sent to the cloud server.
3. The method according to claim 1, characterized in that The determining, based on the health status data compression strategy, target compressed sequence data corresponding to the sensor acquisition node from a plurality of local compressed sequence data includes: Obtaining the current sensor node ID number corresponding to the data to be processed; Determine the compressed sequence data with the ID number of the current sensor node from a plurality of local compressed sequence data to form a candidate compressed sequence data set; Based on the health status data compression strategy, the candidate compression sequence data set is screened to obtain the compression sequence data that has at least one free storage bit and is written as the last data to be processed corresponding to the current sensor node ID number, as the target compression sequence data.
4. The method according to claim 3, characterized in that After the step of determining the compressed sequence data having the ID number of the current sensor node from the plurality of compressed sequence data locally to form a candidate compressed sequence data set, the method further comprises: If it is determined that each compressed sequence data in the candidate compressed sequence data set is in a fully stored state, newly created compressed sequence data is created based on the current sensor node ID number, and the newly created compressed sequence data is used as the target compressed sequence data.
5. The method according to claim 1, characterized in that After the step of obtaining the current cumulative number of feature data of the feature counter if it is determined that the feature data to be processed meets the preset data feature screening condition, adding the number of data corresponding to the data to be processed to the current cumulative number of feature data, and updating the current cumulative number of feature data, the method further includes: If it is determined that the time interval between the current device time and the last counter data sending time is equal to the preset data upload period, the current data cumulative count number corresponding to the cumulative counter and the current feature data cumulative number corresponding to the feature counter are sent to the cloud server; The current data cumulative count and the current feature data cumulative count are reset to 0.
6. A sensor network device data return dimension reduction compression device, configured in a subnet aggregation computing gateway, characterized in that: The subnet convergence computing gateway is connected to multiple sensor acquisition nodes in communication, and the subnet convergence computing gateway is also connected to a cloud server in communication; the sensor network device data return dimension reduction compression device includes: A data type determination unit, used to receive the data to be processed sent by the sensor acquisition node, and obtain the data type of the data to be processed; A first dimensionality reduction compression unit is used to compress and store the data to be processed based on a preset health status data compression strategy if it is determined that the data type is health status data and it is determined that the health status data corresponding to the data to be processed does not exceed a preset node survival threshold; wherein the health status data compression strategy is used to store the health status data of the data to be processed in the form of a status identification value when the health status data of the data to be processed does not exceed a preset node survival threshold; A second dimension reduction compression unit is used for, if it is determined that the data type is statistical filtering data, performing counting statistics on the data to be processed based on a preset counter statistical strategy, and deleting the data to be processed after completing the counting statistics; The compressing and storing the data to be processed based on a preset health status data compression strategy includes: Determining target compressed sequence data corresponding to the sensor acquisition node from a plurality of local compressed sequence data based on the health status data compression strategy; wherein the target compressed sequence data has at least one free storage bit; If it is determined that the health status data does not exceed the node survival threshold and corresponds to normal status data, the to-be-processed data is converted into a first preset status identification value based on the health status data compression strategy, and stored in the target compression sequence data, and the target compression sequence data is updated; If it is determined that the health status data does not exceed the node survival threshold and corresponds to abnormal status data, the to-be-processed data is converted into a second preset status identification value based on the health status data compression strategy, and stored in the target compression sequence data, and the target compression sequence data is updated; The counter statistical strategy includes a preset cumulative counter and a feature counter, and the feature counter stores preset data feature screening conditions; The counting and statistics of the data to be processed based on a preset counter statistics strategy, and deleting the data to be processed after the counting and statistics are completed, includes: Obtaining the current data cumulative count number of the cumulative counter, adding the data number corresponding to the data to be processed to the current data cumulative count number, and updating the current data cumulative count number; Extracting feature data from the data to be processed based on the feature counter to obtain feature data to be processed; If it is determined that the feature data to be processed meets the preset data feature screening condition, the current cumulative number of feature data of the feature counter is obtained, the number of data corresponding to the data to be processed is accumulated to the current cumulative number of feature data, and the current cumulative number of feature data is updated.
7. A computer device, characterized in that: The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the sensor network device data return dimension reduction and compression method according to any one of claims 1 to 5 when executing the computer program.
8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which includes program instructions. When the program instructions are executed by a processor, the sensor network device data return dimension reduction compression method according to any one of claims 1 to 5 can be implemented.
Citation Information
Patent Citations
Data transmission method and device based on database storage system, equipment and medium
CN116185657A
Network data processing method and device and computer readable storage medium
CN117319509A