Multi-source data fusion uploading and receiving method, device, equipment and storage medium

Abstract

The invention relates to the technical field of multi-source data fusion transmission, in particular to a multi-source data fusion uploading and receiving method and device, equipment and a storage medium. Secondly, for each first data set, finding a closest class center from a plurality of classes, and determining an outlier index according to a distance from the closest class center; mapping the plurality of first data sets to a predetermined interval to obtain a plurality of second data sets and a plurality of first extreme value pairs; and finally, encoding the plurality of second data sets according to the encoding fineness determined by the plurality of outlier indexes to obtain a plurality of fused data blocks, and transmitting the plurality of fused data blocks and the plurality of first extreme value pairs. According to the embodiment of the invention, the node fuses the data and then sends the data to the platform server, so that the occupation of public communication system resources is reduced, and after the data is fused, the data transmission efficiency is greatly improved.

Description

Technical Field

[0001] The present invention relates to the technical field of multi-source data fusion transmission, and in particular to a multi-source data fusion uploading and receiving method, device, equipment and storage medium. Background Art

[0002] Smart city is a new form of city built with the goal of more scientific development, more efficient management and better life, supported by information technology and communication technology, and through transparent and sufficient information acquisition, extensive and secure information transmission and effective and scientific information processing, to improve the efficiency of urban operation and the level of public services. Data transmission, as an important part of it, plays a vital role.

[0003] As part of public facilities, lighting poles play a huge role in the construction of smart cities by integrating multiple functional carriers, such as integrating traffic signals and meteorological data collection.

[0004] Currently, data on various carriers on lighting poles are mostly sent and received from the end to the platform through public communication networks, which takes up more communication resources and makes it difficult to ensure the security of data transmission.

[0005] Based on this, it is necessary to develop and design a multi-source data fusion upload method. Summary of the invention

[0006] The embodiments of the present invention provide a multi-source data fusion uploading and receiving method, device, equipment and storage medium, which are used to solve the problem that in the prior art, multi-source data completes data transmission through a public communication network and occupies a large amount of communication resources.

[0007] In a first aspect, an embodiment of the present invention provides a multi-source data fusion uploading method, comprising: Applied to a data fusion node, the data fusion node is connected to multiple terminal signals, and the multi-source data fusion uploading method includes: Acquire multiple first data sets, wherein each first data set corresponds to a data acquisition terminal, and the first data set includes multiple data queues; For each first data set, find the closest class center from a plurality of classes, and determine an outlier index according to the distance to the closest class center, wherein the plurality of classes are obtained based on clustering; Mapping the plurality of first data sets to a predetermined interval to obtain a plurality of second data sets and a plurality of first maximum value pairs, wherein each first data set corresponds to a second data set and a first maximum value pair, and the first maximum value pair includes a maximum value and a minimum value of the first data set; The plurality of second data sets are encoded according to the encoding fineness determined by the plurality of outlier indices to obtain a plurality of fused data blocks, and the plurality of fused data blocks and the plurality of first maximum value pairs are transmitted.

[0008] In a possible implementation, for each first data set, finding the closest class center from multiple classes, and determining the outlier index according to the distance from the closest class center includes: For each first data set, perform the following steps respectively: Acquire multiple classes, wherein the multiple classes are clustered by a kmeans clustering method; Finding a first preset number of multiple class data with the closest Euclidean distance to the first data set from the multiple classes as multiple reference data; Determine the closest class center according to the plurality of reference data, wherein the class to which the closest class center belongs contains the largest number of reference data; The ratio of the distance between the first data set and the closest class center to the maximum class radius is used as the outlier index, wherein the maximum class radius is the maximum value of the distance from the data in the class to the class center.

[0009] In a possible implementation, mapping the plurality of first data sets to a predetermined interval to obtain a plurality of second data sets and a plurality of first maximum value pairs includes: Obtaining an upper limit and a lower limit of the predetermined interval; For each first data set, perform the following steps respectively: Constructing the maximum value and the minimum value of the first data set into a first maximum value pair; The data of the first data set is processed according to a first formula to obtain a second data set, wherein the first formula is:

[0010] In the formula, For the second data set The first elements, is the lower limit of the predetermined interval, is the upper limit of the predetermined interval, is the maximum value of the first data set, is the minimum value of the first data set, For the first data set The first elements.

[0011] In a possible implementation, encoding the plurality of second data sets according to the encoding fineness determined according to the plurality of outlier indices to obtain a plurality of fused data blocks includes: For each second data set, perform the following steps respectively: Determining the number of rows and columns of the fused data block according to the outlier index, wherein the number of rows of the fused data block and the number of columns of the fused data block are both positively correlated with the outlier index; According to the number of rows of the fused data block, the number of columns of the fused data block and the second formula, the second data set is fused to obtain a fused data block, wherein the second formula is:

[0012] In the formula, For the fusion data block Line Column data, For the second data set The first elements, is the total number of the second data set queues, is the total number of data in the queue of the second data set, is a natural constant, is an imaginary unit, is the ratio of pi.

[0013] In a possible implementation manner, the transmitting of the multiple fused data blocks includes: For each fused data block, perform the following steps: Take out data columns from the fused data block in sequence as the array to be transmitted; According to the transmission timing and the number of columns of the array to be transmitted in the fused data block, modulate the multiple data of the array to be transmitted onto multiple subcarriers respectively to obtain multiple modulated subcarriers; Transmitting the multiple modulated subcarriers; If the traversal of the fused data block is not completed, the process jumps to the step of sequentially taking out data columns from the fused data block as the array to be transmitted.

[0014] In a possible implementation manner, modulating multiple data of the array to be transmitted onto multiple subcarriers respectively according to the transmission timing and the number of columns of the array to be transmitted in the fused data block to obtain multiple modulated subcarriers includes: According to a third formula, the plurality of data of the array to be transmitted are modulated onto a plurality of subcarriers respectively, wherein the third formula is:

[0015] In the formula, For the Subcarriers in The waveform of the moment, The first data, is the bias constant, is a sine function, is the period length of the reference fundamental wave, is the frequency of the reference fundamental wave, is the number of columns of the array to be transmitted in the data block, is the initial phase; The transmitting the multiple modulated subcarriers comprises: According to the fourth formula, the multiple modulated subcarriers are synthesized to obtain a synthesized waveform, wherein the fourth formula is:

[0016] In the formula, is the synthetic waveform, To adjust the number of subcarriers, For the The waveform of the subcarriers; The synthesized waveform is up-converted to a predetermined frequency band for transmission.

[0017] In a second aspect, an embodiment of the present invention provides a method for receiving fused data, which is applied to a platform server, wherein the server is connected to a data fusion node signal, and the method for receiving fused data includes: Acquire multiple fused data blocks and multiple first maximum value pairs obtained according to the multi-source data fusion uploading method described in the first aspect or any possible implementation manner of the first aspect; Decoding the multiple fused data blocks to obtain multiple third data sets, wherein each third data set corresponds to a fused data block; For each third data set, the third data set is de-mapped according to the predetermined interval and the first maximum value pair corresponding to the third data set to obtain a fourth data set.

[0018] In a third aspect, an embodiment of the present invention provides a multi-source data fusion uploading device, which is used to implement the multi-source data fusion uploading method described in the first aspect or any possible implementation of the first aspect, and the multi-source data fusion uploading device includes: A data acquisition module, used to acquire a plurality of first data sets, wherein each first data set corresponds to a data acquisition terminal, and the first data set includes a plurality of data queues; A data outlier analysis module, configured to find the closest class center from a plurality of classes for each first data set, and determine an outlier index according to a distance from the closest class center, wherein the plurality of classes are obtained based on clustering; An interval mapping module, used for mapping the plurality of first data sets to a predetermined interval, to obtain a plurality of second data sets and a plurality of first maximum value pairs, wherein each first data set corresponds to a second data set and a first maximum value pair, and the first maximum value pair includes a maximum value and a minimum value of the first data set; as well as, The fusion sending module is used to encode the multiple second data sets according to the coding fineness determined by the multiple outlier indices to obtain multiple fused data blocks, and transmit the multiple fused data blocks and the multiple first maximum value pairs.

[0019] In a fourth aspect, an embodiment of the present invention provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the computer program, it implements the steps of the method described in the first aspect or any possible implementation method of the first aspect.

[0020] In a fifth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method described in the first aspect or any possible implementation of the first aspect are implemented.

[0021] Compared with the prior art, the embodiments of the present invention have the following beneficial effects: The embodiment of the present invention discloses a multi-source data fusion uploading method, which first obtains multiple first data sets, wherein each first data set corresponds to a data acquisition terminal, and the first data set includes multiple data queues; then for each first data set, find the closest class center from multiple classes, and determine the outlier index according to the distance to the closest class center, wherein multiple classes are obtained based on clustering; then map the multiple first data sets to a predetermined interval, obtain multiple second data sets and multiple first maximum value pairs, wherein each first data set corresponds to a second data set and a first maximum value pair, and the first maximum value pair includes the maximum value and the minimum value of the first data set; finally, encode the multiple second data sets according to the coding fineness determined by the multiple outlier indexes, obtain multiple fused data blocks, and transmit the multiple fused data blocks and the multiple first maximum value pairs. The embodiment of the present invention reduces the occupation of public communication system resources by aggregating multiple terminal data to nodes, and sending the data to the platform server after the node fusion. After the data is fused, the efficiency of data transmission is greatly improved. Moreover, when the data is fused, the fusion degree is determined according to the outlier characteristics of the data, and the data is fused according to the demand, taking into account the authenticity of data restoration and the efficiency of data transmission. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0023] Figure 1 is a flow chart of a multi-source data fusion uploading method provided by an embodiment of the present invention; Figure 2 This is an application scenario diagram of the multi-source data fusion uploading method provided by an embodiment of the present invention; Figure 3 It is a functional block diagram of a multi-source data fusion uploading device provided by an embodiment of the present invention; Figure 4 It is a functional block diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0024] In the following description, specific details such as specific system structures and technologies are provided for the purpose of illustration rather than limitation so as to provide a thorough understanding of the embodiments of the present invention. However, it should be clear to those skilled in the art that the present invention may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, and methods are omitted so as not to obscure the description of the present invention with unnecessary details.

[0025] In order to make the purpose, technical solutions and advantages of the present invention more clear, a specific implementation method will be described below in conjunction with the accompanying drawings.

[0026] The following is a detailed description of an embodiment of the present invention. This example is implemented based on the technical solution of the present invention, and provides a detailed implementation method and a specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0027] Figure 1 Flow chart of the multi-source data fusion uploading method provided in an embodiment of the present invention.

[0028] like Figure 1 As shown, it shows a flowchart of the implementation method of multi-source data fusion uploading provided by an embodiment of the present invention, which is described in detail as follows: In step 101, a plurality of first data sets are acquired, wherein each first data set corresponds to a data acquisition terminal, and the first data set includes a plurality of data queues.

[0029] For example, Figure 2As shown, the figure shows an application scenario diagram of an embodiment of the present invention. In order to overcome the defects mentioned above, the lamp pole of the embodiment of the present invention is used as a carrier for transmitting data and is divided into two levels in the data transmission system, one of which is the terminal 203 level. These levels are connected to multiple data acquisition devices, for example, meteorological data acquisition devices, traffic signal data acquisition devices, etc., and each type of collected data is collected in chronological order as a data queue. For example, the temperature data is arranged in the order of time nodes 202 to form a temperature data queue, and the wind speed data is arranged in the order of time nodes 202 to form a wind speed data queue. The multiple data queues collected by each terminal 203 are aggregated to form a data set (a first data set). The data set obtained by each terminal 203 is sent to the node 202 via a short-distance transmission method, for example, a data transmission method based on the 315MHz band. In actual application, the node 202 It may be a terminal 203 with the same functions as other terminals 203, except that the node 202 also has the ability to receive data sets from other terminals 203, fuse these data sets into a fused data block, and then send it to the platform server 201. After receiving the fused data block, the platform server 201 can obtain the data collected by multiple terminals 203 through reverse fusion. The advantage of this is that each terminal 203 does not need to communicate with the server separately, which reduces the occupation of public communication network resources. In addition, since the data is fused before being sent, the data transmission efficiency is high. The server changes from the previous situation where it needs to process the data sent by all terminals 203 at the same time to the situation where it only needs to receive the data from a small number of nodes 202. The pressure on the server to send and receive data is greatly reduced, and the server can directly store these fused data blocks. Since the data volume is reduced after data fusion, the data storage volume is reduced.

[0030] To achieve the above objectives, the embodiments of the present invention are discussed in detail from the following steps.

[0031] In step 102, for each first data set, the closest class center is found from a plurality of classes, and an outlier index is determined according to the distance from the closest class center, wherein the plurality of classes are obtained based on clustering.

[0032] In some implementations, for each first data set, finding the closest class center from multiple classes, and determining the outlier index based on the distance to the closest class center includes: For each first data set, perform the following steps respectively: Acquire multiple classes, wherein the multiple classes are clustered by a kmeans clustering method; Finding a first preset number of multiple class data with the closest Euclidean distance to the first data set from the multiple classes as multiple reference data; Determine the closest class center according to the plurality of reference data, wherein the class to which the closest class center belongs contains the largest number of reference data; The ratio of the distance between the first data set and the closest class center to the maximum class radius is used as the outlier index, wherein the maximum class radius is the maximum value of the distance from the data in the class to the class center.

[0033] Exemplarily, in fact, the data fusion of the present invention is a loss-controllable fusion. However, if the data fusion degree is set to a high level, some data details may be lost (for example, when indicating the temperature trend, the decimal point is erased and the trend effect becomes less obvious). If the fusion degree is reduced, the data transmission volume will increase, which is not conducive to improving transmission efficiency and timeliness.

[0034] In order to overcome the above defects, the embodiment of the present invention adopts a fusion method with a variable fusion rate, specifically, the fusion rate is determined according to the characteristics of the data set itself. Specifically, the embodiment of the present invention determines the outlier index of the first data set by the distance between the first data set and the closest class center.

[0035] That is to say, before determining the outlier index, some historical first data sets are first obtained, and these historical first data sets are clustered into multiple classes through a clustering method (preferably the kmeans clustering method), and the class centers of these multiple classes are then calculated.

[0036] When determining the outlier index, find multiple historical first data sets that are closest to the first data set from the above multiple historical first data sets (class data), for example, find 10 historical first data sets, and we will temporarily call these historical first data sets reference data. The class center that belongs to the most classes among these reference data is taken as the closest class center. For example, in the above example, three of the 10 historical first data sets belong to class A, and the other parts of these 10 historical first data sets may belong to one or two other classes. Then, the class center of class A is determined to be the closest class center.

[0037] This class obviously has a maximum radius, which is the maximum distance between other historical first data sets in the class and the class center. We know that the greater the distance between the first data set and the closest class center, the greater the outlier characteristic, reflecting that the deviation between the first data set and the previous historical first data set is greater. In order to express this characteristic more accurately, the embodiment of the present invention uses the ratio of the distance between the first data set and the closest class center to the maximum radius as the outlier index. Obviously, the larger the outlier index, the lower the fusion degree should be used when fusion.

[0038] In step 103, the multiple first data sets are mapped to a predetermined interval to obtain multiple second data sets and multiple first maximum value pairs, wherein each first data set corresponds to a second data set and a first maximum value pair, and the first maximum value pair includes the maximum value and the minimum value of the first data set.

[0039] In some implementations, mapping the plurality of first data sets to a predetermined interval to obtain a plurality of second data sets and a plurality of first maximum value pairs comprises: Obtaining an upper limit and a lower limit of the predetermined interval; For each first data set, perform the following steps respectively: Constructing the maximum value and the minimum value of the first data set into a first maximum value pair; The data of the first data set is processed according to a first formula to obtain a second data set, wherein the first formula is:

[0040] In the formula, For the second data set The first elements, is the lower limit of the predetermined interval, is the upper limit of the predetermined interval, is the maximum value of the first data set, is the minimum value of the first data set, For the first data set The first elements.

[0041] Exemplarily, the data fusion of the implementation mode of the present invention is actually completed through two-step fusion. The first step is a low-loss fusion (in some cases, the first step can achieve lossless fusion), and its purpose is to complete the data interval unification through data mapping. For example, the data distribution interval of some first data sets is 50000-100000, and the data distribution interval of another first data set sent by the node together with the first data set is 150000-200000. Then, when expressing the data, 50000-200000 is required. This obviously wastes the length of data expression. The implementation mode of the present invention is to map both first data sets to 0-50000 to improve the repeatability of the data.

[0042] To achieve this goal, the present invention first determines the upper and lower limits of the mapped interval, then obtains the maximum and minimum values ​​of each first data set, and uses these two values ​​as the first maximum value pair, and then uses the first formula to complete the mapping of each first data set:

[0043] In the formula, For the second data set The first elements, is the lower limit of the predetermined interval, is the upper limit of the predetermined interval, is the maximum value of the first data set, is the minimum value of the first data set, For the first data set The first elements.

[0044] In step 104, the plurality of second data sets are encoded according to the encoding fineness determined by the plurality of outlier indices to obtain a plurality of fused data blocks, and the plurality of fused data blocks and the plurality of first maximum value pairs are transmitted.

[0045] In some implementations, encoding the plurality of second data sets according to the encoding fineness determined according to the plurality of outlier indices to obtain a plurality of fused data blocks includes: For each second data set, perform the following steps respectively: Determining the number of rows and columns of the fused data block according to the outlier index, wherein the number of rows of the fused data block and the number of columns of the fused data block are both positively correlated with the outlier index; According to the number of rows of the fused data block, the number of columns of the fused data block and the second formula, the second data set is fused to obtain a fused data block, wherein the second formula is:

[0046] In the formula, For the fusion data block Line Column data, For the second data set The first elements, is the total number of the second data set queues, is the total number of data in the queue of the second data set, is a natural constant, is an imaginary unit, is the ratio of pi.

[0047] Exemplarily, during encoding, in some implementations, the encoding fineness is determined according to the outlier index. Specifically, the larger the outlier index, the more rows and columns of the fused data block, and vice versa. The purpose of this is to retain as many details as possible after the data set with a large outlier index is fused.

[0048] For example, in some scenarios, the second dataset is constructed as a OK Before merging the data blocks of the columns, firstly, the number of rows in the data block is , the number of columns of the data block and the scale factor to determine a basic number of rows and the base number of columns :

[0049]

[0050] In the above formula, 0.8 is the proportionality coefficient; Then compare the outlier index with the basic number of rows mentioned above and the base number of columns Multiply, and the result is the number of rows of the fused data block and the number of columns :

[0051]

[0052] In the above formula, is the outlier index.

[0053] After the number of rows and columns of the fused data block is determined, the second data set can be fused using the second formula.

[0054] In some implementations, the transmitting of the plurality of fused data blocks includes: For each fused data block, perform the following steps: Take out data columns from the fused data block in sequence as the array to be transmitted; According to the transmission timing and the number of columns of the array to be transmitted in the fused data block, modulate the multiple data of the array to be transmitted onto multiple subcarriers respectively to obtain multiple modulated subcarriers; Transmitting the multiple modulated subcarriers; If the traversal of the fused data block is not completed, the process jumps to the step of sequentially taking out data columns from the fused data block as the array to be transmitted.

[0055] In some implementations, modulating the plurality of data of the array to be transmitted onto a plurality of subcarriers respectively according to the transmission timing and the number of columns of the array to be transmitted in the fused data block to obtain a plurality of modulated subcarriers includes: According to a third formula, the plurality of data of the array to be transmitted are modulated onto a plurality of subcarriers respectively, wherein the third formula is:

[0056] In the formula, For the Subcarriers in The waveform of the moment, The first data, is the bias constant, is a sine function, is the period length of the reference fundamental wave, is the frequency of the reference fundamental wave, is the number of columns of the array to be transmitted in the data block, is the initial phase; The transmitting the multiple modulated subcarriers comprises: According to the fourth formula, the multiple modulated subcarriers are synthesized to obtain a synthesized waveform, wherein the fourth formula is:

[0057] In the formula, is the synthetic waveform, To adjust the number of subcarriers, For the The waveform of the subcarriers; The synthesized waveform is up-converted to a predetermined frequency band for transmission.

[0058] Exemplarily, as described above, the fusion in the embodiment of the present invention is a two-step fusion, and the fusion in the second step is a loss-controllable fusion. Specifically, according to the outlier index, a row number and a column number of a fused data block are specified for each second data set (the data set obtained by the first fusion) according to the outlier index (see the above-mentioned determination of the number of rows of the fused data block according to the outlier index). and the number of columns In addition, in some other scenarios, regardless of the fusion degree, the number of rows remains unchanged, and only the number of columns of the fused data block is changed. That is to say, in this scenario, the controllability of the fusion degree is achieved only by the number of columns of the fused data block, and the number of columns is positively correlated with the outlier index.

[0059] After determining the number of rows and columns of the fused CNC, the second data set is fused by the second formula to obtain a fused data block. The second formula is:

[0060] In the formula, For the fusion data block Line Column data, For the second data set The first elements, is the total number of the second data set queues, is the total number of data in the queue of the second data set, is a natural constant, is an imaginary unit, is the ratio of pi.

[0061] In terms of sending the fused data block, the embodiment of the present invention extracts data from the fused data block column by column, modulates the data onto multiple subcarriers, and sends the extracted column data in parallel.

[0062] In terms of modulation of the subcarrier according to the extracted column data (array to be transmitted), the third formula is applied:

[0063] In the formula, For the Subcarriers in The waveform of the moment, The first data, is the bias constant, is a sine function, is the period length of the reference fundamental wave, is the frequency of the reference fundamental wave, is the number of columns of the array to be transmitted in the data block, is the initial phase.

[0064] Multiple subcarriers are then waveform synthesized to form a synthetic waveform:

[0065] In the formula, is the synthetic waveform, To adjust the number of subcarriers, For the The waveform of the subcarrier.

[0066] Finally, the synthesized waveform is up-converted to a predetermined band and transmitted.

[0067] The embodiment of the present invention further provides a method for receiving fused data, comprising: Acquire multiple fused data blocks and multiple first maximum value pairs; Decoding the multiple fused data blocks to obtain multiple third data sets, wherein each third data set corresponds to a fused data block; For each third data set, the third data set is de-mapped according to the predetermined interval and the first maximum value pair corresponding to the third data set to obtain a fourth data set.

[0068] By way of example, the receiving process on the platform server side has to be explained here. After receiving the up-converted waveform, the platform server first down-converts the frequency, and then separates the synthesized waveform (for example, Fourier transform) to obtain multiple subcarrier waveforms. Then, the fused data block is obtained through the inverse transformation of the third formula mentioned above. This data block is again de-fused (decoded) to obtain a third data set. In fact, at this point, the third data set obtained is very different from the second data set. Then, the data restoration is completed by de-mapping according to the first maximum value pair corresponding to the third data set.

[0069] The embodiment of the multi-source data fusion uploading method of the present invention first obtains multiple first data sets, wherein each first data set corresponds to a data acquisition terminal, and the first data set includes multiple data queues; then for each first data set, the closest class center is found from multiple classes, and the outlier index is determined according to the distance from the closest class center, wherein multiple classes are obtained based on clustering; then the multiple first data sets are mapped to a predetermined interval to obtain multiple second data sets and multiple first maximum value pairs, wherein each first data set corresponds to a second data set and a first maximum value pair, and the first maximum value pair includes the maximum value and the minimum value of the first data set; finally, the multiple second data sets are encoded according to the coding fineness determined by the multiple outlier indexes, and multiple fused data blocks are obtained, and the multiple fused data blocks and the multiple first maximum value pairs are transmitted. The embodiment of the present invention reduces the occupation of public communication system resources by aggregating multiple terminal data to nodes, and sending the data to the platform server after the node fusion. After the data is fused, the efficiency of data transmission is greatly improved, and when the data is fused, the fusion degree is determined according to the outlier characteristics of the data, and the data is fused according to the demand, taking into account the authenticity of data restoration and the efficiency of data transmission.

[0070] It should be understood that the size of the serial numbers of the steps in the above implementation does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the implementation method of the present invention.

[0071] The following is an implementation of the device of the present invention. For details not described in detail, reference may be made to the corresponding method implementation described above.

[0072] Figure 3 is a functional block diagram of a multi-source data fusion upload device provided by an embodiment of the present invention, referring to Figure 3 The multi-source data fusion uploading device includes: a data acquisition module 301, a data outlier analysis module 302, an interval mapping module 303 and a fusion sending module 304, wherein: The data acquisition module 301 is used to acquire multiple first data sets, wherein each first data set corresponds to a data acquisition terminal, and the first data set includes multiple data queues; A data outlier analysis module 302 is used to find the closest class center from a plurality of classes for each first data set, and determine an outlier index according to the distance from the closest class center, wherein the plurality of classes are obtained based on clustering; An interval mapping module 303 is used to map the multiple first data sets to a predetermined interval to obtain multiple second data sets and multiple first maximum value pairs, wherein each first data set corresponds to a second data set and a first maximum value pair, and the first maximum value pair includes a maximum value and a minimum value of the first data set; The fusion sending module 304 is used to encode the multiple second data sets according to the coding fineness determined by the multiple outlier indices to obtain multiple fused data blocks, and transmit the multiple fused data blocks and the multiple first maximum value pairs.

[0073] Figure 4 is a functional block diagram of an electronic device provided by an embodiment of the present invention. Figure 4 As shown, the electronic device 4 of this embodiment includes: a processor 400 and a memory 401, wherein the memory 401 stores a computer program 402 that can be run on the processor 400. When the processor 400 executes the computer program 402, the steps in the above-mentioned multi-source data fusion uploading method and the embodiment are implemented, for example Figure 1 Steps 101 to 104 are shown.

[0074] Exemplarily, the computer program 402 may be divided into one or more modules / units, and the one or more modules / units are stored in the memory 401 and executed by the processor 400 to implement the present invention.

[0075] The electronic device 4 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The electronic device 4 may include, but is not limited to, a processor 400 and a memory 401. Those skilled in the art will appreciate that Figure 4 It is only an example of the electronic device 4 and does not constitute a limitation of the electronic device 4. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electronic device 4 may also include input and output devices, network access devices, buses, etc.

[0076] The processor 400 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0077] The memory 401 may be an internal storage unit of the electronic device 4, such as a hard disk or memory of the electronic device 4. The memory 401 may also be an external storage device of the electronic device 4, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 4. Further, the memory 401 may also include both an internal storage unit and an external storage device of the electronic device 4. The memory 401 is used to store the computer program 402 and other programs and data required by the electronic device 4. The memory 401 may also be used to temporarily store data that has been output or is to be output.

[0078] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the implementation method can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method implementation method, which will not be repeated here.

[0079] In the above-mentioned embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0080] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0081] In the embodiments provided by the present invention, it should be understood that the disclosed devices / electronic devices and methods can be implemented in other ways. For example, the device / electronic device embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0082] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0083] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0084] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned implementation method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned various methods and device implementation methods can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device that can carry the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium.

[0085] The above-described embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A multi-source data fusion uploading method, characterized in that: Applied to a data fusion node, the data fusion node is connected to multiple terminal signals, and the multi-source data fusion uploading method includes: Acquire multiple first data sets, wherein each first data set corresponds to a data acquisition terminal, and the first data set includes multiple data queues; For each first data set, find the closest class center from a plurality of classes, and determine an outlier index according to the distance to the closest class center, wherein the plurality of classes are obtained based on clustering; Mapping the plurality of first data sets to a predetermined interval to obtain a plurality of second data sets and a plurality of first maximum value pairs, wherein each first data set corresponds to a second data set and a first maximum value pair, and the first maximum value pair includes a maximum value and a minimum value of the first data set; The plurality of second data sets are encoded according to the encoding fineness determined by the plurality of outlier indices to obtain a plurality of fused data blocks, and the plurality of fused data blocks and the plurality of first maximum value pairs are transmitted.

2. The multi-source data fusion uploading method according to claim 1 is characterized in that: For each first data set, finding the closest class center from multiple classes, and determining the outlier index according to the distance from the closest class center, including: For each first data set, perform the following steps respectively: Acquire multiple classes, wherein the multiple classes are clustered by a kmeans clustering method; Finding a first preset number of multiple class data with the closest Euclidean distance to the first data set from the multiple classes as multiple reference data; Determine the closest class center according to the plurality of reference data, wherein the class to which the closest class center belongs contains the largest number of reference data; The ratio of the distance between the first data set and the closest class center to the maximum class radius is used as the outlier index, wherein the maximum class radius is the maximum value of the distance from the data in the class to the class center.

3. The multi-source data fusion uploading method according to claim 1 is characterized in that: The step of mapping the plurality of first data sets to a predetermined interval to obtain a plurality of second data sets and a plurality of first maximum value pairs includes: Obtaining an upper limit and a lower limit of the predetermined interval; For each first data set, perform the following steps respectively: Constructing the maximum value and the minimum value of the first data set into a first maximum value pair; The data of the first data set is processed according to a first formula to obtain a second data set, wherein the first formula is: In the formula, For the second data set The first elements, is the lower limit of the predetermined interval, is the upper limit of the predetermined interval, is the maximum value of the first data set, is the minimum value of the first data set, For the first data set The first elements.

4. The multi-source data fusion uploading method according to any one of claims 1 to 3, characterized in that: The step of encoding the plurality of second data sets according to the encoding fineness determined according to the plurality of outlier indices to obtain a plurality of fused data blocks comprises: For each second data set, perform the following steps respectively: Determining the number of rows and columns of the fused data block according to the outlier index, wherein the number of rows of the fused data block and the number of columns of the fused data block are both positively correlated with the outlier index; According to the number of rows of the fused data block, the number of columns of the fused data block and the second formula, the second data set is fused to obtain a fused data block, wherein the second formula is: In the formula, For the fusion data block Line Column data, For the second data set The first elements, is the total number of the second data set queues, is the total number of data in the queue of the second data set, is a natural constant, is an imaginary unit, is the ratio of pi.

5. The multi-source data fusion uploading method according to any one of claims 1 to 3, characterized in that: The transmitting of the multiple fused data blocks comprises: For each fused data block, perform the following steps: Take out data columns from the fused data block in sequence as the array to be transmitted; According to the transmission timing and the number of columns of the array to be transmitted in the fused data block, modulate the multiple data of the array to be transmitted onto multiple subcarriers respectively to obtain multiple modulated subcarriers; Transmitting the multiple modulated subcarriers; If the traversal of the fused data block is not completed, the process jumps to the step of sequentially taking out data columns from the fused data block as the array to be transmitted.

6. The multi-source data fusion uploading method according to claim 5 is characterized in that: The method of modulating the plurality of data of the array to be transmitted onto a plurality of subcarriers respectively according to the transmission timing and the number of columns of the array to be transmitted in the fused data block to obtain a plurality of modulated subcarriers includes: According to a third formula, the plurality of data of the array to be transmitted are modulated onto a plurality of subcarriers respectively, wherein the third formula is: In the formula, For the Subcarriers in The waveform of the moment, The first data, is the bias constant, is a sine function, is the period length of the reference fundamental wave, is the frequency of the reference fundamental wave, is the number of columns of the array to be transmitted in the data block, is the initial phase; The transmitting the multiple modulated subcarriers comprises: According to the fourth formula, the multiple modulated subcarriers are synthesized to obtain a synthesized waveform, wherein the fourth formula is: In the formula, is the synthetic waveform, To adjust the number of subcarriers, For the The waveform of the subcarriers; The synthesized waveform is up-converted to a predetermined frequency band for transmission.

7. A method for receiving fused data, characterized in that: Applied to a platform server, the server is connected to a data fusion node signal, and the fusion data receiving method includes: Acquire a plurality of fused data blocks and a plurality of first maximum value pairs obtained according to the multi-source data fusion uploading method according to any one of claims 1 to 6; Decoding the multiple fused data blocks to obtain multiple third data sets, wherein each third data set corresponds to a fused data block; For each third data set, the third data set is de-mapped according to the predetermined interval and the first maximum value pair corresponding to the third data set to obtain a fourth data set.

8. A multi-source data fusion uploading device, characterized in that: Used to implement the multi-source data fusion uploading method according to any one of claims 1 to 6, the multi-source data fusion uploading device comprises: A data acquisition module, used to acquire a plurality of first data sets, wherein each first data set corresponds to a data acquisition terminal, and the first data set includes a plurality of data queues; A data outlier analysis module, configured to find the closest class center from a plurality of classes for each first data set, and determine an outlier index according to a distance from the closest class center, wherein the plurality of classes are obtained based on clustering; An interval mapping module, used for mapping the plurality of first data sets to a predetermined interval, to obtain a plurality of second data sets and a plurality of first maximum value pairs, wherein each first data set corresponds to a second data set and a first maximum value pair, and the first maximum value pair includes a maximum value and a minimum value of the first data set; as well as, The fusion sending module is used to encode the multiple second data sets according to the coding fineness determined by the multiple outlier indices to obtain multiple fused data blocks, and transmit the multiple fused data blocks and the multiple first maximum value pairs.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images