A data stream feature integration method, device and electronic equipment

Abstract

The application provides a data stream feature integration method and device and electronic equipment, and relates to the technical field of data security.In the application, first, a first attribute data set corresponding to a first data stream and a second attribute data set corresponding to a second data stream are acquired, based on first attribute data in the first attribute data set and second attribute data corresponding to the first attribute data in the second attribute data set, each attribute mutation index value is calculated, then each attribute mutation index value is fused to obtain an attribute mutation fusion index value, and based on the attribute mutation fusion index value, a first weight value corresponding to the first data stream and a second weight value corresponding to the second data stream are reconstructed; finally, the first data stream and the second data stream are feature-integrated based on the first weight value and the second weight value. In this way, the accuracy of feature integration of the first data stream and the second data stream can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data security, and particularly relates to a data stream feature integration method and device and electronic equipment. BACKGROUND

[0002] Data identification is the basis of enterprise data asset management and data security control. The data management end may receive massive data streams from different data sources in the process of data asset management. The data management end identifies the massive data streams to facilitate better utilization and management of data resources in the later period.

[0003] The accuracy of data identification depends on the accuracy of data features when the data management end identifies the massive data streams. Since any one of the massive data streams has multiple data features, and any two of the massive data streams may have high similarity data features, the data feature set generated by the massive data streams is too large and has redundant features. Therefore, the features of the data streams need to be integrated.

[0004] The existing data stream feature integration method extracts and comprehensively calculates the features of multiple data streams, sets corresponding static weights (fixed weights) for the multiple data streams with correlation, and integrates the features of the multiple data streams according to the comprehensive calculation result.

[0005] However, in the above manner, when the features of the multiple data streams with correlation are integrated, if the feature attributes of one of the multiple data streams change, since the static weight set for the data stream remains unchanged, the accuracy of integrating the features of the data stream is reduced. SUMMARY

[0006] The present application provides a data stream feature integration method, device and electronic equipment for improving the accuracy of feature integration of various data streams. The specific technical solutions are as follows:

[0007] In a first aspect, the present application provides a data stream feature integration method, comprising:

[0008] obtaining a first attribute data set corresponding to a first data stream and a second attribute data set corresponding to a second data stream, wherein the attribute data set is obtained by sampling the attributes of the data stream in real time;

[0009] calculating each attribute mutation index value based on the first attribute data in the first attribute data set and the second attribute data corresponding to the first attribute data in the second attribute data set;

[0010] fusing the attribute mutation indicator values to obtain an attribute mutation fusion indicator value, and reconstructing a first weight value corresponding to the first data stream and a second weight value corresponding to the second data stream based on the attribute mutation fusion indicator value;

[0011] performing feature integration on the first data stream and the second data stream based on the first weight value and the second weight value.

[0012] Based on the above method, the first attribute of the first data stream and the second attribute of the second data stream can be identified and detected in real time. According to the attribute mutation indicator values related to the first attribute and the second attribute, the attribute mutation relationship between the first data stream and the second data stream can be determined. According to the attribute mutation fusion indicator value, the first weight value set for the first data stream and the second weight value set for the second data stream can be adjusted in real time. Not only the redundant feature information between the first data stream and the second data stream is removed, but also the accuracy of feature integration between the first data stream and the second data stream is improved.

[0013] In a possible implementation, the calculating of the attribute mutation indicator values based on the first attribute data in the first attribute data set and the second attribute data corresponding to the first attribute data in the second attribute data set comprises:

[0014] selecting the first attribute data in a set sampling time period from the first attribute data set and selecting the second attribute data in the set sampling time period from the second attribute data set, wherein one attribute data corresponds to one attribute feature;

[0015] calculating a first attribute detail value corresponding to the first attribute data and a second attribute detail value corresponding to the second attribute data based on the first attribute data and the second attribute data, and taking the sum of the first attribute detail value and the second attribute detail value as a first attribute mutation indicator value, wherein the attribute detail value represents the explicitness of the attribute of the attribute data;

[0016] calculating an attribute dependence between the first attribute data and the second attribute data based on the first attribute data and the second attribute data, and taking the attribute dependence as a second attribute mutation indicator value, wherein the attribute dependence represents the correlation between the attributes of the first attribute data and the second attribute data;

[0017] calculating a first frequency of the first attribute data in a set detection sample and a second frequency of the second attribute data in the detection sample based on the first attribute data and the second attribute data, and taking the sum of the first frequency and the second frequency as a third attribute mutation indicator value.

[0018] Based on the above method, the attribute mutation relationship between the first data stream and the second data stream can be determined, thereby providing a basis for subsequent feature integration of the first data stream and the second data stream.

[0019] In a possible implementation, the attribute mutation fusion indicator value is obtained by fusing the attribute mutation indicator values, and the method comprises the following steps:

[0020] The first attribute mutation indicator value is assigned a corresponding first mutation weight value to obtain a first weighted attribute mutation indicator value;

[0021] The second attribute mutation indicator value is assigned a corresponding second mutation weight value to obtain a second weighted attribute mutation indicator value;

[0022] The third attribute mutation indicator value is assigned a corresponding third mutation weight value to obtain a third weighted attribute mutation indicator value;

[0023] The sum of the first weighted attribute mutation indicator value, the second weighted attribute mutation indicator value, and the third weighted attribute mutation indicator value is taken as the attribute mutation fusion indicator value.

[0024] Based on the above method, the attribute mutation fusion indicator value can be obtained, and the first weight value corresponding to the first data stream and the second weight value corresponding to the second data stream can be adjusted according to the numerical interval in which the attribute mutation fusion indicator value is located.

[0025] In a possible implementation, the first weight value corresponding to the first data stream and the second weight value corresponding to the second data stream are reconstructed, and the method comprises the following steps:

[0026] It is judged whether the attribute mutation fusion indicator value is less than or equal to a set attribute mutation value;

[0027] If yes, the first weight value and the second weight value are not changed;

[0028] If no, the first weight value and the second weight value are adjusted.

[0029] Based on the above method, the first weight value corresponding to the first data stream and the second weight value corresponding to the second data stream can be adjusted in real time, thereby improving the accuracy of feature integration of the first data stream and the second data stream.

[0030] In a second aspect, the present application provides a data stream feature integration device, comprising:

[0031] The data acquisition module is configured to acquire a first attribute data set corresponding to the first data stream and a second attribute data set corresponding to the second data stream, wherein the attribute data set is obtained by sampling attributes of the data stream in real time.

[0032] The feature extraction module is configured to calculate attribute mutation indicator values based on first attribute data in the first attribute data set and second attribute data corresponding to the first attribute data in the second attribute data set.

[0033] The data fusion module is configured to fuse the attribute mutation indicator values to obtain an attribute mutation fusion indicator value, and reconstruct a first weight value corresponding to the first data stream and a second weight value corresponding to the second data stream based on the attribute mutation fusion indicator value.

[0034] The feature integration module is configured to perform feature integration on the first data stream and the second data stream based on the first weight value and the second weight value.

[0035] In a possible implementation, the feature extraction module is specifically configured to:

[0036] select first attribute data in a set sampling time period from the first attribute data set and select second attribute data in the set sampling time period from the second attribute data set, wherein one attribute data corresponds to one attribute feature;

[0037] calculate first attribute detail values corresponding to the first attribute data and second attribute detail values corresponding to the second attribute data based on the first attribute data and the second attribute data, and take a sum of the first attribute detail values and the second attribute detail values as a first attribute mutation indicator value, wherein the attribute detail value represents a degree of explicitness of the attribute of the attribute data;

[0038] calculate an attribute dependency between the first attribute data and the second attribute data based on the first attribute data and the second attribute data, and take the attribute dependency as a second attribute mutation indicator value, wherein the attribute dependency represents a degree of association between the attributes of the first attribute data and the second attribute data;

[0039] calculate a first frequency of the first attribute data in a set detection sample and a second frequency of the second attribute data in the detection sample based on the first attribute data and the second attribute data, and take a sum of the first frequency and the second frequency as a third attribute mutation indicator value.

[0040] In a possible implementation, the data fusion module is specifically configured to:

[0041] assigning a corresponding first mutation weight value to the first attribute mutation indicator value to obtain a first weighted attribute mutation indicator value;

[0042] assigning a corresponding second mutation weight value to the second attribute mutation indicator value to obtain a second weighted attribute mutation indicator value;

[0043] assigning a corresponding third mutation weight value to the third attribute mutation indicator value to obtain a third weighted attribute mutation indicator value;

[0044] taking the sum of the first weighted attribute mutation indicator value, the second weighted attribute mutation indicator value and the third weighted attribute mutation indicator value as the attribute mutation fusion indicator value.

[0045] In a possible implementation, the data fusion module is further configured to:

[0046] determine whether the attribute mutation fusion indicator value is less than or equal to a set attribute mutation value;

[0047] if yes, the first weight value and the second weight value are not changed;

[0048] if no, the first weight value and the second weight value are adjusted.

[0049] In a third aspect, the present application provides an electronic device, comprising:

[0050] a memory configured to store a computer program;

[0051] a processor configured to execute the computer program stored in the memory to implement the steps of the data stream feature integration method.

[0052] In a fourth aspect, the present application provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the data stream feature integration method.

[0053] The technical effects of each of the above-mentioned second to fourth aspects and each of the possible solutions of the first aspect can be referred to the above-mentioned technical effect descriptions of the first aspect or the various possible solutions of the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0054] Figure 1 a flow chart of a data stream feature integration method provided by the present application;

[0055] Figure 2 a data stream feature integration system architecture schematic diagram provided by the present application;

[0056] Figure 3A flowchart for verifying a first attribute and a second attribute is provided for the present application.

[0057] Figure 4 A structural schematic diagram of a data stream feature integration device is provided for the present application.

[0058] Figure 5 A structural schematic diagram of an electronic device is provided for the present application. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application will be described in further detail below with reference to the accompanying drawings. The specific operation method in the method embodiment can also be applied to the device embodiment or the system embodiment. It should be noted that in the description of the present application, "multiple" is understood as "at least two". The association relationship of the associated objects is described, which means that there can be three relationships, for example, A and / or B can represent the following three cases: A exists alone, A and B exist together, and B exists alone. A and B are connected, which means that A and B are directly connected and A and B are connected through C. In addition, in the description of the present application, "first", "second", etc. are used only for the purpose of distinguishing the description, and cannot be understood as indicating or implying relative importance, nor can it be understood as indicating or implying order.

[0060] The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0061] Data recognition is the basis of enterprise data asset management and data security control. The data management end may receive a large amount of data streams from different data sources in the process of data asset management. The data management end identifies the large amount of data streams in order to better utilize and manage data resources in the later period.

[0062] The accuracy of data recognition depends on the accuracy of data features when the data management end identifies the large amount of data streams. Since any one of the large amount of data streams has multiple data features, and any two of the large amount of data streams may have high similarity data features, which will make the data feature set generated by the large amount of data streams too large and have redundant features, therefore, the features of the data stream need to be integrated.

[0063] The existing data stream feature integration method is to extract and comprehensively calculate the features of multiple data streams, set corresponding static weights (fixed weights) for multiple data streams with correlation, and integrate the features of multiple data streams according to the comprehensive calculation result.

[0064] However, in the above manner, when the features of the multiple data streams with correlation are integrated, if the feature attribute of one of the multiple data streams changes, since the static weight set for the data stream remains unchanged, the accuracy of integrating the features of the data stream is reduced.

[0065] In view of this, in order to improve the accuracy of integrating the features of a large amount of data streams, the present application provides a data stream feature integration method, which specifically comprises: first, obtaining a first attribute data set corresponding to a first data stream and a second attribute data set corresponding to a second data stream, calculating each attribute mutation index value based on the first attribute data in the first attribute data set and the second attribute data corresponding to the first attribute data in the second attribute data set, then fusing each attribute mutation index value to obtain an attribute mutation fusion index value, and reconstructing a first weight value corresponding to the first data stream and a second weight value corresponding to the second data stream based on the attribute mutation fusion index value; finally, integrating the features of the first data stream and the second data stream based on the first weight value and the second weight value.

[0066] As can be seen, by calculating each attribute mutation index value, the attribute mutation relationship between the first data stream and the second data stream can be determined, and whether to adjust the first weight corresponding to the first data stream and the second weight corresponding to the second data stream is determined according to the size of the attribute mutation fusion index value, i.e. the size of the attribute mutation degree between the first data stream and the second data stream. In this way, the accuracy of integrating the features of the first data stream and the second data stream can be improved.

[0067] Referring to Figure 1 The method comprises:

[0068] S1, obtaining a first attribute data set corresponding to a first data stream and a second attribute data set corresponding to a second data stream.

[0069] Firstly, the method provided by the present application can be applied to Figure 2 as shown in the system architecture, the system architecture comprises a data stream receiving device, a data stream processing device and a target terminal, and the method provided by the present application can run in the data stream processing device.

[0070] The data stream receiving device is used to receive various types of data streams, such as a medical-related data stream A or an education-related data stream B, etc. When receiving various types of data streams, the data stream receiving device can forward the various types of data streams to the data stream processing device. The present application does not make specific limitations on the types and quantities of data streams that the data stream receiving device can receive.

[0071] The data stream processing device is used for identifying features of received various types of data streams, and selecting a corresponding feature integration mode according to an attribute mutation relationship between attributes of the various types of data streams, removing redundant feature information between the various types of data streams, reducing system space occupied by a data feature set generated by the massive data streams, and improving accuracy of feature integration of the various types of data.

[0072] The target terminal can be used for receiving an update message about adjusting respective weights of the various data streams sent by the data stream processing device, and returning feedback information about the weight adjustment to the data stream processing device according to the update message, so that the data stream processing device sets respective weight values of the various data streams according to the weight values adjusted in the feedback information.

[0073] In the embodiment of the application, the data stream processing device first receives a first data stream (data stream A) and a second data stream (data stream B), and takes the first data stream as a sample data stream and the second data stream as a comparison data stream; the data stream processing device can also receive N data streams with a number greater than 2, and each of the N data streams is provided with a corresponding initial weight value, then selects one data stream from the N data streams as a first data stream (sample data stream), takes all the remaining data streams in the N data streams as a second data stream (comparison data stream), and finally performs subsequent feature integration on the N data streams. The application only takes data stream A and data stream B as examples to describe the invention in detail.

[0074] The data stream processing device performs attribute sampling on the first data stream and the second data stream. When performing attribute sampling on the data stream, attribute sampling can be performed on the data stream in real time by setting a fixed sampling time length, or attribute sampling can be performed on the data stream in segments according to a set data size. The application does not make a specific limitation on the way of attribute sampling on the data stream.

[0075] After attribute sampling is performed on the first data stream and the second data stream, a first attribute data set corresponding to the first data stream and a second attribute data set corresponding to the second data stream can be obtained. The first attribute data set contains a plurality of first attribute data collected in a plurality of set sampling time periods, for example, first attribute data collected in a sampling time period of 0-2s and first attribute data collected in a sampling time period of 2-4s.

[0076] Here, it should be pointed out that the attribute data set also includes parameter values of various types associated with attributes. The parameter values of various types are obtained by performing attribute sampling on the data stream, and then analyzing and counting the sampled attributes of the data stream, and can be used for detecting the attributes of the data stream.

[0077] In this way, the first attribute data set corresponding to the first data stream and the second attribute data set corresponding to the second data stream can be acquired, so as to facilitate subsequent calculation of an index value related to attributes of the first data stream and the second data stream.

[0078] S2, based on the first attribute data in the first attribute data set and the second attribute data corresponding to the first attribute data in the second attribute data set, calculating each attribute mutation index value.

[0079] In the embodiment of the application, after the data stream processing device receives the first attribute data set and the second attribute data set, the data stream processing device can select first attribute data in a set sampling time period from the first attribute data set, and select second attribute data in the set sampling time period from the second attribute data set. For example, in the first attribute data set constituted by a sampling time period of 0-10s, first attribute data collected from the first data stream in a sampling time period of 0-2s is selected. It should be noted that attribute data collected in each sampling time period corresponds to an attribute feature.

[0080] In a possible implementation, the data stream processing device can also directly receive the first data stream and the second data stream in the same set sampling time period, and then perform attribute sampling on the first data stream and the second data stream to obtain first attribute data corresponding to the first data stream in the set sampling time period and second attribute data corresponding to the second data stream in the set sampling time period. After attribute sampling on the first data stream and the second data stream, the step of analyzing and counting the sampled attribute of the data stream is also included, which will not be described herein again.

[0081] After the data stream processing device acquires the first attribute data and the second attribute data, the data stream processing device can verify the first attribute (an attribute corresponding to the first attribute data) and the second attribute (an attribute corresponding to the second attribute data) according to parameter values of each type associated with the first attribute included in the first attribute data and parameter values of each type associated with the second attribute included in the second attribute data. For specific steps of verifying the first attribute and the second attribute, refer to Figure 3 As shown in the figure, the steps are as follows:

[0082] S31, calculating the first attribute mutation index value.

[0083] In the embodiment of the application, the data stream processing device can first calculate a first attribute detail value corresponding to the first attribute data and a second attribute detail value corresponding to the second attribute data, and then calculate a sum of the first attribute detail value and the second attribute detail value, and take the sum of the first attribute detail value and the second attribute detail value as the first attribute mutation index value. The attribute detail value represents a parameter value corresponding to a type of attribute in the attribute data, and the first attribute mutation index value K dThe calculation can be performed by the following formula:

[0084]

[0085] wherein DLY i represents a single data attribute explicit dimension index, the explicit dimension index refers to an index capable of objectively measuring the attribute; DLS i represents a single data attribute explicit granularity (upper and lower limits of the explicit dimension index), α i represents a single data attribute detail granularity median, and n is a positive integer greater than or equal to 2.

[0086] Here, it should be pointed out that the above DLY i , DLS i , α i parameter values are obtained by analyzing and counting the sampling attributes of the data stream.

[0087] S32, calculating a second attribute mutation index value.

[0088] In the embodiments of the present application, after the data stream processing device obtains the first attribute data and the second attribute data, the data stream processing device can further calculate an attribute dependency between the first attribute data and the second attribute data according to the parameter values of each type associated with the first attribute contained in the first attribute data and the parameter values of each type associated with the second attribute contained in the second attribute data. The attribute dependency represents the association between the attributes of the first attribute data and the second attribute data, and the attribute dependency R d can be calculated in the following formula:

[0089]

[0090] wherein RQ represents a progressive dimension index of data attribute characteristics; Cd i represents a vector coordinate of data attribute characteristics, Sd represents an offset of data attribute characteristics, Ts is a cut-off value of a longitudinal axis coordinate of data attribute characteristics, and Td is a cut-off value of a transverse axis coordinate of data attribute characteristics.

[0091] Here, it should be pointed out that the above RQ, Cd i , Sd, Td, and Ts parameter values are also obtained by analyzing and counting the sampling attributes of the data stream.

[0092] After the data stream processing device calculates the attribute dependency between the first attribute data and the second attribute data, the data stream processing device can take the obtained attribute dependency as the second attribute mutation index value.

[0093] S33, calculating a third attribute mutation index value.

[0094] In the examples of the present application, after obtaining the first attribute data and the second attribute data, the data stream processing device can further calculate a first frequency of occurrence of the first attribute in the set detection sample and a second frequency of occurrence of the second attribute in the set detection sample according to the parameter values of each type of parameter associated with the first attribute contained in the first attribute data and the parameter values of each type of parameter associated with the second attribute contained in the second attribute data, and take the sum of the first frequency and the second frequency as the third attribute mutation indicator value.

[0095] The third attribute mutation indicator value R S may be calculated according to the following formula:

[0096]

[0097] wherein CAU j represents the capture frequency of the attribute feature; SUC represents the set detection sample capacity; and β j represents the capture success rate of the attribute feature.

[0098] It should be noted here that the CAU j , SUC, and β j parameter values are also obtained by analyzing and counting the sampled attributes of the data stream.

[0099] By calculating the first attribute mutation indicator value, the second attribute mutation indicator value, and the third attribute mutation indicator value, the attribute mutation relationship between the first data stream and the second data stream can be determined, which provides a basis for subsequent feature integration of the first data stream and the second data stream.

[0100] S3, fuse the attribute mutation indicator values to obtain an attribute mutation fusion indicator value, and reconstruct the first weight value corresponding to the first data stream and the second weight value corresponding to the second data stream based on the attribute mutation fusion indicator value.

[0101] In the embodiments of the present application, after obtaining the first attribute mutation indicator value K d , the second attribute mutation indicator value R d , and the third attribute mutation indicator value R S , the data stream processing device can assign a corresponding first mutation weight value K1 to the first attribute mutation indicator value to obtain a first weighted attribute mutation indicator value K d ·K1.

[0102] assign a corresponding second mutation weight value K2 to the second attribute mutation indicator value to obtain a second weighted attribute mutation indicator value R d ·K2; and assign a corresponding third mutation weight value K3 to the third attribute mutation indicator value to obtain a third weighted attribute mutation indicator value R S• K3. In the embodiments of the present application, the size of the mutation weight value can be set according to the actual requirement of the data stream feature integration, and the present application does not make specific limitation thereto.

[0103] After obtaining the above-mentioned weighted attribute mutation indicator values, the data stream processing device can take the sum of the first weighted attribute mutation indicator value, the second weighted attribute mutation indicator value and the third weighted attribute mutation indicator value as the attribute mutation fusion indicator value, and the representation of the attribute mutation fusion indicator value H is as follows:

[0104] H = K d • K1+R d • K2+R S • K3

[0105] According to the attribute mutation fusion indicator value, the attribute mutation degree between the first attribute of the first data stream and the second attribute of the second data stream can be determined.

[0106] In the embodiments of the present application, the attribute mutation degree between the first attribute and the second attribute can be divided into four attribute mutation levels as shown in Table 1, i.e., high, high, moderate and low, and the number of attribute mutation degrees divided by the present application is not specifically limited.

[0107] Attribute mutation fusion index value H Attribute mutation degree H>＝85 Higher 70<＝H<85 High 60<＝H<70 Moderate H<60 Low

[0108] Table 1

[0109] As shown in Table 1, the respective weights corresponding to the feature integration of the first data stream and the second data stream can be set by judging the attribute mutation degree level or the numerical interval of the attribute mutation fusion indicator value H.

[0110] For example, the data stream processing device can judge whether the attribute mutation fusion indicator value is less than or equal to the set attribute mutation value, and the attribute mutation value H0 can be set to 70 or any other arbitrary positive integer greater than or equal to 0, and the present application does not make specific limitation thereto.

[0111] When the data stream processing device determines that the attribute mutation fusion indicator value is less than or equal to the set attribute mutation value, it is determined that the attribute mutation degree between the first attribute and the second attribute is low or moderate, and therefore, the first weight originally set for the first data stream and the second weight set for the second data stream can not be changed, for example, the first weight 0.5 and the second weight 0.5 originally set are not changed.

[0112] When the data stream processing device determines that the attribute mutation fusion index value is greater than the set attribute mutation value, that is, the degree of attribute mutation between the first attribute and the second attribute is high or high, the data stream processing device can increase the first weight 0.5 originally set for the first data stream to any value in the range of 0.50-0.99, and reduce the second weight 0.5 originally set for the second data stream to any value in the range of 0-0.5; the data stream processing device can also send an update message to the target terminal (user terminal) whether to adjust the respective weights corresponding to each data stream; according to the feedback information returned by the user terminal based on the update message, the first weight value corresponding to the first data stream and the second weight value corresponding to the second data stream are determined, and the size of the first weight value and the second weight value can be set according to the actual data stream feature integration demand, which is not limited in the present application. Through the above-mentioned manner, the degree of attribute mutation between the first attribute of the first data stream and the second attribute of the second data stream can be determined, and according to the attribute mutation fusion index value, it can be determined whether to adjust the first weight value originally set for the first data stream and the second weight value set for the second data stream. Not only can the redundant feature information between the first data stream and the second data stream be removed, but also the accuracy of feature integration between the first data stream and the second data stream is improved.

[0113] S4, performing feature integration on the first data stream and the second data stream based on the first weight value and the second weight value.

[0114] In the embodiments of the present application, after the data stream processing device determines the first weight value corresponding to the first data stream and the second weight value corresponding to the second data stream, it can calculate the first product of the first data stream and the first weight value, the second product of the second data stream and the second weight value, and finally calculate the sum of the first product and the second product to realize the feature integration between the first data stream and the second data stream.

[0115] In the embodiments of the present application, when the data stream processing device receives N data streams, it can first select one data stream from the N data streams as the first data stream (sample data stream), and select all the remaining data streams in the N data streams as the second data stream (comparison data stream), and then integrate the N data streams based on the above-mentioned steps of integrating the first data stream and the second data stream, which will not be repeated here.

[0116] In summary, the data stream integration method provided in the application can identify and detect the first attribute of the first data stream and the second attribute of the second data stream in real time, can determine the attribute mutation relationship between the first data stream and the second data stream by calculating the attribute mutation index values, and can adjust the first weight value set for the first data stream and the second weight value set for the second data stream in real time according to the attribute mutation fusion index value, so as to not only remove the redundant feature information between the first data stream and the second data stream, but also improve the accuracy of feature integration between the first data stream and the second data stream.

[0117] Based on the method provided in the above embodiments, the embodiments of the application further provide a data stream feature integration device, as shown in Figure 4 Fig. 1 is a structural schematic diagram of a data stream feature integration device in an embodiment of the application. The device comprises:

[0118] The data acquisition module 401 is configured to acquire a first attribute data set corresponding to a first data stream and a second attribute data set corresponding to a second data stream, wherein the attribute data set is obtained by sampling the attribute of the data stream in real time.

[0119] The feature extraction module 402 is configured to calculate attribute mutation index values based on the first attribute data in the first attribute data set and the second attribute data corresponding to the first attribute data in the second attribute data set.

[0120] The data fusion module 403 is configured to fuse the attribute mutation index values to obtain an attribute mutation fusion index value, and reconstruct a first weight value corresponding to the first data stream and a second weight value corresponding to the second data stream based on the attribute mutation fusion index value.

[0121] The feature integration module 404 is configured to integrate the features of the first data stream and the second data stream based on the first weight value and the second weight value.

[0122] In a possible implementation, the feature extraction module 402 is specifically configured to:

[0123] select the first attribute data in a set sampling time period from the first attribute data set, and select the second attribute data in the set sampling time period from the second attribute data set, wherein one attribute data corresponds to one attribute feature.

[0124] based on the first attribute data and the second attribute data, a first attribute detail value corresponding to the first attribute data and a second attribute detail value corresponding to the second attribute data are calculated, and a sum of the first attribute detail value and the second attribute detail value is taken as a first attribute mutation indicator value, wherein the attribute detail value represents a degree of explicitness of an attribute of the attribute data;

[0125] based on the first attribute data and the second attribute data, an attribute dependence between the first attribute data and the second attribute data is calculated, and the attribute dependence is taken as a second attribute mutation indicator value, wherein the attribute dependence represents a degree of correlation between the attributes of the first attribute data and the second attribute data;

[0126] based on the first attribute data and the second attribute data, a first frequency of the first attribute data in a set of detection samples and a second frequency of the second attribute data in the detection samples are calculated, and a sum of the first frequency and the second frequency is taken as a third attribute mutation indicator value.

[0127] In a possible implementation, the data fusion module 403 is specifically configured to:

[0128] a first mutation weight value corresponding to the first attribute mutation indicator value is assigned, to obtain a first weighted attribute mutation indicator value;

[0129] a second mutation weight value corresponding to the second attribute mutation indicator value is assigned, to obtain a second weighted attribute mutation indicator value;

[0130] a third mutation weight value corresponding to the third attribute mutation indicator value is assigned, to obtain a third weighted attribute mutation indicator value;

[0131] a sum of the first weighted attribute mutation indicator value, the second weighted attribute mutation indicator value, and the third weighted attribute mutation indicator value is taken as the attribute mutation fusion indicator value.

[0132] In a possible implementation, the data fusion module 403 is further configured to:

[0133] it is judged whether the attribute mutation fusion indicator value is less than or equal to a set attribute mutation value;

[0134] if yes, the first weight value and the second weight value are not changed;

[0135] if no, the first weight value and the second weight value are adjusted.

[0136] Based on the same inventive concept, the embodiments of the present application further provide an electronic device, which can realize the functions of the foregoing data flow feature integration apparatus, and the functions of the foregoing data flow feature integration apparatus are not repeated here.Figure 5 The electronic device comprises:

[0137] at least one processor 501, and a memory 502 connected with the at least one processor 501, and the specific connection medium between the processor 501 and the memory 502 is not limited in the embodiments of the present application, Figure 5 The connection between the processor 501 and the memory 502 is taken as an example of connection through a bus 500 in the foregoing. Figure 5 The connection mode between other components is only schematically illustrated, and is not limited. The bus 500 can be divided into an address bus, a data bus, a control bus, and the like, for the convenience of representation, Figure 5 In the foregoing, only one thick line is used to represent the connection, but it does not mean that there is only one bus or only one type of bus. Alternatively, the processor 501 can also be referred to as a controller, and the name is not limited.

[0138] In the embodiments of the present application, the memory 502 stores instructions executable by the at least one processor 501, and the at least one processor 501 can execute the data stream feature integration method discussed in the foregoing by executing the instructions stored in the memory 502. The processor 501 can realize the functions of various modules in the apparatus shown in the foregoing. Figure 4

[0139] The processor 501 is the control center of the apparatus, can connect all parts of the control device through various interfaces and lines, and can realize various functions and process data of the apparatus by running or executing instructions stored in the memory 502 and calling data stored in the memory 502, thereby monitoring the apparatus as a whole.

[0140] In a possible design, the processor 501 can include one or more processing units, and the processor 501 can integrate an application processor and a modem processor, wherein the application processor mainly processes an operating system, a user interface, and an application program, and the modem processor mainly processes wireless communication. It can be understood that the foregoing modem processor can also not be integrated into the processor 501. In some embodiments, the processor 501 and the memory 502 can be implemented on the same chip, and in some embodiments, they can also be respectively implemented on independent chips.

[0141] ​The processor 501 can be a general processor, such as a central processing unit (CPU), a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic component, a discrete hardware component, and can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general processor can be a microprocessor or any conventional processor. The steps of the data stream feature integration method disclosed in the embodiments of the present application can be directly embodied as execution completed by a hardware processor, or executed by a combination of hardware and software modules in the processor.

[0142] The memory 502 is a non-volatile computer readable storage medium, and can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The memory 502 can include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic storage, magnetic disk, optical disk, etc. The memory 502 is any other medium capable of carrying or storing desired program code in the form of instructions or data structures and capable of being accessed by a computer, but is not limited to this. The memory 502 in the embodiments of the present application can also be a circuit or any other device capable of realizing a storage function, used to store program instructions and / or data.

[0143] By designing and programming the processor 501, the code corresponding to the data stream feature integration method introduced in the foregoing embodiments can be fixed into the chip, so that the chip can execute the steps of the data stream feature integration method of the embodiments shown in the running time. Figure 1 How to design and program the processor 501 is a technology known to those skilled in the art, which will not be described here.

[0144] Based on the same inventive concept, the embodiments of the present application also provide a storage medium storing computer instructions, when the computer instructions run on a computer, the computer executes the data stream feature integration method discussed above.

[0145] In some possible implementation, the various aspects of the data stream feature integration method provided by the present application can also be implemented in the form of a program product, which includes program codes for causing the control device to perform the steps of the data stream feature integration method according to various exemplary embodiments of the present application described above in the specification when the program product is run on the device.

[0146] Those skilled in the art should understand that the embodiments of the present application can be provided in the form of a method, a system, or a computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) containing computer-usable program codes.

[0147] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, as well as a combination of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing apparatus generate a means for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus for performing the functions specified in the flowcharts and / or block diagrams.

[0148] These computer program instructions can also be stored in a computer-readable memory that can direct the computer or other programmable data processing apparatus to work in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction apparatus that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus for performing the functions specified in the flowcharts and / or block diagrams.

[0149] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus, so that a series of operation steps are performed on the computer or other programmable data processing apparatus to produce a computer-implemented process, so that the instructions executed on the computer or other programmable data processing apparatus provide a process for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus for performing the functions specified in the flowcharts and / or block diagrams.

[0150] Obviously, many modifications and variations of the present application are possible in light of the above teachings. It is, therefore, to be understood that within the scope of the appended claims and their equivalents, the application can be practiced otherwise than as specifically described.

Claims

1. A data stream feature integration method, characterized in that, include: Obtain the first attribute data set corresponding to the first data stream and the second attribute data set corresponding to the second data stream, wherein the attribute data set is obtained by sampling the attributes of the data stream in real time; Based on the first attribute data in the first attribute data set and the second attribute data corresponding to the first attribute data in the second attribute data set, calculate the mutation index values ​​of each attribute, including: First attribute data within a set sampling time period is selected from the first attribute data set, and second attribute data within the set sampling time period is selected from the second attribute data set, wherein one attribute data corresponds to one attribute feature; Based on the first attribute data and the second attribute data, calculate the first attribute detail value corresponding to the first attribute data and the second attribute detail value corresponding to the second attribute data, and use the sum of the first attribute detail value and the second attribute detail value as the first attribute mutation index value, wherein the attribute detail value represents the degree of explicitness of the attribute data; Based on the first attribute data and the second attribute data, the attribute dependency degree between the first attribute data and the second attribute data is calculated, and the attribute dependency degree is used as the second attribute mutation index value, wherein the attribute dependency degree characterizes the degree of correlation between the attributes of the first attribute data and the second attribute data. Based on the first attribute data and the second attribute data, calculate the first frequency of the first attribute data appearing in the set detection sample and the second frequency of the second attribute data appearing in the detection sample, and use the sum of the first frequency and the second frequency as the third attribute mutation index value; By fusing the attribute mutation index values, an attribute mutation fusion index value is obtained, and based on the attribute mutation fusion index value, the first weight value corresponding to the first data stream and the second weight value corresponding to the second data stream are reconstructed; Feature integration is performed on the first data stream and the second data stream based on the first weight value and the second weight value.

2. The method as described in claim 1, characterized in that, The attribute mutation index values ​​are fused to obtain the attribute mutation fusion index value, including: Assign a corresponding first mutation weight value to the first attribute mutation index value to obtain the first weighted attribute mutation index value; Assign a corresponding second mutation weight value to the second attribute mutation index value to obtain the second weighted attribute mutation index value; Assign a corresponding third mutation weight value to the third attribute mutation index value to obtain the third weighted attribute mutation index value; The sum of the first weighted attribute mutation index value, the second weighted attribute mutation index value, and the third weighted attribute mutation index value is used as the attribute mutation fusion index value.

3. The method as described in claim 1, characterized in that, Reconstructing the first weight value corresponding to the first data stream and the second weight value corresponding to the second data stream includes: Determine whether the attribute mutation fusion index value is less than or equal to the set attribute mutation value; If so, then the first weight value and the second weight value remain unchanged; If not, then adjust the first weight value and the second weight value.

4. A data stream feature integration device, characterized in that, include: The data acquisition module is used to acquire the first attribute data set corresponding to the first data stream and the second attribute data set corresponding to the second data stream, wherein the attribute data set is obtained by sampling the attributes of the data stream in real time; The feature extraction module is used to calculate the value of each attribute mutation index based on the first attribute data in the first attribute data set and the second attribute data corresponding to the first attribute data in the second attribute data set, including: selecting the first attribute data within a set sampling time period from the first attribute data set, and selecting the second attribute data within the set sampling time period from the second attribute data set, wherein one attribute data corresponds to one attribute feature; Based on the first attribute data and the second attribute data, calculate the first attribute detail value corresponding to the first attribute data and the second attribute detail value corresponding to the second attribute data, and use the sum of the first attribute detail value and the second attribute detail value as the first attribute mutation index value, wherein the attribute detail value represents the degree of explicitness of the attribute data; Based on the first attribute data and the second attribute data, the attribute dependency degree between the first attribute data and the second attribute data is calculated, and the attribute dependency degree is used as the second attribute mutation index value, wherein the attribute dependency degree characterizes the degree of correlation between the attributes of the first attribute data and the second attribute data. Based on the first attribute data and the second attribute data, calculate the first frequency of the first attribute data appearing in the set detection sample and the second frequency of the second attribute data appearing in the detection sample, and use the sum of the first frequency and the second frequency as the third attribute mutation index value; The data fusion module is used to fuse the attribute mutation index values ​​to obtain attribute mutation fusion index values, and based on the attribute mutation fusion index values, reconstruct the first weight value corresponding to the first data stream and the second weight value corresponding to the second data stream. The feature integration module is used to integrate features of the first data stream and the second data stream based on the first weight value and the second weight value.

5. The apparatus as described in claim 4, characterized in that, The data fusion module is specifically used for: Assign a corresponding first mutation weight value to the first attribute mutation index value to obtain the first weighted attribute mutation index value; Assign a corresponding second mutation weight value to the second attribute mutation index value to obtain the second weighted attribute mutation index value; Assign a corresponding third mutation weight value to the third attribute mutation index value to obtain the third weighted attribute mutation index value; The sum of the first weighted attribute mutation index value, the second weighted attribute mutation index value, and the third weighted attribute mutation index value is used as the attribute mutation fusion index value.

6. The apparatus as claimed in claim 4, characterized in that, The data fusion module is also used for: Determine whether the attribute mutation fusion index value is less than or equal to the set attribute mutation value; If so, then the first weight value and the second weight value remain unchanged; If not, then adjust the first weight value and the second weight value.

7. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, when executing a computer program stored in the memory, implements the method steps of any one of claims 1-3.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method described in any one of claims 1-3.

Images