Method and system for data feature extraction

Abstract

Systems and methods are provided for data feature extraction based on feedback from a network. The system is generally useful for enabling feature-specific network operations, including one or more of access control, data compression, and selective encryption. Further, methods and systems for data compression based on feature similarity are provided, and may include privacy protection.

Description

Technical Field

[0001] The present invention relates to the field of communication networks, and in particular to a system and method for extracting data features. Background Art

[0002] Open Mobile Alliance (OMA) download digital rights management (DLDRM) focuses on download digital rights management (DRM) (e.g., content container for downloadable objects, as well as DRM key and rights management). OMA DLDRM adopts the 3rd Generation Partnership Project (3 rd OMA DLDRM also adopts a protected stream format, which indicates the use of selective encryption to support OMA DLDRM, and defines how to reconstruct only partially encrypted streams with sufficiently high quality. One goal of these selected formats is to reduce computational complexity. 3GPP also provides several options to support stream security. Current literature discusses that network nodes can process data content to enable artificial intelligence (AI) training and data transmission / data compression. As can be seen from the above, there is a trend that in certain scenarios, the network may process data (for example, perform actions that may include one or more of selective encryption, data compression, and data processing).

[0003] Feature extraction is a process of dimensionality reduction by which an initial set of raw data is reduced to smaller, more manageable groups of data for further processing. Feature extraction involves reducing the number of resources required to describe a large dataset. One major problem when performing analysis on complex data arises from the number of variables involved. Furthermore, feature extraction is a general term for methods of constructing combinations of variables to circumvent these issues while still describing the data with sufficient accuracy. For example, when the input data to be processed is too large and is suspected to be redundant, this input data can be converted into a reduced set of features. It can be expected that the selected features contain relevant information from the input data, so that the desired task can be performed by using this reduced representation, such as the selected features, instead of the complete initial data.

[0004] Feature extraction can be used to reduce the amount of resources required to process data, typically without losing important or relevant information. Prior art implementations extract data features (e.g., based on frequency of occurrence) without considering the usefulness of these extracted data features. As a result, these extracted data features may not be the most useful and may therefore result in poor performance of network operations (e.g., when features are based on access control or data compression).

[0005] Some literature points out that three-dimensional (3D) holographic technology can become a potential scenario for future communication networks. 3D holographic technology is a next-generation media technology that can present postures and facial expressions through holographic displays. The content to be displayed can be obtained through real-time capture, transmission or 3D rendering technology. It should be understood that multiple 3D holographic displays can generate a large amount of data. Storing, transmitting and processing such a large amount of data will have a negative impact on network performance and may also cause scalability problems.

[0006] Various data compression methods have been proposed, especially for data that is transmitted, stored, and processed within a network. Currently, lossy compression algorithms are more suitable for use in 3D holographic displays, where the original data cannot be retrieved from the compressed data, and the data size is permanently reduced by eliminating redundant data. Traditional lossy compression methods typically compress information randomly, which may be detrimental to data analysis. In an attempt to optimize data compression, current lossy compression methods utilize data features (e.g., important data or interesting information related to human contextual attributes) to try to maximize compression gain while avoiding the loss of important information. Current lossy compression techniques may have several problems. For example, when generating or selecting a compression solution based on data features, the accuracy of feature extraction can be a key point in obtaining good compression performance. In addition, current feature extraction is based on the needs of data consumers, which may have one or more of unusual features, specific features, and abnormal features. This may cause another problem because static compression matching may not be feasible for unusual or specific or abnormal features.

[0007] Therefore, new methods and systems associated with data feature extraction are needed to enable feature-specific network operations (which may include data compression or access control) in future networks that are not subject to one or more limitations of the prior art.

[0008] The purpose of the background art is to disclose information that the applicant believes may be relevant to the present invention. It is not necessary to admit, nor should it be construed, that any of the above information constitutes prior art against the present invention. Summary of the invention

[0009] An object of embodiments of the present invention is to provide a method and system for data feature extraction.

[0010] One aspect of the present invention provides a system for data feature extraction. The system includes a feature extraction router and a feature verification center, wherein each of the feature extraction router and the feature verification center has one or more associated memories storing instructions. The instructions are executed by one or more associated processors so that the feature extraction router is used to receive a feature extraction request from a data consumer, extract data features in response to the request, and send the data features to the feature verification center. The instructions are executed by one or more associated processors so that the feature verification center is used to determine whether the data feature belongs to a feature set, wherein the feature set includes existing data features provided offline by a third party; after determining that the data feature does not belong to the feature set, generating feedback indicating a potential data feature, wherein the potential data feature is associated with the data feature and the existing data feature, and sending the feedback to the feature extraction router to re-extract new data features; and after determining that the data feature belongs to the feature set, triggering a network operation that enables the feature based at least in part on a valid data feature set, wherein the valid data feature set is associated with the feature set and the data feature.

[0011] In some embodiments, to determine whether the data feature belongs to the feature set, the feature verification center is also used to calculate the feature similarity between the data feature and the existing data feature, the feature similarity indicates the similarity level, and after determining that the feature similarity exceeds a threshold, it is determined that the data feature belongs to the feature set.

[0012] In some embodiments, the feature extraction router is further configured to receive the feedback and re-extract the new data feature based on the feedback. In some embodiments, the new data feature includes at least one of the potential data features. In some embodiments, the effective data feature set includes at least one of the following: a feature ID list with corresponding feature similarities; and a privacy indication of whether sensitive data is protected, wherein the privacy indication is generated by the feature verification center.

[0013] In some embodiments, the system further comprises a function for implementing a network operation to enable a feature, the feature verification center triggering the function by sending the valid data feature set to the function. In some embodiments, the network operation to enable a feature comprises feature-based data compression. In some embodiments, the function is a compression controller, the compression controller being used to determine a compression solution based on at least one of an input, a compression performance of an existing compression solution, and an indication from a data consumer, wherein the input comprises the valid data feature set, wherein the indication indicates whether the data satisfies necessary service requirements after decompression. In some embodiments, the compression controller is further used to compress the data based on the compression solution to obtain compressed data, and to send the compressed data to the data consumer, wherein in order to obtain the data from the compressed data, the data consumer performs decompression.

[0014] In some embodiments, the system further comprises a compression controller, wherein the compression controller is configured to select a compression solution based on the characteristic data and send the compression solution to a parameter configuration function, wherein the parameter configuration function is configured to generate a parameter configuration based on the compression solution.

[0015] One aspect of the present invention provides a feature extraction method. The method includes: a feature extraction router receives a feature extraction request from a data consumer; the feature extraction router extracts data features in response to the request; and the feature verification center determines whether the data feature belongs to a feature set, wherein the feature set includes existing data features provided offline by a third party. After determining that the data feature does not belong to the feature set, the feature verification center generates feedback indicating a potential data feature, wherein the potential data feature is identified from the data feature and the existing data feature or is associated with the data feature and the existing data feature, and the feature verification center sends the feedback to the feature extraction router to re-extract new data features. After determining that the data feature belongs to the feature set, the feature verification center triggers a network operation that enables the feature based at least in part on a valid data feature set, and the valid data feature set is associated with the feature set and the data feature.

[0016] In some embodiments, when determining whether the data feature belongs to the feature set, the method also includes: the feature verification center calculates the feature similarity between the data feature and the existing data feature, and the feature similarity indicates the similarity level; and after determining that the feature similarity exceeds a threshold, the feature verification center determines that the data feature belongs to the feature set.

[0017] In some embodiments, the method further comprises: the feature extraction router receiving the feedback; and the feature extraction router re-extracting the new data feature according to the feedback. In some embodiments, the new data feature comprises at least one of the potential data features.

[0018] In some embodiments, the valid data feature set includes at least one of: a feature ID list having corresponding feature similarities; and a privacy indication of whether sensitive data is protected, wherein the privacy indication is generated by the feature verification center.

[0019] In some embodiments, the method further comprises the feature verification center triggering the function by sending the valid data feature set to a function for implementing a feature-enabled network operation. In some embodiments, the feature-enabled network operation comprises feature-based data compression. In some embodiments, the function is configured as a compression controller, the method further comprises the compression controller determining a compression solution based on at least one of an input, a compression performance of an existing compression solution, and an indication from a data consumer, wherein the input comprises the valid data feature set, wherein the indication indicates whether the data satisfies necessary service requirements after decompression.

[0020] In some embodiments, the method further includes the compression controller compressing data based on the compression solution to obtain compressed data, and sending the compressed data to the data consumer, wherein, in order to obtain the data from the compressed data, the data consumer performs decompression.

[0021] One aspect of the present invention provides a data feature verification method. The method includes: a feature verification center receives a data feature from a feature extraction router; and the feature verification center determines whether the data feature belongs to a feature set, wherein the feature set includes existing data features provided offline by a third party. After determining that the data feature does not belong to the feature set, the feature verification center generates feedback indicating a potential data feature, wherein the potential data feature is identified from the data feature and the existing data feature or is associated with the data feature and the existing data feature, and the feature verification center sends the feedback to the feature extraction router to re-extract a new data feature. After determining that the data feature belongs to the feature set, the feature verification center triggers a network operation that enables the feature based at least in part on a valid data feature set, and the valid data feature set is associated with the feature set and the data feature.

[0022] In some embodiments, when determining whether the data feature belongs to the feature set, the method also includes: the feature verification center calculates the feature similarity between the data feature and the existing data feature, and the feature similarity indicates the similarity level; and after determining that the feature similarity exceeds a threshold, the feature verification center determines that the data feature belongs to the feature set.

[0023] In some embodiments, the new data feature includes at least one of the potential data features.

[0024] In some embodiments, the valid data feature set includes at least one of the following: a feature ID list having corresponding feature similarities; a privacy indication of whether sensitive data is protected, wherein the privacy indication is generated by the feature verification center.

[0025] One aspect of the present invention provides a device for data feature verification. The device includes: a transceiver module for receiving a data feature from a feature extraction router; and a processing module for determining whether the data feature belongs to a feature set, wherein the feature set includes existing data features provided offline by a third party. After determining that the data feature does not belong to the feature set, the processing module generates feedback indicating a potential data feature, wherein the potential data feature is identified from the data feature and the existing data feature or is associated with the data feature and the existing data feature, and the transceiver module sends the feedback to the feature extraction router to re-extract a new data feature. After determining that the data feature belongs to the feature set, the processing module triggers a network operation of enabling the feature based at least in part on a valid data feature set, and the valid data feature set is associated with the feature set and the data feature.

[0026] In some embodiments, the processing module is used to calculate the feature similarity between the data feature and the existing data feature, and the feature similarity indicates a similarity level; after determining that the feature similarity exceeds a threshold, the processing module is used to determine that the data feature belongs to the feature set.

[0027] In some embodiments, the new data feature includes at least one of the potential data features.

[0028] In some embodiments, the valid data feature set includes at least one of the following: a feature ID list having corresponding feature similarities; a privacy indication of whether sensitive data is protected, wherein the privacy indication is generated by the feature verification center.

[0029] Embodiments are described above in conjunction with various aspects of the present invention, and these embodiments can be implemented based on these aspects. It will be appreciated by those skilled in the art that embodiments can be implemented in conjunction with the aspects in which they are described, but can also be implemented in conjunction with other embodiments of the aspects. When the embodiments are mutually exclusive or incompatible with each other, this will be apparent to those skilled in the art. Some embodiments can be described in conjunction with one aspect, but can also be applied to other aspects, which will be apparent to those skilled in the art. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Other features and advantages of the present invention will become apparent from the following detailed description taken in conjunction with the accompanying drawings, in which:

[0031] Figure 1 A feature extraction system of an embodiment is depicted that may be applied to data compression.

[0032] Figure 2 A feature extraction method of an embodiment is described.

[0033] Figure 3 A feature verification flow chart of an embodiment is depicted.

[0034] Figure 4 Describes an embodiment based at least in part on Figure 3 method.

[0035] Figure 5 A data compression flow chart of an embodiment is depicted.

[0036] Figure 6 A data compression process of an embodiment is depicted.

[0037] Figure 7 A block diagram of an electronic device of an embodiment is depicted that may be used to implement one or more of the methods disclosed herein.

[0038] It should be noted that throughout the drawings, similar features are identified by similar reference numerals. DETAILED DESCRIPTION

[0039] Embodiments of the present invention describe a system for data feature extraction. The system can generally be used to enable feature-specific network operations, such as one or more of access control, data compression, and selective encryption. In addition, embodiments of the present invention provide a method for data compression based on feature similarity. The method can provide privacy protection and can be used for one or more of unusual features, specific features, and abnormal features.

[0040] It should be understood that while data compression may be used throughout the present invention, it is only used as an exemplary application of embodiments of the present invention. It will be reasonably understood by those skilled in the art that embodiments of the present invention will not be limited to data compression.

[0041] According to an embodiment of the present invention, a system for data feature extraction is provided. The system is based on feedback from the network, and network actions based on feature similarity, such as data compression. The system includes a feature extraction router and a feature verification center. The feature extraction router is used to extract data features from received data, and the feature verification center is used to perform feature verification, and determine whether to generate feedback for re-extracting data features based on the verification results. It is easy to understand that the functions of these components can be deployed or enabled by different parties, so that these parties may not trust each other. In this type of scenario, some form of verification of the actions performed will be required, such as using third-party action confirmation or decentralized form of action confirmation.

[0042] According to an embodiment, when a data consumer sends a feature extraction request to the system, the system extracts from the data that the data consumer may request to obtain a data feature, verifies the data feature (the data feature may also be referred to as an extracted data feature), and then triggers a network operation that enables the feature. The network operation that enables the feature may include access control or data compression or other actions that may require feature extraction. For access control, the feature verification center may send the extracted data features to an access controller to request control of data access. For data compression, the feature verification center may send a list of feature IDs with feature similarities to a compression controller. The compression controller may then perform data compression based on feature similarity. As previously described, access control and data compression are merely exemplary use cases associated with data feature extraction and data feature verification associated with the present invention, and the techniques disclosed herein may generally be used to enable feature-specific network operations.

[0043] According to some embodiments, the system of the present invention may further include other network functions, which may include, for example, a compression controller, a parameter configuration function, an access controller, an offline training function, a service controller, and a data consumer.

[0044] Figure 1 The system 100 for extracting data features from data according to an embodiment is described, and the system can be applied to data compression of data. Figure 1 After the data consumer 102 requests a service from the service controller 104, another request including a feature extraction instruction is sent to the feature extraction router 106. The feature extraction router 106 then extracts data features from the data.

[0045] This iterative process will be discussed further below. After extracting the data features, the feature extraction router 106 sends the extracted data features to the feature verification center 108 for feature verification. During the data feature verification, the feature verification center 108 verifies whether the extracted data features belong to a feature set that can be provided or generated through offline training. The feature set may include existing features (e.g., where the existing data features may be data features identified during offline learning, where these existing data features are provided by the offline training function 116). In an optional manner, the feature verification center 108 calculates the feature similarity between the extracted data features and the existing data features, and determines that the data features belong to the feature set after determining that the feature similarity exceeds a threshold. The feature similarity can describe such a similarity level between the extracted data features and the existing data features (e.g., where the existing data features may be data features identified during offline learning, where these existing data features are provided by the offline training function 116). If the extracted data features belong to the feature set, the feature verification center 108 triggers a feature-enabled network operation based at least in part on a valid data feature set, where the valid data feature set is associated with the feature set and the extracted data features. The valid data feature set includes a list of feature IDs with corresponding feature similarities. The feature verification center 108 can trigger a feature-enabled network operation by sending the valid data feature set to the compression controller 110. Otherwise, that is, the extracted data feature does not belong to the feature set, the feature verification center 108 generates feedback about the feature extraction router 106 and sends the feedback to the feature extraction router 106 together with the feature re-extraction request. The feedback can indicate information (e.g., words, phrases, or other features in the data) that at least partially describe potential data features that can be considered for data extraction. The potential data feature that includes the information may have a high probability of being selected as a data feature for subsequent extraction. The potential data feature may be associated with the data feature selected for subsequent extraction and with existing data features that are known or previously extracted. The higher the similarity between the existing data feature and the data feature, the more likely it is that the extracted data feature is included in the feedback as a potential data feature for extraction. Optionally, the extracted data feature can be considered as a potential data feature.

[0046] After receiving the feedback, the feature extraction router 106 extracts new data features for the feedback received from the feature verification center 108. For example, the new data feature is identified and may include at least one potential data feature specified in the feedback from the feature verification center. The new data feature is then sent to the feature verification center 108 for verification. If the new data feature passes the verification process, the feature verification center 108 obtains a valid data feature set including the new data feature. The feature verification center 108 then triggers a feature-enabled network operation based at least in part on the valid data feature set, which includes valid data features that include the new data feature. In this case, it is reasonably understood that the system 100 can improve the accuracy of data feature extraction because the level of similarity between the data feature and the existing data features is low. It can be easily understood that this can be an iterative process in order to identify data features for extraction.

[0047] In some embodiments, the feature verification center 108 can check or evaluate the extracted data features to determine whether the extracted data features are related to private information or information related to people. It is easy to understand that sensitive / interesting data, or data features related to humans, may need to be noted or identified because such information may require a certain degree of protection. Therefore, selective encryption or a combination of encryption and compression can be used to protect sensitive / interesting data. In these cases, a privacy indication of whether the sensitive / interesting data is protected is required, which is generated by the feature verification center 108 and sent to the compression controller 110. In an optional manner, the privacy indication of whether the sensitive data is protected can be included in the valid data feature set together with the feature ID list and its corresponding feature similarity.

[0048] According to some embodiments, after receiving the feature ID list and its corresponding feature similarities and in some cases receiving the privacy indication, the compression controller 110 dynamically selects (or generates) a compression solution based on the input. The input may include the feature ID list with feature similarities, the privacy indication, and the compression performance of existing compression solutions (for example, which may be obtained through an offline training function).

[0049] In some embodiments, the input to the compression controller 110 may also include an indication from the data consumer 102. The indication from the data consumer 102 may indicate whether the decompressed data meets the necessary service requirements previously defined by the request received from the data consumer 102.

[0050] According to an embodiment, the parameter configuration function 114 generates and implements a parameter configuration including one or more parameters for selected compression or compression and encryption of the extracted data features. These parameters can be configured for the feature extraction router 106 and the data consumer 102. It may be noted that the compression controller 110 and the parameter configuration function 114 can be deployed by the same provider. The dashed boxes around the compression controller and the parameter configuration function illustrate this. However, it will be readily understood that the compression controller and the parameter configuration function can be provided by different entities and therefore appropriate operational communication will be required between them.

[0051] In some embodiments, when the data consumer 102 accesses the required data, the permissions related to the access to the required data may need to be checked through access control. In this case, the feature verification center 108 can send the valid data feature together with the data consumer ID (e.g., which can be associated with the data consumer 102) to the access controller 112 to enable access data control after feature verification.

[0052] Figure 2 Depicted is a feature extraction method 200, which may be performed by Figure 1 The method 200 includes: at step 202, the feature extraction router receives a feature extraction request; and at step 204, the feature extraction router extracts a data feature (the data feature may also be referred to as an extracted data feature), wherein, in some embodiments, the data feature is identified at least in part based on feedback from a feature verification center. The method 200 also includes: at step 206, the feature extraction router sends the extracted data feature to the feature verification center; and at step 208, the feature verification center verifies the extracted data feature to obtain a valid data feature. The evaluation or assessment of the extracted data feature by the feature verification center may then be fed back to the feature extraction router, thereby substantially guiding the extraction of the data feature by the feature extraction router. Specifically, the feature verification center determines whether the data feature belongs to a feature set, the feature set including existing data features provided by a third party offline (e.g., offline training function 116). After determining that the data feature does not belong to the feature set, the feature verification center generates feedback indicating a potential data feature. The potential data feature may be identified from the extracted data feature and the existing data feature in the feature set or associated with the extracted data feature and the existing data feature. The feature verification center can send feedback to the feature extraction router to start the feature extraction router to re-extract new data features. After determining that the data feature belongs to the feature set, trigger a network operation to enable the feature, wherein the network operation to enable the feature is at least partially based on the valid data feature set, wherein the valid data feature set is associated with the feature set and the data feature.

[0053] According to some embodiments, this feedback from the feature verification center may indicate information (e.g., words, phrases, or other features in the data) that describes various aspects of a data feature that may be considered as a potential data feature for extraction. The information is generated by the feature verification center, where the information indicates a level of similarity between the extracted data feature and existing data features that are known or previously extracted. Feature similarity may be calculated by the feature verification center and may describe a level of similarity between the extracted data feature and existing data features (e.g., where the existing data features may be data features identified during offline learning, where these existing data features may be provided by an offline training function). The greater the similarity between the extracted data feature and the existing data features in the feature set, the more likely it is that the extracted data feature will be included in the feedback as a potential data feature for extraction.

[0054] In some embodiments, the valid data feature set includes a list of feature IDs with similarity to its own features. The valid data feature set also includes a privacy indication, which is generated by the feature verification center based on the data features. The privacy indication indicates whether the network operation that enables the feature requires sensitive / interesting data protection.

[0055] In some embodiments, the method 200 further includes, at step 210, the feature verification center sending the valid data feature set to the compression controller. The method 200 further includes, at step 212, the compression controller selecting a compression solution based on the input; and at step 214, the compression controller sending the compression solution to a parameter configuration function. The method 200 further includes, at step 216, the parameter configuration function generating a parameter configuration based on the compression solution. In some embodiments, the input may include the valid data feature set, the compression performance of an existing compression solution, an indication indicating whether the decompressed data meets the necessary service requirements previously defined by a request received from a data consumer.

[0056] It is currently known that, typically, data feature extraction is performed by, for example, simply extracting data features based on the frequency of occurrence, without considering the usefulness of the data features. In some cases, these extracted data features may be useless. As a result, poor performance of network operations (e.g., feature-based access control or data compression) may occur. Compared to known systems, the system of the present invention first extracts data features and then verifies the extracted data features for subsequent triggering of appropriate or suitable network operations. According to an embodiment, after extracting the data features, it will be determined whether the data features belong to a feature set, and when the data features do not belong to the feature set, feedback will be generated, and it is the feedback from the feature verification center to the feature extraction router that can improve the accuracy or usefulness of the extracted data features. This interaction between the feature extraction router and the feature verification center can be an iterative process to improve one or more extracted data features, thereby achieving improved network operations, such as data compression.

[0057] According to an embodiment of the present invention, a feature verification method is provided. Figure 1 In the system shown in FIG. 1 , a feature extraction router 106 extracts data features, which are then sent to a feature verification center 108. The process performed by the feature verification center includes determining whether the extracted data features belong to a feature set. The feature set may include one or more existing or previously extracted data features that appear during system operation, and / or may include existing data features identified during offline learning.

[0058] Figure 3 A feature verification flow chart 300 is provided. Note that e.g. Figure 1 The offline training functionality shown in may be helpful for developing or creating a feature set, where the feature set includes one or more data features, where each data feature may be associated with a feature ID and feature content.

[0059] Reference Figure 3 , from e.g. Figure 1 The extracted data features 302 of the feature extraction router 106 are identified or evaluated so as to determine whether the extracted data features belong to a feature set by comparing the feature similarity value with a threshold at step 304. The data features from the feature set may be determined by Figure 1 The offline training function 116 of , or can be identified during system operation and considered suitable for inclusion in the feature set. Subsequently, two options are proposed based on the evaluation of the comparison.

[0060] In some embodiments, at step 304, the feature verification center 108 calculates a data feature similarity level between the data feature 302 and one or more of the data features included in the feature set. For example, the feature set may include data features provided by the offline training function 116. During the determination of the similarity, at step 306, the feature verification center checks whether the feature similarity between the extracted data feature 302 and the feature set exceeds a threshold. If the threshold is exceeded, the feature verification center 108 sends an indication of the feature similarity to the compression controller 110 along with a feature ID that can be associated with a specific data feature associated with the feature set. Based on this information, the compression controller 110 then generates a compression strategy at step 308. If the similarity level does not exceed the threshold, the feature verification center 108 sends feedback to the feature extraction router 106 to continue extracting data features at step 310.

[0061] In some embodiments, feature similarity can be calculated using knowledge-based methods or other similarity estimation methods. For example, similarity estimation methods can be associated with the use of scatter plots, statistical techniques, and piecewise linear interpolation. In some embodiments, subjective Bayesian methods can be used to model concept semantic similarity and obtain semantic similarity of related words. It will be readily appreciated that other concepts of similarity estimation can be used, and those skilled in the art will readily appreciate.

[0062] In some embodiments, the feature verification center 108 checks the extracted data features to determine whether the extracted data features are related to private information or information about a natural person (i.e., a human being). It may be noted that sensitive or personal data associated with data features and data features related to humans generally need to be protected. Therefore, selective encryption or encryption combined with compression or other encryption methods may be used to protect such sensitive or personal data.

[0063] Figure 4 Describes a process based at least in part on Figure 3 The method 400 of data feature verification of the method shown in . The method 400 includes receiving data features at step 402. The method 400 also includes: verifying the extracted data features at step 404; and generating a privacy indication at step 406, and obtaining a valid data feature set for the verified data features.

[0064] Benefits of embodiments of the systems and methods for data feature extraction defined in the present invention include taking into account feedback from the network during subsequent data feature extraction. In this way, the extracted features can be adjusted or refined, for example, based on similarity to previously extracted or known data features. This is an advantage over prior art techniques that do not take into account any type of feedback. In addition, the feedback indicates information (e.g., words, phrases) describing potential data features, where the information can be generated based on feature similarity. Feature similarity can describe the level of similarity of the extracted data features to existing data features (which, in some cases, can be identified during offline learning of the data features).

[0065] According to an embodiment of the present invention, a data compression method is provided. Figure 1 In a system of FIG. 1 , the compression controller 110 may dynamically select or generate a compression solution based on input received from a feature verification center. The input may include a list of feature IDs with feature similarity levels, a privacy indication, compression performance relative to existing compression solutions (which may be defined by offline training samples), and an indication from a data consumer 102. The indication from the data consumer 102 may indicate whether the data, after being decompressed for transmission to the data consumer, satisfies the necessary service requirements of the data consumer 102. One or more of these requirements may be provided by the data consumer in an initial service request.

[0066] Embodiments related to generating compression solutions will now be discussed. Figure 5 , depicting a data compression flow chart of an embodiment. If the compression solution 502 obtained from the offline training sample matches the received feature ID, where, for example, the data feature associated with the received feature ID has been previously compressed using the compression solution 502, the compression controller 110 selects a compression solution or strategy 506 based on the received feature ID and sends the selected compression ID associated with the compression solution to the parameter configuration function 114. The parameter configuration function 114 then performs parameter generation 508 and determines compression parameter configurations 510a and 510b. It should be noted that in some embodiments, the parameter configuration function 114 can be incorporated into the compression controller 110.

[0067] In some embodiments, if the privacy indication defines that data associated with the feature ID requires privacy protection, the compression controller 110 also sends the ID of the encryption algorithm to the parameter configuration function 114, which enables the compression controller to determine or evaluate the encryption parameter configuration.

[0068] In some embodiments, the compression controller 110 may send a partial parameter compression solution or strategy to the parameter configuration function 114. The parameter configuration function 114 may generate parameters 508 for compression and (if necessary) encryption, and then send these generated parameters to the feature extraction router 106 in the form of configuration 510a and to the data consumer in the form of configuration 510b. Upon receiving the respective configurations 510a and 510b, the feature extraction router 106 and the data consumer 102 configure the compression solution based on the received configurations.

[0069] If the compression solution determined from the offline training sample does not match the received feature ID, the compression controller 110 determines the compression solution based on at least one of the input, the compression performance of the existing compression solution, and the indication from the data consumer. The input may include a valid data feature set, and the indication may indicate whether the data meets the necessary service requirements after decompression. The compression controller 110 then generates a preferred compression solution based on feature similarity, privacy indication, compression performance of the existing compression solution (e.g., offline training sample), and indication from the data consumer 102. After generating the preferred compression solution, a compression ID is assigned, and the compression controller 110 sends the compression ID to the parameter configuration function 114 for determining or evaluating the compression parameter configurations 510a and 510b. The compression controller 110 may also send the ID of the encryption algorithm to the parameter configuration function 114 for determining the encryption parameter configuration when the privacy indication indicates that the required data requires privacy protection. In some embodiments, the compression controller 110 may send a partial parameter compression solution or strategy to the parameter configuration function 114. The parameter configuration function 114 may generate parameters 508 for compression and, if necessary, encryption, and then send these generated parameters to the feature extraction router 106 in the form of configuration 510a and to the data consumer in the form of configuration 510b. Upon receiving the respective configurations 510a and 510b, the feature extraction router 106 and the data consumer 102 configure the compression solution based on the received configurations.

[0070] Currently, considerations for data compression are based on extracted features, so current compression controllers obtain sensitive features that may lead to privacy leaks of data owners or data consumers. Therefore, the benefits of embodiments of the systems and methods for data feature compression defined in the present invention include considerations related to data compression based on feature similarity lists, indications, and compression performance of existing compression solutions to better protect privacy. In addition, embodiments can provide compression solutions for any features, including matched features and unmatched features. Unmatched features can be used to define that the extracted data features are not included in the existing data features. Similarly, matched features mean that the extracted data features are included in the existing data features.

[0071] According to an embodiment, a data compression process is provided, wherein when a data consumer 102 requests a data compression service. The network first checks whether the data consumer 102 has the authority to obtain the requested data, and after authorization, the network implements data compression to reduce network bandwidth and communication overhead. However, due to the lack of trust between elements associated with the system, for example, when the elements of the system are provided by multiple non-trusted parties, or when the data consumer does not trust private data, it is desirable to protect sensitive data or personal data related to activities performed by the elements of the system. In addition, sensitive data should be protected during transmission, so a certain degree of encryption can be used to achieve transmission.

[0072] Figure 6 The data compression process 600 of an embodiment is depicted. In the process 600, the data consumer 102, for example, Figure 1 The service controller 104 sends the request 602 to the feature extraction router 106. The request 602 may include information related to the data consumer, such as a data consumer ID, service information indicating which data may be sent to the data consumer 102, and one or more security requirements (if security of information sent to the data consumer is necessary). In some embodiments, the request 602 may be associated with multi-view real-time video data, and the request 602 may also include a natural person's interactive activity, which may provide the data consumer 102 with information indicating a posture, body position, and / or perspective in the video / data.

[0073] After receiving the request 602, the feature extraction router 106 extracts data features based on the service information and the human interaction (data feature extraction 604). Then, the feature extraction router 106 sends a feature verification request 606 to the feature verification center 108, which includes the data features and (if necessary) one or more security requirements.

[0074] The feature verification center 108 implements feature verification to obtain a valid data feature set based on a feature set representing known or previously extracted data features, which may be provided by the offline training function 116. The feature set may include one or more data features and an associated feature ID for each of the one or more data features.

[0075] In some embodiments, if the extracted data feature belongs to the feature set, the feature verification center 108 sends a feature verification response 610 to the feature extraction router 106 , where the feature verification response may indicate a match.

[0076] In some embodiments, the feature verification center 108 calculates a data feature similarity level 608 between the extracted data features and one or more of the data features included in the feature set. For example, the feature set may include data features provided by the offline training function 116. The feature verification center 108 can be used to check the feature similarity level and determine whether the feature similarity level exceeds a threshold. For example, the threshold can be configured as a fixed threshold or a dynamic threshold, which can be defined at least in part based on the extracted data feature characteristics or one or more other parameters. After determining the feature similarity level, the feature verification center sends an appropriate data feature verification response 610 to the feature extraction router 106.

[0077] In some embodiments, if the extracted data feature belongs to a feature set, or if the feature similarity level exceeds the threshold, the feature verification response 610 indicates that the extracted data feature is accepted. However, in the case where the extracted data feature neither belongs to a feature set nor the similarity level exceeds the threshold, the feature verification response may include feedback information related to the extracted data feature. For example, the feedback information may indicate information (e.g., words, phrases, or other data features) that may describe the characteristics of suitable potential data features that may be extracted. The feedback information may be generated by the feature verification center 108 based on previously assessed data feature similarities. For example, the higher the feature similarity of the extracted data feature, the more likely the data feature is to be included in the information feedback as a potential data feature.

[0078] After the feature verification center 108 sends the feature verification response 610 to the feature extraction router 106 , if the feature verification response 610 includes feedback information, the feature extraction router 108 extracts new data features based at least in part on the feedback information.

[0079] According to an embodiment, if the feature verification response 610 indicates that the extracted data features are accepted, the feature verification center 108 sends an access request 612 to the access controller 112. The access request 612 includes the data consumer ID and the extracted data features.

[0080] According to an embodiment, the access controller 112 may implement access control based on the extracted data features and send an access response 614 back to the feature verification center 108. The access response 614 may indicate the rights of the data consumer 102 for the requested data, which may be based at least in part on the information received in the initial service request. If the access response 614 indicates that the data consumer 102 does not have the rights to obtain the requested data, the feature verification center 108 denies the service request and sends a service response 616 back to the data consumer 102. The response 616 indicates that the data consumer 102 does not have the rights to obtain the requested service.

[0081] According to an embodiment, if the access response 614 indicates that the data consumer 102 has permission, the feature verification center 108 can check the extracted data features to determine whether the extracted data features include any information that can be considered private or information whose privacy will be protected. For example, the feature verification center can determine whether the extracted data features are related to a person's private information or to a person's interesting information. If privacy of certain information associated with the extracted data features is desired or required, selective encryption or encryption combined compression can be used to protect at least the private data associated with the extracted data features. The feature verification center 108 then sends a compression request 618 to the compression controller 110. The compression request 618 may include a set of valid data features, a list of feature IDs with corresponding feature similarity levels, and a privacy indication indicating whether encryption or other forms of privacy protection are required for data compression before being transmitted to the data consumer.

[0082] In some embodiments, where an existing compression solution that matches the received feature ID is known (e.g., a compression solution determined or configured based on an offline training sample), the compression controller 110 sends a parameter request 624 to the parameter configuration function 114. The parameter request may include the compression ID and some parameters about the compression. If the privacy indication indicates that at least partial data protection is required for the data compression, the parameter request may also include an encryption ID. In some embodiments, the offline training function sample may include information including the feature ID, the compression ID, and the corresponding compression performance associated with the compression method associated with the compression ID.

[0083] In some embodiments, in the case where a known compression solution is not available, the compression controller 110 is configured to determine a compression solution based on at least one of an input, compression performance of an existing compression solution, and an indication from a data consumer. The input may include a set of valid data features, and the indication may indicate whether the data meets the necessary service requirements after decompression. The compression controller 110 may select or generate a desired or optimal compression solution based on one or more of feature similarity, privacy indication, compression performance of an existing compression solution, and an indication from a data consumer 102.

[0084] In some embodiments, the indication from the data consumer 102 may indicate whether the extracted data features satisfy the requirements of the service request originally received from the data consumer 102 after the extracted data features are decompressed.

[0085] In some embodiments, existing optimization algorithms (eg, a knapsack optimization algorithm) may be used to select or identify a desired or appropriate or optimized compression solution.

[0086] In some embodiments, in the event that selection or generation of a suitable or optimized compression solution fails, the compression controller 110 may request information indicating feature similarities and then send a compression response 622 back to the feature verification center 108. The compression response may include a request for more feature IDs having associated feature similarity levels between specific selected data features of the feature set and the extracted data features. The process for selecting or generating a suitable compression solution may then be performed again.

[0087] After obtaining a successful compression selection / generation, the compression controller 110 sends a parameter request 624 to the parameter configuration function 114. The parameter request 624 may include a compression ID, and some parameters related to the selected compression solution. The parameter request 624 may also include an encryption ID for use in the case where the privacy indication indicates that the data compression requires privacy protection of some information associated with the extracted data features.

[0088] In some embodiments, the parameter configuration function 114 then generates parameters for compression (or encryption) and may send these parameters to the feature extraction router 106 (e.g., by transmitting a configuration message 626a) and the data consumer 102 (e.g., by transmitting a configuration message 626b). The configuration messages 626a and 626b may provide the feature extraction router and the data consumer, respectively, with appropriate parameters that conform to the respective configurations of the selected compression solution.

[0089] In some embodiments, the feature verification center can use a text frequency (TF) method to determine the data features for extraction. The TF method can be used to measure the number of times a term appears in a data packet. For example, the occurrence of a term in a group of data packets can be divided by the total terms present in the group of data packets to determine the term frequency. Subsequently, all terms can be sorted based on the term frequency, and terms with higher frequency of occurrence (i.e., term frequency) can be selected as data features for extraction. In some embodiments, there may be some terms that, although they are not the same, can be considered to have a similarity level. The similarity level of these terms can be determined, and the similarity level is then transmitted to the feature extraction router as feedback information, thereby providing feedback from the feature verification center to the feature extraction router for subsequent data feature extraction.

[0090] According to an embodiment, the compression controller dynamically selects or generates a compression solution based on input (e.g., feature data). The input or feature data may include a list of feature IDs with feature similarity levels, a privacy indication, compression performance related to an existing compression solution (which may be determined by an offline training function), and an indication from a data consumer. The privacy indication may indicate that the feature verification center will send a list of feature IDs instead of sensitive feature content, and / or encrypt information of data features that have been identified as privacy, thereby protecting privacy after responding to a data extraction request from a data consumer. Since the input may include an indication of compression performance related to an existing compression solution, the system can provide a suitable compression solution for a variety of different extracted features (e.g., including matched features and unmatched features).

[0091] In some embodiments, data features can be extracted based on feedback from the network. The system and method according to the present invention can be applied to applications including Internet of Things (IoT) and Internet of Vehicles (IoV) scenarios, or can be applied to applications such as satellite communications. In these types of applications, the destination identifier in the data packet can be an identifier of a user equipment (UE) or an identifier of a terminal device (such as an IoT device, a wearable device, or a vehicle device or an onboard device).

[0092] Figure 7Schematic diagram of an electronic device 700, such as a computing device or system, for different embodiments of the present invention, which electronic device can perform any or all steps of the above-mentioned methods and features described herein. For example, one or more of the feature extraction router, feature verification center, compression controller, and parameter configuration function can be configured as an electronic device. It will be readily understood that two or more of the feature extraction router, feature verification center, compression controller, and parameter configuration function can be configured in a single electronic device. In addition, one or more of the feature extraction router, feature verification center, compression controller, and parameter configuration function can be configured as an electronic device and can be configured by multiple electronic devices.

[0093] As shown, the device includes a processor 710, a memory 720, a non-transient mass storage element 730, an I / O interface 740, a network interface 750, and a transceiver 760, all of which are coupled in a communication manner via a bidirectional bus 770. According to certain embodiments, any or all of the elements in the depicted elements may be utilized, or only a subset of these elements may be utilized. In addition, the device 700 may include multiple instances of certain elements, such as multiple processors, memories, or transceivers. In addition, the elements of the hardware device may be directly coupled to other elements without a bidirectional bus.

[0094] The memory 720 may include any type of non-transient or non-transient memory, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), read-only memory (ROM), any combination thereof, etc. The mass storage element 730 may include any type of non-transient storage device, such as a solid state drive, a hard disk drive, a disk drive, an optical drive, a USB drive, or any computer program product for storing data and machine executable program code. According to some embodiments, the memory 720 or the mass storage element 730 may have instructions (e.g., machine-readable instructions) recorded thereon, which may be executed by the processor 710 to perform any of the aforementioned method steps described above.

[0095] The actions associated with the methods described herein can be implemented as coded instructions in a computer program product. In other words, a computer program product is a computer-readable medium in which software codes are recorded to perform these methods when the computer program product is loaded into a memory and executed by a processor of a computing device.

[0096] The actions associated with the methods described herein may be implemented as coded instructions in a plurality of computer program products. For example, the first portion of the method may be performed using one computing device, while the second portion of the method may be performed using another computing device, a server, etc. In this case, each computer program product is a computer-readable medium in which software code is recorded to perform the appropriate portion of the method when the computer program product is loaded into a memory and executed on a processor of a computing device.

[0097] In addition, each step of these methods can be executed on any computing device, such as a personal computer, server, PDA, etc., according to one or more of one or more program elements, modules or objects generated from any programming language such as C++, Java, etc., or a part of one or more program elements, modules or objects. In addition, each step or a file or object that implements each step, etc. can be executed by dedicated hardware or a circuit module designed for this purpose.

[0098] Although the present invention has been described with reference to the specific features and embodiments of the present invention, it is apparent that various modifications and combinations may be made to the present invention without departing from the present invention. Therefore, the specification and the drawings are only regarded as an explanation of the present invention as defined by the appended claims, and are intended to cover any and all modifications, variations, combinations or equivalents falling within the scope of the present invention.

Claims

1. A system for feature extraction, characterized in that: The system includes a feature extraction router and a feature verification center, wherein each of the feature extraction router and the feature verification center has one or more associated memories storing instructions, wherein the instructions are executed by one or more associated processors such that: The feature extraction router is used to: Receive feature extraction requests from data consumers; extracting data features in response to the request; sending the data feature to the feature verification center; and The feature verification center is used to: Determining whether the data feature belongs to a feature set, wherein the feature set includes existing data features provided offline by a third party; After determining that the data feature does not belong to the feature set, generating feedback indicating a potential data feature, wherein the potential data feature is associated with the data feature and the existing data feature, and sending the feedback to the feature extraction router to re-extract a new data feature; After determining that the data feature belongs to the feature set, triggering a feature-enabled network operation based at least in part on a valid data feature set associated with the feature set and the data feature.

2. The system according to claim 1, characterized in that When determining whether the data feature belongs to the feature set, the feature verification center is further used to: calculating a feature similarity between the data feature and the existing data feature, the feature similarity indicating a similarity level; After determining that the feature similarity exceeds a threshold, determining that the data feature belongs to the feature set.

3. The system according to claim 1 or 2, characterized in that: The feature extraction router is also used to: receiving the feedback; The new data features are re-extracted based on the feedback.

4. The system according to claim 3, characterized in that The new data feature includes at least one of the potential data features.

5. The method according to any one of claims 1 to 4, characterized in that The valid data feature set includes at least one of the following: A list of feature IDs with corresponding feature similarities; A privacy indication of whether sensitive data is protected, wherein the privacy indication is generated by the feature verification center.

6. The system according to any one of claims 1 to 5, characterized in that The system also includes a function for implementing a network operation for enabling a feature, the feature verification center triggering the function by sending the valid data feature set to the function.

7. The system according to claim 6, characterized in that The feature-enabled network operation includes feature-based data compression.

8. The system according to claim 6 or 7, characterized in that: The function is a compression controller, which is used to: determining a compression solution based on at least one of an input, compression performance of an existing compression solution, and an indication from a data consumer; Wherein, the input includes the valid data feature set; The indication indicates whether the data meets necessary service requirements after decompression.

9. The system according to claim 8, characterized in that The compression controller is also used for: compressing the data based on the compression solution to obtain compressed data, and sending the compressed data to the data consumer; Therein, in order to obtain the data from the compressed data, the data consumer performs decompression.

10. A feature extraction method, characterized in that: The method comprises: The feature extraction router receives feature extraction requests from data consumers; The feature extraction router extracts data features in response to the request; The feature verification center determines whether the data feature belongs to a feature set, wherein the feature set includes existing data features provided offline by a third party; After determining that the data feature does not belong to the feature set, the feature verification center generates feedback indicating a potential data feature, wherein the potential data feature is identified from or associated with the data feature and the existing data feature, and the feature verification center sends the feedback to the feature extraction router to re-extract a new data feature; After determining that the data feature belongs to the feature set, the feature verification center triggers a feature-enabled network operation based at least in part on a valid data feature set associated with the feature set and the data feature.

11. The method according to claim 10, characterized in that When determining whether the data feature belongs to the feature set, the method further includes: The feature verification center calculates feature similarity between the data feature and the existing data feature, the feature similarity indicating a similarity level; After determining that the feature similarity exceeds a threshold, the feature verification center determines that the data feature belongs to the feature set.

12. The method according to claim 10 or 11, characterized in that: The method further comprises: The feature extraction router receives the feedback; The feature extraction router re-extracts the new data features according to the feedback.

13. The method according to claim 12, characterized in that The new data feature includes at least one of the potential data features.

14. The method according to any one of claims 10 to 13, characterized in that The valid data feature set includes at least one of the following: A list of feature IDs with corresponding feature similarities; A privacy indication of whether sensitive data is protected, wherein the privacy indication is generated by the feature verification center.

15. The method according to any one of claims 10 to 14, characterized in that The method also includes the feature verification center triggering the function by sending the valid data feature set to the function for implementing a feature-enabled network operation.

16. The system according to claim 15, characterized in that The feature-enabled network operation includes feature-based data compression.

17. The method according to claim 15 or 16, characterized in that The function is configured as a compression controller, the method further comprising: The compression controller determines a compression solution based on at least one of an input, compression performance of an existing compression solution, and an indication from a data consumer; Wherein, the input includes the valid data feature set; The indication indicates whether the data meets necessary service requirements after decompression.

18. The method according to claim 17, characterized in that Also includes: The compression controller compresses the data based on the compression solution to obtain compressed data, and sends the compressed data to the data consumer; Therein, in order to obtain the data from the compressed data, the data consumer performs decompression.

19. A data feature verification method, characterized in that: include: The feature verification center receives data features from the feature extraction router; The feature verification center determines whether the data feature belongs to a feature set, wherein the feature set includes existing data features provided offline by a third party; After determining that the data feature does not belong to the feature set, the feature verification center generates feedback indicating a potential data feature, wherein the potential data feature is identified from or associated with the data feature and the existing data feature, and the feature verification center sends the feedback to the feature extraction router to re-extract a new data feature; After determining that the data feature belongs to the feature set, the feature verification center triggers a feature-enabled network operation based at least in part on a valid data feature set associated with the feature set.

20. The method according to claim 19, characterized in that When the feature verification center determines whether the data feature belongs to the feature set, the feature verification center is further used to: The feature verification center calculates feature similarity between the data feature and the existing data feature, the feature similarity indicating a similarity level; After determining that the feature similarity exceeds a threshold, the feature verification center determines that the data feature belongs to the feature set.

21. The method according to claim 19 or 20, characterized in that The new data feature includes at least one of the potential data features.

22. The method according to any one of claims 19 to 21, characterized in that The valid data feature set includes at least one of the following: A list of feature IDs with corresponding feature similarities; A privacy indication of whether sensitive data is protected, wherein the privacy indication is generated by the feature verification center.

23. A device for data feature verification, characterized in that: The device comprises: a transceiver module for receiving data features from a feature extraction router; A processing module is used to determine whether the data feature belongs to a feature set, wherein the feature set includes existing data features provided offline by a third party, wherein Upon determining that the data feature does not belong to the feature set, the processing module generates feedback indicating a potential data feature, wherein the potential data feature is identified from or associated with the data feature and the existing data feature, and the transceiver module sends the feedback to the feature extraction router to re-extract a new data feature; After determining that the data feature belongs to the feature set, the processing module triggers a feature-enabled network operation based at least in part on a valid data feature set associated with the feature set and the data feature.

24. The device according to claim 23, characterized in that The processing module is used to calculate the feature similarity between the data feature and the existing data feature, and the feature similarity indicates a similarity level; After determining that the feature similarity exceeds a threshold, the processing module is used to determine that the data feature belongs to the feature set.

25. The device according to claim 23 or 24, characterized in that The new data feature includes at least one of the potential data features.

26. The device according to any one of claims 23 to 25, characterized in that The valid data feature set includes at least one of the following: A list of feature IDs with corresponding feature similarities; A privacy indication of whether sensitive data is protected, wherein the privacy indication is generated by the feature verification center.

27. A device, characterized in that: The method comprises at least one processor, wherein the processor is configured to execute instructions stored in a memory, and when the instructions are executed by the processor, the method according to any one of claims 19 to 22 is implemented.

28. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, which, when executed by a computer, implement the method according to any one of claims 19 to 22.

Images