Data encryption method, device, computer equipment and storage medium
By performing feature extraction on multimedia data to generate encryption keys and combining reference key factor encryption processing, the problem of poor multimedia data encryption in the prior art is solved, and more efficient data security is achieved.
Patent Information
- Application Number
- CN202110016072.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-01-04
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2041-01-04
AI Technical Summary
The prior art is difficult to effectively improve the encryption effect and encryption performance in multimedia data encryption, resulting in insufficient data security.
By extracting the multimedia data feature, the target feature vector is generated as the encryption key, and the target encryption key is generated in combination with the reference key factor to encrypt the multimedia data.
It significantly improves the encryption effect and encryption performance of multimedia data, prevents data from being maliciously tampered with or stolen, and improves data security.
Smart Images

Figure CN114726506B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of Internet technology, specifically to the field of computer technology, and in particular to a data encryption method, a data encryption device, a computer device and a computer storage medium. Background Art
[0002] At present, in the scenarios of multimedia data storage or data transmission, the multimedia data to be processed is usually encrypted to improve the data security of the multimedia data; the so-called data encryption refers to the data processing process of converting plaintext into ciphertext through encryption algorithm. Here, plaintext refers to the original or unencrypted data, and ciphertext refers to the data obtained by encrypting the plaintext. Based on this, how to better encrypt multimedia data has become a research hotspot. Summary of the invention
[0003] The embodiments of the present invention provide a data encryption method, device, terminal and storage medium, which can better encrypt multimedia data, improve encryption effect and encryption performance, and thus improve the data security of multimedia data.
[0004] In one aspect, an embodiment of the present invention provides a data encryption method, the method comprising:
[0005] Acquire multimedia data to be processed;
[0006] Performing feature extraction processing on the multimedia data to obtain a target feature vector of the multimedia data;
[0007] generating a target encryption key for the multimedia data according to the target feature vector; and,
[0008] The target encryption key is used to perform data encryption processing on the multimedia data to obtain target encrypted data.
[0009] On the other hand, an embodiment of the present invention provides a data encryption device, the device comprising:
[0010] An acquisition unit, used for acquiring multimedia data to be processed;
[0011] A processing unit, configured to perform feature extraction processing on the multimedia data to obtain a target feature vector of the multimedia data;
[0012] an encryption unit, configured to generate a target encryption key for the multimedia data according to the target feature vector; and
[0013] The encryption unit is further used to perform data encryption processing on the multimedia data using the target encryption key to obtain target encrypted data.
[0014] In one implementation, when the encryption unit is used to generate a target encryption key for the multimedia data according to the target feature vector, it may be specifically used to:
[0015] Obtaining a reference key factor, wherein the reference key factor includes at least one of the following factors: a device identifier, a timestamp, a signature version number, and a random number;
[0016] A target encryption key for the multimedia data is generated according to a key generation strategy and based on the reference key factor and the target feature vector.
[0017] In another embodiment, when the encryption unit is used to generate the target encryption key of the multimedia data according to the key generation strategy and based on the reference key factor and the target feature vector, it can be specifically used to:
[0018] Arranging each factor in the reference key factor and the target feature vector according to a preset arrangement order to obtain a target encryption key for the multimedia data; or,
[0019] The reference key factor and the target feature vector are used to construct an initial key, and a preset private key is used to perform data encryption processing on the initial key to obtain a target encryption key for the multimedia data.
[0020] In another embodiment, the multimedia data is a face image of a target user acquired by the terminal calling a camera component, and the terminal and the server are built with the same feature extraction model; accordingly, the processing unit can also be used for:
[0021] Sending the target encrypted data and the facial image to the server, allowing the server to obtain a reference feature vector of the facial image through the feature extraction model, using a reference encryption key generated according to the reference feature vector to perform data encryption processing on the facial image, and when the obtained reference encrypted data is consistent with the target encrypted data, performing identity recognition on the facial image to return the identity information of the target user to the terminal;
[0022] Receive the identity information returned by the server, and perform business processing according to the identity information, the business processing including: face payment processing.
[0023] In yet another embodiment, when the processing unit is used to perform feature extraction processing on the multimedia data to obtain the target feature vector of the multimedia data, it can be specifically used to:
[0024] Acquire a feature extraction model, wherein the feature extraction model includes at least: a first network sub-model and a second network sub-model having different processing capabilities;
[0025] Calling the first network sub-model to perform feature extraction processing on the multimedia data to obtain M first vectors and a confidence level of each first vector, where M is a positive integer;
[0026] Calling the second network sub-model to perform feature extraction processing on the multimedia data to obtain N second vectors and a confidence level of each second vector, where N is a positive integer;
[0027] According to the confidence of each first vector and the confidence of each second vector, a target feature vector of the multimedia data is selected from the M first vectors and the N second vectors.
[0028] In yet another embodiment, when the processing unit is used to select the target feature vector of the multimedia data from the M first vectors and the N second vectors according to the confidence of each first vector and the confidence of each second vector, it can be specifically used to:
[0029] According to the confidences of the first vectors and the second vectors, a vector with the largest confidence is selected from the M first vectors and the N second vectors as the target feature vector of the multimedia data; or,
[0030] A first vector whose confidence is greater than a first threshold value among the M first vectors is used as a candidate vector, and a second vector whose confidence is greater than a second threshold value among the N second vectors is used as a candidate vector; and any candidate vector is selected from all the candidate vectors as a target feature vector of the multimedia data.
[0031] In another embodiment, when the processing unit is used to call any network sub-model to perform feature extraction processing on the multimedia data, it can be specifically used to:
[0032] Acquire a feature enhancement strategy adapted to any one of the network sub-models, where any one of the network sub-models is the first network sub-model or the second network sub-model;
[0033] According to the feature enhancement strategy, the multimedia data is subjected to feature enhancement processing to obtain enhanced data;
[0034] Call any of the network sub-models to perform feature extraction processing on the enhanced data.
[0035] In another embodiment, the multimedia data is a face image, and the enhanced data includes: an enhanced image corresponding to the face image; accordingly, when the processing unit is used to perform feature enhancement processing on the multimedia data according to the feature enhancement strategy to obtain the enhanced data, it can be specifically used to:
[0036] According to the block parameters indicated by the feature enhancement strategy, split the target image into multiple image blocks, and traverse the multiple image blocks;
[0037] Calculating the gradient of each pixel in the current image block currently traversed, and constructing a directional gradient histogram of the current image block according to the gradient of each pixel;
[0038] According to the directional gradient histogram of the current image block, an image vector of the current image block is obtained to obtain the enhanced image; the enhanced image includes the image vectors of each image block, and the image vector of any image block is used to represent the enhanced image features of any image block.
[0039] In another embodiment, the target feature vector is obtained by calling a feature extraction model to extract features from the multimedia data; accordingly, the processing unit may also be used to:
[0040] Acquire an initial model to be trained, wherein the initial model at least includes: a first initial sub-model and a second initial sub-model, and the processing capabilities of the initial sub-models in the initial model are different from each other;
[0041] Acquire a training sample set corresponding to the initial model, wherein the training sample set includes at least the following sample data: first sample data adapted to the processing capability of the first initial sub-model, and second sample data adapted to the processing capability of the second initial sub-model;
[0042] Using each sample data in the training sample set to perform model training on the initial model to obtain an intermediate model; and using any sample data in the training sample set to perform model evaluation on the intermediate model;
[0043] If the intermediate model passes the model evaluation, the intermediate model is used as the feature extraction model.
[0044] In another embodiment, the training sample set further includes a label category of each sample data; accordingly, when the processing unit is used to perform model training on the initial model using each sample data in the training sample set to obtain an intermediate model, it can be specifically used to:
[0045] Traversing each sample data in the training sample set;
[0046] Calling a first initial sub-model in the initial model to perform feature extraction on the current sample data currently traversed, and determining a first prediction category of the current sample data according to the extracted first feature vector;
[0047] Calling a second initial sub-model in the initial model to extract features of the current sample data, and determining a second prediction category of the current sample data according to the extracted second feature vector;
[0048] According to the first prediction category, the second prediction category and the labeled category of the current sample data, the model parameters of the first initial model and the model parameters of the second initial model are updated to obtain an intermediate model.
[0049] In another embodiment, when the processing unit is used to use any sample data in the training sample set to perform model evaluation on the intermediate model, it can be specifically used to:
[0050] Calling the intermediate model to perform feature extraction on any sample data in the training sample set to obtain a sample feature vector of the any sample data and a confidence level of the sample feature vector;
[0051] If the confidence of the sample feature vector is greater than or equal to the confidence threshold, determining that the intermediate model passes the model evaluation;
[0052] If the confidence of the sample feature vector is less than the confidence threshold, it is determined that the intermediate model has failed the model evaluation.
[0053] In another aspect, an embodiment of the present invention provides a computer device, the computer device comprising an input interface and an output interface, the computer device further comprising:
[0054] a processor adapted to implement one or more instructions; and,
[0055] A computer storage medium storing one or more instructions, wherein the one or more instructions are suitable for being loaded by the processor and executing the following steps:
[0056] Acquire multimedia data to be processed;
[0057] Performing feature extraction processing on the multimedia data to obtain a target feature vector of the multimedia data;
[0058] generating a target encryption key for the multimedia data according to the target feature vector; and,
[0059] The target encryption key is used to perform data encryption processing on the multimedia data to obtain target encrypted data.
[0060] In another aspect, an embodiment of the present invention provides a computer storage medium, wherein the computer storage medium stores one or more instructions, wherein the one or more instructions are suitable for being loaded by a processor and executing the following steps:
[0061] Acquire multimedia data to be processed;
[0062] Performing feature extraction processing on the multimedia data to obtain a target feature vector of the multimedia data;
[0063] generating a target encryption key for the multimedia data according to the target feature vector; and,
[0064] The target encryption key is used to perform data encryption processing on the multimedia data to obtain target encrypted data.
[0065] According to the embodiment of the present invention, for the multimedia data to be processed, the multimedia data may be firstly feature extracted to obtain the target feature vector of the multimedia data. Secondly, the target feature vector may be used as a key factor to generate a target encryption key for the multimedia data in a targeted manner, thereby ensuring the uniqueness of the target encryption key and effectively improving the reliability of the target encryption key. Since the target encryption key and the data features of the multimedia data are closely related, the target encryption key may be used to perform data encryption processing on the multimedia data, which may effectively improve the encryption effect and encryption performance of the data, thereby effectively preventing the multimedia data from being maliciously tampered with or stolen, and effectively improving the data security of the multimedia data. BRIEF DESCRIPTION OF THE DRAWINGS
[0066] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.
[0067] Figure 1a This is an application scenario diagram of a data encryption solution provided by an embodiment of the present invention;
[0068] Figure 1b This is an application scenario diagram of another data encryption solution provided by an embodiment of the present invention;
[0069] Figure 2 It is a flowchart of a data encryption method provided by an embodiment of the present invention;
[0070] Figure 3a It is a schematic diagram of a model training process provided by an embodiment of the present invention;
[0071] Figure 3b is a corresponding schematic diagram of an initial model and a feature extraction model provided by an embodiment of the present invention;
[0072] Figure 4is a flow chart of a data encryption method provided by another embodiment of the present invention;
[0073] Figure 5a is a model structure diagram of a first initial sub-model in an initial model provided by an embodiment of the present invention;
[0074] Figure 5b is a model structure diagram of a second initial sub-model in an initial model provided by an embodiment of the present invention;
[0075] Figure 6 It is a structural schematic diagram of a data encryption device provided by an embodiment of the present invention;
[0076] Figure 7 It is a structural schematic diagram of a computer device provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0077] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.
[0078] With the continuous development of Internet technology, AI (Artificial Intelligence) technology has also been better developed. The so-called AI refers to the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science; it mainly produces a new intelligent machine that can respond in a similar way to human intelligence by understanding the essence of intelligence, so that the intelligent machine has multiple functions such as perception, reasoning and decision-making. Correspondingly, AI technology is a comprehensive discipline, which mainly includes several major directions such as computer vision technology (Computer Vision, CV), speech processing technology, natural language processing technology and machine learning (Machine Learning, ML) / deep learning.
[0079] Among them, machine learning is the core of AI and is the basis for making computer devices intelligent. Its applications are spread across all fields of artificial intelligence. The so-called machine learning is a multi-disciplinary interdisciplinary subject involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines; it specializes in studying how computer devices simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and learning by teaching.
[0080] Based on the machine learning / deep learning technology in the above AI technology, an embodiment of the present invention proposes a data encryption scheme to better encrypt data. In a specific implementation, the data encryption scheme can be executed by a target computer device; the so-called computer device refers to any device with data computing capabilities, such as a terminal or a server. Among them, the terminal may include but is not limited to: a smart phone, a tablet computer, a laptop computer, a desktop computer, a face-swiping payment device (a device that automatically deducts electronic resources from a resource account associated with the identity information after performing face recognition to obtain identity information), an automatic cash register device, and the like. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and basic cloud computing services such as big data and artificial intelligence platforms, and the like.
[0081] Specifically, the general principle of the data encryption scheme is as follows: First, a feature extraction model can be pre-trained based on machine learning / deep learning technology; when there is multimedia data to be encrypted, the multimedia data can be input into the feature extraction model for feature extraction processing. After the feature extraction model outputs the processing result of the multimedia data, the processing result can be used as an encryption security factor to perform data encryption processing on the multimedia data in a targeted manner, thereby obtaining the encrypted data (i.e., ciphertext) corresponding to the multimedia data. It should be noted that the feature extraction model mentioned here can be a model that only extracts features from the input data; in this case, the processing result output by the feature extraction model can include: the feature vector output by any network layer in the feature extraction model. Alternatively, the feature extraction model can also be a model that extracts features from the input data and performs related processing (such as category recognition processing, gesture recognition, etc.) based on the extracted features; in this case, the processing result output by the feature extraction model can include at least one of the following: the feature vector output by any network layer in the feature extraction model, and the object category information obtained by the feature extraction model performing category recognition on the multimedia data, etc.
[0082] In practical applications, the above-mentioned data encryption scheme can be applied to different scenarios according to actual needs, such as data storage scenarios, data transmission scenarios, etc. Among them, the data storage scenario refers to: a scenario in which multimedia data is stored in the local space of a computer device. For example, the data storage scenario can be: after the terminal obtains the multimedia data input by the user, the multimedia data needs to be encrypted and saved in the local space; or, the terminal responds to the user's operation of downloading a certain multimedia data, downloads the multimedia data and encrypts and stores it in the local space of the terminal; or, the server receives a certain multimedia data and needs to encrypt and store the multimedia data in the local space of the server, etc. In the data storage scenario, the target computer device for executing the data encryption scheme refers to: a computer device that stores the multimedia data. The data transmission scenario refers to: a scenario in which multimedia data is transmitted from one computer device to another computer device. For example, the data transmission scenario can be: after a terminal collects the multimedia data of the user, the multimedia data needs to be encrypted and transmitted to a server or another terminal; or, the server encrypts and transmits a certain multimedia data stored by itself to other servers or terminals, etc. In the data transmission scenario, the target computer device for executing the data encryption scheme refers to: a computer device that sends the multimedia data.
[0083] The following uses the data encryption scheme in the face-swiping payment scenario based on face recognition as an example to explain the application process of the data encryption scheme. The so-called face recognition refers to the technology of exchanging facial multimedia information (such as image frames containing faces) for identity information corresponding to faces; and the face-swiping payment scenario based on face recognition means: the face-swiping payment device collects the face image containing the user's face, encrypts the face image and transmits it to the server side for identity recognition, and automatically deducts the corresponding electronic resources from the resource account associated with the recognized identity information based on the items purchased by the user; the data transmission scenario of payment can be realized without the user operating his own mobile terminal (such as mobile phone, tablet). See Figure 1a As shown, the specific application process of the data encryption scheme is as follows:
[0084] When a user needs to pay for a target electronic resource corresponding to a target item resource in order to purchase the target item resource, the user can aim the target face of the user at the camera component of the face-scanning payment device (i.e., the target computer device), such as Figure 1aAs shown. At this time, the face-scanning payment device can call the relevant interface of the camera component to collect the facial image of the target face; after collecting the facial image of the target face, the face-scanning payment device can display the collected facial image in the user interface. Then, the face-scanning payment device can use the data encryption scheme to encrypt the facial image to obtain encrypted data; specifically, the feature extraction model can be called to perform feature extraction on the facial image, and the feature vector output by the feature extraction model can be used to encrypt the facial image to obtain encrypted data. After obtaining the encrypted data, the face-scanning payment device can send the encrypted data to the server; during this processing, the face-scanning payment device can output a prompt message (such as "Processing...") in the user interface to prompt the user that payment processing is in progress, such as Figure 1b shown.
[0085] Correspondingly, after receiving the encrypted data, the server side can obtain a facial image based on the encrypted data; and perform facial recognition on the target face in the facial image to obtain the identity information corresponding to the target face. Then, the recognized identity information is returned to the face-scanning payment device, so that the face-scanning payment device can perform facial payment processing based on the identity information; specifically, the face-scanning payment device can obtain the target resource account associated with the identity information, and then deduct the target electronic resource corresponding to the target item resource purchased by the user from the target resource account. After successfully deducting the target electronic resource, the face-scanning payment device can also output a payment success prompt in the user interface; for example, the payment success prompt can be Figure 1b The text message "Payment successful" is shown in the figure.
[0086] From the above description, it can be seen that the data encryption scheme proposed in the embodiment of the present invention realizes data encryption processing of encrypted data (such as face images) by introducing integrated network learning on the encryption security factor, using the output of the feature extraction model constructed by the combination structure of one or more neural networks as part of the encryption security factor, which can effectively improve the encryption effect and encryption performance of the data, thereby effectively preventing the data from being maliciously tampered with or stolen, and effectively improving the data security.
[0087] Based on the above description of the data encryption scheme, an embodiment of the present invention proposes a data encryption method, which can be executed by the target computer device mentioned above. Figure 2 , the data encryption method may include the following steps S201-S204:
[0088] S201: Acquire multimedia data to be processed.
[0089] The multimedia data may be any type of data. For example, the multimedia data may be a face image, a landscape image, a gesture image, etc. under the image type; another example, the multimedia data may be a message text, a text message, etc. under the text type; another example, the multimedia data may be a Word file, an Excel file, a PDF file, etc. under the file type; another example, the multimedia data may be audio data, video data, etc. under the audio and video type.
[0090] S202, performing feature extraction processing on the multimedia data to obtain a target feature vector of the multimedia data.
[0091] In the specific implementation process, the target computer device may first obtain a feature extraction model; then, the feature extraction model may be called to perform feature extraction processing on the multimedia data to obtain the target feature vector of the multimedia data. The feature extraction model is related to the type of multimedia data; when the multimedia data is image data, the feature extraction model is a model that can extract features from images; when the multimedia data is audio and video data, the feature extraction model is a model that can extract features from audio and video data; when the multimedia data is text data, the feature extraction model is a model that can extract features from text, and so on. In addition, the feature extraction model is generated in advance after using machine learning / deep learning technology to perform sample collection, feature engineering, model selection, model training, and model evaluation processes, such as Figure 3a shown.
[0092] The following will explain the various processes involved in generating a feature extraction model:
[0093] 1) Sample collection refers to the operation of collecting a training sample set for model training, which includes multiple training samples. In addition, the training sample and the multimedia data belong to the same type; specifically, if the multimedia data is image data, then the training sample is an image; if the multimedia data is audio and video data, then the training sample is audio and video data, and so on.
[0094] 2) Feature engineering refers to the use of signal data processing technology to obtain, process and extract meaningful feature data and attributes from each training sample in the training sample set for model training.
[0095] 3) Model selection refers to the operation of selecting and building an initial model based on preset conditions. The preset conditions here may include at least one of the following: ① Whether there is a residual network substructure, which can be used to prevent model overfitting; ② Whether it contains Batch Normalization, which can be used to prevent gradient vanishing during model training and accelerate model training.
[0096] In one implementation, the embodiment of the present invention can mainly adopt a binary classification method for classification; then based on this preset condition, a resnet (residual) network model can be selected as the backbone network to construct an initial model. In other words, the initial model in this implementation is constructed based on a resnet network model.
[0097] In another embodiment, since the processing capacity of each model after model training is usually fixed, when data beyond the processing capacity of the model is input, the processing effect of the model on the data will deteriorate; for example, for a model that can only process images with a resolution less than 640×480, if the model is called to perform feature extraction processing on an image with a resolution of 1920×1080, the accuracy of the extracted feature vector will be low. Therefore, in order to ensure that a more accurate feature vector can be obtained later, two or more initial sub-models with different processing capabilities can also be used to construct the initial model. For example, the initial model can be constructed using the first initial sub-model and the second initial sub-model; the first initial sub-model here can be built based on the resnet network model, and the second initial sub-model can be built based on the desnet network model. Compared with the resnet network model, DenseNet proposes a more radical dense connection mechanism: that is, all layers are connected to each other, specifically, each layer will accept all the previous layers as its additional input. In other words, the initial model under this embodiment may include multiple initial sub-models with different processing capabilities.
[0098] 4) Model training refers to the operation of training the initial model using data processed by feature engineering to obtain a feature extraction model. It should be noted that if the initial model includes two or more initial sub-models with different processing capabilities, the feature extraction model obtained after model training includes: the network sub-models after each initial sub-model is trained, such as Figure 3bAs shown. In addition, before training the initial model, the TensorFlow framework can be selected as the training framework, and the hyperparameters and data required by each initial sub-model in the initial model can be selected. Taking the initial model including two initial sub-models (i.e., the first initial sub-model and the second initial sub-model) as an example, the hyperparameters and data required for the first initial sub-model can be set as follows: batch = 128, epoch = 1000, learning rate = 0.0001; the hyperparameters and data required for the second initial sub-model can be set as follows: batch = 128, epoch = 5000, learning rate = 0.0001. Among them, batch is a hyperparameter used to define the number of samples that need to be processed before updating the internal model parameters; epoch is a hyperparameter used to define the number of times the training sample is repeatedly passed and used during the model training process, or the number of times the learning algorithm works in the overall training sample; learning rate can be called the learning rate, which is mainly used to indicate the error allowed each time the model parameters are updated.
[0099] 5) Model evaluation refers to the operation of evaluating the model capability of the trained feature extraction model using test data; specifically, the feature extraction model can be called to perform feature extraction processing on the test data, and the extracted feature vectors can be voted to evaluate the model capability of the feature extraction model. If the feature extraction model passes the model evaluation, the feature extraction model can be directly saved as a file in pb (Protocol Buffers, a lightweight and efficient structured data storage format) format for subsequent use; if the feature extraction model does not pass the model evaluation, the feature extraction model can be trained again to update the model parameters of the feature extraction model until the updated feature extraction model passes the model evaluation.
[0100] S203: Generate a target encryption key for the multimedia data according to the target feature vector.
[0101] Since the target feature vector is extracted from the multimedia data, a target encryption key closely related to the multimedia data can be generated according to the target feature vector to ensure the uniqueness and reliability of the target encryption key. In the specific implementation process, the specific implementation of step S203 may include any of the following:
[0102] In one implementation, the target computer device may directly use the target feature vector as the target encryption key for the multimedia data.
[0103] In another embodiment, the target computer device may first obtain a reference key factor, where the reference key factor may include at least one of the following factors: a magic number (magic_num), a device identifier (device_info), a timestamp (timestamp), a signature version number (sign_version), a random number (random), and an encryption counter value (counter). The magic number refers to data used to mark the format of the encryption protocol; the device identifier refers to data used to identify the target computer device, such as the device serial number (International Mobile Equipment Identity, IMEI); the timestamp refers to the time when the target computer device generates the target encryption key; the random number refers to a number randomly generated by a random algorithm, and the encryption counter value refers to a value that increases with the number of encryptions.
[0104] Then, the target computer device can generate a target encryption key for multimedia data according to the key generation strategy and based on the reference key factor and the target feature vector. In a specific implementation, the various factors in the reference key factor and the target feature vector can be arranged in a preset arrangement order to obtain the target encryption key for multimedia data. In another specific implementation, the reference key factor and the target feature vector can also be used to construct an initial key; specifically, the various factors in the reference key factor and the target feature vector can be arranged in a preset arrangement order to obtain the initial key. Then, the initial key is encrypted using a preset private key to obtain the target encryption key for multimedia data.
[0105] The preset arrangement order mentioned above can be set according to the requirements. For example, the reference key factors include magic number, device identification, timestamp, signature version number, random number and encryption count value. The factors and the target feature vector (expressed as "integrated network factor") are arranged according to the preset arrangement order. The target encryption key obtained can be as follows:
[0106] {magic_num}{device_info}{sign_version}{timestamp}{counter}{random}{Ensemble learning safety factor}
[0107] In another embodiment, if the target computer device subsequently needs to transmit encrypted data about multimedia data to other devices (such as a server), the target computer device may perform key negotiation with other devices based on the target feature vector to obtain a target encryption key. Specifically, the target computer device may first use an elliptic curve algorithm to calculate the target feature vector to obtain a first value; then, a key negotiation request carrying the first value and the elliptic curve algorithm is sent to other devices. Other devices may respond to the key negotiation request, generate a key random number and use an elliptic curve algorithm to calculate the key random number to obtain a second value; then, the second value is returned to the target computer device, and a decryption key may also be generated using the key random number and the first value. Accordingly, the target computer device may generate a target encryption key based on the target feature vector and the second value.
[0108] It should be noted that the above are only exemplary examples of several specific implementation methods for generating the target encryption key, and are not exhaustive.
[0109] S204, using the target encryption key to perform data encryption processing on the multimedia data to obtain target encrypted data.
[0110] In the specific implementation process, the target computer device can select an encryption algorithm according to the demand, and use the encryption algorithm to encrypt the multimedia data with the target encryption key to obtain the target encrypted data. The encryption algorithm here may include but is not limited to: DES (Data Encryption Standard) algorithm, 3DES algorithm, AES (Advanced Encryption Standard) algorithm, RSA (RSA algorithm) algorithm, DSA (Digital Signature Algorithm) algorithm, etc.
[0111] Optionally, if the multimedia data is obtained in a data storage scenario, the target computer device may store the target encrypted data in the local space of the target computer device after obtaining the target encrypted data. If the multimedia is obtained in a data transmission scenario, the target computer device may also send the target encrypted data to other devices after obtaining the target encrypted data. Specifically, if the target computer device generates the target encryption key by key negotiation, the feature extraction model does not need to be deployed in other devices; in this case, the target computer device may only send the target encrypted data to other devices, so that other devices can use the decryption key generated during the key negotiation process to decrypt the target encrypted data and obtain the multimedia data.
[0112] If the target computer device generates the target encryption key according to the key generation strategy or directly uses the target feature vector as the target encryption key, the feature extraction model needs to be deployed in other devices; in this case, when the target computer device sends the target encrypted data to other devices, it also needs to send the multimedia data to other devices, so that other devices can detect whether the multimedia data has been tampered with based on the multimedia data and the target encrypted data, thereby ensuring the security of the multimedia data. Specifically, other devices can call the feature extraction model to obtain the reference feature vector of the multimedia data, generate a reference encryption key based on the reference feature vector, and use the reference encryption key to perform data encryption processing on the multimedia data to obtain the reference encrypted data. Then, it can be determined whether the reference encrypted data and the target encrypted data are consistent. If they are consistent, it is determined that the multimedia data has not been tampered with; otherwise, it is determined that the multimedia data has been tampered with. After determining that the multimedia data has not been tampered with, other devices can store the multimedia data or perform subsequent related processing based on the multimedia data.
[0113] For example, assume that the target computer device is a terminal, and the multimedia data is a facial image obtained by the terminal calling a camera component to collect the face of the target user, and the terminal and the server have the same built-in feature extraction model. In this case, the target computer device can send the target encrypted data and the facial image to the server, so that the server obtains the reference feature vector of the facial image through the feature extraction model, uses the reference encryption key generated according to the reference feature vector to perform data encryption processing on the facial image, and when the obtained reference encrypted data is consistent with the target encrypted data, performs identity recognition on the facial image to return the identity information of the target user to the terminal. Accordingly, the terminal can receive the identity information returned by the server and perform business processing according to the identity information. The business processing here may include any of the following: facial payment processing, facial attendance processing, etc.
[0114] According to the embodiment of the present invention, for the multimedia data to be processed, the multimedia data may be firstly feature extracted to obtain the target feature vector of the multimedia data. Secondly, the target feature vector may be used as a key factor to generate a target encryption key for the multimedia data in a targeted manner, thereby ensuring the uniqueness of the encryption key and effectively improving the reliability of the encryption key. Since the target encryption key and the data features of the multimedia data are closely related, the target encryption key may be used to perform data encryption processing on the multimedia data, which may effectively improve the encryption effect and encryption performance of the data, thereby effectively preventing the multimedia data from being maliciously tampered with or stolen, and effectively improving the data security of the multimedia data.
[0115] See also Figure 4 , is a flowchart of another data encryption method provided by an embodiment of the present invention, which can be executed by the target computer device mentioned above. Figure 4 , the data encryption method may include the following steps S401-S407:
[0116] S401: Acquire multimedia data to be processed.
[0117] S402, obtaining a feature extraction model.
[0118] The feature extraction model may include at least: a first network sub-model and a second network sub-model with different processing capabilities. The processing capability mentioned here may refer to the maximum size of input data that the model can process more accurately. For example, if the first network sub-model and the second network sub-model are both models for processing images, then the processing capability of the first network sub-model may be: processing images with a resolution less than 640×480, and the processing capability of the second network sub-model may be: processing images with a resolution of 1920×1080.
[0119] As can be seen from the foregoing, the feature extraction model is obtained by pre-training the model based on machine learning technology; specifically, the training generation process of the feature extraction model may include the following steps s11-s15:
[0120] s11, obtaining an initial model to be trained, the initial model may include at least: a first initial sub-model and a second initial sub-model, and the processing capabilities of the initial sub-models in the initial model are different.
[0121] In one embodiment, a first initial sub-model can be built based on the resnet network model; see Figure 5a As shown, the first initial sub-model may include: Input (input module), Stem (preprocessing module), 5×inception-resenet-A (residual Inception module A), Reduction-A (scaling module A), 10×inception-resenet-B (residual Inception module B), Reduction-B (scaling module B), 5×inception-resenet-C (residual Inception module C), Average Pooling (average pooling module), Dropout (exit module) and Softmax (normalization module), etc.
[0122] Among them, Input is mainly used to receive data input to the first initial sub-model. Stem is mainly used to pre-process the data before entering the Inception module, which may include multiple convolution operations and 2 maximum pooling operations. The scaling module is mainly used to scale the residual inception module, change the width and depth of the residual inception module connected to its input end, so as to avoid instability in the residual inception module. Average Pooling is mainly used to perform average pooling on the output data of the residual Inception module C. Dropout is mainly used to process data with a certain probability (for example Figure 5a The 0.8 in the “Keep 0.8” mentioned in the above statement ignores a certain number of neurons so that the ignored neurons do not participate in data processing, thereby reducing overfitting. Softmax is mainly used to normalize the output data of Dropout. The processing result of Softmax can be directly used as the output result of the first initial sub-model.
[0123] In another embodiment, a second initial sub-model can be constructed based on the Desnet network model; see Figure 5a As shown, the second initial sub-model may include: a plurality of convolution modules (such as Figure 5b ), multiple space_to_depth modules (feature transfer modules), multiple maximum modules (feature selection modules) and Convert to RGBA module (feature conversion module). It should be noted that the embodiment of the present invention is mainly described by taking multimedia data as image type data as an example, so the second initial sub-model may include the Convert to RGBA module; in other embodiments, if the multimedia data is other types of data, the Convert to RGBA module may not be included in the second initial sub-model.
[0124] in, Figure 5b The value in any white module in represents the number and size of convolution kernels involved in any white module when performing convolution operation; for example, Figure 5bThe white module in the first row has a corresponding value of "16 3×3", which means that the size of the convolution kernel involved in the convolution operation of the white module is 3×3, and the number of convolution kernels is 16. The space_to_depth module is mainly used to move some features of the input data (space data) of the module from the width dimension and the height dimension to the depth dimension. The value in the space_to_depth module is used to represent the parameters involved in the feature transfer. The Convert to RGBA module is mainly used to perform feature conversion on the input data of the module to obtain the features of the input data of the module in the following channels: R channel (red channel), G channel (green channel), B channel (blue channel) and A channel (transparent channel); Figure 5b The value of K in is equal to the number of channels. For RGBA channels, K is equal to 4. The maximum module is mainly used to select the maximum input data from the input data of the maximum module as the output of the module. It should be noted that Figure 5b Each white module (i.e., convolution module) in the network may be connected to a normalization module, but there is no limitation on this.
[0125] s12, obtain the training sample set corresponding to the initial model.
[0126] Among them, the training sample set includes at least the following sample data: the first sample data adapted to the processing capability of the first initial sub-model, and the second sample data adapted to the processing capability of the second initial sub-model. Specifically, if each initial sub-model in the initial model is a model for processing images, then each sample data in the training sample set may include: RGB three-color images of faces obtained from the device end, and the resolution of each image is related to the camera component of the device (such as a camera), and the maximum resolution supported by the camera component can generally be used as the resolution of the image. If each initial sub-model in the initial model is a model for processing audio data, then each sample data in the training sample set may include: a large amount of audio data obtained from the device end or the Internet, and so on.
[0127] s13, using each sample data in the training sample set to train the initial model to obtain an intermediate model.
[0128] In one embodiment, the training sample set also includes annotated feature vectors of each sample data; accordingly, the specific implementation of step s13 may be: calling the initial model to extract features of each sample data to obtain a predicted feature vector; adjusting the model parameters of the initial model in the direction of reducing the difference between the annotated feature vector and the predicted feature vector of each sample data to obtain an intermediate model.
[0129] In another embodiment, the training sample set also includes the labeled categories of each sample data; accordingly, the specific implementation of step s13 may be: traverse each sample data in the training sample set; call the first initial sub-model in the initial model to extract features of the current sample data currently traversed, and determine the first prediction category of the current sample data according to the extracted first feature vector; call the second initial sub-model in the initial model to extract features of the current sample data, and determine the second prediction category of the current sample data according to the extracted second feature vector. Then, the model parameters of the first initial model and the model parameters of the second initial model may be updated according to the first prediction category, the second prediction category and the labeled category of the current sample data to obtain an intermediate model; specifically, the prediction category with a larger confidence may be selected from the first prediction category and the second prediction category according to the confidence of the first prediction category and the confidence of the second prediction category, and the model parameters of the first initial model and the model parameters of the second initial model may be updated in the direction of reducing the difference between the labeled category and the selected prediction category to obtain an intermediate model.
[0130] The specific implementation of calling the first initial sub-model in the initial model to extract features from the current sample data being traversed can be: directly calling the first initial sub-model in the initial model to extract features from the current sample data being traversed; or, performing feature enhancement processing on the current sample data being traversed to obtain enhanced sample data, and then calling the first initial sub-model in the initial model to extract features from the enhanced sample data. It should be noted that when performing feature enhancement processing on the current sample data, different enhancement processing methods can be selected according to the data type of the current sample data to perform feature enhancement processing.
[0131] For example, if the current sample data is a sample image, when the feature enhancement processing is performed on the current sample data, the currently traversed sample image can be divided into multiple image blocks according to the first resolution (such as 60×60) corresponding to the first initial sub-model; the directional gradient histogram (Histogram Of Gradient, HOG) of each image block is obtained, and the image vector of each image block is obtained according to the directional gradient histogram of each image block. The image vector of any image block is a one-dimensional vector obtained by adding the first and last vectors of each HOG feature vector after extracting multiple HOG features from the directional gradient histogram of any image block; and the enhanced sample data includes the image vector of each image block.
[0132] For any image block, the specific implementation method of obtaining the directional gradient histogram of any image block can be: calculate the gradient of each pixel point in any image block, and the gradient of any pixel point can include the gradient value of any pixel point in the horizontal direction and the vertical direction; then, the amplitude value and gradient direction of each pixel point can be calculated according to the gradient of each pixel point, and the directional gradient histogram of any image block can be constructed using the calculated amplitude value and gradient direction of each pixel point. Among them, the amplitude value and gradient direction of each pixel point can be calculated using the sobel algorithm shown in the following formula:
[0133]
[0134]
[0135] Wherein, Ix and Iy represent the gradient values of any pixel in the horizontal direction and vertical direction respectively, M(x, y) represents the amplitude value of the gradient of any pixel, and θ(x, y) represents the gradient direction of any pixel.
[0136] For another example, if the current sample data is audio data, when the feature enhancement processing is performed on the current sample data, the librosa algorithm (an audio data processing algorithm) can be used to perform audio enhancement processing on the currently traversed audio data to obtain enhanced sample data. Among them, the audio enhancement processing may include at least one of the following: noise enhancement (NoiseAugmentation), time shift enhancement (Time Shift Augmentation), pitch shift enhancement (Pitch ShiftAugmentation), and so on. Noise enhancement here refers to: adding a random noise segment to the data to be processed; time shift enhancement refers to: randomly shifting the audio signal by rolling the signal along the time axis; pitch shift enhancement refers to: randomly rolling the audio signal in the data to be processed within the range of ±5% around the frequency axis, and performing a surround conversion to retain all information.
[0137] It should be noted that the specific principle of calling the second initial sub-model to extract features from the current sample data is similar to the specific principle of the first initial sub-model to perform feature extraction processing, and will not be repeated here. However, when the current sample data is a sample image, and feature enhancement processing is required for the current sample data so that the second initial sub-model performs feature extraction processing on the enhanced sample data, the currently traversed sample image is divided into multiple image blocks according to the second resolution (such as 240×240) corresponding to the second initial sub-model, and the other steps are similar to the above description.
[0138] s14, use any sample data in the training sample set to evaluate the intermediate model.
[0139] In the specific implementation process, the intermediate model can be called to perform feature extraction on any sample data in the training sample set to obtain the sample feature vector of any sample data and the confidence of the sample feature vector. Specifically, the intermediate model can be directly called to perform feature extraction on any sample data in the training sample set to obtain the sample feature vector of any sample data and the confidence of the sample feature vector. Alternatively, feature enhancement processing can be performed on any sample data. The specific feature enhancement processing can be found in the relevant description of step s13 above, which will not be repeated here; then, the intermediate model can be called to extract any enhanced sample data to obtain the sample feature vector of any sample data and the confidence of the sample feature vector. If the confidence of the sample feature vector is greater than or equal to the confidence threshold, it is determined that the intermediate model has passed the model evaluation; if the confidence of the sample feature vector is less than the confidence threshold, it is determined that the intermediate model has not passed the model evaluation.
[0140] It should be noted that, from the above, if any sample data is an image, it can be divided into blocks and the image vectors of each image block can be obtained when performing feature enhancement processing. In this case, the image vectors of each image block can be respectively input into the intermediate model for prediction to obtain multiple sample feature vectors and the confidence of each sample feature vector; then, the confidence of each sample feature vector can be compared with the confidence threshold, and voting can be used to determine whether the intermediate model passes the model evaluation based on the comparison result. If the majority of voters agree that the intermediate model passes the model evaluation based on the comparison result, it can be determined that the intermediate model passes the model evaluation; otherwise, it is determined that the intermediate model does not pass the model evaluation; the voters here can be devices, neurons in the intermediate model, and so on.
[0141] s15, if the intermediate model passes the model evaluation, the intermediate model is used as the feature extraction model. It should be noted that if the intermediate model does not pass the model evaluation, the intermediate model can be used as the initial model, and the above steps s12-s14 are repeatedly executed until the intermediate model passes the model evaluation.
[0142] S403, calling the first network sub-model to perform feature extraction processing on the multimedia data to obtain M first vectors and the confidence of each first vector, where M is a positive integer.
[0143] S404, calling the second network sub-model to perform feature extraction processing on the multimedia data to obtain N second vectors and the confidence of each second vector, where N is a positive integer.
[0144] In the specific implementation process of steps S403-S404, since the principles of calling the first network sub-model and calling the second network sub-model for feature extraction are similar, for the sake of convenience, any network sub-model is used as an example for explanation below. The specific implementation method of calling any network sub-model to perform feature extraction on multimedia data can be: obtaining a feature enhancement strategy adapted to any network sub-model, the any network sub-model being the first network sub-model or the second network sub-model; secondly, performing feature enhancement on the multimedia data according to the feature enhancement strategy to obtain enhanced data; then, any network sub-model can be called to perform feature extraction on the enhanced data.
[0145] Among them, when the multimedia data is a face image, the enhanced data may include: an enhanced image corresponding to the face image; accordingly, according to the feature enhancement strategy, the multimedia data is subjected to feature enhancement processing, and the specific implementation method for obtaining the enhanced data may be: according to the block parameters indicated by the feature enhancement strategy, the target image is split into multiple image blocks, and multiple image blocks are traversed; the gradient of each pixel point in the current image block currently traversed is calculated, and a directional gradient histogram of the current image block is constructed according to the gradient of each pixel point; according to the directional gradient histogram of the current image block, an image vector of the current image block is obtained to obtain an enhanced image; the enhanced image includes the image vectors of each image block, and the image vector of any image block is used to represent the enhanced image feature of any image block.
[0146] S405 , selecting a target feature vector of the multimedia data from the M first vectors and the N second vectors according to the confidence of each first vector and the confidence of each second vector.
[0147] In one embodiment, according to the confidence of each first vector and the confidence of each second vector, a vector with the largest confidence can be selected from the M first vectors and the N second vectors as the target feature vector of the multimedia data. In another embodiment, a first vector with a confidence greater than a first threshold value among the M first vectors can be used as a candidate vector, and a second vector with a confidence greater than a second threshold value among the N second vectors can be used as a candidate vector; then, any candidate vector is selected from all the candidate vectors as the target feature vector of the multimedia data, or the average of the candidate vectors is calculated as the target feature vector of the multimedia data.
[0148] S406: Generate a target encryption key for the multimedia data according to the target feature vector.
[0149] S407, using the target encryption key to perform data encryption processing on the multimedia data to obtain target encrypted data.
[0150] In the embodiment of the present invention, the specific implementation of steps S406-S407 can refer to the relevant description of steps S203-S204 in the above-mentioned embodiment of the invention, which will not be repeated here.
[0151] According to the embodiment of the present invention, for the multimedia data to be processed, the multimedia data may be firstly feature extracted to obtain the target feature vector of the multimedia data. Secondly, the target feature vector may be used as a key factor to generate a target encryption key for the multimedia data in a targeted manner, thereby ensuring the uniqueness of the encryption key and effectively improving the reliability of the encryption key. Since the target encryption key and the data features of the multimedia data are closely related, the target encryption key may be used to perform data encryption processing on the multimedia data, which may effectively improve the encryption effect and encryption performance of the data, thereby effectively preventing the multimedia data from being maliciously tampered with or stolen, and effectively improving the data security of the multimedia data.
[0152] Based on the description of the above data encryption method embodiment, the present invention also discloses a data encryption device, which can be a computer program (including program code) running on the target computer device mentioned above. Figure 2 or Figure 4 See the method shown in Figure 6 , the data encryption device can run the following units:
[0153] An acquisition unit 601 is used to acquire multimedia data to be processed;
[0154] The processing unit 602 is used to perform feature extraction processing on the multimedia data to obtain a target feature vector of the multimedia data;
[0155] an encryption unit 603, configured to generate a target encryption key for the multimedia data according to the target feature vector; and
[0156] The encryption unit 603 is further configured to perform data encryption processing on the multimedia data using the target encryption key to obtain target encrypted data.
[0157] In one implementation, when the encryption unit 603 is used to generate a target encryption key for the multimedia data according to the target feature vector, it may be specifically used to:
[0158] Obtaining a reference key factor, wherein the reference key factor includes at least one of the following factors: a device identifier, a timestamp, a signature version number, and a random number;
[0159] A target encryption key for the multimedia data is generated according to a key generation strategy and based on the reference key factor and the target feature vector.
[0160] In another embodiment, when the encryption unit 603 is used to generate the target encryption key of the multimedia data according to the key generation strategy and based on the reference key factor and the target feature vector, it can be specifically used to:
[0161] Arranging each factor in the reference key factor and the target feature vector according to a preset arrangement order to obtain a target encryption key for the multimedia data; or,
[0162] The reference key factor and the target feature vector are used to construct an initial key, and a preset private key is used to perform data encryption processing on the initial key to obtain a target encryption key for the multimedia data.
[0163] In another embodiment, the multimedia data is a face image of a target user acquired by the terminal calling a camera component, and the terminal and the server are built with the same feature extraction model; accordingly, the processing unit 602 can also be used for:
[0164] Sending the target encrypted data and the facial image to the server, allowing the server to obtain a reference feature vector of the facial image through the feature extraction model, using a reference encryption key generated according to the reference feature vector to perform data encryption processing on the facial image, and when the obtained reference encrypted data is consistent with the target encrypted data, performing identity recognition on the facial image to return the identity information of the target user to the terminal;
[0165] Receive the identity information returned by the server, and perform business processing according to the identity information, the business processing including: face payment processing.
[0166] In another embodiment, when the processing unit 602 is used to perform feature extraction processing on the multimedia data to obtain the target feature vector of the multimedia data, it can be specifically used to:
[0167] Acquire a feature extraction model, wherein the feature extraction model includes at least: a first network sub-model and a second network sub-model having different processing capabilities;
[0168] Calling the first network sub-model to perform feature extraction processing on the multimedia data to obtain M first vectors and a confidence level of each first vector, where M is a positive integer;
[0169] Calling the second network sub-model to perform feature extraction processing on the multimedia data to obtain N second vectors and a confidence level of each second vector, where N is a positive integer;
[0170] According to the confidence of each first vector and the confidence of each second vector, a target feature vector of the multimedia data is selected from the M first vectors and the N second vectors.
[0171] In yet another embodiment, when the processing unit 602 is used to select the target feature vector of the multimedia data from the M first vectors and the N second vectors according to the confidence of each first vector and the confidence of each second vector, it can be specifically used to:
[0172] According to the confidences of the first vectors and the second vectors, a vector with the largest confidence is selected from the M first vectors and the N second vectors as the target feature vector of the multimedia data; or,
[0173] A first vector whose confidence is greater than a first threshold value among the M first vectors is used as a candidate vector, and a second vector whose confidence is greater than a second threshold value among the N second vectors is used as a candidate vector; and any candidate vector is selected from all the candidate vectors as a target feature vector of the multimedia data.
[0174] In another embodiment, when the processing unit 602 is used to call any network sub-model to perform feature extraction processing on the multimedia data, it can be specifically used to:
[0175] Acquire a feature enhancement strategy adapted to any one of the network sub-models, where any one of the network sub-models is the first network sub-model or the second network sub-model;
[0176] According to the feature enhancement strategy, the multimedia data is subjected to feature enhancement processing to obtain enhanced data;
[0177] Call any of the network sub-models to perform feature extraction processing on the enhanced data.
[0178] In another embodiment, the multimedia data is a face image, and the enhanced data includes: an enhanced image corresponding to the face image; accordingly, when the processing unit 602 is used to perform feature enhancement processing on the multimedia data according to the feature enhancement strategy to obtain the enhanced data, it can be specifically used to:
[0179] According to the block parameters indicated by the feature enhancement strategy, split the target image into multiple image blocks, and traverse the multiple image blocks;
[0180] Calculating the gradient of each pixel in the current image block currently traversed, and constructing a directional gradient histogram of the current image block according to the gradient of each pixel;
[0181] According to the directional gradient histogram of the current image block, an image vector of the current image block is obtained to obtain the enhanced image; the enhanced image includes the image vectors of each image block, and the image vector of any image block is used to represent the enhanced image features of any image block.
[0182] In another embodiment, the target feature vector is obtained by calling a feature extraction model to extract features from the multimedia data; accordingly, the processing unit 602 may also be used to:
[0183] Acquire an initial model to be trained, wherein the initial model at least includes: a first initial sub-model and a second initial sub-model, and the processing capabilities of the initial sub-models in the initial model are different from each other;
[0184] Acquire a training sample set corresponding to the initial model, wherein the training sample set includes at least the following sample data: first sample data adapted to the processing capability of the first initial sub-model, and second sample data adapted to the processing capability of the second initial sub-model;
[0185] Using each sample data in the training sample set to perform model training on the initial model to obtain an intermediate model; and using any sample data in the training sample set to perform model evaluation on the intermediate model;
[0186] If the intermediate model passes the model evaluation, the intermediate model is used as the feature extraction model.
[0187] In another embodiment, the training sample set further includes a label category of each sample data; accordingly, when the processing unit 602 is used to perform model training on the initial model using each sample data in the training sample set to obtain an intermediate model, it can be specifically used to:
[0188] Traversing each sample data in the training sample set;
[0189] Calling a first initial sub-model in the initial model to perform feature extraction on the current sample data currently traversed, and determining a first prediction category of the current sample data according to the extracted first feature vector;
[0190] Calling a second initial sub-model in the initial model to extract features of the current sample data, and determining a second prediction category of the current sample data according to the extracted second feature vector;
[0191] According to the first prediction category, the second prediction category and the labeled category of the current sample data, the model parameters of the first initial model and the model parameters of the second initial model are updated to obtain an intermediate model.
[0192] In another embodiment, when the processing unit 602 is used to use any sample data in the training sample set to perform model evaluation on the intermediate model, it can be specifically used to:
[0193] Calling the intermediate model to perform feature extraction on any sample data in the training sample set to obtain a sample feature vector of the any sample data and a confidence level of the sample feature vector;
[0194] If the confidence of the sample feature vector is greater than or equal to the confidence threshold, determining that the intermediate model passes the model evaluation;
[0195] If the confidence of the sample feature vector is less than the confidence threshold, it is determined that the intermediate model has failed the model evaluation.
[0196] According to one embodiment of the present application, Figure 2 or Figure 4 Each step involved in the method shown can be performed by Figure 6 The data encryption device shown in FIG. 1 is executed by each unit in the data encryption device. For example, Figure 2 The steps S201-S202 shown in FIG. 1 can be performed by Figure 6 The acquisition unit 601 and the processing unit 602 shown in FIG. 6 are used to perform the steps S203-S204. Figure 6 Executed by the encryption unit 603 shown in . Figure 4 The step S401 shown in FIG. 4 can be performed by Figure 6 The acquisition unit 601 shown in FIG. 1 is executed, and steps S402-S405 can be performed by Figure 6 The processing unit 601 shown in FIG. 1 is executed, and steps S406-S407 can be performed by Figure 6 , and so on.
[0197] According to another embodiment of the present application, Figure 6 The various units in the data encryption device shown can be combined into one or several other units separately or in whole, or one (some) of the units can be further divided into multiple smaller units in function, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present invention. The above units are divided based on logical functions. In practical applications, the function of one unit can also be realized by multiple units, or the functions of multiple units can be realized by one unit. In other embodiments of the present invention, other units can also be included based on the image processing device. In practical applications, these functions can also be implemented with the assistance of other units, and can be implemented by the collaboration of multiple units.
[0198] According to another embodiment of the present application, the program can be executed by running on a general computing device such as a computer including a central processing unit (CPU), a random access memory medium (RAM), a read-only memory medium (ROM), and other processing elements and storage elements. Figure 2 or Figure 4 A computer program (including program code) for each step involved in the corresponding method shown in Figure 6 The data encryption device shown in the embodiment of the present invention is used to implement the data encryption method of the embodiment of the present invention. The computer program can be recorded on a computer-readable recording medium, for example, and loaded into the above-mentioned computing device through the computer-readable recording medium and run therein.
[0199] According to the embodiment of the present invention, for the multimedia data to be processed, the multimedia data may be firstly feature extracted to obtain the target feature vector of the multimedia data. Secondly, the target feature vector may be used as a key factor to generate a target encryption key for the multimedia data in a targeted manner, thereby ensuring the uniqueness of the encryption key and effectively improving the reliability of the encryption key. Since the target encryption key and the data features of the multimedia data are closely related, the target encryption key may be used to perform data encryption processing on the multimedia data, which may effectively improve the encryption effect and encryption performance of the data, thereby effectively preventing the multimedia data from being maliciously tampered with or stolen, and effectively improving the data security of the multimedia data.
[0200] Based on the description of the above method embodiment and device embodiment, the embodiment of the present invention also provides a computer device. Figure 7 The computer device at least includes a processor 701, an input interface 702, an output interface 703, and a computer storage medium 704. The processor 701, the input interface 702, the output interface 703, and the computer storage medium 704 in the computer device may be connected via a bus or other means.
[0201] The computer storage medium 704 can be stored in the memory of the computer device, and the computer storage medium 704 is used to store a computer program, and the computer program includes program instructions, and the processor 701 is used to execute the program instructions stored in the computer storage medium 704. The processor 701 (or CPU (Central Processing Unit)) is the computing core and control core of the computer device, which is suitable for implementing one or more instructions, and is specifically suitable for loading and executing one or more instructions to implement the corresponding method flow or corresponding function; in one embodiment, the processor 701 described in the embodiment of the present invention can be used to perform a series of data encryption processing, specifically including: receiving video stream data about the target object uploaded by the terminal, and the video stream data at least includes: obtaining multimedia data to be processed; performing feature extraction processing on the multimedia data to obtain the target feature vector of the multimedia data; generating a target encryption key about the multimedia data according to the target feature vector; and using the target encryption key to perform data encryption processing on the multimedia data to obtain target encrypted data, etc.
[0202] The embodiment of the present invention also provides a computer storage medium (Memory), which is a memory device in a computer device for storing programs and data. It is understandable that the computer storage medium here can include both built-in storage media in the computer device and, of course, extended storage media supported by the computer device. The computer storage medium provides a storage space, which stores the operating system of the computer device. In addition, one or more instructions suitable for being loaded and executed by the processor 701 are also stored in the storage space, and these instructions can be one or more computer programs (including program codes). It should be noted that the computer storage medium here can be a high-speed RAM memory, or a non-volatile memory (non-volatile memory), such as at least one disk storage; optionally, it can also be at least one computer storage medium located away from the aforementioned processor.
[0203] In one embodiment, the processor 701 may load and execute one or more instructions stored in a computer storage medium to implement the corresponding steps of the method in the above data encryption method embodiment; in a specific implementation, the processor 701 loads one or more instructions in a computer storage medium and executes the following steps:
[0204] Acquire multimedia data to be processed;
[0205] Performing feature extraction processing on the multimedia data to obtain a target feature vector of the multimedia data;
[0206] generating a target encryption key for the multimedia data according to the target feature vector; and,
[0207] The target encryption key is used to perform data encryption processing on the multimedia data to obtain target encrypted data.
[0208] In one implementation, when generating a target encryption key for the multimedia data according to the target feature vector, the one or more instructions may be loaded and specifically executed by the processor 701:
[0209] Obtaining a reference key factor, wherein the reference key factor includes at least one of the following factors: a device identifier, a timestamp, a signature version number, and a random number;
[0210] A target encryption key for the multimedia data is generated according to a key generation strategy and based on the reference key factor and the target feature vector.
[0211] In another embodiment, when generating the target encryption key of the multimedia data according to the key generation strategy and based on the reference key factor and the target feature vector, the one or more instructions may be loaded and specifically executed by the processor 701:
[0212] Arranging each factor in the reference key factor and the target feature vector according to a preset arrangement order to obtain a target encryption key for the multimedia data; or,
[0213] The reference key factor and the target feature vector are used to construct an initial key, and a preset private key is used to perform data encryption processing on the initial key to obtain a target encryption key for the multimedia data.
[0214] In another embodiment, the multimedia data is a facial image of a target user acquired by the terminal calling a camera component, and the terminal and the server are equipped with the same feature extraction model; accordingly, the one or more instructions may also be loaded and specifically executed by the processor 701:
[0215] Sending the target encrypted data and the facial image to the server, allowing the server to obtain a reference feature vector of the facial image through the feature extraction model, using a reference encryption key generated according to the reference feature vector to perform data encryption processing on the facial image, and when the obtained reference encrypted data is consistent with the target encrypted data, performing identity recognition on the facial image to return the identity information of the target user to the terminal;
[0216] Receive the identity information returned by the server, and perform business processing according to the identity information, the business processing including: face payment processing.
[0217] In another embodiment, when performing feature extraction processing on the multimedia data to obtain a target feature vector of the multimedia data, the one or more instructions may be loaded and specifically executed by the processor 701:
[0218] Acquire a feature extraction model, wherein the feature extraction model includes at least: a first network sub-model and a second network sub-model having different processing capabilities;
[0219] Calling the first network sub-model to perform feature extraction processing on the multimedia data to obtain M first vectors and a confidence level of each first vector, where M is a positive integer;
[0220] Calling the second network sub-model to perform feature extraction processing on the multimedia data to obtain N second vectors and a confidence level of each second vector, where N is a positive integer;
[0221] According to the confidence of each first vector and the confidence of each second vector, a target feature vector of the multimedia data is selected from the M first vectors and the N second vectors.
[0222] In another embodiment, when selecting the target feature vector of the multimedia data from the M first vectors and the N second vectors according to the confidence of each first vector and the confidence of each second vector, the one or more instructions may be loaded and specifically executed by the processor 701:
[0223] According to the confidences of the first vectors and the second vectors, a vector with the largest confidence is selected from the M first vectors and the N second vectors as the target feature vector of the multimedia data; or,
[0224] A first vector whose confidence is greater than a first threshold value among the M first vectors is used as a candidate vector, and a second vector whose confidence is greater than a second threshold value among the N second vectors is used as a candidate vector; and any candidate vector is selected from all the candidate vectors as a target feature vector of the multimedia data.
[0225] In another embodiment, when calling any network sub-model to perform feature extraction processing on the multimedia data, the one or more instructions may be loaded and specifically executed by the processor 701:
[0226] Acquire a feature enhancement strategy adapted to any one of the network sub-models, where any one of the network sub-models is the first network sub-model or the second network sub-model;
[0227] According to the feature enhancement strategy, the multimedia data is subjected to feature enhancement processing to obtain enhanced data;
[0228] Call any of the network sub-models to perform feature extraction processing on the enhanced data.
[0229] In another embodiment, the multimedia data is a face image, and the enhanced data includes: an enhanced image corresponding to the face image; accordingly, when the multimedia data is subjected to feature enhancement processing according to the feature enhancement strategy to obtain the enhanced data, the one or more instructions may be loaded and specifically executed by the processor 701:
[0230] According to the block parameters indicated by the feature enhancement strategy, split the target image into multiple image blocks, and traverse the multiple image blocks;
[0231] Calculating the gradient of each pixel in the current image block currently traversed, and constructing a directional gradient histogram of the current image block according to the gradient of each pixel;
[0232] According to the directional gradient histogram of the current image block, an image vector of the current image block is obtained to obtain the enhanced image; the enhanced image includes the image vectors of each image block, and the image vector of any image block is used to represent the enhanced image features of any image block.
[0233] In another embodiment, the target feature vector is obtained by calling a feature extraction model to extract features from the multimedia data; accordingly, the one or more instructions may also be loaded and specifically executed by the processor 701:
[0234] Acquire an initial model to be trained, wherein the initial model at least includes: a first initial sub-model and a second initial sub-model, and the processing capabilities of the initial sub-models in the initial model are different from each other;
[0235] Acquire a training sample set corresponding to the initial model, wherein the training sample set includes at least the following sample data: first sample data adapted to the processing capability of the first initial sub-model, and second sample data adapted to the processing capability of the second initial sub-model;
[0236] Using each sample data in the training sample set to perform model training on the initial model to obtain an intermediate model; and using any sample data in the training sample set to perform model evaluation on the intermediate model;
[0237] If the intermediate model passes the model evaluation, the intermediate model is used as the feature extraction model.
[0238] In another embodiment, the training sample set further includes a label category of each sample data; accordingly, when the initial model is trained using each sample data in the training sample set to obtain an intermediate model, the one or more instructions may be loaded and specifically executed by the processor 701:
[0239] Traversing each sample data in the training sample set;
[0240] Calling a first initial sub-model in the initial model to perform feature extraction on the current sample data currently traversed, and determining a first prediction category of the current sample data according to the extracted first feature vector;
[0241] Calling a second initial sub-model in the initial model to extract features of the current sample data, and determining a second prediction category of the current sample data according to the extracted second feature vector;
[0242] According to the first prediction category, the second prediction category and the labeled category of the current sample data, the model parameters of the first initial model and the model parameters of the second initial model are updated to obtain an intermediate model.
[0243] In another embodiment, when any sample data in the training sample set is used to perform model evaluation on the intermediate model, the one or more instructions may be loaded and specifically executed by the processor 701:
[0244] Calling the intermediate model to perform feature extraction on any sample data in the training sample set to obtain a sample feature vector of the any sample data and a confidence level of the sample feature vector;
[0245] If the confidence of the sample feature vector is greater than or equal to the confidence threshold, determining that the intermediate model passes the model evaluation;
[0246] If the confidence of the sample feature vector is less than the confidence threshold, it is determined that the intermediate model has failed the model evaluation.
[0247] According to the embodiment of the present invention, for the multimedia data to be processed, the multimedia data may be firstly feature extracted to obtain the target feature vector of the multimedia data. Secondly, the target feature vector may be used as a key factor to generate a target encryption key for the multimedia data in a targeted manner, thereby ensuring the uniqueness of the encryption key and effectively improving the reliability of the encryption key. Since the target encryption key and the data features of the multimedia data are closely related, the target encryption key may be used to perform data encryption processing on the multimedia data, which may effectively improve the encryption effect and encryption performance of the data, thereby effectively preventing the multimedia data from being maliciously tampered with or stolen, and effectively improving the data security of the multimedia data.
[0248] It should be noted that according to one aspect of the present application, a computer program product or a computer program is also provided, the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the above Figure 2 or Figure 4 The method provided in various optional aspects of the data encryption method embodiment shown.
[0249] Furthermore, it should be understood that what is disclosed above is only a preferred embodiment of the present invention, and certainly cannot be used to limit the scope of rights of the present invention. Therefore, equivalent changes made according to the claims of the present invention are still within the scope of the present invention.
Claims
1. A data encryption method, It is characterized in that include: Acquire multimedia data to be processed; Acquire a feature extraction model, wherein the feature extraction model includes at least: a first network sub-model and a second network sub-model having different processing capabilities; the processing capability of any network sub-model refers to: a maximum size of input data supported by the corresponding network sub-model for processing; Calling the first network sub-model to perform feature extraction processing on the multimedia data to obtain M first vectors and a confidence level of each first vector, where M is a positive integer; Calling the second network sub-model to perform feature extraction processing on the multimedia data to obtain N second vectors and a confidence level of each second vector, where N is a positive integer; According to the confidence of each first vector and the confidence of each second vector, a target feature vector of the multimedia data is selected from the M first vectors and the N second vectors; wherein the target feature vector is any one of the following: a vector with the largest confidence, any candidate vector, or an average of the candidate vectors; the candidate vector is a first vector with a confidence greater than a first threshold or a second vector with a confidence greater than a second threshold; generating a target encryption key for the multimedia data according to the target feature vector; and, The target encryption key is used to perform data encryption processing on the multimedia data to obtain target encrypted data.
2. The method according to claim 1, It is characterized in that The step of generating a target encryption key for the multimedia data according to the target feature vector comprises: Obtaining a reference key factor, wherein the reference key factor includes at least one of the following factors: a device identifier, a timestamp, a signature version number, and a random number; A target encryption key for the multimedia data is generated according to a key generation strategy and based on the reference key factor and the target feature vector.
3. The method according to claim 2, It is characterized in that The step of generating a target encryption key for the multimedia data according to the key generation strategy and based on the reference key factor and the target feature vector includes: Arranging each factor in the reference key factor and the target feature vector according to a preset arrangement order to obtain a target encryption key for the multimedia data; or, The reference key factor and the target feature vector are used to construct an initial key, and a preset private key is used to perform data encryption processing on the initial key to obtain a target encryption key for the multimedia data.
4. The method according to claim 2 or 3, It is characterized in that The multimedia data is a facial image of a target user acquired by the terminal calling a camera component, and the terminal and the server are built with the same feature extraction model; The method further comprises: Sending the target encrypted data and the facial image to the server, allowing the server to obtain a reference feature vector of the facial image through the feature extraction model, using a reference encryption key generated according to the reference feature vector to perform data encryption processing on the facial image, and when the obtained reference encrypted data is consistent with the target encrypted data, performing identity recognition on the facial image to return the identity information of the target user to the terminal; Receive the identity information returned by the server, and perform business processing according to the identity information, the business processing including: face payment processing.
5. The method according to claim 1, It is characterized in that The step of selecting the target feature vector of the multimedia data from the M first vectors and the N second vectors according to the confidence of each first vector and the confidence of each second vector includes: According to the confidences of the first vectors and the second vectors, a vector with the largest confidence is selected from the M first vectors and the N second vectors as the target feature vector of the multimedia data; or, A first vector whose confidence is greater than a first threshold value among the M first vectors is used as a candidate vector, and a second vector whose confidence is greater than a second threshold value among the N second vectors is used as a candidate vector; and any candidate vector is selected from all the candidate vectors as a target feature vector of the multimedia data.
6. The method according to claim 1, It is characterized in that Calling any network sub-model to perform feature extraction processing on the multimedia data includes: Acquire a feature enhancement strategy adapted to any one of the network sub-models, where any one of the network sub-models is the first network sub-model or the second network sub-model; According to the feature enhancement strategy, the multimedia data is subjected to feature enhancement processing to obtain enhanced data; Call any of the network sub-models to perform feature extraction processing on the enhanced data.
7. The method according to claim 6, It is characterized in that The multimedia data is a face image, and the enhanced data includes: an enhanced image corresponding to the face image; and the feature enhancement processing is performed on the multimedia data according to the feature enhancement strategy to obtain the enhanced data, including: According to the block parameters indicated by the feature enhancement strategy, split the target image into multiple image blocks, and traverse the multiple image blocks; Calculating the gradient of each pixel in the current image block currently traversed, and constructing a directional gradient histogram of the current image block according to the gradient of each pixel; According to the directional gradient histogram of the current image block, an image vector of the current image block is obtained to obtain the enhanced image; the enhanced image includes the image vectors of each image block, and the image vector of any image block is used to represent the enhanced image features of any image block.
8. The method according to claim 1, It is characterized in that The target feature vector is obtained by calling a feature extraction model to perform feature extraction on the multimedia data; The method further comprises: Acquire an initial model to be trained, wherein the initial model at least includes: a first initial sub-model and a second initial sub-model, and the processing capabilities of the initial sub-models in the initial model are different from each other; Acquire a training sample set corresponding to the initial model, wherein the training sample set includes at least the following sample data: first sample data adapted to the processing capability of the first initial sub-model, and second sample data adapted to the processing capability of the second initial sub-model; Using each sample data in the training sample set to perform model training on the initial model to obtain an intermediate model; and using any sample data in the training sample set to perform model evaluation on the intermediate model; If the intermediate model passes the model evaluation, the intermediate model is used as the feature extraction model.
9. The method according to claim 8, It is characterized in that The training sample set also includes the labeled category of each sample data; the initial model is trained using each sample data in the training sample set to obtain an intermediate model, including: Traversing each sample data in the training sample set; Calling a first initial sub-model in the initial model to perform feature extraction on the current sample data currently traversed, and determining a first prediction category of the current sample data according to the extracted first feature vector; Calling a second initial sub-model in the initial model to extract features of the current sample data, and determining a second prediction category of the current sample data according to the extracted second feature vector; According to the first prediction category, the second prediction category and the labeled category of the current sample data, the model parameters of the first initial model and the model parameters of the second initial model are updated to obtain an intermediate model.
10. The method according to claim 8, It is characterized in that The adopting any sample data in the training sample set to perform model evaluation on the intermediate model comprises: Calling the intermediate model to perform feature extraction on any sample data in the training sample set to obtain a sample feature vector of the any sample data and a confidence level of the sample feature vector; If the confidence of the sample feature vector is greater than or equal to the confidence threshold, determining that the intermediate model passes the model evaluation; If the confidence of the sample feature vector is less than the confidence threshold, it is determined that the intermediate model has failed the model evaluation.
11. A data encryption device, It is characterized in that include: An acquisition unit, used for acquiring multimedia data to be processed; A processing unit, used to obtain a feature extraction model, the feature extraction model at least comprising: a first network sub-model and a second network sub-model with different processing capabilities; the processing capability of any network sub-model refers to: the maximum size of input data supported by the corresponding network sub-model for processing; calling the first network sub-model to perform feature extraction processing on the multimedia data to obtain M first vectors and the confidence of each first vector, M being a positive integer; calling the second network sub-model to perform feature extraction processing on the multimedia data to obtain N second vectors and the confidence of each second vector, N being a positive integer; selecting a target feature vector of the multimedia data from the M first vectors and the N second vectors according to the confidence of each first vector and the confidence of each second vector; wherein the target feature vector is any one of the following: a vector with the largest confidence, any candidate vector or the average of each candidate vector; the candidate vector refers to a first vector with a confidence greater than a first threshold or a second vector with a confidence greater than a second threshold; an encryption unit, configured to generate a target encryption key for the multimedia data according to the target feature vector; and The encryption unit is further used to perform data encryption processing on the multimedia data using the target encryption key to obtain target encrypted data.
12. A computer device comprising an input interface and an output interface, It is characterized in that Also includes: a processor adapted to implement one or more instructions; as well as, A computer storage medium storing one or more instructions, wherein the one or more instructions are suitable for being loaded by the processor and executing the data encryption method according to any one of claims 1 to 10.
13. A computer storage medium, It is characterized in that The computer storage medium stores one or more instructions, and the one or more instructions are suitable for being loaded by a processor and executing the data encryption method according to any one of claims 1-10.
Citation Information
Patent Citations
Face image biological key generating method
CN103886235A
Data management method and system based on biological key
CN110391898A
Portrait biological characteristic privacy protection and decryption method
CN111723395A
Privacy-Enhanced Biometrics-Secret Binding Scheme
US20160269178A1
Network of biometrically secure devices with enhanced privacy protection
US20180108020A1
