A privacy decision recommendation system based on image data and deep learning

Abstract

The present invention provides a privacy decision recommendation system based on image data and deep learning. Model library: used to learn a large number of social privacy samples in advance through deep learning technology, and generate multiple deep learning models for determining different privacy levels; semantic recognition module: used to convert the surface information of the image in the image data into semantic information when receiving an instruction to send image data; privacy recognition module: used to import the semantic information into the model library and determine the privacy level of the image data; decision module: used to feedback the image sending decision to the user after the privacy level is determined; wherein the image sending decision includes: immediate sending decision, prohibiting sending decision and partial sending decision.

Description

Technical Field

[0001] The present invention relates to the field of privacy processing technology, and in particular to a privacy decision recommendation system based on image data and deep learning. Background Art

[0002] At present, with the development of social network and smart phone technology, people can easily share the images they take on social network. However, if such sharing is not controlled or guided, it may cause various privacy leaks, and even cause personal injury or infringement, various cyber crimes or fraud. As pointed out by relevant surveys and research, many teenagers like to share or show off their own, friends' or classmates' private photos on social networks, but they don't know that doing so will cause various privacy leaks, and some even cause lifelong negative impact or harm. It is also reported that 78% of network-related criminals use social network media to find victims. Criminals can query or deduce various information about the victim based on social network pictures (and related auxiliary information). Current social networking sites do not provide effective privacy decision recommendation tools for ordinary users (especially teenagers) to remind them that when posting social network images, the user himself or the social network access control system makes decisions. Therefore, it is urgent to develop privacy decision recommendation methods and systems for sharing digital images on social networks.

[0003] In the prior art, the privacy protection of images in social networks is mainly achieved through user-defined access control permissions, anonymization, image encryption, and self-destruction. However, since mobile terminals already support real-time sharing of photos and videos, users may not realize the insufficiency of permission settings until after uploading the images, or accidentally send images with private information due to drunkenness. At this time, the images have already been uploaded, which will bring about the risk of privacy leakage that is difficult to eliminate in the large environment of social networks. Summary of the invention

[0004] The present invention provides a privacy decision recommendation system based on image data and deep learning, which is used to solve the problem that users are likely to realize the insufficiency of permission settings only after uploading images, or accidentally send images with private information due to drunkenness and unclear consciousness. At this time, the image has already been uploaded, which will bring about difficult-to-eliminate privacy leakage risks in the large environment of social networks.

[0005] A privacy decision recommendation system based on image data and deep learning, including:

[0006] Model library: used to pre-learn a large number of social privacy samples through deep learning technology and generate multiple deep learning models for different privacy level judgments;

[0007] Semantic recognition module: used for converting the surface information of the image in the image data into semantic information when receiving the instruction to send the image data;

[0008] Privacy identification module: used for importing the semantic information into the model library to determine the privacy level of the image data;

[0009] Decision module: used to send a decision to the user after the privacy level is determined;

[0010] The image sending decision includes: immediate sending decision, prohibiting sending decision and partial sending decision.

[0011] As an embodiment of the present invention: the system further includes:

[0012] Privacy determination module: used to obtain the setting standards of different privacy levels set by users based on the user's historical privacy setting information;

[0013] The set standards include: picture content standards, social type standards and picture storage location standards;

[0014] Classifier construction module: according to the set standard, establish an image classifier based on image information extraction; wherein,

[0015] The picture information includes: an image surface information classifier, a social type classifier, and a picture storage classifier;

[0016] The picture classifier includes: an information content classifier and a picture type classifier;

[0017] Sample screening module: screening the user's historical privacy data according to the image classifier to determine the social privacy sample;

[0018] Level classification module: used to perform privacy awareness calculation and privacy trust calculation on different social privacy samples, and classify privacy levels according to the calculation results to determine social privacy samples of different social privacy levels;

[0019] The social privacy samples include: low-risk privacy samples, medium-risk privacy samples and high-risk privacy samples.

[0020] As an embodiment of the present invention: the level division module includes:

[0021] Data extraction unit: used to determine the social program existing on the user terminal, establish an API crawling interface according to the social program, and obtain user data; wherein,

[0022] The user data is image data generated during the social interaction;

[0023] Data preprocessing unit: used to preprocess the user data and determine the semantic content of the user data; wherein,

[0024] The preprocessing includes: image-text conversion processing, word segmentation processing, privacy labeling processing, TF-IDF processing and image named entity classification processing;

[0025] Calculation unit: performing privacy awareness calculation on the semantic content to determine a privacy awareness value, and performing privacy trust calculation on the semantic content to determine a privacy trust value;

[0026] A level division unit: a binary decision tree is established according to the privacy trust value and the privacy awareness value to determine the sensitivity values ​​of different image data, and privacy level division is performed according to the sensitivity values; wherein,

[0027] The privacy levels include: low-risk privacy, medium-risk privacy, and high-risk privacy;

[0028] The sensitivity value has a corresponding sensitivity value threshold according to the privacy level.

[0029] As an embodiment of the present invention: the model library includes:

[0030] Deep learning model building unit: pre-set deep learning models based on different privacy levels;

[0031] The deep learning models include: a low-risk privacy model, a medium-risk privacy model, and a high-risk privacy model;

[0032] Deep learning training unit: used to import the social privacy samples into the deep learning model, perform data iterative calculation, and determine the optimal parameters of the model after iterative calculation;

[0033] Model library unit: used to generate a deep learning model library for deep learning based on the optimal parameters of the model.

[0034] As an embodiment of the present invention: the semantic recognition module includes:

[0035] Instruction recognition unit: used to monitor the privacy of the social software on the user terminal and determine whether there is an image issuing instruction;

[0036] Feedback unit: used to generate a feedback signal when there is an image issuing instruction;

[0037] Annotation unit: used for determining the image address of the image to be sent when sensing the feedback signal, and semantically annotating the surface information of the image according to the image address;

[0038] A word segmentation processing unit: used to segment the surface information of the image according to the semantic annotation to generate a vocabulary set;

[0039] Semantic conversion unit: used for performing semantic conversion on each image according to the vocabulary set to generate semantic text of each image.

[0040] As an embodiment of the present invention: the privacy identification module includes:

[0041] A matching unit is used to perform steady-state matching of a deep learning model according to the semantic information, and determine a unique deep learning model corresponding to the semantic information; wherein,

[0042] The steady-state matching includes: performing steady-state calculations on the semantic information and different deep learning models to determine a steady-state degree, and determining a unique deep recognition model according to the steady-state degree;

[0043] Importing unit: used for importing the semantic information into the unique deep learning model to determine the privacy level value;

[0044] Level determination unit: used to determine the corresponding privacy level according to the privacy level value.

[0045] As an embodiment of the present invention: the system further includes a decision module, the decision module including:

[0046] Discrete attribute determination unit: used to match the corresponding discrete model according to the privacy level of the image data and determine the discrete value;

[0047] A continuous attribute determination unit: used to match a corresponding continuous model according to the privacy level of the image data and determine a continuous value;

[0048] A data set generating unit: used for generating a corresponding semantic generalization data set according to the discrete value and the continuous value;

[0049] Adaptive privacy budget allocation unit: performing privacy budgeting according to the generalized data set, constructing a privacy decision tree, and determining images that can be sent and images that cannot be sent in the image data;

[0050] A subdivision scheme selection unit: performing privacy decision subdivision according to the privacy decision tree and generating an image sending scheme;

[0051] Decision-making unit: according to the image sending plan, sends the image sending plan to the user terminal interface and obtains the selection instruction.

[0052] As an embodiment of the present invention: the instant issuance decision further includes: obtaining information of the image receiving end, determining whether the receiving end has privacy acquisition authorization, and executing an instant privacy decision when the image receiving end has privacy acquisition authorization;

[0053] The prohibition decision further includes: recording the issuing instruction according to the risk level of the image data, and generating an information email with the prohibition decision, and sending the information email to the user mailbox;

[0054] The partial issuance decision also includes: assessing the risk level of the images in the image data, determining the images with low risk levels based on the assessment results, and issuing the images with low risk levels, and storing the images with medium risk levels and high risk levels in the corresponding to-be-issued spaces.

[0055] As an embodiment of the present invention, the system further includes:

[0056] Decision matrix unit: used to obtain all decisions issued by the image and establish a hesitant fuzzy decision matrix based on the attributes of each decision;

[0057] A decision comprehensive value calculation unit: used to determine the comprehensive decision value of each decision behavior according to the hesitant fuzzy decision matrix;

[0058] Sorting unit: used for performing average weighted calculation on the comprehensive decision value to generate a decision sequence;

[0059] Decision-making scheme determination unit: used to determine the optimal decision-making scheme according to the decision sequence.

[0060] As an embodiment of the present invention: the system further includes:

[0061] Privacy label setting module: users submit pictures containing protected objects and corresponding semantic labels;

[0062] Label image matching module: used to identify images and obtain the semantic content of the protected objects in the images;

[0063] The identified semantic content and the corresponding semantic tags are stored in a semantic sample library, and semantic information extraction models for different images are set based on the semantic sample library;

[0064] Rule setting module: the user selects a corresponding set of semantic samples from the semantic information extraction model and sets the privacy level of each semantic sample;

[0065] Privacy level matching module: The user stores a new picture in the rule setting module for image recognition, obtains the privacy level of the new picture, and performs semantic sample calibration in the rule setting module according to the privacy level of the new picture.

[0066] The beneficial effects of the present invention are:

[0067] The present invention judges the privacy level of image data, filters the judgment results, and pushes the image data with the highest ranking to the user; performs semantic analysis on the image data, and combines the image data to be protected with the corresponding semantic information; uses a large number of samples in the early stage of learning through deep learning technology to learn the privacy data in the social network, thereby increasing the credibility of the image privacy decision. The present invention uses human-computer interaction as the technical basis, refines deterministic text privacy rules, and based on this, analyzes the privacy semantic relationship layer by layer to achieve rapid matching of complex image privacy relationships. The present invention not only solves the problem of automatic understanding and matching of privacy semantics of complex digital images. The present invention proposes a multi-source social network privacy decision model based on multimodal deep learning to solve the problem of difficult joint analysis of image privacy semantics in complex and diverse social environments. The present invention proposes and designs a privacy protection method based on a random response mechanism to solve the risk of privacy leakage of user training data in deep learning privacy decision models.

[0068] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description and the accompanying drawings.

[0069] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0071] Figure 1 This is a system composition diagram of a privacy decision recommendation system based on image data and deep learning in an embodiment of the present invention;

[0072] Figure 2 This is a sample processing diagram of a privacy decision recommendation system based on image data and deep learning in an embodiment of the present invention;

[0073] Figure 3 This is a new image calibration map of a privacy decision recommendation system based on image data and deep learning in an embodiment of the present invention. DETAILED DESCRIPTION

[0074] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0075] This solution automatically calculates the privacy level through deep learning methods and makes corresponding decision recommendations based on the calculation results of the privacy level, so that the privacy level of image information recommended by the decision is higher.

[0076] Example 1

[0077] As attached Figure 1 As shown, the present invention provides a privacy decision recommendation system based on image data and deep learning, including:

[0078] Model library: used to pre-learn a large number of social privacy samples through deep learning technology and generate multiple deep learning models for different privacy level judgments;

[0079] Semantic recognition module: used for converting the surface information of the image in the image data into semantic information when receiving the instruction to send the image data;

[0080] Privacy identification module: used for importing the semantic information into the model library to determine the privacy level of the image data;

[0081] Decision module: used to send a decision to the user after the privacy level is determined;

[0082] The image sending decision includes: immediate sending decision, prohibiting sending decision and partial sending decision.

[0083] The principle of the above technical solution is:

[0084] The difference between the privacy decision-making method of the present invention and the prior art lies mainly in the semantic transformation and the establishment of a deep model. The prior art has a single model, but the main innovation of the present invention is that the present invention establishes multiple deep learning models that do not require registration and judgment, such as: a low-level privacy decision recommendation model, a medium-level privacy decision recommendation model, and a high-level privacy decision recommendation model; through multiple models, different images are determined to correspond to different privacy levels in a semantic recognition manner. Different privacy levels determine that when different images are sent, some can be sent and some cannot be sent, and when sending multiple images, there is also a problem of partial sending. In the prior art, the social method mostly uses the user's manual method to judge the privacy level of image data. When the user's privacy protection ability is not strong or when he is unconscious, it leads to the leakage of privacy information; the protection of image privacy only supports the surface information of the image, and is not combined with semantic information. There is less learning of privacy samples in the early stage, so the credibility in the decision recommendation process is low. The present invention judges the privacy level of image data, filters the judgment results, and pushes the image data with the highest ranking to the user; performs semantic analysis on image data, and combines the image data to be protected with the corresponding semantic information; uses deep learning technology to use a large number of samples in the early stage of learning to learn the privacy data in the social network, thereby increasing the credibility of image privacy decisions. The present invention refines deterministic text privacy rules through human-computer interaction, and based on this, analyzes privacy semantic relationships layer by layer to achieve rapid matching of complex image privacy relationships. The present invention proposes an image-source-only social network decision model based on image-text matching deep learning to solve the problem of automatic understanding and matching of privacy semantics of complex digital images. The present invention proposes a multi-source social network privacy decision model based on multimodal deep learning to solve the problem of difficult joint analysis of image privacy semantics in complex and diverse social environments. The present invention proposes and designs a privacy protection method based on a random response mechanism to solve the risk of privacy leakage of user training data in a deep learning privacy decision model.

[0085] The beneficial effects of the present invention are:

[0086] The present invention judges the privacy level of image data, filters the judgment results, and pushes the image data with the highest ranking to the user; performs semantic analysis on the image data, and combines the image data to be protected with the corresponding semantic information; uses a large number of samples in the early stage of learning through deep learning technology to learn the privacy data in the social network, thereby increasing the credibility of the image privacy decision. The present invention uses human-computer interaction as the technical basis, refines deterministic text privacy rules, and based on this, analyzes the privacy semantic relationship layer by layer to achieve rapid matching of complex image privacy relationships. The present invention not only solves the problem of automatic understanding and matching of privacy semantics of complex digital images. The present invention proposes a multi-source social network privacy decision model based on multimodal deep learning to solve the problem of difficult joint analysis of image privacy semantics in complex and diverse social environments. The present invention proposes and designs a privacy protection method based on a random response mechanism to solve the risk of privacy leakage of user training data in deep learning privacy decision models.

[0087] Example 2

[0088] As an embodiment of the present invention: Figure 2 As shown, the system also includes:

[0089] Privacy determination module: used to obtain the setting standards of different privacy levels set by users based on the user's historical privacy setting information;

[0090] The set standards include: picture content standards, social type standards and picture storage location standards;

[0091] The user's historical privacy setting information is obtained in advance, and then the type and standard of the set privacy information are determined; for example: when a user sets his or her own life photos as privacy, if the picture content involves the user's life photos, the life information is the standard of the picture content, and the life information may be level 2 privacy. In addition, some pictures may be the user's work content, and the user's work may be level 1 privacy content. The social type standard is different social channels such as WeChat and blind dating. For the blind dating social channel, it may be level 2 privacy of the social type, and there may be encrypted places in the user's mobile phone, and these encrypted places may belong to the user's level 3 privacy;

[0092] Classifier construction module: according to the set standard, establish an image classifier based on image information extraction; wherein,

[0093] The picture information includes: an image surface information classifier, a social type classifier, and a picture storage classifier;

[0094] The image classifier for image information extraction is also a classifier that divides information through images; the image surface information classifier is to classify information based on the elements displayed on the surface of the image. For example, if the image is a workplace, the classification type is work; and when the element displayed on the surface of the image is a forest, the classification type may be scenery. The social type classifier is to determine the type of social channel based on the elements on the image.

[0095] The picture classifier includes: an information content classifier and a picture type classifier;

[0096] Sample screening module: screening the user's historical privacy data according to the image classifier to determine the social privacy sample;

[0097] Level classification module: used to perform privacy awareness calculation and privacy trust calculation on different social privacy samples, and classify privacy levels according to the calculation results to determine social privacy samples of different social privacy levels;

[0098] The social privacy samples include: low-risk privacy samples, medium-risk privacy samples and high-risk privacy samples. Privacy awareness calculation is that users set privacy features according to the judgment criteria for privacy, build a screening model through privacy features, and determine privacy awareness through the screening model judgment; then establish privacy entropy calculation, and calculate the trust through the privacy entropy value.

[0099] The principle of the above technical solution is: because in the prior art, there is a problem that when obtaining privacy samples, there is little sample data, so the solution adopted by the present invention is to divide different levels of privacy data in the form of a classifier, and then classify and store them after division. The different social privacy samples stored in the classification can determine all privacy samples through privacy awareness calculation and privacy trust calculation based on user habits.

[0100] Example 3

[0101] As an embodiment of the present invention: the level division module includes:

[0102] Data extraction unit: used to determine the social program existing on the user terminal, establish an API crawling interface according to the social program, and obtain user data; wherein,

[0103] The user data is image data generated during the social interaction;

[0104] The API crawling interface is a predefined full-cycle interface that is used to comprehensively obtain all data in the user's terminal device, thereby performing a comprehensive test of all data in the user's mobile terminal.

[0105] Data preprocessing unit: used to preprocess the user data and determine the semantic content of the user data; wherein,

[0106] The preprocessing includes: image-text conversion (that is, converting the image content into text), word segmentation (if there are texts in the image or text, these texts are segmented and divided into multiple technical terms), privacy annotation (that is, annotating the privacy content in the image), TF-IDF processing (belonging to text mining technology, it is to protect user privacy in the user terminal) and image named entity classification (that is, after the image has a specific classification, the image is classified by the specific classification name);

[0107] Calculation unit: perform privacy awareness calculation on the semantic content to determine the privacy awareness value, and perform privacy trust calculation on the semantic content to determine the privacy trust value; this is different from the privacy awareness calculation of the privacy sample in Example 3, which is a privacy content calculation for each private content, and then calculates the privacy trust value based on the privacy content. The privacy trust value is whether the private content is trustworthy and whether it is really private content. The privacy awareness value is to calculate the image content to determine whether it is private and what the degree of privacy is. The degree of privacy is the privacy awareness value.

[0108] Level classification unit: According to the privacy trust value and the privacy awareness value, a binary decision tree is established to determine the sensitivity values ​​of different image data, and privacy levels are classified according to the sensitivity values; wherein the sensitivity value is also the privacy value of the privacy content to the user, that is, the value of the degree of privacy.

[0109] The privacy levels include: low-risk privacy, medium-risk privacy, and high-risk privacy;

[0110] The sensitivity value has a corresponding sensitivity value threshold according to the privacy level.

[0111] Because the present invention needs to establish a variety of deep learning models with different privacy levels, it is necessary to obtain a variety of samples. The present invention establishes an API interface, obtains image data through the API interface, processes the image data, and divides the privacy data into different privacy levels based on the processing results. The privacy division is based on the sensitivity value, which is mainly divided manually. The present invention can not only obtain personal privacy by analyzing a single or multiple images on a social network, but also obtain intelligence in various fields such as national politics, economy, and military by analyzing a single or multiple images (and related text and other environmental information) posted to forums, social networks, or other Internet environments. This means that the present invention only uses the API interface to establish a model.

[0112] In a specific embodiment, it also includes: determining the privacy awareness value is calculated by the following steps:

[0113] Step 1: Determine whether the content complies with the privacy requirements based on the semantic content:

[0114] in, represents the privacy value; represents the historical privacy type determination coefficient; represents the historical privacy content determination coefficient; Indicates Type characteristics of image data; Indicates Semantic features of image data; The decision function is used to determine the Whether the type characteristics of the image data meet the privacy type determination coefficient, and determine the privacy type value; To determine the Whether the type characteristics of the image data meet the privacy content determination coefficient is used to determine the privacy content value; whether it is privacy content is determined by the sum of the content and privacy type; only when both are met, or only one is met, but the sum must be greater than 1. Otherwise, it is worth being relieved that neither of them is private. , Indicates the amount of private data.

[0115] Step 2 performs content matching and calculates the privacy awareness value:

[0116]

[0117] in, represents the correlation coefficient, when hour, Indicates The image features of the privacy images with different privacy levels are pre-trained image features for comparison.

[0118] The above-mentioned technical solution performs a matching algorithm when the privacy awareness value is determined, and then calculates the image features of the privacy images with different historical levels through the matching algorithm. When the two are different, the relevant value is judged based on the prior inertia formula, and the corresponding privacy awareness value is determined according to the relevant value. Through this privacy awareness value, more caution is exercised when sending pictures.

[0119] Example 4

[0120] As an embodiment of the present invention: the model library includes:

[0121] Deep learning model building unit: pre-set deep learning models based on different privacy levels;

[0122] The deep learning models include: a low-risk privacy model, a medium-risk privacy model, and a high-risk privacy model;

[0123] Deep learning training unit: used to import the social privacy samples into the deep learning model, perform data iterative calculation, and determine the optimal parameters of the model after iterative calculation;

[0124] The most important parameter of the model is the reference parameter to determine whether the privacy data is low-risk privacy, medium-risk privacy or high-risk privacy. This reference parameter is an optimal reference parameter. In actual calculation, we can get a threshold. There are different thresholds for low-risk privacy, medium-risk privacy or high-risk privacy. The maximum value among the thresholds is the optimal model parameter;

[0125] Model library unit: used to generate a deep learning model library for deep learning based on the optimal parameters of the model.

[0126] The principle of the above technical solution is:

[0127] Because the present invention needs to set up different multiple deep learning model libraries, this model library can not only determine the privacy level, but also continuously optimize the identification of privacy data in an iterative calculation manner, so as to obtain the most appropriate privacy data as much as possible. At present, users may bring various privacy security threats when sharing images in social networks, and the current social network application system fails to provide effective decision support tools. The existing theoretical models and methods have many shortcomings in terms of privacy decision-making in social network image sharing. In view of this application demand and the current status of theoretical research, the present invention comprehensively runs methods and means such as human-computer interaction, deep learning, and differential privacy, studies and designs privacy decision models in network image sharing, explores and solves related scientific problems, and innovates multimedia information security research content and methods, providing core algorithms and technical support for privacy security applications of social network image information. Therefore, the present invention establishes a variety of different deep learning models. At this stage, there are also some shortcomings, such as most of them are based on manual labeling, rather than automatic recognition based on image semantics, which increases the burden on users. Many users are unwilling or even unable to perform manual labeling, which makes the method invalid. For example, when a user is drunk, he cannot complete effective manual labeling, and some elderly or teenagers cannot complete accurate labeling. At the same time, image information can be used to analyze privacy levels, such as "the more people in the image, the higher the privacy level" and "the brighter the image, the lower the privacy level", but these rules are not universal.

[0128] Based on the above considerations, this project intends to use the rapidly developing deep learning technology to build a social network image privacy decision model, and use the advantages of deep learning technology in image privacy semantic understanding and matching to design a more accurate and effective privacy decision model to solve the above problems.

[0129] Embodiment 5:

[0130] As an embodiment of the present invention: the semantic recognition module includes:

[0131] Instruction recognition unit: used to monitor the privacy of the social software on the user terminal and determine whether there is an image issuing instruction;

[0132] Since the present invention is mainly for the detection of image data, the image data in the user terminal will be monitored, and the image mainly monitored is sent from the user terminal.

[0133] Feedback unit: used to generate a feedback signal when there is an image issuing instruction; the feedback instruction refers to the feedback issued after the image issuing instruction is generated, but there is no such setting in the prior art, and it can only be judged by real-time monitoring whether an image is issued.

[0134] Annotation unit: used for determining the image address of the image to be sent when sensing the feedback signal, and semantically annotating the surface information of the image according to the image address;

[0135] Semantic annotation of image surface information through image address means annotating the type, content and specific image information of the image through the image address. This annotation is a specific content annotation and type annotation of the image.

[0136] The word segmentation processing unit is used to segment the surface information of the image according to the semantic annotation and generate a vocabulary set; the function of the word segmentation set is mainly to segment the key information of the image information.

[0137] Semantic conversion unit: used to perform semantic conversion on each image according to the vocabulary set to generate a semantic text for each image. This semantic text is an image description for each image, which is a specific content description and type description of the image.

[0138] The beneficial effect of the above technical solution is that one of the most important capabilities of the present invention is the ability to realize semantic recognition and semantic labeling. In this way, it can assist deep learning and convert each image into semantic text. Through these semantic texts, more accurate training can be achieved, and more accurate risk level judgment can be achieved.

[0139] Embodiment 6:

[0140] As an embodiment of the present invention: the privacy identification module includes:

[0141] A matching unit is used to perform steady-state matching of a deep learning model according to the semantic information, and determine a unique deep learning model corresponding to the semantic information; wherein,

[0142] The steady-state matching includes: performing steady-state calculations on the semantic information and different deep learning models to determine a steady-state degree, and determining a unique deep recognition model according to the steady-state degree;

[0143] Steady-state matching means using a deep learning model to identify images. When determining the privacy level, only one level will be identified. Two different levels will not be identified for one image. This ensures that there is only one privacy level when identifying.

[0144] Importing unit: used for importing the semantic information into the unique deep learning model to determine the privacy level value;

[0145] Level determination unit: used to determine the corresponding privacy level according to the privacy level value.

[0146] When the present invention performs privacy identification, the main ability of the model is stability, because only when the model is stable can the model recognize deep information faster and more directly. In addition, when the present invention recognizes image data, different data will be assessed for risk levels through different deep learning models, so that the risk levels can be subdivided.

[0147] Embodiment 7:

[0148] As an embodiment of the present invention: the system further includes a decision module, the decision module including:

[0149] Discrete attribute determination unit: used to match the corresponding discrete model according to the privacy level of the image data and determine the discrete value; the discrete model is to reflect the overall discreteness of the image data. This discrete type mainly reflects the discreteness of the different contents of the image data. The image data includes many images, including different privacy levels, different contents, and different image types. The discrete model is mainly to reflect this discreteness.

[0150] Continuous attribute determination unit: used to match the corresponding continuous model and determine the continuous value according to the privacy level of the image data; the present invention matches the continuous model according to the privacy level. The continuous model is a model of image continuity within a privacy level. There is a certain correlation with the image association, but it is not a complete correlation. For example, the first image is a person facing a red light, and the second image is the person running a red light. This is continuity. It takes time as the reference axis, and the events that occur are things before and after the time reference axis.

[0151] Dataset generation unit: used to generate a corresponding semantic generalization data set according to the discrete value and the continuous value; semantic generalization data set; semantic generalization data set is based on the discrete value and the continuous value. After the discrete value and the continuous value are determined, each discrete value and the continuous value can determine similar or similar image data. These similar or similar image data can increase the semantic set of similar image data by determining their semantics, so that in the present invention, we construct a privacy decision tree, have more semantic samples when making a privacy budget, and make more detailed decisions when making privacy decisions.

[0152] Adaptive privacy budget allocation unit: performing privacy budgeting according to the generalized data set, constructing a privacy decision tree, and determining images that can be sent and images that cannot be sent in the image data;

[0153] A subdivision scheme selection unit: performing privacy decision subdivision according to the privacy decision tree and generating an image sending scheme;

[0154] Decision-making unit: according to the image sending plan, sends the image sending plan to the user terminal interface and obtains the selection instruction. The image sending plan is the instruction of which images can be sent and which images cannot be sent, and is also a kind of dialogue and strategy for image sending.

[0155] When establishing a decision, the present invention only has three decision-making methods: sending an image, not sending an image, and sending a portion of an image. However, if there are multiple images, it is necessary to determine the attributes and continuity of each image, and finally determine which images to send through a subdivision scheme to generate a corresponding sending decision.

[0156] Embodiment 8:

[0157] As an embodiment of the present invention: the instant issuance decision also includes: obtaining information of the image receiving end, determining whether the receiving end has privacy acquisition authorization, and executing an instant privacy decision when the image receiving end has privacy acquisition authorization; that is, only when the image receiving end has the user's privacy data authorization can it receive the image, otherwise it cannot receive the image with privacy data.

[0158] The prohibition decision also includes: recording the issuance instruction according to the risk level of the image data, and generating an information email with a prohibition decision, and the information email is sent to the user's mailbox; when a prohibition decision is made, there may be someone intruding and snooping on the privacy, so when the present invention has a prohibition decision, a record will be made through an information email, allowing the user to monitor whether the terminal device is invaded.

[0159] The partial issuance decision also includes: assessing the risk level of the images in the image data, determining the low-risk level images according to the assessment results, and issuing the low-risk level images, and storing the medium-risk level images and high-risk level images in the corresponding waiting-to-send space. The waiting-to-send space is a preparation space, and there is a delay in issuance. During this delay, the user can judge whether these medium-risk and high-risk images can be issued.

[0160] As an embodiment of the present invention, the system further includes:

[0161] Decision matrix unit: used to obtain all decisions issued by the image, and to establish a hesitant fuzzy decision matrix based on the attributes of each decision; the hesitant fuzzy decision matrix is ​​a detection matrix for judging the attributes of the decision and for possible inaccuracies in the decision. There may be multiple decision schemes when making fuzzy decisions, which are based on the attributes of the decision. This attribute is determined by the trust value. If the privacy level of the receiver and the image sent matches, the trust value is high. If the privacy level of the receiver and the image sent does not match, that is, the receiver does not have enough authority to receive private data, the trust value is low. The fuzzy decision matrix is ​​composed of the fusion of the fuzzy algorithm and the trust value of the decision scheme.

[0162] Decision comprehensive value calculation unit: used to determine the comprehensive decision value of each decision behavior according to the hesitant fuzzy decision matrix; the comprehensive decision value is determined by the fuzzy decision value of each decision scheme obtained by the fuzzy decision matrix and the number of images issued by the decision behavior and the privacy level. The comprehensive decision coefficients of images with different privacy levels are different. The comprehensive decision value is obtained by multiplying the privacy images of different levels by the coefficients, multiplying them by the number of images, and dividing them by the fuzzy decision value.

[0163] Sorting unit: used to perform average weighted calculation on the comprehensive decision values ​​to generate a decision order; average weighted calculation is an existing calculation method, which is mainly used to make a larger difference between different comprehensive decision values ​​so as to select the best decision plan.

[0164] Decision-making scheme determination unit: used to determine the optimal decision-making scheme according to the decision sequence.

[0165] When executing decisions, the present invention actually has decisions of issuing, not issuing, and partially issuing for any image, but the priority of each decision is different, so the present invention actually determines which images can be issued and which images cannot be issued in the form of priority, generates a decision order, and determines which plan has the highest priority through the decision order.

[0166] As an embodiment of the present invention: Figure 3 As shown, the system also includes:

[0167] Privacy label setting module: users submit pictures containing protected objects and corresponding semantic labels; semantic labels are labels of pictures of objects to be protected, which are used to describe the picture information of the protected objects.

[0168] Label image matching module: used to identify images and obtain the semantic content of the protected objects in the images;

[0169] The identified semantic content and the corresponding semantic tags are stored in a semantic sample library, and semantic information extraction models for different images are set based on the semantic sample library;

[0170] Rule setting module: the user selects a corresponding set of semantic samples from the semantic information extraction model and sets the privacy level of each semantic sample;

[0171] Privacy level matching module: The user stores a new picture in the rule setting module for image recognition, obtains the privacy level of the new picture, and performs semantic sample calibration in the rule setting module according to the privacy level of the new picture.

[0172] Because the privacy judgment in the present invention is based on the actual situation of the user, the user's terminal device may add new pictures. Of course, the data set for model training will also generate new data sets. Therefore, for new images, the present invention will re-calibrate the semantic samples according to the labels and set the privacy level for the new pictures.

[0173] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A privacy decision recommendation system based on image data and deep learning, characterized in that: include: Model library: used to pre-learn a large number of social privacy samples through deep learning technology and generate multiple deep learning models for different privacy level judgments; Semantic recognition module: used for converting the surface information of the image in the image data into semantic information when receiving the instruction to send the image data; Privacy identification module: used for importing the semantic information into the model library to determine the privacy level of the image data; Privacy identification module: used to send feedback image to the user to make a decision after the privacy level is determined; The image sending decision includes: immediate sending decision, prohibiting sending decision and partial sending decision; The system further comprises: Privacy determination module: used to obtain the setting standards of different privacy levels set by users based on the user's historical privacy setting information; The set standards include: picture content standards, social type standards and picture storage location standards; Classifier construction module: according to the set standard, establish an image classifier based on image information extraction; wherein, The picture information includes: an image surface information classifier, a social type classifier, and a picture storage classifier; The picture classifier includes: an information content classifier and a picture type classifier; Sample screening module: screening the user's historical privacy data according to the image classifier to determine the social privacy sample; Level classification module: used to perform privacy awareness calculation and privacy trust calculation on different social privacy samples, and classify privacy levels according to the calculation results to determine social privacy samples of different social privacy levels; The social privacy samples include: low-risk privacy samples, medium-risk privacy samples and high-risk privacy samples; The system also includes a decision module, which includes: Discrete attribute determination unit: used to match the corresponding discrete model according to the privacy level of the image data and determine the discrete value; A continuous attribute determination unit: used to match a corresponding continuous model according to the privacy level of the image data and determine a continuous value; A data set generating unit: used for generating a corresponding semantic generalization data set according to the discrete value and the continuous value; Adaptive privacy budget allocation unit: performing privacy budgeting according to the generalized data set, constructing a privacy decision tree, and determining images that can be sent and images that cannot be sent in the image data; A subdivision scheme selection unit: performing privacy decision subdivision according to the privacy decision tree and generating an image sending scheme; Decision-making unit: according to the image sending plan, sends the image sending plan to the user terminal interface and obtains the selection instruction.

2. A privacy decision recommendation system based on image data and deep learning as claimed in claim 1, characterized in that: The level classification module includes: Data extraction unit: used to determine the social program existing on the user terminal, establish an API crawling interface according to the social program, and obtain user data; wherein, The user data is image data generated during the social interaction; Data preprocessing unit: used to preprocess the user data and determine the semantic content of the user data; wherein, The preprocessing includes: image-text conversion processing, word segmentation processing, privacy labeling processing, TF-IDF processing and image named entity classification processing; Calculation unit: performing privacy awareness calculation on the semantic content to determine a privacy awareness value, and performing privacy trust calculation on the semantic content to determine a privacy trust value; A level division unit: a binary decision tree is established according to the privacy trust value and the privacy awareness value to determine the sensitivity values ​​of different image data, and privacy level division is performed according to the sensitivity values; wherein, The privacy levels include: low-risk privacy, medium-risk privacy, and high-risk privacy; The sensitivity value has a corresponding sensitivity value threshold according to the privacy level.

3. The privacy decision recommendation system based on image data and deep learning as claimed in claim 1, characterized in that: The model library includes: Deep learning model building unit: pre-set deep learning models based on different privacy levels; The deep learning models include: a low-risk privacy model, a medium-risk privacy model, and a high-risk privacy model; Deep learning training unit: used to import the social privacy samples into the deep learning model, perform data iterative calculation, and determine the optimal parameters of the model after iterative calculation; Model library unit: used to generate a deep learning model library for deep learning based on the optimal parameters of the model.

4. The privacy decision recommendation system based on image data and deep learning as claimed in claim 1, characterized in that: The semantic recognition module comprises: Instruction recognition unit: used to monitor the privacy of the social software on the user terminal and determine whether there is an image issuing instruction; Feedback unit: used to generate a feedback signal when there is an image issuing instruction; Annotation unit: used for determining the image address of the image to be sent when sensing the feedback signal, and semantically annotating the surface information of the image according to the image address; A word segmentation processing unit: used to segment the surface information of the image according to the semantic annotation to generate a vocabulary set; Semantic conversion unit: used for performing semantic conversion on each image according to the vocabulary set to generate semantic text of each image.

5. The privacy decision recommendation system based on image data and deep learning as claimed in claim 1, characterized in that: The privacy identification module includes: A matching unit is used to perform steady-state matching of a deep learning model according to the semantic information, and determine a unique deep learning model corresponding to the semantic information; wherein, The steady-state matching includes: performing steady-state calculations on the semantic information and different deep learning models to determine a steady-state degree, and determining a unique deep recognition model according to the steady-state degree; Importing unit: used for importing the semantic information into the unique deep learning model to determine the privacy level value; Level determination unit: used to determine the corresponding privacy level according to the privacy level value.

6. The privacy decision recommendation system based on image data and deep learning as claimed in claim 1, characterized in that: The instant issuance of the decision further includes: obtaining information of the image receiving end, determining whether the receiving end has privacy acquisition authorization, and executing the instant privacy decision when the image receiving end has privacy acquisition authorization; The prohibition decision further includes: recording the issuing instruction according to the risk level of the image data, and generating an information email with the prohibition decision, and sending the information email to the user mailbox; The partial issuance decision also includes: assessing the risk level of the images in the image data, determining the images with low risk levels based on the assessment results, and issuing the images with low risk levels, and storing the images with medium risk levels and high risk levels in the corresponding to-be-issued spaces.

7. The privacy decision recommendation system based on image data and deep learning as claimed in claim 1, characterized in that: The system further comprises: Decision matrix unit: used to obtain all decisions issued by the image and establish a hesitant fuzzy decision matrix based on the attributes of each decision; A decision comprehensive value calculation unit: used to determine the comprehensive decision value of each decision behavior according to the hesitant fuzzy decision matrix; Sorting unit: used for performing average weighted calculation on the comprehensive decision value to generate a decision sequence; Decision-making scheme determination unit: used to determine the optimal decision-making scheme according to the decision sequence.

8. The privacy decision recommendation system based on image data and deep learning as claimed in claim 1, characterized in that: The system further comprises: Privacy label setting module: users submit pictures containing protected objects and corresponding semantic labels; Label image matching module: used to identify images and obtain the semantic content of the protected objects in the images; The identified semantic content and the corresponding semantic tags are stored in a semantic sample library, and semantic information extraction models for different images are set based on the semantic sample library; Rule setting module: the user selects a corresponding set of semantic samples from the semantic information extraction model and sets the privacy level of each semantic sample; Privacy level matching module: The user stores a new picture in the rule setting module for image recognition, obtains the privacy level of the new picture, and performs semantic sample calibration in the rule setting module according to the privacy level of the new picture.

Images