Privacy protection method and device, electronic equipment and storage medium

Abstract

Embodiments of the present disclosure provide a privacy protection method and device, electronic equipment and storage medium, and relate to the technical field of privacy protection. The method comprises: obtaining shooting data corresponding to a current shooting behavior; obtaining context information based on the shooting data, the context information representing at least one of the willingness of a photographed person, the social relationship between the photographed person and the shooter, and the scene type of the shooting environment; determining whether there is a privacy invasion risk in the current shooting behavior according to the context information; and executing a first privacy protection strategy in the case that there is a privacy invasion risk in the current shooting behavior.

Description

Technical Field

[0001] This disclosure relates to the field of privacy protection technology, and more specifically, to a privacy protection method, privacy protection device, electronic device, and storage medium. Background Technology

[0002] Currently, shooting devices equipped with high-definition cameras are widely used in daily life and work. These devices can continuously capture environmental data, providing photographers with a convenient way to record and share. However, while enjoying the convenience, unauthorized shooting and data collection can also cause many inconveniences for photographers, preventing them from shooting without any concerns.

[0003] Therefore, how to determine whether a photographer's actions violate the privacy of others has become an urgent problem to be solved. Summary of the Invention

[0004] This disclosure provides a new technology solution for privacy protection methods, devices, electronic devices, and storage media.

[0005] According to a first aspect of the present disclosure, a privacy protection method is provided, comprising: Obtain the shooting data corresponding to the current shooting action; Based on the shooting data, contextual information is obtained, which represents at least one of the following: the subject's willingness, the social relationship between the subject and the photographer, and the scene type of the shooting environment. Based on the contextual information, determine whether the current filming activity poses a risk of privacy violation; Given the current risks of privacy violations associated with the filming process, the primary privacy protection strategy should be implemented.

[0006] Optionally, before acquiring the shooting data corresponding to the current shooting action, the method further includes: Obtain the preview image data acquired when the image acquisition module is started; Privacy element features are extracted from the preview image data to obtain privacy element feature data; Determine the privacy violation risk corresponding to the privacy element feature data; The acquisition of shooting data corresponding to the current shooting action includes: If the risk of privacy violation is less than a preset risk threshold, the shooting data corresponding to the current shooting behavior is obtained.

[0007] Optionally, the method further includes: If the privacy violation risk is greater than or equal to a preset risk threshold, a second privacy protection strategy will be implemented. The second privacy protection strategy includes sending a notification to the photographer that the current filming behavior violates their privacy.

[0008] Optionally, determining whether the current shooting behavior poses a privacy violation risk based on the context information includes: The context information is matched with preset privacy rules to obtain the matching result; Based on the matching results, it is determined whether the current shooting behavior poses a risk of privacy infringement.

[0009] Optionally, the implementation of a first privacy protection strategy when the current filming activity poses a risk of privacy infringement includes: If the current shooting behavior poses a risk of privacy violation, the level of privacy violation risk will be determined based on the matching results; Implement the first privacy protection strategy corresponding to the privacy violation risk level.

[0010] Optionally, executing the first privacy protection strategy corresponding to the privacy infringement risk level includes: If the privacy violation risk level is Level 1, the photographer is prompted that the current filming activity violates privacy, or provided with at least one of the following interfaces: continue filming and cancel filming; and / or, When the privacy violation risk level is the second risk level, a prompt is output to the photographer indicating that the current shooting behavior violates privacy, and the image area corresponding to the privacy element in the shooting data is blurred. The risk level of the first risk level is less than that of the second risk level.

[0011] Optionally, the captured data includes at least two of image data, audio data, and context data, and obtaining context information based on the captured data includes: Based on the image data, image features are determined. The image features include at least one of the following: the facial expression features of the subject in the current shooting frame, the orientation features of the subject, the body posture adjustment features of the subject in cooperation with the shooting, the distance features between the photographer and the subject, and the feature of multiple people in the current shooting frame. Based on the audio data, acoustic features are determined, including at least one of the following: speech features representing the subject's willingness to be filmed, and human voice acoustic features that conform to the frequency band of laughter and the waveform characteristics of human voice. Based on the context data, context features are determined, which include at least one of scene features and shooting mode features associated with the current shooting behavior; The context information is determined based on at least two of the image features, the acoustic features, and the context features.

[0012] According to a second aspect of the present disclosure, a privacy protection device is provided, comprising: The first acquisition module is used to acquire the shooting data corresponding to the current shooting action; The second acquisition module is used to obtain contextual information based on the shooting data. The contextual information represents at least one of the following: the subject's willingness, the social relationship between the subject and the photographer, and the scene type of the shooting environment. The determination module is used to determine, based on the context information, whether the current shooting behavior poses a risk of privacy infringement. The execution module is used to implement the first privacy protection strategy when there is a risk of privacy violation in the current shooting behavior.

[0013] According to a third aspect of the present disclosure, an electronic device is provided, including a privacy protection device as described in a second aspect of the present disclosure, or... The electronic device includes a memory and a processor, the memory being used to store computer instructions, and the processor being used to retrieve the computer instructions from the memory to perform the method as described in the first aspect of the present disclosure.

[0014] According to a fourth aspect of the present disclosure, a storage medium is provided that stores computer program instructions thereon, which, when executed by a processor, implement the method described in the first aspect of the present disclosure.

[0015] This disclosure provides an embodiment of a proactive, real-time privacy protection scheme. It acquires shooting data from the shooting terminal during the shooting process, obtains contextual information based on this data, and determines whether the current shooting behavior poses a privacy violation risk. If a privacy violation risk exists, a first privacy protection strategy is executed. This disclosure provides a proactive, real-time privacy protection scheme that prevents the photographer from unintentionally violating the privacy of the subject during shooting, allowing the photographer to shoot without worry, while also protecting the privacy of the subject. Furthermore, establishing a privacy protection mechanism at the data collection source avoids the inherent defects of traditional post-processing solutions.

[0016] Other features and advantages of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments of the present disclosure and, together with their description, serve to explain the principles of the present disclosure.

[0018] Figure 1 This is a flowchart illustrating a privacy protection method provided in an embodiment of this disclosure.

[0019] Figure 2 This is a schematic diagram of the structure of a privacy protection device provided in an embodiment of this disclosure.

[0020] Figure 3 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this disclosure. Detailed Implementation

[0021] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of the present disclosure.

[0022] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit this disclosure or its application or use.

[0023] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0024] In all the examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0025] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0026] This disclosure relates to a privacy protection method, apparatus, system, electronic device, and storage medium. First, a brief description is given of a shooting terminal system that can apply the technical solution provided in this disclosure. This system may include: an image acquisition module, a processor, a memory, an alert module, and a display module.

[0027] The image acquisition module can be, for example, a camera. The image acquisition module can acquire preview image data before shooting (i.e., during the preview stage) or during shooting. The processor can be a Central Processing Unit (CPU), a Digital Signal Processor (DSP), or a Field-Programmable Gate Array (FPGA) for executing the privacy protection method described in this disclosure. The memory can include Random Access Memory (RAM) and Read-Only Memory (ROM) for storing computer instructions and data. The alert module can be a vibration motor or a speaker for alerting the user according to the processor's instructions. The display module can be a liquid crystal display (LCD) or an organic light-emitting diode (OLED) display for displaying the captured image and intervention prompts.

[0028] In related technologies, some privacy protection solutions focus on protecting the privacy of the photographer using the photographer's own device, while others focus on detecting being photographed using the subject's device. However, there is a lack of solutions that analyze whether the photographer's actions infringe on the privacy of others at the photographer's own device level. The act of photographing and collecting data carries the risk of infringing on the privacy of others, causing considerable inconvenience to the photographer and preventing them from photographing without any concerns.

[0029] Based on this, the present disclosure provides a privacy protection method. This privacy protection method is applied to the photographer's terminal device. Figure 1 This is a schematic flowchart of a privacy protection method provided in an embodiment of this disclosure. Figure 1 As shown, the method may include steps S110 to S140.

[0030] Step S110: Obtain the shooting data corresponding to the current shooting action.

[0031] The shooting data in the embodiments of this disclosure is shooting data acquired in real time during the shooting process.

[0032] In some examples, the captured data may include at least one of image data, audio data, and contextual data. Image data may be a sequence of image frames captured by the capturing terminal during the capturing process. Audio data may be audio signals captured by the capturing terminal during the capturing process. Contextual data may include the location data and device attitude data of the capturing terminal. Location data may be obtained through the Global Positioning System (GPS) and characterizes the geographical location of the capturing terminal. Device attitude data may be obtained through a gyroscope and accelerometer and characterizes the tilt angle and motion state of the capturing terminal.

[0033] Step S120: Obtain contextual information based on the captured data.

[0034] The contextual information in the embodiments of this disclosure may characterize at least one of the following: the subject's willingness, the social relationship between the subject and the photographer, and the scene type of the shooting environment.

[0035] In some examples, the subject's willingness can be either proactive or passive. The social relationship between the subject and the photographer can include no social relationship (i.e., strangers), a general social relationship (e.g., acquaintances), a friendship, or a close relationship (e.g., family). The type of shooting environment can include public places, private places, commercial areas, and sensitive areas (e.g., schools, hospitals, etc.).

[0036] Step S130: Based on the context information, determine whether the current shooting behavior poses a risk of privacy infringement.

[0037] For example, when the subject's willingness is passive, or there is no social interaction or minimal social interaction between the subject and the photographer, or the shooting scene is the subject's private scene, it indicates that the current shooting behavior poses a risk of privacy infringement on the subject.

[0038] In some examples, step S130 may include: matching contextual information with preset privacy rules to obtain matching results; and determining whether the current shooting behavior poses a risk of privacy infringement based on the matching results.

[0039] Preset privacy rules can be privacy violation judgment rules based on laws, regulations and social norms.

[0040] For example, a privacy rule engine can be preset, with a preset set of privacy rules built into it. Contextual information can be input into the privacy rule engine to obtain matching results.

[0041] For example, the preset privacy rules may include: if the subject's facial expressions indicate a negative attitude and the scene is the subject's private place, then the current filming behavior is determined to have a privacy violation risk; if the subject's facial expressions indicate a positive attitude and voice features indicating consent to being filmed are identified, then the current filming behavior is determined not to have a privacy violation risk.

[0042] For example, preset privacy rules may include: if a person's private parts are photographed without their explicit consent, the current photographing behavior is deemed to have a risk of privacy violation; or, if the person being photographed is a public figure and the scene being photographed is a public place, the current photographing behavior is deemed not to have a risk of privacy violation.

[0043] Step S140: If the current shooting behavior poses a risk of privacy infringement, implement the first privacy protection strategy.

[0044] In some examples, the primary privacy protection strategy may include differentiated interventions based on the level of privacy violation risk. This approach can balance privacy protection with minimizing interference with the photographer's normal shooting activities, thus achieving a balance between privacy protection and shooting convenience.

[0045] In some examples, the preset privacy rules are configured with a privacy violation risk level for each rule. The privacy violation risk level of the current shooting behavior can be determined by matching contextual information with the preset privacy rules.

[0046] In some examples, the privacy violation risk level can include a first risk level, a second risk level, and a third risk level. The first risk level is less risky than the second risk level, and the third risk level is less risky than the first risk level. For example, the third risk level is low risk, the first risk level is medium risk, and the second risk level is high risk.

[0047] For example, in a scenario where the privacy violation risk level is Level 1, the photographer is provided with both a continue shooting interface and a cancel shooting interface. The continue shooting interface could be a button displayed on the display module, which the photographer can click to continue shooting. The cancel shooting interface could be another button displayed on the display module, which the photographer can click to cancel the current shooting action. By providing both continue and cancel shooting interfaces, the photographer is given the final decision-making power while ensuring that the decision is made with the photographer's full knowledge.

[0048] For example, in cases where the privacy violation risk level is the highest, a prompt can be sent to the photographer, such as a slight vibration or visual cue. The visual cue can briefly explain the basis for the judgment, such as "A pedestrian was detected involuntarily entering the frame from behind."

[0049] For example, in cases where the privacy violation risk level is Level 2, the image regions corresponding to privacy elements in the captured data are blurred. Privacy elements may include, for example, faces, license plates, and text. Blurring can be achieved, for example, using a Gaussian blur algorithm or a mosaic algorithm. By blurring the image regions corresponding to privacy elements, the privacy of the subject can be protected while preserving the overall content of the captured image.

[0050] For example, in cases where the privacy violation risk level is the second level, a prompt such as a strong vibration can also be sent to the photographer.

[0051] For example, in cases where the privacy violation risk level is Level 2, an interface for deleting data can also be provided to the photographer. If the user requests recovery, additional steps can still be taken to confirm the recovery of the original image.

[0052] For example, if the privacy infringement risk level is level three, no privacy infringement risk warning will be output.

[0053] This embodiment of the disclosure acquires shooting data corresponding to the current shooting behavior collected in real time by the shooting terminal during the shooting process. Based on this shooting data, contextual information is obtained, and a first privacy protection strategy is executed according to the contextual information. This can not only prevent the photographer from unintentionally infringing on the privacy of the subject during the shooting process, allowing the photographer to shoot without any worries, but also protect the privacy of the subject. In addition, establishing a privacy protection mechanism at the source of data collection avoids the inherent defects of traditional post-processing solutions.

[0054] In some embodiments of this disclosure, before step S110, the privacy protection method of this disclosure may further include: acquiring preview image data acquired when the shooting terminal starts the image acquisition module; extracting privacy element features from the preview image data to obtain privacy element feature data; and determining the privacy infringement risk corresponding to the privacy element feature data.

[0055] In some examples, privacy-preserving features can be extracted using lightweight edge-side neural network models. For instance, these models could be MobileNet or ShuffleNet. Privacy-preserving feature data is a collection of privacy-preserving features. This data can include facial features, license plate features, text features, etc. By extracting and analyzing these privacy-preserving features, rather than storing the raw images, the risk of privacy breaches can be reduced.

[0056] In some examples, the risk of privacy violation can be calculated using a risk assessment model. This model can calculate a risk score based on the number and type of privacy elements in the privacy element feature data. For instance, if multiple faces are detected in the preview image data and the scene type is a private location, the risk score is high; if no privacy elements are detected in the preview image data or the scene type is a public location, the risk score is low.

[0057] In this embodiment of the disclosure, the preview image data is a real-time image frame sequence displayed on the display module from the time the photographer starts the image acquisition module of the shooting terminal until the photographer presses the shooting button.

[0058] In this embodiment, a privacy violation risk threshold can be preset. If the privacy violation risk is less than the preset threshold, the shooting data corresponding to the current shooting action is acquired for subsequent contextual analysis.

[0059] In some embodiments of this disclosure, a second privacy protection strategy is implemented when the risk of privacy violation is greater than or equal to a preset privacy violation risk threshold. For example, the second privacy protection strategy may include outputting a warning to the photographer indicating a privacy violation risk associated with the current shooting activity. This warning may be issued via a subtle vibration from a reminder module or by displaying text information via a display module. By reminding the photographer before shooting, the occurrence of privacy violations can be effectively reduced.

[0060] In some examples, a second privacy protection strategy is implemented when the risk of privacy violation is greater than or equal to a first preset privacy violation risk threshold but less than a second preset privacy violation risk threshold.

[0061] In other examples, a third privacy protection strategy is implemented when the risk of privacy violation is greater than or equal to a second preset privacy violation risk threshold. For instance, the third privacy protection strategy may include disabling the camera function, which can adapt to higher-risk scenarios and directly block the source of privacy leakage.

[0062] In some embodiments of this disclosure, the privacy protection method may further include a photographer feedback learning mechanism. Specifically, it acquires the photographer's feedback and decision results regarding the first privacy protection strategy. The feedback may include the photographer's operation record of choosing to continue shooting or cancel shooting. Based on the feedback and decision results, the parameters for acquiring subsequent contextual information and the execution threshold of the privacy protection strategy are adjusted. For example, if the photographer repeatedly chooses to continue shooting in a specific scene type, the system can learn the importance of that scene type to the photographer, reduce the risk assessment score for that scene type, and thus reduce interference with the photographer. By continuously learning the photographer's shooting habits and scene characteristics, the accuracy of privacy judgment can be continuously optimized.

[0063] In some embodiments of this disclosure, step S120 may include: determining image features based on image data.

[0064] The image features in this embodiment may include at least one of the following: the facial expression features of the subject in the current shooting frame, the orientation features of the subject, the body posture adjustment features of the subject in cooperation with the shooting, the distance features between the photographer and the subject, and the feature of multiple people in the current shooting frame.

[0065] For example, acquiring image features based on image data can include: identifying the facial expressions of the subject within the current frame using computer vision algorithms. Facial expressions can include types such as smiling, frowning, and surprise. By analyzing these facial expressions, the subject's attitude towards being photographed can be inferred. For instance, a smiling expression usually indicates a positive attitude towards being photographed, while a frowning or evasive expression usually indicates a negative attitude.

[0066] For example, obtaining image features based on image data may also include identifying the subject's orientation characteristics. Orientation characteristics can be determined using face detection algorithms to determine the subject's facial orientation angle. If the subject's face is facing the photographer, it usually indicates that the subject is aware of the act of being photographed; if the subject's face is turned away from the photographer, it usually indicates that the subject is unaware of the act of being photographed.

[0067] For example, acquiring image features based on image data may also include: identifying the subject's body posture adjustments in cooperation with the photo shoot. Body posture adjustment features may include behaviors such as the subject posing for the photo and adjusting their standing position. By analyzing body posture adjustment features, it can be inferred whether the subject actively cooperated with the photo shoot.

[0068] For example, obtaining image features based on image data can also include identifying distance features between the photographer and the subject. Distance features can be calculated using depth estimation algorithms to determine the physical distance between the photographer and the subject. A closer distance generally indicates a stronger social relationship between the photographer and the subject, while a greater distance generally indicates a weaker social relationship.

[0069] For example, obtaining image features based on image data can also include: identifying multiple people in the current shooting frame. Multiple people in the frame can be identified by counting the number of faces in the current shooting frame using a face detection algorithm. If multiple people are in the frame simultaneously, it usually indicates that the current shooting activity may be a group photo scene.

[0070] Step S120 may also include: determining acoustic features based on audio data.

[0071] The acoustic features in this example may include at least one of the following: speech features that characterize the subject's willingness to be photographed, and human voice acoustic features that conform to the frequency band of laughter and the waveform characteristics of human voice.

[0072] For example, determining acoustic features based on audio data can include: identifying speech features that characterize the subject's willingness to be filmed using a speech recognition algorithm. Speech features can include keywords spoken by the subject, such as speech indicating consent to being filmed. By recognizing these keywords, it can be determined whether the subject has explicitly consented to being filmed.

[0073] For example, determining acoustic features based on audio data may also include: identifying human voice acoustic features that match the frequency band of laughter and the waveform characteristics of human voices. The frequency band of laughter can be preset. The waveform characteristics of human voices can be extracted from the time and frequency domain features of the audio signal. By recognizing laughter, it can be inferred that the subject tends to have a more positive attitude towards the filming.

[0074] Step S120 may also include: determining contextual features based on contextual data.

[0075] The contextual features in this example may include at least one of the scene features and shooting mode features associated with the current shooting action.

[0076] For example, determining contextual features based on contextual data can include: identifying scene features associated with the current shooting action based on location data. Scene features can include scene types such as public or private locations. Public locations can include parks, squares, etc., while private locations can include residences, offices, private clubs, etc. By analyzing the scene type, it can be determined whether the current shooting action infringes on the privacy of the person being photographed.

[0077] Scene types can also include commercial areas or sensitive areas. Commercial areas can include shopping malls, supermarkets, restaurants, etc., while sensitive areas can include schools, hospitals, etc. Filming in sensitive areas usually requires higher privacy protection requirements.

[0078] For example, determining contextual features based on contextual data may also include: determining the shooting mode features of the shooting terminal based on device posture data. Shooting mode features may include selfie mode and other-shot mode. If the camera of the shooting terminal is facing the photographer, it usually indicates that the current mode is selfie mode; if the camera of the shooting terminal is facing the external environment, it usually indicates that the current mode is other-shot mode. Selfie mode usually indicates that the current shooting behavior is a group photo scenario, and the subject's willingness to be photographed is likely proactive.

[0079] Step S120 may further include: determining contextual information based on at least two of the image features, acoustic features, and contextual features.

[0080] The embodiments disclosed herein obtain contextual information through a multimodal fusion method, which can improve the accuracy of contextual information and thereby reduce misjudgments of privacy infringement risks.

[0081] This disclosure also provides a privacy protection device. Figure 2 This is a schematic diagram of the structure of a privacy protection device provided in an embodiment of this disclosure. Figure 2 As shown, the privacy protection device 200 may include a first acquisition module 210, a second acquisition module 220, a determination module 230, and an execution module 240.

[0082] The first acquisition module 210 can be used to acquire shooting data corresponding to the current shooting behavior collected in real time by the shooting terminal.

[0083] The second acquisition module 220 can be used to obtain contextual information based on the shooting data. This contextual information represents at least one of the following: the subject's willingness, the social relationship between the subject and the photographer, and the scene type of the shooting environment.

[0084] The determination module 230 can be used to determine whether the current shooting behavior poses a risk of privacy infringement based on contextual information.

[0085] The execution module 240 can be used to execute a first privacy protection strategy when there is a risk of privacy violation in the current shooting behavior.

[0086] In some examples, the first acquisition module 210 may include a preview image acquisition submodule, a feature extraction submodule, and a risk determination submodule. The preview image acquisition submodule can be used to acquire preview image data collected when the shooting terminal starts the image acquisition module. The feature extraction submodule can be used to extract privacy element features from the preview image data to obtain privacy element feature data. The risk determination submodule can be used to determine the privacy infringement risk corresponding to the privacy element feature data, and acquire the shooting data corresponding to the current shooting behavior if the privacy infringement risk is less than a preset risk threshold.

[0087] In some examples, the second acquisition module 220 may include an image feature determination submodule, an acoustic feature determination submodule, a context feature determination submodule, and a situational information determination submodule. The image feature determination submodule can be used to determine image features based on image data. Image features may include at least one of the following: facial expression features of the subject in the current shooting frame, the subject's orientation features, the subject's body posture adjustment features in cooperation with the shooting, the distance features between the photographer and the subject, and features of multiple people appearing in the current shooting frame. The acoustic feature determination submodule can be used to determine acoustic features based on audio data. Acoustic features may include at least one of the following: speech features representing the subject's willingness to be photographed, and human voice acoustic features conforming to the frequency band of laughter and the waveform features of human voices. The context feature determination submodule can be used to determine context features based on context data. Context features may include at least one of the following: scene features associated with the current shooting behavior and shooting mode features of the shooting terminal. The situational information determination submodule can be used to determine situational information based on at least two of the image features, acoustic features, and context features.

[0088] In some examples, the determination module 230 may include a matching submodule and a risk assessment submodule. The matching submodule can be used to match contextual information with preset privacy rules to obtain a matching result. The risk assessment submodule can be used to determine whether the current shooting behavior poses a privacy violation risk based on the matching result.

[0089] In some examples, execution module 240 may include a risk level determination submodule and a policy execution submodule. The risk level determination submodule can be used to determine the privacy violation risk level based on the matching results if there is a privacy violation risk in the current shooting behavior. The policy execution submodule can be used to execute a first privacy protection policy corresponding to the privacy violation risk level.

[0090] The privacy protection device provided in this disclosure can execute the privacy protection method provided in the foregoing method embodiments. It has the corresponding functional modules for executing the method and the beneficial effects can be referred to in the foregoing method embodiments.

[0091] This disclosure also provides an electronic device. This electronic device may include the privacy protection device described in the foregoing embodiments. Alternatively, as... Figure 3 As shown, the electronic device 300 includes a memory 310 and a processor 320. The memory 310 is used to store computer instructions, and the processor 320 is used to retrieve the computer instructions from the memory 310 to execute the privacy protection method described in the embodiments of this disclosure.

[0092] In some examples, the electronic device can be a smart wearable device such as smart glasses, smart bracelet, or smartwatch.

[0093] In other examples, the electronic device can be a mobile terminal such as a smartphone.

[0094] In some examples, processor 320 may be a central processing unit, microprocessor, digital signal processor, or other chip with data processing capabilities. Memory 310 may be a storage medium such as random access memory, read-only memory, or flash memory.

[0095] This disclosure also provides a storage medium storing computer program instructions, which, when executed by a processor, can implement the aforementioned privacy protection method. In some examples, the storage medium may be a non-volatile computer-readable storage medium. This non-volatile computer-readable storage medium may be a read-only memory, flash memory, hard disk, optical disk, or other storage media.

[0096] This disclosure can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of this disclosure.

[0097] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination of the foregoing. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0098] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0099] The computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source or object programs written in any combination of one or more programming languages, including object-oriented programming languages ​​(such as Smalltalk, C++, etc.) and conventional procedural programming languages ​​(such as the "C" language or similar programming languages). The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network (e.g., a local area network or a wide area network), or it may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays, or programmable logic arrays, can execute computer-readable program instructions to implement various aspects of the embodiments of this disclosure by utilizing state information from the computer-readable program instructions.

[0100] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0101] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0102] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0103] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions. It will be known to those skilled in the art that implementation in hardware, implementation in software, and implementation in a combination of software and hardware are equivalent.

[0104] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, and are not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein. The scope of this disclosure is defined by the appended claims.

Claims

1. A privacy protection method, characterized in that, include: Obtain the shooting data corresponding to the current shooting action; Based on the shooting data, contextual information is obtained, which represents at least one of the following: the subject's willingness, the social relationship between the subject and the photographer, and the scene type of the shooting environment. Based on the contextual information, determine whether the current filming activity poses a risk of privacy violation; Given the current risks of privacy violations associated with the filming process, the primary privacy protection strategy should be implemented.

2. The method according to claim 1, characterized in that, Before acquiring the shooting data corresponding to the current shooting action, the method further includes: Obtain the preview image data acquired when the image acquisition module is started; Privacy element features are extracted from the preview image data to obtain privacy element feature data; Determine the privacy violation risk corresponding to the privacy element feature data; The acquisition of shooting data corresponding to the current shooting action includes: If the risk of privacy violation is less than a preset risk threshold, the shooting data corresponding to the current shooting behavior is obtained.

3. The method according to claim 2, characterized in that, The method further includes: If the privacy violation risk is greater than or equal to a preset risk threshold, a second privacy protection strategy will be implemented. The second privacy protection strategy includes sending a notification to the photographer that the current filming behavior violates their privacy.

4. The method according to claim 1, characterized in that, The step of determining whether the current shooting behavior poses a privacy violation risk based on the context information includes: The context information is matched with preset privacy rules to obtain the matching result; Based on the matching results, it is determined whether the current shooting behavior poses a risk of privacy infringement.

5. The method according to claim 4, characterized in that, In cases where the current filming activity poses a risk of privacy infringement, the first privacy protection strategy is implemented, including: If the current shooting behavior poses a risk of privacy violation, the level of privacy violation risk will be determined based on the matching results; Implement the first privacy protection strategy corresponding to the privacy violation risk level.

6. The method according to claim 5, characterized in that, The execution of the first privacy protection strategy corresponding to the privacy infringement risk level includes: If the privacy violation risk level is Level 1, the photographer is prompted that the current filming activity violates privacy, or provided with at least one of the following interfaces: continue filming and cancel filming; and / or, When the privacy violation risk level is the second risk level, a prompt is output to the photographer indicating that the current shooting behavior violates privacy, and the image area corresponding to the privacy element in the shooting data is blurred. The risk level of the first risk level is less than that of the second risk level.

7. The method according to any one of claims 1 to 6, characterized in that, The captured data includes at least two of the following: image data, audio data, and context data. The process of obtaining context information based on the captured data includes: Based on the image data, image features are determined. The image features include at least one of the following: the facial expression features of the subject in the current shooting frame, the orientation features of the subject, the body posture adjustment features of the subject in cooperation with the shooting, the distance features between the photographer and the subject, and the feature of multiple people in the current shooting frame. Based on the audio data, acoustic features are determined, including at least one of the following: speech features representing the subject's willingness to be filmed, and human voice acoustic features that conform to the frequency band of laughter and the waveform characteristics of human voice. Based on the context data, context features are determined, which include at least one of scene features and shooting mode features associated with the current shooting behavior; The context information is determined based on at least two of the image features, the acoustic features, and the context features.

8. A privacy protection device, characterized in that, include: The first acquisition module is used to acquire the shooting data corresponding to the current shooting action; The second acquisition module is used to obtain contextual information based on the shooting data. The contextual information represents at least one of the following: the subject's willingness, the social relationship between the subject and the photographer, and the scene type of the shooting environment. The determination module is used to determine, based on the context information, whether the current shooting behavior poses a risk of privacy infringement. The execution module is used to implement the first privacy protection strategy when there is a risk of privacy violation in the current shooting behavior.

9. An electronic device, characterized in that, The electronic device includes the privacy protection device as described in claim 8; or... The electronic device includes a memory and a processor, the memory being used to store computer instructions, and the processor being used to retrieve the computer instructions from the memory to perform the method as described in any one of claims 1 to 7.

10. A storage medium storing computer program instructions thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method of any one of claims 1 to 7.

Images