Method for protecting portrait privacy of tourist based on face processing

Abstract

The invention discloses a method for protecting portrait privacy of a tourist based on face processing. The method comprises the following steps: S1, performing face detection processing on each frame of an original video played by the tourist; s2, determining positions and sizes of five sense organs of the tourist according to a face detection result; s3, matching a proper mask according to the positions and sizes of the five sense organs of the tourist; s4, covering the face of each frame of tourist face in the original video with a mask; s5, associating the mask-wearing video with the original video through a label, and storing the mask-wearing video and the original video; s6, playing the video with the mask when the tourist previews the video, and protecting the privacy of the tourist; and S7, when the tourist selects the original video, providing the original video for the tourist to download. According to the invention, the personal privacy information of the tourist is not acquired or spread by an unauthorized third party; the requirements of tourists for viewing or purchasing personal images can be met while privacy is protected, and the overall experience of the tourists is improved; the system is suitable for various public places, especially tourist-intensive places such as scenic spots, playgrounds, museums and the like.

Description

Technical Field

[0001] The present invention relates to the field of portrait privacy, and particularly to a method for protecting the portrait privacy of tourists based on face processing. Background Art

[0002] Currently, images taken by tourists in public places such as scenic spots, amusement parks, museums, etc. are often displayed or used for commercial purposes. Especially in public display areas, other tourists can easily obtain these images through shooting, screenshotting, or downloading without purchase or unauthorized authorization, resulting in the facial features and personal images of tourists being obtained or misused by unauthorized others, and there is a risk of portrait information leakage.

[0003] The privacy protection requirements of different tourists may vary. Some tourists may hope to keep their portraits confidential throughout the process, while some tourists may only hope to protect their privacy during public display. Existing technologies are difficult to provide flexible and customizable privacy protection solutions and cannot meet the needs of different tourists. Summary of the Invention

[0004] The purpose of the present invention is to propose a method for protecting the portrait privacy of tourists based on face processing for the above technical problems.

[0005] A method for protecting the portrait privacy of tourists based on face processing includes the following sub-steps: S1: Perform face detection processing on the original video of tourists' play; S2: Determine the positions and / or sizes of the facial features of tourists from the face detection results; S3: Match the masks at the corresponding positions through the positions and / or sizes of the facial features of tourists; S4: Cover the masks on the faces of tourists in the original video frames; S5: Establish an association relationship between the masked video and the original video and store them in the cloud respectively; S6: Play the masked video during video preview in public spaces to protect the privacy of tourists; S7: When tourists need to download the original video, find the original video through the association relationship and provide the original video for tourists to download.

[0006] Further, a method for protecting the portrait privacy of tourists based on face processing, the step S1 includes the following sub-steps: S11: Upload the original video of tourists' play to the data processing center through a camera; S12: The data processing center captures video frames through the video file; S13: Perform face detection on each frame through the face detection algorithm of the convolutional neural network to determine whether there is a face in the image.

[0007] Furthermore, a method for protecting the privacy of tourist portraits based on face processing, the step S2 includes the following sub-steps: S21: Use a deep learning model to predict a series of key points in the face area; The key points include ears, eyebrows, eyes, nose and mouth; S22: Calculate the specific position coordinates of the key points.

[0008] Furthermore, a method for protecting the privacy of tourist portraits based on face processing, the step S3 includes the following sub-steps: S31: Determine the size and position of the mask through the key point coordinates and generate a tourist mask; S32: Process each frame of the video through real-time video editing technology, put the mask on the tourist's face through face tracking technology, and generate a video of the tourist wearing the mask.

[0009] Furthermore, a method for protecting the privacy of tourist portraits based on face processing, step S5 includes the following sub-steps: S51: Establishing an association relationship between the original video and the mask-wearing video.

[0010] Further, a method for protecting the privacy of tourist portraits based on face processing, step S6 includes the following sub-steps: In the public display area, the preview image displayed to visitors before purchasing the image is the mask-wearing video; In the public display area, when other visitors view the images, the preview images displayed cannot obtain the specific facial features of others.

[0011] Further, a method for protecting the privacy of tourist portraits based on face processing, said S7 comprises the following sub-steps: S71: Visitors select images of their interest in the image display area; S72: After the tourist meets the downloading conditions, the system automatically matches the tourist's image and removes the privacy protection processing in the image; S73: The visitor obtains his original video through the terminal device.

[0012] Beneficial effects of the present invention: (1) Privacy protection: By effectively controlling images during the display stage, we ensure that visitors’ personal privacy information is not obtained or disseminated by unauthorized third parties; (2) Visitor experience: While protecting privacy, it can meet the needs of visitors to view or purchase personal images, thus improving the overall experience of visitors; (3) Fully automated: fully automated shooting and automatic image protection processing; (4)Wide applicability: Applicable to various public places, especially places with dense tourists such as scenic spots, amusement parks, museums, etc., which can effectively solve the contradiction between tourist privacy protection and image preview display. Brief Description of the Drawings

[0013] Figure 1 is a flowchart of a method for protecting tourist portrait privacy based on face processing. Detailed Implementation Manner

[0014] For a clearer understanding of the technical features, objectives, and effects of the present invention, the specific implementation manner of the present invention will now be described with reference to the accompanying drawings.

[0015] As shown in the attached Figure 1 figures, a method for protecting tourist portrait privacy based on face processing includes the following sub-steps: S1: Perform face detection processing on the original video of tourists' play; S2: Determine the positions and / or sizes of the facial features of tourists from the face detection results; S3: Match the masks at the corresponding positions through the positions and / or sizes of the facial features of tourists; S4: Cover the masks on the faces of tourists in the original video frames; S5: Establish an association relationship between the masked video and the original video, and perform cloud storage respectively; S6: Play the masked video during video preview in public spaces to protect tourist privacy; S7: When tourists need to download the original video, find the original video through the association relationship and provide the original video for tourists to download.

[0016] Further, a method for protecting tourist portrait privacy based on face processing, the step S1 includes the following sub-steps: S11: Upload the original video of tourists' play to the data processing center through a camera; S12: The data processing center captures video frames through the video file; S13: Perform face detection on each frame through the face detection algorithm of the convolutional neural network to determine whether there is a face in the image.

[0017] Further, a method for protecting tourist portrait privacy based on face processing, the step S2 includes the following sub-steps: S21: Use a deep learning model to predict a series of key points within the face area; The key points include ears, eyebrows, eyes, nose, and mouth; S22: Calculate the specific position coordinates of the key points.

[0018] Further, a method for protecting the portrait privacy of tourists based on face processing, the step S3 includes the following sub-steps: S31: Determine the size and position of the mask through key point coordinates to generate a tourist mask; S32: Process each frame of the video through real-time video editing technology, and wear the mask on the tourist's face through face tracking technology to generate a video of the tourist wearing a mask.

[0019] Further, a method for protecting the portrait privacy of tourists based on face processing, the step S5 includes the following sub-steps: S51: Establish an association relationship between the original video and the video with a mask.

[0020] Further, a method for protecting the portrait privacy of tourists based on face processing, the step S6 includes the following sub-steps: In the public display area, before the tourist purchases the image, the preview image shown is the video with a mask; In the public display area, when other tourists watch the image, the preview image shown cannot obtain the specific facial features of others.

[0021] Further, a method for protecting the portrait privacy of tourists based on face processing, the S7 includes the following sub-steps: S71: The tourist selects the image he is interested in in the image display area; S72: After the tourist meets the download conditions, the system automatically matches the tourist's image and removes the privacy protection process in the image; S73: The tourist obtains his original video through the terminal device.

[0022] Specific Embodiment 1 Privacy Protection Control before Shooting Specific Embodiment 1 Privacy Protection for Image Display after Shooting I. Function Overview To protect the privacy and security of tourists in the image preview session, this system adopts dynamic face detection and 3D model overlay technology to perform real-time occlusion processing on the face area in the image that has not been purchased. Through the overlay of elements such as virtual masks, cartoon glasses, and facial special effects, it ensures that the tourist's facial information cannot be directly recognized by the outside world, while retaining the interest and preview value of the image.

[0023] II. Technical Implementation Process 1. Face Detection and Location Algorithm Framework: Based on the YOLO (You Only Look Once) real-time object detection algorithm, detect the face position and bounding box in the image frame by frame.

[0024] Precision Optimization: Adopt a lightweight network structure to support robust detection under multi-angle and occlusion conditions, with a single-frame processing time ≤ 50 ms (1080p resolution).

[0025] 2. Facial Feature Analysis Feature Point Extraction: Extract 5 key feature points, including ears, eyebrows, eyes, nose, and mouth, from the cropped face region through a pre-trained neural network model, such as ResNet-50 or MobileNetV3.

[0026] 3D Pose Estimation: Combine the Perspective-n-Point algorithm to calculate the scaling ratio, rotation angle, and translation parameters of the face in 3D space, and construct the basis for 3D model adaptation.

[0027] 3. Virtual Element Overlay Rendering Model Adaptation: Dynamically adjust prefabricated 3D virtual elements, such as masks, glasses, decorations, etc., according to parameters to ensure complete matching with the size, angle, and perspective relationship of the original face.

[0028] Real-time Rendering: Based on the OpenGL or WebGL rendering engine, overlay virtual elements onto the original video frame with a transparent channel, support light and shadow fusion and edge anti-aliasing processing, and achieve a realistic occlusion effect.

[0029] III. Examples of Privacy Processing Effects Processing type Effect description Virtual mask Semi-transparent mask that dynamically fits the facial contour (such as animal face painting, sci-fi style, etc.) Cartoon glasses Exaggerated frames that adapt to the size of the human face (such as round sunglasses, pixel-style glasses, etc.) Facial decoration Superimposed dynamic effects (such as virtual stickers, dynamic light spots, funny beards, etc.) Specific Example 2: The generated protected video and the original video are stored separately and an association relationship is established 1. System Basic Framework: Video Acquisition Module: Deployed in the camera group in the scenic area or park, used to collect real-time video streams of tourists' play; - Data Processing Center: Includes a face detection unit, a key point positioning unit, a mask generation unit, and a video processing unit; Dual Video Storage Module: Adopt a cloud distributed storage system to store the original video and the protected video separately, and record the mapping relationship through an association database; Access Control Module: Provide a public preview interface and an authorized download interface, supporting hierarchical permission management.

[0030] 2. Dual Video Association Storage Each video generates a unique UUID as the association identifier, and the data is written into the MongoDB sharding cluster with the primary key as video_id, and a spatio-temporal composite index (timestamp + camera ID) is established. The pseudo-code is as follows: { "video_id": "VID_20231001_1000_CAM007", "original": { "bucket": "oss-original-bucket", "path": " / zone3 / cam007 / 2023-10-01 / 10:00:00.mp4", "size": "1280x720", "duration": 120.5 }, "masked": { "bucket": "oss-masked-bucket", "path": " / zone3 / cam007 / 2023-10-01 / 10:00:00_masked.mp4" }, "faces": { "track_id": "FACE_001", "keypoints": [ [x1,y1], ..., [x68,y68]], "mask_template": "mask_style_05.obj" } }。

[0031] 3. Hierarchical Access and Download Control Public Preview Restriction: When playing the protected video on the scenic area display screen, the following processing is performed: 1. Add dynamic masks or facial occlusion special effects (including the text "Preview Only" and the timestamp); 2. Authorized Download Process Step 1: The tourist swipes their face or enters the ticket ID at the terminal, and the system retrieves the list of protected videos; Step 2: After selecting the original video, complete the payment by scanning the code with WeChat / Alipay; Step 3: The system verifies the payment serial number, extracts the original video address from the associated library, and generates a download link; Step 4: Encrypt the original video before downloading, and send the encryption key to the tourist's mobile phone separately via SMS.

[0032] ​This solution adopts a method for protecting the portrait privacy of tourists based on face processing, effectively controlling the images during the display stage to ensure that the personal privacy information of tourists is not obtained or disseminated by unauthorized third parties; while protecting privacy, it can meet the needs of tourists to view or purchase their personal images, enhancing the overall experience of tourists; automatically shooting throughout the process and automatically processing image protection; applicable to various public places, especially tourist-intensive places such as scenic spots, amusement parks, and museums, and can effectively resolve the contradiction between tourist privacy protection and image preview display.

[0033] The above shows and describes the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments. What is described in the above embodiments and the specification only illustrates the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention will have various changes and improvements, and these changes and improvements all fall within the scope of the present invention claimed. The scope of protection claimed by the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for protecting the privacy of tourist portraits based on face processing, characterized in that: It includes the following sub-steps: S1: Perform face detection on the original video of tourists playing; S2: determining the location and / or size of the facial features of the tourist from the face detection result; S3: Match the mask at the corresponding position according to the location and / or size of the tourist's facial features; S4: Covering the mask onto the tourist’s face in the original video frame; S5: Establish an association relationship between the mask-wearing video and the original video, and store them in the cloud separately; S6: Play the mask-wearing video during video preview in public spaces to protect visitors’ privacy; S7: When a visitor needs to download the original video, the original video is found through the association relationship, and the original video is provided for the visitor to download.

2. A method for protecting the privacy of tourist portraits based on face processing according to claim 1, characterized in that: The step S1 comprises the following sub-steps: S11: Upload the original video of the tourists' play to the data processing center through the camera; S12: The data processing center captures video frames through video files; S13: Perform face detection on each frame using a convolutional neural network face detection algorithm to determine whether there is a face in the image.

3. The method for protecting the privacy of tourist portraits based on face processing according to claim 1, characterized in that: The step S2 comprises the following sub-steps: S21: Use a deep learning model to predict a series of key points in the face area; The key points include ears, eyebrows, eyes, nose and mouth; S22: Calculate the specific position coordinates of the key points.

4. The method for protecting the privacy of tourist portraits based on face processing according to claim 1, characterized in that: The step S3 comprises the following sub-steps: S31: Determine the size and position of the mask through the key point coordinates and generate a tourist mask; S32: Process each frame of the video through real-time video editing technology, put the mask on the tourist's face through face tracking technology, and generate a video of the tourist wearing the mask.

5. The method for protecting the privacy of tourist portraits based on face processing according to claim 1, characterized in that: The step S5 comprises the following sub-steps: S51: Establishing an association relationship between the original video and the mask-wearing video.

6. The method for protecting the privacy of tourist portraits based on face processing according to claim 1, characterized in that: The step S6 comprises the following sub-steps: In the public display area, the preview image displayed to visitors before purchasing the image is the mask-wearing video; In the public display area, when other visitors view the images, the preview images displayed cannot obtain the specific facial features of others.

7. The method for protecting the privacy of tourist portraits based on face processing according to claim 1, characterized in that: The S7 comprises the following sub-steps: S71: Visitors select images of their interest in the image display area; S72: After the tourist meets the downloading conditions, the system automatically matches the tourist's image and removes the privacy protection processing in the image; S73: The visitor obtains his original video through the terminal device.

Images