Video data processing method and device, storage medium and electronic equipment

Abstract

The invention discloses a video data processing method and device, a storage medium and electronic equipment, and relates to the technical field of smart home, and the video data processing method comprises the steps: determining a target distance boundary value according to the region type of a region where monitoring equipment is located; when it is detected that a target object exists in the target video data collected by the monitoring equipment, determining a first distance between the target object and the monitoring equipment, and determining a size relationship between the first distance and a target distance boundary value; and under the condition that the size relationship indicates that the first distance is greater than the target distance boundary value, privacy processing is performed on a to-be-privatized area in the target video data, and the to-be-privatized area is an area determined according to the target object.

Description

Technical Field

[0001] The present application relates to the field of smart home technology, and more specifically, to a method and device for processing video data, a storage medium, and an electronic device. Background Art

[0002] In modern home security systems, surveillance cameras are widely used in areas requiring monitoring, such as doorways and balconies. Their primary function is to capture and record all activity within their field of view, allowing users to review it at any time and safeguarding their homes. However, when surveillance cameras are installed at residential doorways, particularly those in close proximity or with intersecting lines of sight with neighbors' doors, existing cameras often fail to intelligently distinguish between activity at one's own door and that at a neighbor's door, leading to a significant privacy issue.

[0003] Specifically, existing surveillance camera systems, whether using motion detection-based intelligent monitoring or traditional continuous recording modes, tend to monitor activity within their entire field of view. While this comprehensive monitoring strategy ensures a complete record of activity in front of one's home, it can also inadvertently capture private activities such as neighbors' comings and goings, and visitors. This risk of privacy violation is particularly significant when a neighbor's activity area overlaps with the surveillance area in front of one's home.

[0004] Regarding the problem in related technologies where surveillance cameras record all activities within the shooting range, which may infringe on the privacy of neighbors, no effective solution has been proposed. Summary of the Invention

[0005] The embodiments of the present application provide a method and apparatus for processing video data, a storage medium, and an electronic device to at least solve the problem in related technologies that surveillance cameras record all activities within the shooting range, which may infringe on the privacy of neighbors.

[0006] According to one embodiment of the present application, a method for processing video data is provided, comprising: determining a target distance threshold value based on the area type of an area where a monitoring device is located; upon detecting the presence of a target object in target video data collected by the monitoring device, determining a first distance between the target object and the monitoring device, and determining a magnitude relationship between the first distance and the target distance threshold value; and, upon the magnitude relationship indicating that the first distance is greater than the target distance threshold value, performing privacy processing on a to-be-privatized area in the target video data, wherein the to-be-privatized area is an area determined based on the target object.

[0007] In an exemplary embodiment, performing privacy processing on the area to be privatized in the target video data includes: detecting an area corresponding to a target object in each frame of the target video data, and determining the area corresponding to the target object as the area to be privatized corresponding to each frame of the image; adjusting the color of pixels in the area to be privatized corresponding to each frame of the image to a target color, or mosaicking the pixels in the area to be privatized corresponding to each frame of the image.

[0008] In an exemplary embodiment, performing privacy processing on the area to be privatized in the target video data includes: obtaining a background image of the area where the monitoring device is located from historical video data collected by the monitoring device; extracting a partial image corresponding to the area to be privatized from the background image; filling the area to be privatized with pixels of the partial image; and smoothly transitioning the pixels of the filled area to be privatized in the target video data with pixels of an adjacent area in the target video data through a target operation, wherein the adjacent area is an area adjacent to the area to be privatized in other areas of the target video data.

[0009] In an exemplary embodiment, after determining the target distance demarcation value based on the area type of the area where the monitoring device is located, the method further includes: obtaining a first frame image in which the target object is detected in the video data collected by the monitoring device, and determining the image features of the target object based on the first frame image; determining feature points of the target object based on the image features; tracking the target object in the video data based on the feature points, and determining the last frame image in which the target object is detected in the video data; and segmenting the video data based on the first frame image and the last frame image to obtain the target video data.

[0010] In an exemplary embodiment, before performing privacy processing on the to-be-private area in the target video data, the method further includes: determining the object type of the target object and the number of the target objects; and determining a processing method for the target video data based on the object type of the target object and the number of the target objects.

[0011] In an exemplary embodiment, determining a processing method for the target video data based on the object type and the number of target objects includes: when there are multiple target objects and each target object is of a privacy object type, determining the processing method to be deleting the target video data, wherein the privacy objects are objects requiring privacy protection; when there are multiple target objects and some of the target objects are of a privacy object type, determining the processing method to be performing privacy processing on to-be-privatized areas in the target video data, wherein the to-be-privatized areas are areas corresponding to some of the target objects; when there is only one target object and the object type of the target object is a privacy object, determining the processing method to be deleting the target video data; and when there is only one target object and the object type of the target object is a surveillance object, determining the processing method to not process the target video data, wherein the surveillance object is an object not requiring privacy protection.

[0012] In an exemplary embodiment, a target distance boundary value is determined based on the area type of the area where the monitoring device is located, including: determining a first distance boundary value corresponding to the area type, and determining whether a setting operation sent by a first object is received, wherein the setting operation is used to set the distance boundary value of the monitoring device to a second distance boundary value; in a case where the setting operation sent by the first object is received, determining the target distance boundary value based on the first distance boundary value and the second distance boundary value; in a case where the setting operation sent by the first object is not received, determining the first distance boundary value as the target distance boundary value.

[0013] According to another embodiment of the present application, a video data processing device is further provided, comprising: a first determination module for determining a target distance threshold value based on the area type of the area where the monitoring device is located; a second determination module for, upon detecting the presence of a target object in target video data collected by the monitoring device, determining a first distance between the target object and the monitoring device, and determining a magnitude relationship between the first distance and the target distance threshold value; and a processing module for performing privacy processing on a to-be-privatized area in the target video data, when the magnitude relationship indicates that the first distance is greater than the target distance threshold value, wherein the to-be-privatized area is an area determined based on the target object.

[0014] According to another aspect of the embodiments of the present application, a computer-readable storage medium is provided, in which a computer program is stored. The computer program is configured to execute the above-mentioned video data processing method when running.

[0015] According to another aspect of an embodiment of the present application, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the video data processing method through the computer program.

[0016] In an embodiment of the present application, a target distance demarcation value is determined based on the area type of the area where the monitoring device is located; when a target object is detected in the target video data collected by the monitoring device, a first distance between the target object and the monitoring device is determined, and the size relationship between the first distance and the target distance demarcation value is determined; when the size relationship indicates that the first distance is greater than the target distance demarcation value, the area to be privatized in the target video data is privatized, wherein the area to be privatized is an area determined based on the target object; in an embodiment of the present application, through the "target distance demarcation value", combined with the specific application scenario of the monitoring device, it is intelligently judged whether the target object activity may infringe on privacy, thereby effectively avoiding improper infringement of privacy while ensuring the monitoring effect. The above technical solution solves the problem that the surveillance camera records all activities within the shooting range, which may infringe on the privacy of neighbors. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0018] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0019] Figure 1 This is a schematic diagram of the hardware environment of a method for processing video data according to an embodiment of the present application;

[0020] Figure 2 is a flowchart of a method for processing video data according to an embodiment of the present application;

[0021] Figure 3 is a schematic diagram of an activity area according to an embodiment of the present application;

[0022] Figure 4 1 is a structural block diagram of a video data processing device according to an embodiment of the present application (I);

[0023] Figure 5This is a structural block diagram (II) of a video data processing device according to an embodiment of the present application. DETAILED DESCRIPTION

[0024] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0025] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in a sequence other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0026] According to one aspect of the embodiment of the present application, a method for processing video data is provided. The method for processing video data is widely used in smart home (Smart Home), smart home, smart home device ecology, smart residential (Intelligence House) ecology and other whole-house intelligent digital control application scenarios. Optionally, in this embodiment, the above-mentioned method for processing video data can be applied to Figure 1 In the hardware environment shown in FIG. 1 , which is composed of a terminal device 102 and a server 104. Figure 1 As shown, the server 104 is connected to the terminal device 102 via a network, and can be used to provide services (such as application services, etc.) for the terminal or the client installed on the terminal. A database can be set up on the server or independently of the server to provide data storage services for the server 104. Cloud computing and / or edge computing services can be configured on the server or independently of the server to provide data computing services for the server 104.

[0027] The aforementioned network may include, but is not limited to, at least one of the following: a wired network and a wireless network. The aforementioned wired network may include, but is not limited to, at least one of the following: a wide area network, a metropolitan area network, and a local area network. The aforementioned wireless network may include, but is not limited to, at least one of the following: Wi-Fi (Wireless Fidelity) and Bluetooth. The terminal device 102 may be, but is not limited to, a PC, a mobile phone, a tablet computer, a smart air conditioner, a smart range hood, a smart refrigerator, a smart oven, a smart stove, a smart washing machine, a smart water heater, a smart washing machine, a smart dishwasher, a smart projection device, a smart TV, a smart clothes drying rack, smart curtains, smart audio and video, a smart socket, a smart speaker, a smart fresh air device, smart kitchen and bathroom equipment, smart bathroom equipment, a smart sweeping robot, a smart window cleaning robot, a smart mopping robot, a smart air purifier, a smart steamer, a smart microwave oven, a smart kitchen treasure, a smart purifier, a smart water dispenser, a smart door lock, etc.

[0028] In this embodiment, a method for processing video data is provided, which is applied to the above-mentioned terminal device. Figure 2 4 is a flowchart of a method for processing video data according to an embodiment of the present application, the process comprising the following steps:

[0029] Step S202, determining a target distance threshold value according to the area type of the area where the monitoring device is located;

[0030] Different monitoring scenarios (such as public places, doorsteps, and private offices) require different privacy protection requirements. For example, when monitoring a doorstep, it's important to protect the privacy of neighbors, while monitoring in public places may prioritize comprehensiveness and security. Therefore, the target distance threshold should be set based on the type of area the monitoring device is located in to accommodate different privacy protection requirements.

[0031] Through user settings or automatic system learning, a reasonable target distance threshold is preset for different area types (such as the doorstep of a residential area). This threshold is less than or equal to the minimum distance between the monitoring device and potential privacy protection objects (such as a neighbor's door), ensuring that only activities within a safe distance are recorded.

[0032] Step S204: when a target object is detected in the target video data collected by the monitoring device, determining a first distance between the target object and the monitoring device, and determining a magnitude relationship between the first distance and the target distance threshold;

[0033] Advanced computer vision technologies, such as motion detection and object recognition, are used to detect target objects in surveillance videos. Furthermore, through deep learning, binocular ranging technology, or depth-of-field cameras, the first distance between the target object and the surveillance device is calculated in real time. The calculated first distance is then compared with a preset target distance threshold to determine the relative size.

[0034] Step S206 : When the size relationship indicates that the first distance is greater than the target distance threshold, privacy processing is performed on the area to be privatized in the target video data, wherein the area to be privatized is an area determined according to the target object.

[0035] Once the initial distance between the target object and the monitoring device is determined to be greater than the target distance threshold, the system automatically identifies the area occupied by the target object in the video image (the area to be privatized). This area is then subjected to privacy processing, using common methods such as blurring, mosaicking, and complete blocking to remove or obscure sensitive personal information. The privacy-processed video data is then saved for viewing by the monitoring system user, while ensuring the privacy of non-monitored subjects such as neighbors.

[0036] If the first distance between the target object and the monitoring device is less than the target distance threshold, the target object is considered to be within a pre-defined "public" or "non-privacy" area and does not violate privacy protection requirements. In this case, the system will determine that the video content involving the target object does not need to be privacy-protected and will instead treat it as a normal monitored object, maintaining the integrity of the video content.

[0037] Through the above steps, the target distance demarcation value is determined according to the area type of the area where the monitoring device is located; when it is detected that there is a target object in the target video data collected by the monitoring device, the first distance between the target object and the monitoring device is determined, and the size relationship between the first distance and the target distance demarcation value is determined; when the size relationship indicates that the first distance is greater than the target distance demarcation value, the area to be privatized in the target video data is privatized, wherein the area to be privatized is the area determined according to the target object; in the embodiment of the present application, through the "target distance demarcation value", combined with the specific application scenario of the monitoring device, it is intelligently judged whether the target object activity may violate privacy, thereby effectively avoiding improper privacy infringement while ensuring the monitoring effect. The above technical solution solves the problem that the surveillance camera records all activities within the shooting range, which may violate the privacy of neighbors.

[0038] To better understand step S206, step S206 can be implemented in the following manner: detecting an area corresponding to a target object in each frame of the target video data, and determining the area corresponding to the target object as the area to be privatized corresponding to each frame of the target video data; adjusting the color of pixels in the area to be privatized corresponding to each frame of the target video data to a target color, or mosaicking the pixels in the area to be privatized corresponding to each frame of the target video data.

[0039] In continuously recorded surveillance videos, the position of the target object (an individual or activity that may involve privacy concerns) changes dynamically. Therefore, it is necessary to detect the target object's position in each frame in real time. For example, this involves analyzing each frame of the target video data through methods such as object recognition and moving target detection. Deep learning models (such as YOLO and SSD) are used to accurately identify the target object and delineate its corresponding area in the image.

[0040] After identifying the target object, its specific location needs to be determined for privacy protection. Implementation method: Based on the target object area identified in the previous step, it is marked as the area to be protected. After determining the area to be protected, measures need to be taken to remove or cover up the private information in it to protect individual privacy. Specifically:

[0041] Color adjustment: Replace the colors of all pixels in the area to be privatized with a uniform target color (such as black, gray, etc.).

[0042] Mosaicing: Mosaicing is performed on the pixels in the area to be privatized, that is, the pixels in the target area are sampled and blurred to form a mosaic effect.

[0043] To better understand step S206, step S206 may also be implemented in the following manner: obtaining a background image of the area where the monitoring device is located from historical video data collected by the monitoring device; extracting a partial image corresponding to the area to be privatized from the background image; filling the area to be privatized with pixels of the partial image; and smoothly transitioning the pixels of the filled area to be privatized in the target video data with pixels of an adjacent area in the target video data through a target operation, wherein the adjacent area is an area adjacent to the area to be privatized in other areas of the target video data.

[0044] In surveillance videos, the background parts of the screen (static environments such as walls, floors, furniture, etc.) are relatively stable in different time periods. However, in privacy protection processing, color adjustment or mosaic processing often destroys the continuity and visual effect of the background. Therefore, in an embodiment of the present application, a time period with no human activity or privacy-sensitive events is selected from the historical video data of the monitoring device, and a stable and clear background image is extracted or synthesized through image processing technology (such as background modeling, inter-frame difference analysis, etc.). This background image should fully reflect the normal environment of the monitored area to ensure the naturalness of the subsequent filling process and the authenticity of the information.

[0045] Based on the location and size of the privacy-protected area previously identified in the target video data, a portion of the image corresponding to it is precisely cropped from the background image. This cropped portion should match the size and shape of the privacy-protected area to facilitate seamless infilling. Using image processing techniques, the portion of the image extracted from the background image is precisely overlaid onto the privacy-protected area in each frame of the target video data. This process requires ensuring image alignment and matching, as well as accurate pixel replication, to ensure a natural blend of the infilled area with the surrounding environment and avoid unnatural visual effects.

[0046] Using the smooth transition operation in image fusion technology, the edges of the padded area are processed to gradually transition with the pixels in the surrounding adjacent areas (i.e., the non-privacy-treated areas), forming a natural boundary. This includes, but is not limited to, using techniques such as feathering, blurring, brightness, and color adjustment to eliminate edge artifacts and ensure visual consistency and naturalness across the entire image.

[0047] By filling the privacy-delisted areas with historical background images, this solution not only protects sensitive information but also maintains the overall visual quality and information integrity of the video. Through precise image processing technology, this solution maximizes the practicality and aesthetics of the video while protecting personal privacy, meeting users' dual needs for both information integrity and privacy security in surveillance video.

[0048] In an exemplary embodiment, after determining the target distance demarcation value based on the area type of the area where the monitoring device is located, it also includes: obtaining a first frame image in which the target object is detected in the video data collected by the monitoring device, and determining the image features of the target object based on the first frame image; determining feature points of the target object based on the image features; tracking the target object in the video data based on the feature points, and determining the last frame image in which the target object is detected in the video data; and segmenting the video data according to the first frame image and the last frame image to obtain the target video data.

[0049] Through real-time analysis of surveillance video, object recognition and motion detection technologies are used to identify the first frame in the video data where the target object first appears. Based on this first frame, deep learning models (such as convolutional neural networks (CNNs)) are then used to extract the target object's image features, including but not limited to shape, color, texture, and motion trajectory. These features will serve as the basis for subsequent tracking and privacy protection processing.

[0050] Based on the extracted image features, a series of feature points on the target object are further located and marked, which can be eyes, nose, the four corners of the license plate, etc. These feature points should be highly unique and stable so that they can be accurately identified and located in subsequent frames of the video.

[0051] Using feature point tracking techniques (such as optical flow and feature point matching), the target object's movement trajectory in the video is continuously tracked until the target object leaves the monitoring screen or its feature points can no longer be accurately identified. This process records the target object's complete behavior sequence in the video until the last frame is determined, thus capturing the target object's complete activity cycle in the video.

[0052] Based on the determined starting frame (first frame) and ending frame (last frame) involving the target object activity, the surveillance video data is accurately segmented to obtain a video segment containing only the target object activity, namely the target video data.

[0053] Through intelligent and precise tracking capabilities and efficient management of video data, it can not only accurately identify the appearance and disappearance of target objects to ensure targeted privacy protection, but also effectively manage video data through feature point tracking and video segmentation, reduce system resource consumption, and improve the processing efficiency and quality of monitoring information.

[0054] In an exemplary embodiment, before performing privacy processing on the to-be-private area in the target video data, the method further includes: determining the object type of the target object and the number of the target objects; and determining a processing method for the target video data according to the object type of the target object and the number of the target objects.

[0055] Specifically, determining a processing method for the target video data according to the object type of the target object and the number of target objects includes: when there are multiple target objects and the object type of each target object is a privacy object, determining the processing method is to delete the target video data, wherein the privacy object is an object requiring privacy protection; when there are multiple target objects and the object type of some target objects is a privacy object, determining the processing method is to perform privacy processing on a to-be-privatized area in the target video data, wherein the to-be-privatized area is an area corresponding to the some target objects; when there is only one target object and the object type of the target object is a privacy object, determining the processing method is to delete the target video data; when there is only one target object and the object type of the target object is a surveillance object, determining the processing method is to not process the target video data, wherein the surveillance object is an object not requiring privacy protection.

[0056] When the system confirms the existence of multiple privacy objects in the surveillance video through target detection and classification technology, it automatically determines that deleting the entire video data is the most reliable privacy protection measure to ensure that all privacy-related activities are not recorded and replayed.

[0057] When both private and non-private objects exist in a video, a balance must be struck between protecting privacy and maintaining the integrity of the surveillance information. Specifically, private objects and their locations in the video are identified and determined. Then, only the areas corresponding to these objects are privacy-enhanced, such as pixelation, occlusion, or replacement, while retaining the information of other non-private objects. This ensures the value and analyzability of the surveillance video after privacy protection.

[0058] When the system detects the presence of a single privacy object in a video, based on the privacy protection priority, the system will decide to delete the video data to avoid any privacy leakage risk.

[0059] In video surveillance, certain objects (such as couriers, maintenance workers, users of monitoring equipment, etc.) may not be subject to privacy protection. When it is identified that a single monitoring object rather than a private object appears in the video, it will be determined that there is no need to perform privacy processing on the video data in order to fully retain the information and activities of the monitored object and meet the monitoring needs.

[0060] This embodiment of the application dynamically selects the most appropriate privacy protection strategy based on the type and number of target objects, preventing inappropriate privacy leaks while ensuring the integrity and availability of monitoring information. By introducing this intelligent decision-making mechanism, the system can more flexibly respond to different monitoring scenarios and improve the effectiveness and efficiency of privacy protection.

[0061] In an exemplary embodiment, a target distance boundary value is determined based on the area type of the area where the monitoring device is located, including: determining a first distance boundary value corresponding to the area type, and determining whether a setting operation sent by a first object is received, wherein the setting operation is used to set the distance boundary value of the monitoring device to a second distance boundary value; in a case where the setting operation sent by the first object is received, determining the target distance boundary value based on the first distance boundary value and the second distance boundary value; in a case where the setting operation sent by the first object is not received, determining the first distance boundary value as the target distance boundary value.

[0062] Based on an analysis of the type of area where the monitoring device is located, an appropriate distance threshold, known as the first distance threshold, is preset or determined through machine learning as the default privacy protection range. For example, for residential hallway monitoring, the first distance threshold might be set to the distance from the monitoring device to a neighbor's door, ensuring that no neighbor's private information is inadvertently collected.

[0063] In the embodiment of the present application, a user interface is also provided to allow the user (first object) to send a setting operation to manually adjust the distance threshold value of the monitoring device to the second distance threshold value. This setting operation provides a flexible way for the user to adjust the privacy protection range to meet personalized needs.

[0064] After receiving the user's setting operation, the system comprehensively considers the first distance threshold (default setting) and the second distance threshold (user-defined setting) to determine a more reasonable target distance threshold. Determining this target distance threshold may involve algorithmic analysis, such as calculating the average or weighted average of the two thresholds, or selecting a value as the target distance threshold based on specific rules to achieve optimal privacy protection and monitoring functions.

[0065] If the system does not receive any user-defined settings within the preset time, it will automatically adopt the first distance threshold as the target distance threshold and implement the default privacy protection process. This mechanism ensures that even without active user interaction, the system can provide reasonable and necessary privacy protection based on the environment type.

[0066] In order to better understand the process of the above-mentioned video data processing method, the implementation method flow of the above-mentioned video data processing is described below in combination with an optional embodiment, but it is not used to limit the technical solution of the embodiment of the present application.

[0067] In this embodiment, a method for processing video data is provided, which specifically includes:

[0068] In surveillance videos, accurate identification and processing are achieved by measuring the distance between the moving target and the surveillance equipment, such as Figure 3As shown, specifically:

[0069] Based on the user's specific usage scenario, a distance threshold value D is preset. The value of D is set to be smaller than the actual distance from the surveillance camera to the neighbor's door, which serves as the boundary between privacy protection and surveillance functions.

[0070] Utilizing advanced video image change detection technology and image feature extraction technology, it detects and locates moving objects in the image, marks them as objects A and B, starts real-time tracking, and records the object's active video clips for temporary storage.

[0071] During the tracking process, a depth-of-field camera is used to obtain the real-time distance d between the object A and the camera. A and the real-time distance d between object B and the camera B , and combines this data with the extracted image feature information to form a comprehensive target data record.

[0072] The tracking is continued by using the features of object A and object B until they completely disappear from the picture. Then, the distance d between object A and the camera during the entire tracking process is analyzed. A Whether it has crossed the preset threshold value D, and the distance d between object B and the camera B Whether it has crossed the preset demarcation value D, if object A always remains outside the D value, the system will automatically delete the video clip and related data corresponding to object A to avoid meaningless information storage. If object B always remains within the D value, the system will store the video clip and related data corresponding to object B.

[0073] In surveillance videos, the background, such as hallways, walls, and closed doors of neighbors' homes, while not carrying sensitive information, is crucial for creating a coherent and informative video clip. To ensure this context is preserved, background elements in the video are identified and saved, serving as the basis for compensating for image loss during subsequent privacy processing.

[0074] For private content that does not involve surveillance needs, the video recording is directly deleted. However, in some cases, such as live surveillance or when the surveillance target and the privacy protection object are in the same frame, it is necessary to protect the privacy of the video while displaying it.

[0075] After identifying and tracking the active target, the active area of object A in each frame is accurately obtained. For these areas, pixel modification technology is used to turn them black or apply a mosaic effect. This eliminates visual information of sensitive entities while retaining the basic scene information of the video, achieving the purpose of privacy protection.

[0076] Alternatively, you can select the parts of the saved background image that match the privacy area, and through detailed processing such as feathering effect and brightness adjustment, seamlessly blend these background images into the privacy area, making the restored video picture more natural and the visual effect close to the original video.

[0077] In an embodiment of the present application, "active objects" and "background" are distinguished in privacy processing of surveillance video images; distance detection and distance boundaries are used to distinguish between parts of the image that require privacy processing and parts that require monitoring; in privacy processing of surveillance video images, features of "active objects" are extracted to track the active objects, thereby preserving the images before the "active objects" enter the boundary; and compensation and repair are performed on the privacy-processed images, using the background to compensate for the missing parts of the image.

[0078] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of each embodiment of the present application.

[0079] Figure 4 is a structural block diagram of a video data processing device according to an embodiment of the present application; Figure 4 Shown, including:

[0080] A first determining module 42 is configured to determine a target distance threshold value according to the area type of the area where the monitoring device is located;

[0081] A second determining module 44 is configured to, upon detecting that a target object exists in the target video data collected by the monitoring device, determine a first distance between the target object and the monitoring device, and determine a magnitude relationship between the first distance and the target distance threshold;

[0082] The processing module 46 is configured to perform privacy processing on a to-be-private area in the target video data when the size relationship indicates that the first distance is greater than the target distance threshold, wherein the to-be-private area is an area determined according to the target object.

[0083] Through the above-mentioned device, the target distance demarcation value is determined according to the area type of the area where the monitoring device is located; when it is detected that there is a target object in the target video data collected by the monitoring device, the first distance between the target object and the monitoring device is determined, and the size relationship between the first distance and the target distance demarcation value is determined; when the size relationship indicates that the first distance is greater than the target distance demarcation value, the area to be privatized in the target video data is privatized, wherein the area to be privatized is the area determined according to the target object; in the embodiment of the present application, through the "target distance demarcation value", combined with the specific application scenario of the monitoring device, it is intelligently judged whether the target object activity may violate privacy, thereby effectively avoiding improper privacy infringement while ensuring the monitoring effect. The above-mentioned technical solution solves the problem that the surveillance camera records all activities within the shooting range, which may violate the privacy of neighbors.

[0084] In an exemplary embodiment, the processing module 46 is configured to detect an area corresponding to a target object in each frame of the target video data, and determine the area corresponding to the target object as the area to be privatized corresponding to each frame of the image; adjust the color of pixels in the area to be privatized corresponding to each frame of the image to a target color, or mosaic the pixels in the area to be privatized corresponding to each frame of the image.

[0085] In an exemplary embodiment, the processing module 46 is configured to obtain a background image of the area where the monitoring device is located from historical video data collected by the monitoring device; extract a partial image corresponding to the area to be privatized from the background image; fill the area to be privatized with pixels of the partial image; and smoothly transition the pixels of the filled area to be privatized in the target video data with pixels of an adjacent area in the target video data through a target operation, wherein the adjacent area is an area adjacent to the area to be privatized in other areas of the target video data.

[0086] In an exemplary embodiment, Figure 5 As shown, the above-mentioned device also includes: an acquisition module 52, which is used to acquire the first frame image in which the target object is detected in the video data collected by the monitoring device, and determine the image features of the target object based on the first frame image; determine the feature points of the target object according to the image features; track the target object in the video data based on the feature points, and determine the last frame image in which the target object is detected in the video data; and segment the video data according to the first frame image and the last frame image to obtain the target video data.

[0087] In an exemplary embodiment, the processing module 46 is further configured to determine the object type of the target object and the number of the target objects; and determine a processing method for the target video data according to the object type of the target object and the number of the target objects.

[0088] In an exemplary embodiment, the processing module 46 is further configured to, when there are multiple target objects and each target object is a privacy object, determine that the processing method is to delete the target video data, wherein the privacy objects are objects requiring privacy protection; when there are multiple target objects and some of the target objects are privacy objects, determine that the processing method is to perform privacy processing on the to-be-privatized area in the target video data, wherein the to-be-privatized area is the area corresponding to the some of the target objects; when there is only one target object and the object type of the target object is a privacy object, determine that the processing method is to delete the target video data; when there is only one target object and the object type of the target object is a monitoring object, determine that the processing method is to not process the target video data, wherein the monitoring object is an object not requiring privacy protection.

[0089] In an exemplary embodiment, the first determination module 42 is used to determine the first distance boundary value corresponding to the area type, and to determine whether a setting operation sent by a first object is received, wherein the setting operation is used to set the distance boundary value of the monitoring device to a second distance boundary value; in the case of receiving the setting operation sent by the first object, the target distance boundary value is determined based on the first distance boundary value and the second distance boundary value; in the case of not receiving the setting operation sent by the first object, the first distance boundary value is determined to be the target distance boundary value.

[0090] An embodiment of the present application further provides a storage medium, which includes a stored program, wherein the program executes any of the above methods when it is run.

[0091] Optionally, in this embodiment, the storage medium may be configured to store program codes for executing the following steps:

[0092] S1, determining a target distance threshold value according to the area type of the area where the monitoring device is located;

[0093] S2, when detecting that a target object exists in the target video data collected by the monitoring device, determining a first distance between the target object and the monitoring device, and determining a magnitude relationship between the first distance and the target distance threshold;

[0094] S3. When the size relationship indicates that the first distance is greater than the target distance threshold, perform privacy processing on the area to be privatized in the target video data, wherein the area to be privatized is an area determined according to the target object.

[0095] An embodiment of the present application further provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0096] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0097] Optionally, in this embodiment, the processor may be configured to execute the following steps through a computer program:

[0098] S1, determining a target distance threshold value according to the area type of the area where the monitoring device is located;

[0099] S2, when detecting that a target object exists in the target video data collected by the monitoring device, determining a first distance between the target object and the monitoring device, and determining a magnitude relationship between the first distance and the target distance threshold;

[0100] S3. When the size relationship indicates that the first distance is greater than the target distance threshold, perform privacy processing on the area to be privatized in the target video data, wherein the area to be privatized is an area determined according to the target object.

[0101] Optionally, in this embodiment, the above-mentioned storage medium may include but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and other media that can store program codes.

[0102] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps in any one of the above method embodiments are implemented.

[0103] An embodiment of the present application further provides another computer program product, comprising a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any of the above method embodiments are implemented.

[0104] An embodiment of the present application also provides a computer program, which includes computer instructions, which are stored in a computer-readable storage medium; a processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps of any of the above method embodiments.

[0105] Optionally, specific examples in this embodiment may refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0106] Obviously, those skilled in the art should understand that the modules or steps of the present application described above can be implemented using a general-purpose computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices. Alternatively, they can be implemented using program code executable by the computing device, so that they can be stored in a storage device and executed by the computing device. In some cases, the steps shown or described can be performed in a different order than herein, or they can be made into separate integrated circuit modules, or multiple modules or steps can be made into a single integrated circuit module for implementation. Thus, the present application is not limited to any specific combination of hardware and software.

[0107] The above is only a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for processing video data, characterized in that: include: Determine the target distance threshold value based on the area type of the area where the monitoring device is located; In the case where a target object is detected in the target video data collected by the monitoring device, determining a first distance between the target object and the monitoring device, and determining a magnitude relationship between the first distance and the target distance threshold value; When the size relationship indicates that the first distance is greater than the target distance threshold, privacy processing is performed on the area to be privatized in the target video data, wherein the area to be privatized is an area determined according to the target object.

2. The method for processing video data according to claim 1, wherein: Performing privacy processing on the area to be privatized in the target video data, including: Detecting an area corresponding to a target object in each frame of the target video data, and determining the area corresponding to the target object as the area to be privatized corresponding to each frame of the image; The color of the pixel points of the area to be privatized corresponding to each frame of image is adjusted to the target color, or the pixel points of the area to be privatized corresponding to each frame of image are mosaicked.

3. The method for processing video data according to claim 1, wherein: Performing privacy processing on the area to be privatized in the target video data, including: Acquire a background image of the area where the monitoring device is located from historical video data collected by the monitoring device; Extracting a partial image corresponding to the area to be privatized from the background image; Filling the pixel points of the partial image into the area to be privatized; A target operation is performed to smoothly transition pixel points of the to-be-private area in the filled target video data with pixel points of an adjacent area in the target video data, wherein the adjacent area is an area adjacent to the to-be-private area in other areas of the target video data.

4. The method for processing video data according to claim 1, wherein: After determining the target distance threshold value according to the area type of the area where the monitoring device is located, the method further includes: Acquire a first frame image in which the target object is detected from the video data collected by the monitoring device, and determine image features of the target object based on the first frame image; determining feature points of the target object according to the image features; Tracking the target object in the video data based on the feature points, and determining a last frame image in the video data in which the target object is detected; The video data is segmented according to the first frame image and the last frame image to obtain the target video data.

5. The method for processing video data according to claim 1, wherein: Before performing privacy processing on the area to be privacy-protected in the target video data, the method further includes: determining an object type of the target object and a number of the target objects; A processing method for the target video data is determined according to the object type of the target object and the number of the target objects.

6. The method for processing video data according to claim 5, characterized in that: Determining a processing method for the target video data according to the object type of the target object and the number of the target objects includes: In a case where there are multiple target objects and the object type of each target object is a privacy object, determining that the processing method is to delete the target video data, wherein the privacy object is an object requiring privacy protection; When there are multiple target objects and the object types of some of the target objects are private objects, determining the processing method is to perform privacy processing on the to-be-private area in the target video data, wherein the to-be-private area is the area corresponding to the some of the target objects; When the number of the target object is one and the object type of the target object is a privacy object, determining that the processing manner is to delete the target video data; When the number of the target object is one and the object type of the target object is a monitoring object, the processing manner is determined to be not processing the target video data, wherein the monitoring object is an object that does not require privacy protection.

7. The method for processing video data according to claim 1, wherein: The target distance threshold is determined based on the area type of the area where the monitoring equipment is located, including: Determining a first distance threshold value corresponding to the area type, and determining whether a setting operation sent by a first object is received, wherein the setting operation is used to set the distance threshold value of the monitoring device to a second distance threshold value; Upon receiving the setting operation sent by the first object, determining the target distance threshold value according to the first distance threshold value and the second distance threshold value; In a case where no setting operation sent by the first object is received, the first distance threshold value is determined to be the target distance threshold value.

8. A video data processing device, characterized in that: include: A first determining module is used to determine a target distance threshold value according to an area type of an area where the monitoring device is located; a second determining module, configured to, upon detecting that a target object exists in the target video data collected by the monitoring device, determine a first distance between the target object and the monitoring device, and determine a magnitude relationship between the first distance and the target distance threshold; a processing module configured to perform privacy processing on a to-be-private area in the target video data when the size relationship indicates that the first distance is greater than the target distance threshold, wherein the to-be-private area is an area determined according to the target object.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein the method according to any one of claims 1 to 7 is executed when the program is executed.

10. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to execute the method according to any one of claims 1 to 7 through the computer program.

Images