Image occlusion method, computer device and computer readable storage medium

Abstract

The application discloses an image occlusion method, a computer device and a computer readable storage medium. The method comprises the following steps: determining detection regions in which respective objects in an image frame are located, wherein the union set of the detection regions covers the image frame, and the distinctness of the objects in different detection regions is different; identifying target objects in each object in the detection regions by using object category attributes matched with the distinctness corresponding to the detection regions; and performing occlusion processing on at least part of the regions of the target objects in the detection regions based on the distinctness corresponding to the detection regions. The above scheme can improve the image occlusion accuracy.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image occlusion method, a computer device, and a computer-readable storage medium. Background Technology

[0002] With the continuous development of image processing technology, it has been applied to various application scenarios, such as intelligent transportation, monitoring scenarios, and medical imaging. The images or video frames in these application scenarios may contain information about the target, and the demand for privacy protection is becoming increasingly urgent. In order to avoid leaking information about the target, it is necessary to occlude certain targets in the image or video frame.

[0003] Currently, the occlusion method typically involves occluding a fixed area within an image or video frame to achieve the occlusion effect on a target within that fixed area. However, since the occlusion area is fixed, this method cannot accurately occlude the target. Summary of the Invention

[0004] The main technical problem addressed by this application is to provide an image occlusion method, computer device, and computer-readable storage medium that can improve the accuracy of image occlusion.

[0005] To address the aforementioned issues, a first aspect of this application provides an image occlusion method. This method includes: determining the detection regions where each object in an image frame is located, wherein the union of the detection regions covers the image frame, and the clarity of objects in different detection regions varies; identifying target objects within each object in the detection regions using object category attributes that match the clarity of the detection regions; and occluding at least a portion of the target objects in the detection regions based on the clarity of the detection regions.

[0006] To address the aforementioned problems, a second aspect of this application provides a computer device comprising a memory and a processor coupled to each other, wherein the memory stores program data and the processor executes the program data to implement any step of the aforementioned image occlusion method.

[0007] To address the aforementioned problems, a third aspect of this application provides a computer-readable storage medium storing program data executable by a processor, the program data being used to implement any step of the aforementioned image occlusion method.

[0008] The above scheme determines the detection regions where each object in the image frame is located. The union of the detection regions covers the image frame, and the clarity of objects in different detection regions varies. Based on the different clarity of objects in different detection regions, object category attributes that match the clarity level can be matched. Different object category attributes can be matched to different detection regions. By using object category attributes that match the clarity level of the detection region, target objects can be identified within each object in the detection region. Target objects in different detection regions can be identified in different ways. Then, based on the clarity level of the detection region, at least a portion of the target object in the detection region is occluded. Occlusion processing can be performed separately for target objects in detection regions with different clarity levels, ensuring that target objects of each clarity level are occluded, avoiding omission of target objects, and improving the accuracy of image occlusion. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in this application, the accompanying drawings required in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Among them:

[0010] Figure 1 This is a flowchart illustrating the first embodiment of the image occlusion method of this application;

[0011] Figure 2 This application Figure 1 A flowchart illustrating an embodiment of step S11;

[0012] Figure 3 This is a schematic diagram of an embodiment of image occlusion processing for an image frame in this application;

[0013] Figure 4 This is a flowchart illustrating the second embodiment of the image occlusion method of this application;

[0014] Figure 5 This is a schematic diagram of an embodiment of multiple target detection methods used in this application to detect a reference object region;

[0015] Figure 6 This application Figure 4 A flowchart illustrating an embodiment of step S22;

[0016] Figure 7 This is a schematic diagram of an embodiment of the image frame distribution in this application;

[0017] Figure 8 This is a schematic diagram of an embodiment of the multiple detection areas defined in this application;

[0018] Figure 9 This is a flowchart illustrating the third embodiment of the image occlusion method of this application;

[0019] Figure 10 This is a schematic diagram of an embodiment of image occlusion processing in the first detection area of ​​this application;

[0020] Figure 11 This is a flowchart illustrating the fourth embodiment of the image occlusion method of this application;

[0021] Figure 12 This is a schematic diagram of an embodiment of the image occlusion processing for the second detection area of ​​this application;

[0022] Figure 13 This is a flowchart illustrating the fifth embodiment of the image occlusion method of this application;

[0023] Figure 14 This is a schematic diagram of an embodiment of image occlusion processing in the third detection area of ​​this application;

[0024] Figure 15 This is a schematic diagram of an embodiment of image occlusion processing for multiple detection areas in this application;

[0025] Figure 16 This is a schematic diagram of the structure of an embodiment of the image occlusion device of this application;

[0026] Figure 17 This is a schematic diagram of the structure of an embodiment of the computer device of this application;

[0027] Figure 18 This is a schematic diagram of the structure of an embodiment of the computer-readable storage medium of this application. Detailed Implementation

[0028] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0029] The terms "first" and "second" in this application are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such processes, methods, products, or apparatus.

[0030] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0031] In this document, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " generally indicates that the preceding and following related objects have an "or" relationship. Furthermore, "many" in this document means two or more. Moreover, the term "at least one" in this document means any combination of at least two of any one or more of a plurality of objects. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0032] This application provides the following embodiments, and each embodiment is described in detail below.

[0033] Please see Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the image occlusion method of this application. The method may include the following steps:

[0034] S11: Determine the detection region where each object in the image frame is located, wherein the union of the detection regions covers the image frame, and the clarity of the objects in different detection regions is different.

[0035] The image frame can be a specific image frame from the target video, which can be captured by a camera device on the target area. The specific content of the target area is not limited and can be set according to the actual needs of the application scenario. The target area can be any one or more of the following public scenes: nursing homes, hospitals, supermarkets, entrances / exits of a certain area, street scenes, stations, waiting rooms, airports, etc. The target video can contain one or more video image frames, and each video frame can contain one or more objects. This application uses the inclusion of multiple objects as an example for illustration, but it does not impose any limitations on this.

[0036] In some implementations, the object can be any subject that may require image occlusion, privacy protection, or image processing. For example, the object mentioned above can be a vehicle, a person, an animal, or an object, and this application does not impose any limitations on this.

[0037] In some implementations, the object can be a subject of interest in the application scenario. In actual application, the specific physical object referred to by the object can be determined according to actual needs. For example, in scenarios such as banks and shopping malls, the object can refer to people; in scenarios such as roads and bridges, the object can refer to vehicles and pedestrians; in scenarios such as forests and grasslands, the object can refer to animals. This application does not impose any restrictions on this.

[0038] In some implementations, the image frame can be divided into multiple detection regions beforehand, such that the union of the detection regions covers the image frame. Since the scenes or areas captured by each camera device are different, different detection regions can be divided separately for the image frames captured by each camera device, and this application does not impose any restrictions on this.

[0039] In some implementations, the sharpness of objects varies across different detection areas. That is, the sharpness of objects that can be distinguished differs in each detection area, and multiple detection areas can be defined according to the object's sharpness. Different sharpness can also be characterized by different object distances from the imaging device. It is understood that multiple detection areas can be defined based on the object's distance from the imaging device.

[0040] In some implementations, the image frame can be divided into multiple detection regions according to a preset time period to update the detection regions corresponding to the video frame. For example, the preset time period is 1 minute, 10 minutes, 20 minutes, etc., and this application does not limit it.

[0041] Based on the predefined detection regions, target detection is performed on the image frame to determine the detection region of each object in the image frame based on the location of the object.

[0042] In some embodiments, please refer to Figure 2This embodiment can further extend step S11 of the above embodiment. Determining the detection region where each object in the image frame is located can include the following steps:

[0043] S111: Perform target detection based on image frames to obtain the target position of the object in the image frame.

[0044] Object detection can be performed on image frames. Object detection can detect the object region in the image frame and obtain the target position of the object in the image frame. The target position can be the center position of the object region, or the upper left corner position, etc. This application does not limit it.

[0045] S112: Determine the detection area where the object is located based on the target position of the object.

[0046] By using the target location of the object in the image frame and the boundaries of the multiple detection regions defined above, the detection region where the object is located can be determined.

[0047] In some implementations, step S11 can also be performed by using the above-mentioned multiple target detection methods to detect targets in the image frame, thereby obtaining the object region of the object in the image frame under each target detection method, and determining the target position of the object in the image frame based on the object region, thereby determining the detection area where the object is located. The multiple target detection methods have different detection granularities for the objects contained in the image frame.

[0048] In this embodiment, objects contained in an image frame can be divided into detection regions so that image occlusion processing can be performed on each detection region separately.

[0049] In some implementations, step S11 can be based on the region boundaries of the multiple detection regions defined above, and the region boundaries of the multiple detection regions can be determined in the current image frame so that in subsequent steps, target detection can be performed on each detection region and image occlusion processing can be performed on the objects contained in each detection region.

[0050] S12: Identify the target object within each object in the detection area by using the object category attribute that matches the clarity of the detection area.

[0051] Since the clarity of objects varies in different detection areas, object category attributes that match the clarity of each detection area can be set. In other words, different object category attributes can be set for different clarity of objects in different detection areas. The object type attribute can be attribute information that can identify the object, such as the object's action attribute features, external attribute features, part attribute features, etc. This application does not limit this.

[0052] Taking a human as an example, the action attribute features can be movement features, gait features, etc., the external attribute features can be human body contour features, head and shoulder contour features, clothing and decoration features, etc., and the part attribute features can be human face features. This application does not impose any restrictions on these.

[0053] Taking a vehicle as an example, the action attribute features can be vehicle speed features, the external attribute features can be vehicle body outline features, vehicle body attribute features (such as color), etc., and the part attribute features can be license plate features, vehicle logo features, etc. This application does not impose any restrictions on these.

[0054] It is understood that this application does not restrict the setting of matching object category attributes for the clarity levels of each detection area. The object type attribute that can be identified in each detection area can be used as the object type attribute matching the clarity level of that detection area. For example, if external attribute features can be identified in a detection area but location attribute features cannot, then the object category attribute matching the clarity level of the detection area can be the external attribute feature. Similarly, if both external attribute features and location attribute features can be identified in a detection area, then the object category attribute matching the clarity level of the detection area can be both the external attribute feature and the location attribute feature.

[0055] Based on the object category attributes that match the clarity of different detection areas, the target objects in each detection area can be identified separately. This allows for the development of different recognition schemes for each detection area, enabling the identification of target objects in multiple detection areas in various ways.

[0056] In some implementations, the object requiring image occlusion can be designated as the target object. In this step, the target object corresponding to each detection region can be identified from among the multiple objects contained in each detection region. And / or, non-target objects (objects that do not require image occlusion) can be identified from among the objects contained in each detection region, and the objects in the detection region other than the non-target objects can be designated as the target objects corresponding to the detection region.

[0057] S13: Based on the clarity of the detection area, at least a portion of the target object in the detection area is occluded.

[0058] Based on the clarity of each detection area, different occlusion methods can be used for the target objects identified in each detection area. For example, if the clarity of a detection area is low and the specific target object cannot be identified, objects similar to the target object can be occluded to avoid privacy leaks. Conversely, if the clarity of a detection area is high and the target object can be accurately identified, a portion of the target object can be occluded.

[0059] In some implementations, occlusion processing of at least a portion of the area may include at least one of the following: occluding an object region containing the target object in the detection area, occluding a preset region containing the target object in the detection area, and occluding a preset detail region containing the target object in the detection area.

[0060] The preset area is a region within the object area containing the target object that has a preset proportion, such as a region with a 1 / 3 height. The preset detail area is a region within the object area containing the target object that has preset parts, such as a face area or a license plate area.

[0061] In some implementations, occlusion processing may include at least one of the following: overlaying mosaic, overlaying color blocks, blurring, low opacity processing, weak encoding, etc. It is understood that the occlusion processing mentioned in this application is not limited to these, and may also include other methods that can occlude or hide at least a part of the target object.

[0062] The following example illustrates image occlusion using a scenario requiring image occlusion processing. Please refer to [link / reference]. Figure 3 For example, in supermarket applications, it's necessary to obscure customers but not staff. Customers within the image frame can be identified as the target objects. Because customers are diverse, staff (not the target objects) can be identified from the multiple objects contained in the image frame. All other objects besides staff (such as customer A and customer B) are then identified as the target objects, and at least a portion of the target objects are obscured.

[0063] In this embodiment, by determining the detection regions where each object in the image frame is located, and the union of the detection regions covers the image frame, and the clarity of objects in different detection regions is different, object category attributes that match the different clarity of objects in different detection regions can be matched according to the different clarity of objects. Different object category attributes can be matched according to different detection regions, thereby using object category attributes that match the clarity of the detection regions to identify target objects within each object in the detection regions. Target objects in different detection regions can be identified in different ways. Then, based on the clarity of the detection regions, at least a portion of the target objects in the detection regions is occluded. Occlusion processing can be performed separately for target objects in detection regions with different clarity levels, ensuring that target objects of each clarity level are occluded, avoiding omission of target objects, and improving the accuracy of image occlusion.

[0064] Please see Figure 4 , Figure 4This is a flowchart illustrating a second embodiment of the image occlusion method of this application. The method may include the following steps:

[0065] S21: Perform target detection on the reference image frame based on multiple target detection methods to obtain the reference object region under each target detection method; wherein, the detection granularity of the reference objects contained in the reference image frame is different for each of the multiple target detection methods.

[0066] In some implementations, before step S11 above, that is, before determining the detection area where each object in the image frame is located, steps S21 to S22 of this embodiment can be performed to divide the reference image frame into multiple detection areas at a preset time period.

[0067] In some implementations, the image frame of the above embodiment can be used as a reference image frame. After dividing the reference image frame into multiple detection regions, the steps S11 to S13 described above are performed on the reference image frame.

[0068] In some embodiments, the reference image frame is a different video image frame from the image frame in the above embodiments. The reference image frame and the image frame can be video image frames obtained by capturing the same target area, or video image frames captured by the same camera device on the same target area. This application does not impose any limitations on this.

[0069] In some implementations, the reference image frame is a video image frame in the target video that is located before the aforementioned image frame, so that after dividing the reference image into multiple detection regions, steps S11 to S13 are executed according to the divided detection regions to perform occlusion processing on the target objects in each detection region of the aforementioned image frame.

[0070] In this step, multiple target detection methods can be used to perform target detection on the reference image frame to obtain the reference object region under each target detection method. The detection granularity of the reference objects contained in the reference image frame varies among the multiple target detection methods. The detection granularity can be the detection pixel requirement. For example, the detection granularity of the multiple target detection methods includes detection pixels greater than 0 pixels (pixels, the smallest independent element that makes up an image), detection pixels greater than 60 pixels, detection pixels greater than 80 pixels, etc.

[0071] In some implementations, multiple target detection methods are used to detect moving targets, the target as a whole, and target details, where the target details are at least a part of the target as a whole.

[0072] The target detection method for detecting moving targets is used to detect objects with dynamic characteristics, or regions where movement occurs. It is understood that the difference between the current image frame and the previous image frame can be used to obtain the region where movement occurs, which can then be considered the moving target, i.e., the region where the moving target is located. The target detection method for detecting moving targets can employ dynamic region algorithms, such as optical flow, and this application does not impose any restrictions on this.

[0073] The target detection method for detecting the entire target is used to detect objects with overall characteristics (such as human characteristics or vehicle characteristics). Alternatively, the target detection method for detecting the entire target is used to further detect objects with overall characteristics based on objects with dynamic characteristics, thereby obtaining the region where the entire object is located. The target detection method for detecting the entire target can employ human detection algorithms or vehicle detection algorithms, etc., and this application does not impose any restrictions on this.

[0074] The target detection method for detecting target details is used to detect the detailed characteristics of an object (e.g., facial features, license plate features). Alternatively, it can be used to detect objects with overall features (e.g., human body features, vehicle body features) and then further detect objects with detailed features (e.g., facial features, license plate features) to obtain the region where the detailed features of the object are located. The target detection method for detecting target details can employ face detection algorithms or license plate recognition algorithms, etc., and this application does not impose any restrictions on this.

[0075] If the technical solution of this application involves personal information, the product using this technical solution has clearly informed the user of the personal information processing rules and obtained the user's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using this technical solution has obtained the user's separate consent before processing the sensitive personal information, and also meets the requirement of "express consent". For example, at personal information collection devices such as camera equipment, a clear and prominent sign is set up to inform the user that they have entered the scope of personal information collection and that personal information will be collected. If the user voluntarily enters the collection scope, it is deemed that they have agreed to the collection of their personal information; or on the personal information processing device, the personal information processing rules are clearly informed through signs / information, and the user's authorization is obtained through pop-up information or by asking the user to upload their personal information. The personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the types of personal information processed.

[0076] In some implementations, taking a person as an example, the various target detection methods described above can be illustrated in the table below:

[0077] Dynamic Region Algorithm >0 pixels Detecting moving targets Human detection algorithm >60 pixels Overall detection target Face detection algorithm >80 pixels Detect target details

[0078] Table 1. Multiple Target Detection Methods

[0079] In some implementations, please refer to Figure 5 Multiple target detection methods detect moving targets, overall targets, and details of the same or different objects in a reference image frame. In other words, for the same object, at least one of these three aspects can be detected. In some applications, the detected moving targets, overall targets, and details are displayed or not displayed in the reference image frame as detection boxes. The detection boxes can represent the area where the object is located, i.e., the reference object area. Furthermore, the area of ​​the reference object area can represent the size of the object in the reference image frame.

[0080] S22: Based on the distribution of the reference object region under various target detection methods, multiple detection regions are obtained.

[0081] Various object detection methods can be used to detect the reference object region where the object is located in the reference image frame, and multiple detection regions can be obtained based on the distribution of the reference object region in the reference image frame.

[0082] In some implementations, please refer to Figure 6 The above step S22 in this embodiment may include the following steps:

[0083] S221: Obtain the distribution of the reference object region in a preset direction of the reference image frame under various target detection methods, wherein the distribution includes the number of detected reference object regions and / or the detection area of ​​the reference object regions.

[0084] Among them, the reference object region under various target detection methods can include at least one of moving targets, the target as a whole, and target details.

[0085] Taking the detection of a person as an example of the above-mentioned target detection methods, the dynamic region algorithm has a small detection granularity and strong detection capability, and can detect small targets at a distance, that is, targets with poor clarity. Therefore, the number of moving targets detected in the reference image frame is relatively large, but it cannot distinguish between the target as a whole and the target details.

[0086] Because human detection algorithms have a larger detection granularity than moving target detection algorithms, they can perform human detection at closer ranges, meaning they can detect objects with higher clarity than moving targets, thus identifying the entire object. In subsequent occlusion processing, these objects can have their overall attribute characteristics identified, such as clothing attributes, allowing for the identification of objects with specific attribute characteristics, such as company uniforms or standardized work clothes.

[0087] Because face detection algorithms have a large detection granularity, they can perform high-resolution detection, such as close-up face target recognition, which means identifying target details. In subsequent occlusion processing, this type of object can be identified based on the target details.

[0088] It can obtain the distribution of moving targets, the target as a whole, and target details in a preset direction in a reference image frame. The distribution includes the number of detections in the reference object region and / or the detection area of ​​the reference object region.

[0089] In some implementations, please refer to Figure 7 The distribution of the number of detected objects in a preset direction within a reference image frame, including moving targets, overall targets, and target details. The preset direction can be the direction from farthest to closest objects in the reference image frame; or, the preset direction can be a pre-input reference direction, such as the Y-axis direction of the reference image frame; or, the preset direction can be the extension direction of a preset region, such as the direction along the road in a road detection scenario. This application does not impose any restrictions on this.

[0090] S222: Determine the boundaries of multiple detection areas by utilizing the distribution of preset directions.

[0091] Based on the distribution of the number of detections of moving targets, the target as a whole, and the target details in a preset direction of the reference image frame, and / or the distribution of the detection area of ​​the reference object region, the boundaries of multiple detection regions in the reference image frame can be determined. When performing image occlusion processing in subsequent image frames, occlusion processing can be performed separately according to these multiple detection regions.

[0092] In some implementations, the area boundary can be determined by at least one of the following methods.

[0093] In response to the target as a whole reaching a first region boundary requirement in a preset direction, a first boundary position is determined in the preset direction where the target as a whole reaches the first region boundary requirement. This is the position along the preset direction in the reference image frame; for example, if the preset direction is the Y-axis, then the first boundary position is a certain position on the Y-axis. This first boundary position is used as the boundary between the first detection region and the second detection region. The clarity of the reference object in the first detection region is less than that in the second detection region. The first region boundary requirement includes at least one of the following: the number of detected targets is greater than a first quantity threshold, or the detected area of ​​the target as a whole meets a first area threshold condition.

[0094] In response to the target detail reaching the second region boundary requirement in a preset direction, a second boundary position is determined in the preset direction where the target detail reaches the second region boundary requirement, and this second boundary position is used as the region boundary between the second detection region and the third detection region. Specifically, the sharpness of the reference object in the second detection region is less than that in the third detection region; the second region boundary requirement includes at least one of the following: the number of detected target details is greater than a second number threshold, or the detected area of ​​the target detail meets a second area threshold condition.

[0095] As an example, let's take the Y-axis of the image frame with a preset direction as the reference. The region boundaries of each detection area can be obtained using the following formula:

[0096]

[0097] Among them, [Y m ,Y n ) represents the boundary between detection region m and detection region n. md Y represents the boundary between the first and second detection areas. body Y represents the boundary between the second and third detection areas. face This represents the maximum value of the Y-axis in the reference image frame, which is also the height of the reference image frame. N body N represents the total number of detections for the target. face The number of target details detected is represented by N1, which is the first threshold value, and N2 is the second threshold value. W*H represents the width and height of the reference object region (e.g., moving target, target as a whole, target details), which is also the detection area of ​​the reference object region. The first area threshold condition is [60 pixels, 80 pixels), and the second area threshold condition is [80 pixels, T), where T is the maximum area of ​​the reference image frame or the set maximum detection area, which is not limited in this application.

[0098] S223: Based on region delineation, multiple detection regions are obtained by dividing the reference image frame.

[0099] Based on the region boundaries determined by the above method, for example, multiple detection regions can be obtained by dividing the reference image frame according to the above formula (1). The multiple detection regions may include a first detection region, a second detection region, and a third detection region.

[0100] Among them, the first detection region [0, Y] md In the first detection area, the detection area of ​​the reference object region is within the range of [0 pixels, 60 pixels). The clarity of this first detection area is to distinguish moving targets. It can be understood that the first detection area can be represented as the area where the object is far away.

[0101] Second detection area [Y] md ,Y body In the second detection region, the detection area of ​​the reference object region is within the range of [60 pixels, 80 pixels). This second detection region corresponds to a level of clarity that can distinguish the entire target, including its overall features and moving targets. It can be understood that the second detection region can be represented as the region within the object's distance range.

[0102] Third detection area [Y] body ,Y face In the third detection region, the detection area of ​​the reference object region is within the range of [80 pixels, H). This third detection region corresponds to the level of clarity required to distinguish target details. In addition to distinguishing moving targets and overall features, it can also distinguish detailed features. It can be understood that the third detection region can be represented as the region where the object is close.

[0103] As an example, please refer to Figure 8 The reference image frame is divided into three detection regions: a first detection region, a second detection region, and a third detection region. The sharpness of objects varies in each region. Furthermore, moving targets can be detected in the first detection region, moving targets and the target as a whole can be detected in the second detection region, and moving targets, the target as a whole, and target details can be detected in the third detection region.

[0104] In this embodiment, target detection is performed on the reference image frame using multiple target detection methods to obtain reference object regions under each method. The detection granularity of the reference objects in the reference image frame varies among the multiple target detection methods. Based on the distribution of reference object regions under each target detection method, multiple detection regions are obtained. Multiple detection regions can be divided based on different levels of clarity. This method can automatically divide the reference image frame into multiple detection regions by analyzing the data of the detection results of multiple target detection methods, so that subsequent image occlusion processing can be performed separately on the multiple detection regions.

[0105] After dividing the multiple detection regions based on the above embodiments, steps S11 to S13 can be executed to perform occlusion processing on the image. Steps S12 to S13 are further extended below, and specific examples are provided in the following embodiments.

[0106] Please see Figure 9 , Figure 9 This is a flowchart illustrating a third embodiment of the image occlusion method of this application. For the first detection region, the method may include the following steps:

[0107] S31: Perform moving target detection in the first detection area and identify the moving targets contained in the first detection area.

[0108] The first detection area corresponds to the clarity level of distinguishing moving targets; that is, the first detection area can only distinguish moving targets, i.e., targets that are in motion. It can detect "microscopic moving targets" in the image frame, such as people, which may or may not be people. Understandably, the first detection area identifies more moving targets than other detection areas.

[0109] Moving target detection is performed in the first detection area to identify the moving targets contained in the first detection area. Since the first detection area can only detect moving targets, the object category attribute that matches the clarity level of the first detection area is set to include moving targets.

[0110] S32: The moving targets contained in the first detection area are taken as the target objects of the first detection area.

[0111] Since the first detection area cannot distribute the overall features, attribute features, and detailed features of the target object, nor can it identify whether the moving target is a person, all moving targets identified by the first detection area can be regarded as the target objects of the first detection area, that is, the objects that need to be occluded.

[0112] S33: In response to the clarity of the detection area, to distinguish the moving target, the object area containing the target object in the detection area is occluded.

[0113] Please see Figure 10 The object region containing the target object can be selected from the first detection region, that is, the object region in the first detection region can be directly used as the region that needs to be occluded, and the object region of the target object can be occluded.

[0114] In this embodiment, since the first detection area can only detect moving targets, the number of pixels of the target in the image is small, for example, only 0% to 20% of the vertical proportion of the image (this is just an example and is not explicitly stated to be a fixed proportion), it is almost impossible to distinguish the whole object. By taking all the detected moving targets as the target objects, the target objects can be occluded first to avoid the leakage of relevant information of target objects in distant areas.

[0115] Please see Figure 11 , Figure 11 This is a flowchart illustrating the fourth embodiment of the image occlusion method of this application. For the second detection region, the method may include the following steps:

[0116] S41: Obtain the object category attribute that matches the clarity level of the objects contained in each detection area.

[0117] By performing attribute detection on the object regions of each object in each detection region, the object category attributes corresponding to the clarity level of the objects contained in each detection region can be obtained.

[0118] The second detection area corresponds to a level of clarity that distinguishes the entire target, meaning that in addition to recognizing movement, it can also identify the overall features of the object. The object category attribute matching the level of clarity of the second detection area includes a first attribute feature that distinguishes the entire target. This object category attribute refers to attribute features that can be identified by distinguishing the entire target. For example, the first attribute feature can be a motion attribute feature, an external attribute feature, etc. It is understood that the first attribute feature can also be other attribute features that distinguish the entire target, and this application does not limit this. For example, taking a person as an example, the first attribute feature can be human body features, decorative features, etc.

[0119] S42: Compare the object category attributes corresponding to the objects contained in the detection area with the preset object category attributes, and identify the target object from each object in the detection area.

[0120] The first attribute features of the object contained in the second detection region are compared with preset first attribute features to obtain a first comparison result. The preset first attribute features are attribute features of a preset target that needs to be occluded, such as decorative features (e.g., clothing attributes). The decorative features of the object in the identified second detection region are compared with the decorative features of the preset target (i.e., the preset first attribute features) to obtain a first comparison result. If the comparison similarity is greater than a preset similarity threshold, the first comparison result can be considered consistent.

[0121] In some implementations, if the preset target has a first attribute feature, such as a decorative feature, like work clothes (e.g., shop assistant, chef, firefighter uniform, school uniform), or the clothing features of the preset target, the target object can be roughly determined based on the first attribute feature of the preset target to eliminate the possibility of missing the target object and improve the accuracy of occlusion processing of the target object.

[0122] Based on the first comparison result, the target object is identified from each object in the second detection region. There are two possible scenarios: First, if the target is one that requires occlusion processing, then objects whose first comparison result matches are considered target objects in the second detection region. Second, if the target does not require occlusion processing, then objects whose first comparison result does not match are considered target objects in the second detection region. This method can identify the target object within each object in the second detection region.

[0123] Taking a human as an example, the number of targets detected in the second detection region is less than the number of moving targets detected in the first detection region using the dynamic region detection algorithm. However, the overall target detection in the second detection region provides a more accurate identification of the target object. Through overall target detection, it is possible to determine with a high probability that the object is a human target (e.g., 95% certainty). Simultaneously, based on the decorative attribute features built into the human detection algorithm, it is possible to distinguish between some "target people" and "non-target people," thus excluding some other individuals and adjusting the privacy protection strategy for image occlusion processing.

[0124] S43: In response to the clarity of the detection area and to distinguish the target as a whole, a preset area containing the target object in the detection area is occluded; wherein, the preset area is a region in the object area containing the target object with a preset proportion.

[0125] Please see Figure 12 Taking a person as an example, the clarity of the overall target detected in the second detection area is defined as the overall target resolution, ranging from 20% to 45% of the vertical proportion of the image frame (this is just an example and not a fixed proportion). The pixels of the target object in this second detection area generally meet the detection range for "human features," but may not meet the pixel detection range for face detection and recognition, thus failing to identify the specified preset target person. Therefore, identification can be performed based on decorative attribute features. The preset area containing the target object in the second detection area is then occluded; this preset area is a region within the object area containing the target object that is within a preset proportion, for example, one-third of the target object's height. This allows for image occlusion of target objects of a certain identity to protect privacy.

[0126] Please see Figure 13 , Figure 13 This is a flowchart illustrating the fifth embodiment of the image occlusion method of this application. For the second detection region, the method may include the following steps:

[0127] S51: Obtain the object category attribute that matches the clarity level of the objects contained in each detection area.

[0128] By performing attribute detection on the object regions of each object in each detection region, the object category attributes corresponding to the clarity level of the objects contained in each detection region can be obtained.

[0129] The third detection area corresponds to a clarity level of distinguishing target details. Since distinguishing target details can identify both the overall and detailed features of an object, the object category attribute matching the clarity level of the third detection area includes at least one of a first attribute feature and a second attribute feature for distinguishing target details. The first attribute feature can refer to the first attribute features for distinguishing the overall target, such as human body features or clothing features. The second attribute feature can be an attribute feature that can be identified by distinguishing target details, and this attribute feature is different from the first attribute feature. For example, the second attribute feature is a facial feature, ear feature, or head and shoulder feature. This application uses decorative features as the first attribute feature and facial features as the second attribute feature as an example for illustration, but this application is not limited to this. For example, face detection and recognition analysis can be performed on objects in the third detection area to detect the accurate facial information of objects in the third detection area image, which is then compared and recognized with the preset second attribute feature.

[0130] S52: Compare the object category attributes corresponding to the objects contained in the detection area with the preset object category attributes, and identify the target object from each object in the detection area.

[0131] Because the third detection region can identify both overall and detailed targets—for example, it has a larger detection granularity (e.g., 80 pixels)—it can detect clearer images and more detailed information, and the number of detailed targets detected is less than the number of overall targets detected in the second detection region described above. The third detection region can also acquire the object's first and second attribute features. Therefore, the target object in the third detection region can be identified using the first and / or second attribute features, and the correlation between them. The correlation between the first and second attribute features can be represented as attribute features belonging to the same preset target.

[0132] In some implementations, the first attribute features of the objects contained in the third detection region can be compared with preset first attribute features to obtain a second comparison result. Based on the second comparison result, the target object is identified from each object in the third detection region. There are two possible scenarios: First, if the preset target is a target that needs occlusion processing, then objects whose second comparison results match are considered target objects in the third detection region. Second, if the preset target is a target that does not need occlusion processing, then objects whose second comparison results do not match are considered target objects in the third detection region. This method can identify the target object within each object in the third detection region.

[0133] In some implementations, the second attribute features of the objects contained in the third detection region are compared with preset second attribute features to obtain a third comparison result; based on the third comparison result, the target object is identified from each object in the third detection region. This comparison method can refer to the comparison of the first attribute features described above, and will not be elaborated here.

[0134] The second attribute feature can be the object's part attribute feature (such as facial features). Based on the facial features identified by the facial recognition algorithm, it can directly and accurately distinguish between target personnel and non-target personnel, so as to accurately exclude other personnel and determine the target object that needs image occlusion processing for privacy protection strategies.

[0135] In some implementations, in the third detection area, that is, in the near area, the face detection and recognition algorithm requires relatively clear facial information. Facial features may not be detected due to actions such as people looking down or turning their heads. The algorithm can combine the above-mentioned overall target and first attribute features to identify the target object by associating facial features, overall target and first attribute features.

[0136] In some implementations, the first attribute features can be compared first, and then the target object selected based on the first attribute features can be compared with the second attribute features to identify the final target object. Alternatively, the first attribute features and the second attribute features can be compared simultaneously, and the intersection or union of the target objects identified by the two comparison methods can be taken as the target object. The choice can be made based on the specific application scenario, and this application does not impose any restrictions on it.

[0137] S53: In response to the clarity level corresponding to the detection area, in order to distinguish the target details, a preset detail area containing the target object in the detection area is occluded; wherein, the preset detail area is a preset part of the object area containing the target object.

[0138] Please see Figure 14 In the third detection region, the object area is relatively large, and the detected and distinguishable target details account for 45% to 100% of the vertical proportion of the image frame (this is only an example and is not explicitly stated to be a fixed proportion). The pixels of the target object in this region can meet the detection and recognition range of the second attribute feature, and can identify the target person of interest or the target person who needs image occlusion processing, thus achieving image occlusion processing for a target object. Specifically, the third detection region contains a preset detail area of ​​the target object, which is occluded; the preset detail area is a region of a preset part in the object region containing the target object, or the preset detail area can be the region to which the second attribute feature belongs, for example, if the preset part is the face, the preset detail area is the region containing the face.

[0139] As an example, please refer to Figure 15 Image occlusion processing can be performed on target objects in each detection area of ​​the image frame in the manner described above. For example, attribute features such as human body, clothing, and face can be used to perform image occlusion processing on part of the people in each detection area of ​​the image frame.

[0140] In accordance with the above embodiments, this application provides an image processing apparatus. Please refer to... Figure 16 , Figure 16 This is a schematic diagram of an embodiment of the image occlusion device of this application. The image processing device 30 includes a region module 61, a recognition module 62, and an occlusion module 63.

[0141] The region module 61 is used to determine the detection region where each object in the image frame is located. The union of the detection regions covers the image frame, and the clarity of the objects in different detection regions is different.

[0142] The recognition module 62 is used to identify target objects within each object in the detection area by utilizing object category attributes that match the clarity level corresponding to the detection area.

[0143] The occlusion module 63 is used to occlude at least a portion of the target object in the detection area based on the clarity of the detection area.

[0144] The specific implementation of this embodiment can be referred to the implementation process of the above embodiments, and will not be repeated here.

[0145] Regarding the above embodiments, this application provides a computer device; please refer to [link / reference]. Figure 17 , Figure 13 This is a schematic diagram of the structure of a computer device according to an embodiment of the present application. The computer device 70 includes a memory 71 and a processor 72, wherein the memory 71 and the processor 72 are coupled to each other. The memory 71 stores program data, and the processor 72 is used to execute the program data to implement the steps of any embodiment of the image occlusion method described above.

[0146] In this embodiment, processor 72 can also be referred to as a CPU (Central Processing Unit). Processor 72 may be an integrated circuit chip with signal processing capabilities. Processor 72 can also be a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component. The general-purpose processor can be a microprocessor, or processor 72 can be any conventional processor.

[0147] The methods described in the above embodiments can be implemented as computer programs; therefore, this application proposes a computer-readable storage medium. Please refer to [link to relevant documentation]. Figure 18 , Figure 18 This is a schematic diagram of a computer-readable storage medium according to an embodiment of the present application. The computer-readable storage medium 80 stores program data 81 that can be executed by a processor. The program data 81 can be executed by the processor to implement the steps of any embodiment of the image occlusion method described above.

[0148] In this embodiment, the computer-readable storage medium 80 can be a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, or it can be a server that stores the program data 81. The server can send the stored program data 81 to other devices for execution, or it can run the stored program data 81 itself.

[0149] In the several embodiments provided in this application, it should be understood that the disclosed methods and apparatus can be implemented in other ways. For example, the apparatus implementations described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0150] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0151] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0152] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application.

[0153] Obviously, those skilled in the art should understand that the modules or steps of this application described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, and thus stored in a computer-readable storage medium for execution by a computing device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Therefore, this application is not limited to any particular hardware and software combination.

[0154] The above description is merely an embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. An image occlusion method, characterized by, The method comprises: performing target detection on a reference image frame based on a plurality of target detection manners respectively to obtain reference object regions under various target detection manners; wherein the detection granularity of a reference object contained in the reference image frame is different for the plurality of target detection manners; dividing a plurality of detection regions based on the distribution of the reference object regions under various target detection manners; determining detection regions in which each object in an image frame is located, wherein the union of the detection regions covers the image frame, and the clarity of the objects in different detection regions is different; identifying a target object in each object in the detection region using an object category attribute that matches the clarity corresponding to the detection region; performing occlusion processing on at least part of the target object in the detection region based on the clarity corresponding to the detection region.

2. The method of claim 1, wherein, The determination of the detection regions in which each object in the image frame is located comprises: performing target detection based on the image frame to obtain a target position of the object in the image frame; determining the detection region in which the object is located based on the target position of the object.

3. The method of claim 1, wherein, The division of a plurality of detection regions based on the distribution of the reference object regions under various target detection manners comprises: obtaining the distribution of the reference object regions in a preset direction of the reference image frame under various target detection manners, wherein the distribution comprises the number of detections of the reference object regions and / or the detection area of the reference object regions; determining the region boundary of a plurality of detection regions using the distribution in the preset direction; dividing a plurality of detection regions based on the region boundary.

4. The method of claim 3, wherein, The plurality of target detection manners are respectively used to detect a moving object, an object as a whole, and object details, and the determination of the region boundary of a plurality of detection regions using the distribution in the preset direction comprises at least one of the following: in response to the object as a whole meeting a first region boundary requirement in the preset direction, determining a first boundary position at which the object as a whole meets the first region boundary requirement in the preset direction, and using the first boundary position as the region boundary of a first detection region and a second detection region; in response to the object details meeting a second region boundary requirement in the preset direction, determining a second boundary position at which the object details meet the second region boundary requirement in the preset direction, and using the second boundary position as the region boundary of the second detection region and a third detection region; wherein the clarity of the reference object in the first detection region is less than the clarity of the reference object in the second detection region, and the clarity of the reference object in the second detection region is less than the clarity of the reference object in the third detection region. The first region boundary requirement comprises at least one of the following: a detection number of the target whole is greater than a first number threshold, and a detection area of the target whole satisfies a first area threshold condition; and the second region boundary requirement comprises at least one of the following: a detection number of the target detail is greater than a second number threshold, and a detection area of the target detail satisfies a second area threshold condition.

5. The method of claim 1, wherein, The detection region comprises a first detection region, the first detection region corresponds to the resolution motion target, and an object category attribute matched with the resolution of the first detection region comprises a motion target; The target object is identified in each object in the detection region by using the object category attribute matched with the resolution of the detection region, and the identification comprises: The motion target detection is performed on the first detection region, and the motion target contained in the first detection region is identified. The motion target contained in the first detection region is taken as the target object of the first detection region.

6. The method of claim 1, wherein, The detection region comprises a second detection region and / or a third detection region; The target object is identified in each object in the detection region by using the object category attribute matched with the resolution of the detection region, and the identification comprises: The object category attribute matched with the resolution of the object contained in each detection region is obtained. The object category attribute of the object contained in the detection region is compared with a preset object category attribute, and the target object is identified in each object in the detection region.

7. The method of claim 6, wherein, The detection region comprises the second detection region, the second detection region corresponds to the resolution target whole, and an object category attribute matched with the resolution of the second detection region comprises a first attribute feature of the resolution target whole; The object category attribute of the object contained in the detection region is compared with a preset object category attribute, and the target object is identified in each object in the detection region, and the identification comprises: The first attribute feature of the object contained in the second detection region is compared with a preset first attribute feature, and a first comparison result is obtained. The target object is identified in each object in the second detection region based on the first comparison result.

8. The method of claim 6, wherein, The detection region comprises the third detection region, the third detection region corresponds to the resolution target detail, and an object category attribute matched with the resolution of the third detection region comprises at least one of a first attribute feature and a second attribute feature of the resolution target detail; The object category attribute of the object contained in the detection region is compared with a preset object category attribute, and the target object is identified in each object in the detection region, and the identification comprises: The first attribute feature of the object contained in the third detection region is compared with a preset first attribute feature, and a second comparison result is obtained; the target object is identified in each object in the third detection region based on the second comparison result; and / or, The second attribute feature of the object contained in the third detection area is compared with a preset second attribute feature, and a third comparison result is obtained; and based on the third comparison result, a target object is identified from each object in the third detection area.

9. The method of claim 1, wherein, The at least partial region of the target object in the detection area is subjected to the occlusion processing based on the clarity degree corresponding to the detection area, including at least one of: In response to the clarity degree corresponding to the detection area being resolution of a moving target, an object region containing the target object in the detection area is subjected to the occlusion processing; In response to the clarity degree corresponding to the detection area being resolution of a target as a whole, a preset region containing the target object in the detection area is subjected to the occlusion processing; wherein the preset region is a region of a preset proportion in the object region containing the target object; In response to the clarity degree corresponding to the detection area being resolution of target details, a preset detail region containing the target object in the detection area is subjected to the occlusion processing; wherein the preset detail region is a region of a preset part in the object region containing the target object.

10. A computer device, comprising: The memory and the processor are coupled to each other, the memory stores program data, and the processor is configured to execute the program data to implement the steps of the method in any one of claims 1 to 9.

11. A computer readable storage medium, characterized in that, The memory stores program data capable of being executed by the processor, and the program data is configured to implement the steps of the method in any one of claims 1 to 9.

Images