Camera monitoring method and device

Abstract

This application provides a camera monitoring method and apparatus. The camera monitoring method includes: when a trigger condition for entering a privacy mode is triggered, adjusting the camera's hardware parameters to preset hardware parameters, wherein the image captured by the camera under the preset hardware parameters has lower clarity than the image captured by the camera in a focused state; and using the camera with adjusted hardware parameters to monitor a target scene in privacy mode. This application adjusts the camera's hardware parameters and then monitors when the trigger condition for entering a privacy mode is triggered. Monitoring in privacy mode yields a blurrier image than the image captured in a focused state. Because the adjustment is to the camera's hardware parameters, it is impossible to reconstruct the image captured by the camera in privacy mode, thereby improving the camera's ability to protect privacy.

Description

Technical Field

[0001] This application mainly relates to the field of display technology, and specifically to a camera monitoring method and device. Background Technology

[0002] With the development of smart homes, indoor cameras are increasingly being used, such as adding cameras to electric fans / air conditioners to detect human positions, automatically adjust airflow, and even perform indoor surveillance. However, as privacy concerns grow, the application of indoor cameras faces challenges. Thermal imaging, pyroelectric motion sensors, and radar solutions can address privacy issues to some extent. However, thermal imaging and radar solutions are costly, motion sensors can only detect human movement and cannot distinguish between humans and animals, and are susceptible to temperature-related factors. Some solutions use mechanical obstruction, but this renders the camera ineffective. Traditional camera applications operate in a focused state, producing a clear image. This clear image is highly susceptible to privacy leaks. While some applications use blurring or pixelation techniques to blur the image, this pixelation is still a processing of the clear image, making it vulnerable to leaks. Furthermore, the original clear image can be restored through various methods, reducing the camera's ability to protect privacy.

[0003] In other words, existing technologies for camera surveillance have a relatively weak ability to protect privacy. Summary of the Invention

[0004] This application provides a camera monitoring method and apparatus, which aims to solve the problem that the privacy protection capability of camera monitoring in the prior art is weak.

[0005] Firstly, this application provides a camera monitoring method, the camera monitoring method comprising: When the conditions for entering privacy mode are triggered, the hardware parameters of the camera are adjusted to preset hardware parameters, wherein the image captured by the camera under the preset hardware parameters has a lower resolution than the image captured by the camera in focus mode. The target scene is monitored in privacy mode using a camera with the hardware parameters adjusted.

[0006] Optionally, the preset hardware parameters include the target position of the lens, and adjusting the camera's hardware parameters to the preset hardware parameters when the trigger condition for entering privacy mode is triggered includes: When the conditions for entering privacy mode are triggered, the camera lens is moved to the target position, wherein the target position is different from the lens position of the camera in focus state.

[0007] Optionally, the preset hardware parameters include a preset focal length of the lens, the lens being made of metamaterials, and adjusting the camera's hardware parameters to the preset hardware parameters when the trigger condition for entering privacy mode is triggered includes: When the conditions for entering privacy mode are triggered, the focal length of the camera is adjusted to a preset focal length, wherein the preset focal length is different from the focal length of the camera in the focused state.

[0008] Optionally, before moving the camera lens to the target location when the trigger condition for entering privacy mode is triggered, the following steps are included: Acquire the first scene image of the target scene captured by the camera in focus; Perform facial recognition on the first scene image; The identified facial images are matched with pre-stored user facial images; When a target face image matching the user's face image exists in the first scene image, the trigger condition for entering privacy mode is determined.

[0009] Optionally, moving the camera lens to the target location when the trigger condition for entering privacy mode is triggered includes: When the conditions for entering privacy mode are triggered, the current object distance between the target face image in the target scene and the camera is obtained; The current depth of field range is determined based on the current object distance and the preset object distance depth-of-field mapping relationship; The initial moving position is determined based on the current depth of field range, wherein the depth of field range of the camera lens at the initial moving position does not coincide with the current depth of field range; Determine the target location based on the initial movement position.

[0010] Optionally, determining the target location based on the initial movement position includes: Acquire a second scene image captured by the camera at its initial moving position; Determine whether the sharpness of the second scene image exceeds a preset value; If the clarity of the second scene image does not exceed a preset value, then the initial movement position is determined as the target position.

[0011] Optionally, determining the initial movement position as the target position if the clarity of the second scene image does not exceed a preset value includes: If the clarity of the second scene image does not exceed the preset value, then face recognition is performed; If face recognition fails, the initial movement position will be set as the target position.

[0012] Optionally, the camera monitoring method further includes: Acquire a fourth scene image captured by the camera at the target location; Perform human contour recognition on the fourth scene image to obtain the number of human contours in the fourth scene image; Perform human contour recognition on the first scene image to obtain the number of human contours in the first scene image; Determine whether the number of human body outlines in the fourth scene image is the same as the number of human body outlines in the first scene image; If the number of human figures in the fourth scene image is different from the number of human figures in the first scene image, then exit privacy mode.

[0013] Secondly, this application provides a camera monitoring device, the camera monitoring device comprising: The parameter adjustment unit is used to adjust the hardware parameters of the camera to preset hardware parameters when the triggering condition for entering the privacy mode is triggered, wherein the image captured by the camera under the preset hardware parameters has a lower resolution than the image captured by the camera in the focus state. The monitoring unit is used to monitor the target scene in privacy mode using a camera with adjusted hardware parameters.

[0014] Optionally, the preset hardware parameters include the target position of the lens, and the parameter adjustment unit is used for: When the conditions for entering privacy mode are triggered, the camera lens is moved to the target position, wherein the target position is different from the lens position of the camera in focus state.

[0015] Optionally, the preset hardware parameters include the preset focal length of the lens, the lens being made of metamaterials, and the parameter adjustment unit being used for: When the conditions for entering privacy mode are triggered, the focal length of the camera is adjusted to a preset focal length, wherein the preset focal length is different from the focal length of the camera in the focused state.

[0016] Optionally, the parameter adjustment unit is used for: Acquire the first scene image of the target scene captured by the camera in focus; Perform facial recognition on the first scene image; The identified facial images are matched with pre-stored user facial images; When a target face image matching the user's face image exists in the first scene image, the trigger condition for entering privacy mode is determined.

[0017] Optionally, the parameter adjustment unit is used for: When the conditions for entering privacy mode are triggered, the current object distance between the target face image in the target scene and the camera is obtained; The current depth of field range is determined based on the current object distance and the preset object distance depth-of-field mapping relationship; The initial moving position is determined based on the current depth of field range, wherein the depth of field range of the camera lens at the initial moving position does not coincide with the current depth of field range; Determine the target location based on the initial movement position.

[0018] Optionally, the parameter adjustment unit is used for: Acquire a second scene image captured by the camera at its initial moving position; Determine whether the sharpness of the second scene image exceeds a preset value; If the clarity of the second scene image does not exceed a preset value, then the initial movement position is determined as the target position.

[0019] Optionally, the parameter adjustment unit is used for: If the clarity of the second scene image does not exceed the preset value, then face recognition is performed; If face recognition fails, the initial movement position will be set as the target position.

[0020] Optionally, the monitoring unit is used for: Acquire a fourth scene image captured by the camera at the target location; Perform human contour recognition on the fourth scene image to obtain the number of human contours in the fourth scene image; Perform human contour recognition on the first scene image to obtain the number of human contours in the first scene image; Determine whether the number of human body outlines in the fourth scene image is the same as the number of human body outlines in the first scene image; If the number of human figures in the fourth scene image is different from the number of human figures in the first scene image, then exit privacy mode.

[0021] Thirdly, this application provides a smart device, the smart device comprising: One or more processors; Memory; and One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement the camera monitoring method described in any one of the first aspects.

[0022] Fourthly, this application provides a computer-readable storage medium storing a plurality of instructions adapted for loading by a processor to perform the steps of the camera monitoring method described in any one of the first aspects.

[0023] This application provides a camera monitoring method and apparatus. The camera monitoring method includes: when a trigger condition for entering a privacy mode is triggered, adjusting the camera's hardware parameters to preset hardware parameters, wherein the image captured by the camera under the preset hardware parameters has lower clarity than the image captured by the camera in a focused state; and using the camera with adjusted hardware parameters to monitor a target scene in privacy mode. This application adjusts the camera's hardware parameters and then monitors when the trigger condition for entering a privacy mode is triggered. Monitoring in privacy mode yields a blurrier image than the image captured in a focused state. Because the adjustment is to the camera's hardware parameters, it is impossible to reconstruct the image captured by the camera in privacy mode, thereby improving the camera's ability to protect privacy. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 A schematic diagram of a scenario for the control system of the display screen provided in an embodiment of this application; Figure 2 This is a schematic flowchart of an embodiment of the camera monitoring method provided in this application. Figure 3 This is a flowchart illustrating the process before moving the camera lens to the target position when a trigger condition for entering privacy mode is detected in one embodiment of the camera monitoring method provided in this application. Figure 4 This is a schematic diagram of the process of moving the camera lens to the target position when a trigger condition for entering privacy mode is detected in one embodiment of the camera monitoring method provided in this application. Figure 5 This is a diagram illustrating the depth of field of the camera; Figure 6 This is a diagram illustrating the lens adjustment parameters of a camera. Figure 7 This is a schematic diagram of an embodiment of the camera monitoring device provided in this application. Figure 8This is a schematic diagram of an embodiment of the smart device provided in this application. Detailed Implementation

[0026] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0027] In the description of this application, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientation or positional relationships based on the orientation or positional relationships shown in the accompanying drawings, are used only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include one or more features. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.

[0028] In this application, the term "exemplary" is used to mean "used as an example, illustration, or description." Any embodiment described as "exemplary" in this application is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use this application. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that this application can be made without using these specific details. In other instances, well-known structures and processes are not described in detail to avoid obscuring the description of this application with unnecessary detail. Therefore, this application is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed in this application.

[0029] This application provides a camera monitoring method and apparatus, which will be described in detail below.

[0030] Please see Figure 1 , Figure 1This is a schematic diagram of a camera monitoring system provided in an embodiment of this application. The camera monitoring system may include a smart device 100, which integrates a camera monitoring device.

[0031] In this embodiment, the smart device 100 can be an independent server, a server network, or a server cluster. For example, the smart device 100 described in this embodiment includes, but is not limited to, a computer, a network host, a single network server, a set of multiple network servers, or a cloud server composed of multiple servers. The cloud server is composed of a large number of computers or network servers based on cloud computing.

[0032] In this embodiment, the aforementioned smart device 100 can be a general-purpose smart device or a dedicated smart device. In specific implementations, the smart device 100 can be a desktop computer, a portable computer, a network server, a PDA (Personal Digital Assistant), a mobile phone, a tablet computer, a wireless terminal device, a communication device, an embedded device, a car, an air conditioner, etc. This embodiment does not limit the type of smart device 100.

[0033] Those skilled in the art will understand that Figure 1 The application environment shown is merely one application scenario of the solution in this application and does not constitute a limitation on the application scenario of the solution in this application. Other application environments may include more than one application scenario. Figure 1 The number of smart devices shown is more or less, for example Figure 1 Only one smart device is shown in the image. It is understood that the camera monitoring system may also include one or more other smart devices capable of processing data, which are not specifically limited here.

[0034] In addition, such as Figure 1 As shown, the camera monitoring system may also include a memory 200 for storing data.

[0035] It should be noted that, Figure 1 The schematic diagram of the camera monitoring system shown is merely an example. The camera monitoring system and scenario described in this application are intended to more clearly illustrate the technical solutions of this application and do not constitute a limitation on the technical solutions provided in this application. As those skilled in the art will know, with the evolution of camera monitoring systems and the emergence of new business scenarios, the technical solutions provided in this application are also applicable to similar technical problems.

[0036] First, this application provides a camera monitoring method, which includes: when a trigger condition for entering a privacy mode is triggered, adjusting the camera's hardware parameters to preset hardware parameters, wherein the image captured by the camera under the preset hardware parameters has a lower resolution than the image captured by the camera in a focused state; and using the camera with adjusted hardware parameters to monitor the target scene in privacy mode.

[0037] like Figure 2 As shown, Figure 2 This is a schematic flowchart of an embodiment of the camera monitoring method provided in this application, which includes the following steps S201~S202: S201. When the trigger condition for entering privacy mode is triggered, the camera's hardware parameters are adjusted to preset hardware parameters, wherein the image clarity of the camera under the preset hardware parameters is lower than the image clarity of the camera under the focus state.

[0038] Image sharpness refers to the clarity of details and boundaries in an image.

[0039] In-focus mode, when a photograph is taken, the subject is clearly imaged. This is achieved through a series of focusing actions, resulting in a successfully formed image on the image sensor. Simply put, it means the image is in focus and sharp. There is usually a visual or audible notification when the image is in focus. Focusing, also known as autofocus, is the process of adjusting the focal distance using a camera. Digital cameras typically offer several focusing modes: autofocus, manual focus, and multiple focus modes. Focusing is also called adjusting the distance between the subject and the camera lens. It involves adjusting the camera's focusing mechanism to achieve a sharp image of the subject.

[0040] In one specific embodiment, when the user issues a voice command to "enter privacy mode", the conditions for entering privacy mode are triggered, and the camera's hardware parameters are adjusted to preset hardware parameters.

[0041] In another specific embodiment, when a user issues a preset gesture command, the conditions for entering privacy mode are triggered, and the camera's hardware parameters are adjusted to preset hardware parameters. For example, when the user issues a preset gesture command such as "waving" at the camera, the conditions for entering privacy mode are triggered.

[0042] In this embodiment, the preset hardware parameters include the target position of the lens. When the trigger condition for entering privacy mode is triggered, the camera lens is moved to the target position, which is different from the lens position when the camera is in focus. When the camera lens moves to the target position, the camera is no longer in focus, resulting in a less clear image. Furthermore, the blurry image is caused by the change in hardware parameters, making it impossible to obtain a clear image in privacy mode later, thus better protecting privacy.

[0043] In another specific embodiment, the preset hardware parameters include the preset focal length of the lens, which is made of metamaterials. When the trigger condition for entering privacy mode is triggered, the camera's focal length is adjusted to the preset focal length, which is different from the focal length of the camera when in focus. Metamaterials, derived from the Latin root "meta-" meaning "beyond, alternative," refer to a class of man-made materials with special properties that do not exist in nature. They possess unique properties, such as altering the normal properties of light and electromagnetic waves, effects that are impossible with conventional materials. Metamaterials are not particularly special in composition; their peculiar properties stem from their precise geometry and size. Their microstructures, smaller than the wavelength they affect, can thus influence waves. Preliminary research on metamaterials focuses on negative refractive index metamaterials. Focal length, in an optical system, is a measure of the convergence or divergence of light; it refers to the distance from the optical center of the lens to the focal point where parallel light converges. Optical systems with shorter focal lengths have a better ability to focus light than those with longer focal lengths. Simply put, focal length is the distance between the focal point and the center of the mirror. In a camera, a focal length f < image distance < 2f is required for an image to form.

[0044] When the camera lens changes focus, the camera is no longer in focus, resulting in a less clear image. The blurry image is caused by the change in hardware parameters, and a clear image in privacy mode cannot be obtained later, thus better protecting privacy.

[0045] S202. Monitor the target scene in privacy mode using a camera with adjusted hardware parameters.

[0046] The camera can be installed on mobile phones, cars, and air conditioners, among other things. Target scenarios can include indoor spaces, car interiors, etc. After adjusting the hardware parameters, the camera, with its adjusted parameters, can monitor the target scenario in privacy mode. The blurry images captured in privacy mode are due to changes in hardware parameters; clear images from privacy mode cannot be obtained later, thus better protecting privacy.

[0047] Furthermore, after adjusting the hardware parameters, the camera with the adjusted parameters is used to capture images of the target scene, resulting in privacy mode images. The resolution of these privacy mode images is then reduced to a preset value, and the reduced-resolution privacy mode images are stored. By combining hardware parameter adjustment with post-processing to obtain privacy mode images, privacy can be better protected.

[0048] Furthermore, to accurately determine whether the conditions for entering privacy mode are triggered, please refer to... Figure 3 In one specific embodiment, when a trigger condition for entering privacy mode is detected, the camera lens is moved to the target position, prior to which steps S301-S304 are performed: S301. Acquire the first scene image of the target scene captured by the camera in focus.

[0049] In one specific embodiment, the camera is a camera with autofocus functionality. When the camera is powered on, it is controlled to automatically focus, bringing the camera into a focused state. The camera then takes a picture of the target scene in the focused state, acquiring a first scene image, which is a clear image.

[0050] S302. Perform face recognition on the first scene image.

[0051] S303. Match the identified face image with the pre-stored user face image.

[0052] Specifically, user facial images are pre-collected; these images can be user facial images uploaded by the camera registrant. In other embodiments, multiple historical images obtained from monitoring the target scene in privacy mode are acquired, facial recognition is performed on these historical images to obtain multiple historical facial images, and a predetermined number of historical facial images, ranked from highest to lowest frequency, are selected as the preset user facial images.

[0053] S304. When there is no target face image matching the user's face image in the first scene image, determine the trigger condition for entering the privacy mode.

[0054] When a face matching the user's face image is found in the first scene image, it indicates that there is a familiar person in the scene. This determines the trigger condition for entering privacy mode and protects the privacy of the familiar person.

[0055] Furthermore, when no target face image matching the user's face image exists in the first scene image, it is determined whether a voice is detected within a preset time. If a voice is detected, voiceprint recognition is performed on the voice to determine whether the speaker belongs to a user in a preset user set. If the speaker belongs to a user in the preset user set, the trigger condition for entering the privacy mode is determined. The voices and face images of each user in the preset user set can be pre-stored. Voiceprint recognition, a type of biometric technology, also known as speaker recognition, includes speaker identification and speaker confirmation. Voiceprint recognition converts sound signals into electrical signals, which are then identified by a computer. Different tasks and applications use different voiceprint recognition technologies; for example, identification technology may be needed to narrow down the scope of criminal investigations, while confirmation technology is required for bank transactions.

[0056] Furthermore, to accurately determine the degree of hardware parameter adjustment, refer to... Figure 4 In one specific embodiment, when a trigger condition for entering privacy mode is detected, moving the camera lens to the target location may include steps S401-S404: S401. When the trigger condition for entering privacy mode is triggered, obtain the current object distance between the target face image in the target scene and the camera.

[0057] In one specific embodiment, the camera is a depth camera. The camera can be a monocular or a binocular camera. When the conditions for entering privacy mode are triggered, the current object distance between the target face image in the target scene and the camera is obtained. The current object distance is the distance between the camera lens and the user's face.

[0058] With the gradual development of disruptive technologies such as machine vision and autonomous driving, applications using depth cameras for object recognition, behavior recognition, and scene modeling are becoming increasingly common. Depth cameras can be considered the eyes of terminals and robots. Also known as 3D cameras, depth cameras, as the name suggests, can detect the depth of field in the shooting space, which is the biggest difference between them and ordinary cameras. Ordinary color cameras capture images that show and record all objects within their field of view, but the recorded data does not include the distance of these objects from the camera. They can only determine which objects are far away and which are close through semantic analysis of the image, but without precise data. Depth cameras solve this problem precisely. Through the data acquired by depth cameras, we can accurately know the distance of each point in the image from the camera. Adding this to the (x,y) coordinates of that point in the 2D image, we can obtain the three-dimensional spatial coordinates of each point in the image. These three-dimensional coordinates can then be used to reconstruct the real scene and enable applications such as scene modeling.

[0059] S402. Determine the current depth of field range based on the current object distance and the preset object distance-depth of field mapping relationship.

[0060] like Figure 5 As shown, Figure 5 This is a diagram illustrating the depth of field of the camera. The preset object distance-depth of field mapping relationship satisfies the following: Foreground depth ΔL1 = FδL^2 / (f^2 + FδL) The depth of field ΔL2 = FδL^2 / (f^2 - FδL) Depth of field ΔL=ΔL1+ΔL2=(2f^2FδL^2) / (f^4-F^2δ^2L^2) δ—Diameter of the permissible circle of confusion F - the aperture value of the lens. f - Lens focal length L—Object distance ΔL1—Foreground Depth ΔL2 — Depth of field ΔL — Depth of field Depth of field (DOF) refers to the range of distances in front of and behind a subject that allow for a sharp image to be captured in front of the camera lens or other imager. Aperture, lens, and the distance from the focal plane to the subject are important factors affecting depth of field. After focusing, the distance between the subject and the focal point that results in a sharp image is called the depth of field. In front of the lens (before and after the focal point), there is a certain length of space. When a subject is located within this space, its image on the film lies within the same circle of confusion. The length of this space containing the subject is called the depth of field. In other words, the blurriness of the image of a subject within this space on the film is within the allowable circle of confusion; the length of this space is the depth of field.

[0061] As can be seen from the formulas above, the larger the aperture, the shallower the depth of field; the smaller the aperture, the deeper the depth of field. The longer the focal length of the lens, the shallower the depth of field; the shorter the focal length, the deeper the depth of field. Changing the distance between the subject and the background does not change the depth of field, but only determines whether the background is blurred and the degree of blurring. The farther the distance, the deeper the depth of field; the closer the distance (not less than the minimum shooting distance), the shallower the depth of field.

[0062] In one specific embodiment, the foreground depth ΔL1 and background depth ΔL2 are determined based on the current object distance and a preset object distance depth-of-field mapping relationship, and the current foreground depth range is determined based on the foreground and background depths. For example, if the current position of the user's face is A, the current foreground depth range is [a1, b1], where a1 = A + ΔL1 and b1 = A - ΔL2.

[0063] S403. Determine the initial moving position based on the current depth of field range, wherein the depth of field range of the camera lens at the initial moving position does not coincide with the current depth of field range.

[0064] like Figure 6 As shown, Figure 6 This is a diagram illustrating the lens adjustment parameters of a camera. Figure 6 In the image, the camera is in focus, out of focus, and out of focus state from top to bottom.

[0065] In one specific embodiment, the current lens position is obtained, and a position at a first preset distance from the current lens position is determined as a candidate lens position. The depth of field range corresponding to the candidate lens position is calculated. It is determined whether the depth of field range corresponding to the candidate lens position overlaps with the current depth of field range. If the depth of field range corresponding to the candidate lens position does not overlap with the current depth of field range, the candidate lens position is determined as the initial movement position. If the depth of field range corresponding to the candidate lens position at least partially overlaps with the current depth of field range, the candidate lens position is determined as the current lens position, and a position at a second preset distance from the candidate lens position is determined as a candidate lens position. Iterative calculations are performed until the depth of field range corresponding to the candidate lens position does not overlap with the current depth of field range. The first preset distance is greater than the second preset distance, which can gradually reduce the search distance and accurately determine the initial movement position. Of course, the first preset distance can also be equal to the second preset distance.

[0066] For example, the current depth of field range corresponding to the current object distance is [a1, b1]; when the lens moves to the initial moving position, the depth of field range of the camera lens at the initial moving position is calculated based on the corresponding object distance as [a2, b2]. When a1 is greater than b2 or b1 is less than a2, the current depth of field range corresponding to the current object distance and the depth of field range of the lens at the initial moving position do not coincide.

[0067] S404. Determine the target position based on the initial movement position.

[0068] In one specific embodiment, the initial movement position is determined as the target position.

[0069] In another specific embodiment, determining the target location based on the initial movement position may include: (1) Obtain the second scene image captured by the camera at the initial moving position.

[0070] (2) Determine that the clarity of the second scene image does not exceed the preset value.

[0071] Specifically, the sharpness of the second scene image is calculated using no-reference image quality assessment algorithms such as the Brenner gradient function and the Tenengrad gradient function. In no-reference image quality assessment, image sharpness is a crucial indicator of image quality, closely corresponding to human subjective perception; low sharpness indicates blurriness. This paper discusses and analyzes several commonly used and representative sharpness algorithms for no-reference image quality assessment applications, providing a basis for selecting sharpness algorithms in practical applications. The Brenner gradient function is the simplest gradient evaluation function; it simply calculates the square of the gray-level difference between two adjacent pixels. The Sobel operator is used to extract the gradient values ​​in the horizontal and vertical directions respectively.

[0072] In another specific embodiment, a first scene image captured by the camera at the current lens position is acquired. This first scene image is a clear image. A PSNR index is calculated based on the second scene image and the first scene image. If the PSNR index does not exceed a preset PSNR value, the clarity of the second scene image is determined to be within the preset range. PSNR is an abbreviation for "Peak Signal to Noise Ratio," an objective standard for evaluating images. It has limitations and is generally used in engineering projects to measure the difference between the maximum signal and background noise. PSNR is an objective indicator of image quality. Generally, a PSNR > 40dB indicates image quality close to the original image; a PSNR between 20dB and 30dB indicates poor image quality; and a PSNR < 20dB indicates unacceptable image quality. For example, the preset PSNR value is 24.2dB.

[0073] (3) If the clarity of the second scene image does not exceed the preset value, the initial moving position is determined as the target position.

[0074] If the clarity of the second scene image does not exceed a preset value, it means the clarity of the image taken at that location meets the requirements, and the initial movement position is determined as the target position. If the clarity of the second scene image exceeds a preset value, it means the clarity of the image taken by the lens at that location does not meet the requirements and cannot protect privacy, then the process proceeds as follows: the candidate lens position is determined as the current lens position, and the position at a second preset distance from the candidate lens position is determined as the candidate lens position. Iterative calculations are performed until the depth of field range corresponding to the candidate lens position does not overlap with the current depth of field range, and then the candidate lens position is determined as the initial movement position.

[0075] Furthermore, if the clarity of the second scene image does not exceed a preset value, face recognition is performed on the second scene image. If face recognition fails, the initial movement position is determined as the target position.

[0076] Furthermore, if the clarity of the second scene image does not exceed a preset value, multiple third scene images are captured for the target scene. Face recognition is performed on each of the multiple third scene images, and the percentage of recognized faces in the multiple third scene images is calculated. If the percentage of recognized faces is lower than a first preset percentage, the initial movement position is determined as the target position. If the percentage of recognized faces is lower than the first preset percentage, it means that the image captured by the camera cannot accurately perform face recognition, thus effectively protecting privacy.

[0077] Furthermore, if the clarity of the second scene image does not exceed a preset value, multiple third scene images are captured for the target scene. Face recognition is performed on each of the multiple third scene images, and the percentage of recognized faces in the multiple third scene images is calculated. If the first percentage is lower than a first preset percentage, human contour recognition is performed on the multiple third scene images, and the percentage of recognized human contours in the multiple third scene images is calculated. If the second percentage is higher than a second preset percentage, the initial movement position is determined as the target position. The first and second preset percentages can be set according to specific circumstances. If the first percentage is lower than the first preset percentage, it means that the captured image is blurry enough that face recognition cannot accurately identify the face. If the second percentage is higher than the second preset percentage, it means that human contour recognition can be performed. That is, face recognition cannot be performed in privacy mode to protect privacy, but human contour recognition can be performed at the same time, so that smart devices such as home air conditioners can recognize human contours and adjust themselves accordingly.

[0078] Among them, face recognition is the identification of a person's face, while body contour recognition is the identification of the entire human body. Specifically, a face recognition neural network is trained using a face image training set, and a body contour recognition neural network is trained using a body contour image training set.

[0079] Furthermore, after monitoring the target scene in privacy mode using a camera with adjusted hardware parameters, a fourth scene image captured by the camera at the target location is acquired. Human contour recognition is performed on the fourth scene image to obtain the number of human contours in the fourth scene image. Human contour recognition is also performed on the first scene image to obtain the number of human contours in the first scene image. It is then determined whether the number of human contours in the fourth scene image is the same as the number of human contours in the first scene image. If the number of human contours in the fourth scene image is different from the number of human contours in the first scene image, the privacy mode is exited. Specifically, the camera lens is moved to a focused position; or the camera's focal length is adjusted to a focused position.

[0080] Furthermore, it is determined whether the number of human body outlines in the fourth scene image is the same as the number of human body outlines in the first scene image; if the number of human body outlines in the fourth scene image is different from the number of human body outlines in the first scene image, it is determined whether the number of human body outlines in the fourth scene image is less than the number of human body outlines in the first scene image; if the number of human body outlines in the fourth scene image is less than the number of human body outlines in the first scene image, it indicates that someone has left the screen and exits the privacy mode.

[0081] This application provides a camera monitoring method. When the trigger condition for entering privacy mode is triggered, the camera's hardware parameters are adjusted to preset hardware parameters. The image captured by the camera under these preset hardware parameters has lower clarity than the image captured by the camera in focus mode. The adjusted camera is then used to monitor a target scene in privacy mode. This application adjusts the camera's hardware parameters and then monitors the scene. Monitoring in privacy mode results in a blurrier image than the image captured in focus mode. Because the adjustment is to the camera's hardware parameters, it is impossible to reconstruct the image captured in privacy mode, thereby improving the camera's ability to protect privacy.

[0082] To better implement the camera monitoring method in the embodiments of this application, based on the camera monitoring method, the embodiments of this application also provide a camera monitoring device, which is integrated into a smart device, such as... Figure 7 As shown, the camera monitoring device 500 includes: The parameter adjustment unit 501 is used to adjust the hardware parameters of the camera to preset hardware parameters when the triggering condition for entering the privacy mode is triggered, wherein the image captured by the camera under the preset hardware parameters has a lower resolution than the image captured by the camera in the focus state. The monitoring unit 502 is used to monitor the target scene in privacy mode using a camera with the hardware parameters adjusted.

[0083] Optionally, the preset hardware parameters include the target position of the lens, and the parameter adjustment unit 501 is used for: When the conditions for entering privacy mode are triggered, the camera lens is moved to the target position, wherein the target position is different from the lens position of the camera in focus state.

[0084] Optionally, the preset hardware parameters include the preset focal length of the lens, the lens being made of metamaterials, and the parameter adjustment unit 501 being used for: When the conditions for entering privacy mode are triggered, the focal length of the camera is adjusted to a preset focal length, wherein the preset focal length is different from the focal length of the camera in the focused state.

[0085] Optionally, the parameter adjustment unit 501 is used for: Acquire the first scene image of the target scene captured by the camera in focus; Perform facial recognition on the first scene image; The identified facial images are matched with pre-stored user facial images; When a target face image matching the user's face image exists in the first scene image, the trigger condition for entering privacy mode is determined.

[0086] Optionally, the parameter adjustment unit 501 is used for: When the conditions for entering privacy mode are triggered, the current object distance between the target face image in the target scene and the camera is obtained; The current depth of field range is determined based on the current object distance and the preset object distance depth-of-field mapping relationship; The initial moving position is determined based on the current depth of field range, wherein the depth of field range of the camera lens at the initial moving position does not coincide with the current depth of field range; Determine the target location based on the initial movement position.

[0087] Optionally, the parameter adjustment unit 501 is used for: Acquire a second scene image captured by the camera at its initial moving position; Determine whether the sharpness of the second scene image exceeds a preset value; If the clarity of the second scene image does not exceed a preset value, then the initial movement position is determined as the target position.

[0088] Optionally, the parameter adjustment unit 501 is used for: If the clarity of the second scene image does not exceed the preset value, then face recognition is performed; If face recognition fails, the initial movement position will be set as the target position.

[0089] Optionally, the monitoring unit 502 is used for: Acquire a fourth scene image captured by the camera at the target location; Perform human contour recognition on the fourth scene image to obtain the number of human contours in the fourth scene image; Perform human contour recognition on the first scene image to obtain the number of human contours in the first scene image; Determine whether the number of human body outlines in the fourth scene image is the same as the number of human body outlines in the first scene image; If the number of human figures in the fourth scene image is different from the number of human figures in the first scene image, then exit privacy mode.

[0090] This application provides a camera monitoring device that, when a condition for entering a privacy mode is triggered, adjusts the camera's hardware parameters to preset hardware parameters. The image captured by the camera under these preset hardware parameters has lower clarity than an image captured when the camera is in focus. The device then monitors a target scene using the adjusted camera in privacy mode. By adjusting the camera's hardware parameters and then monitoring when the privacy mode is triggered, this application obtains a more blurred image than an image captured in focus mode. Since the adjustment is to the camera's hardware parameters, it is impossible to reconstruct the image captured in privacy mode, thereby improving the camera's ability to protect privacy.

[0091] This application also provides a smart device that integrates any of the camera monitoring devices provided in this application. The smart device includes: One or more processors; Memory; and One or more applications, wherein the applications are stored in memory and configured to be executed by a processor as steps of the camera monitoring method in any of the above embodiments of the camera monitoring method.

[0092] like Figure 8 As shown, it illustrates a structural schematic diagram of the smart device involved in the embodiments of this application, specifically: The intelligent device may include components such as a processor 601 with one or more processing cores, a memory 602 with one or more computer-readable storage media, a power supply 603, and an input unit 604. Those skilled in the art will understand that the intelligent device structure shown in the figures does not constitute a limitation on the intelligent device, and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein: The processor 601 is the control center of the intelligent device. It connects various parts of the intelligent device via various interfaces and lines, and performs various functions and processes data by running or executing software programs and / or modules stored in the memory 602, and by calling data stored in the memory 602, thereby providing overall monitoring of the intelligent device. Optionally, the processor 601 may include one or more processing cores; the processor 601 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor. Preferably, the processor 601 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, physical interface, and application programs, and the modem processor mainly handles wireless communication. It is understood that the aforementioned modem processor may not be integrated into the processor 601.

[0093] The memory 602 can be used to store software programs and modules. The processor 601 executes various functional applications and data processing by running the software programs and modules stored in the memory 602. The memory 602 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created based on the use of the smart device, etc. In addition, the memory 602 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 602 may also include a memory controller to provide the processor 601 with access to the memory 602.

[0094] The smart device also includes a power supply 603 that supplies power to the various components. Preferably, the power supply 603 can be logically connected to the processor 601 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 603 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0095] The smart device may also include an input unit 604, which can be used to receive input digital or character information, and generate keyboard, mouse, joystick, optical or trackball signal inputs related to physical settings and function control.

[0096] Although not shown, the smart device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 601 in the smart device loads the executable files corresponding to the processes of one or more applications into the memory 602 according to the following instructions, and the processor 601 runs the applications stored in the memory 602 to realize various functions, as follows: When the conditions for entering privacy mode are triggered, the camera's hardware parameters are adjusted to preset hardware parameters. The image captured by the camera under the preset hardware parameters has a lower resolution than the image captured by the camera in focus. The camera with adjusted hardware parameters is then used to monitor the target scene in privacy mode.

[0097] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0098] Therefore, embodiments of this application provide a computer-readable storage medium, which may include: read-only memory (ROM), random access memory (RAM), a disk, or an optical disk, etc. A computer program is stored thereon, and the computer program is loaded by a processor to execute the steps in any of the camera monitoring methods provided in embodiments of this application. For example, the computer program loaded by the processor can execute the following steps: When the conditions for entering privacy mode are triggered, the camera's hardware parameters are adjusted to preset hardware parameters. The image captured by the camera under the preset hardware parameters has a lower resolution than the image captured by the camera in focus. The camera with adjusted hardware parameters is then used to monitor the target scene in privacy mode.

[0099] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the detailed descriptions of other embodiments above, which will not be repeated here.

[0100] In practice, each of the above units or structures can be implemented as an independent entity or can be arbitrarily combined to be implemented as the same or several entities. For the specific implementation of each of the above units or structures, please refer to the previous method embodiments, which will not be repeated here.

[0101] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0102] The above provides a detailed description of a camera monitoring method and apparatus provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A camera monitoring method, characterized by, The camera monitoring method includes: When the conditions for entering privacy mode are triggered, the hardware parameters of the camera are adjusted to preset hardware parameters, wherein the image captured by the camera under the preset hardware parameters has a lower resolution than the image captured by the camera in focus mode. The target scene is monitored in privacy mode using a camera with the hardware parameters adjusted. The preset hardware parameters include the target position of the lens. When the trigger condition for entering privacy mode is triggered, adjusting the camera's hardware parameters to the preset hardware parameters includes: When the conditions for entering privacy mode are triggered, the camera lens is moved to the target position, wherein the target position is different from the lens position of the camera in focus state; The camera monitoring method also includes: Acquire a fourth scene image captured by the camera at the target location; Perform human contour recognition on the fourth scene image to obtain the number of human contours in the fourth scene image; Human contour recognition is performed on the first scene image to obtain the number of human contours in the first scene image, where the first scene image is captured by the camera in a focused state; Determine whether the number of human body outlines in the fourth scene image is the same as the number of human body outlines in the first scene image; If the number of human figures in the fourth scene image is different from the number of human figures in the first scene image, then exit privacy mode.

2. The camera monitoring method of claim 1, wherein, When the conditions for entering privacy mode are triggered, the camera lens is moved to the target location. Before this, the following steps are included: Acquire the first scene image of the target scene captured by the camera in focus; Perform facial recognition on the first scene image; The identified facial images are matched with pre-stored user facial images; When a target face image matching the user's face image exists in the first scene image, the trigger condition for entering privacy mode is determined.

3. The camera monitoring method of claim 2, wherein, When the trigger condition for entering privacy mode is triggered, moving the camera lens to the target position includes: When the conditions for entering privacy mode are triggered, the current object distance between the target face image in the target scene and the camera is obtained; The current depth of field range is determined based on the current object distance and the preset object distance depth-of-field mapping relationship; The initial moving position is determined based on the current depth of field range, wherein the depth of field range of the camera lens at the initial moving position does not coincide with the current depth of field range; Determine the target location based on the initial movement position.

4. The camera monitoring method of claim 3, wherein, The process of determining the target location based on the initial movement position includes: Acquire a second scene image captured by the camera at its initial moving position; Determine whether the sharpness of the second scene image exceeds a preset value; If the clarity of the second scene image does not exceed a preset value, then the initial movement position is determined as the target position.

5. The camera monitoring method according to claim 4, characterized in that, If the clarity of the second scene image does not exceed a preset value, then the initial movement position is determined as the target position, including: If the clarity of the second scene image does not exceed the preset value, then face recognition is performed; If face recognition fails, the initial movement position will be set as the target position.

6. The camera monitoring method according to claim 1, characterized in that, The preset hardware parameters include the preset focal length of the lens, which is made of metamaterials. When the trigger condition for entering privacy mode is triggered, adjusting the camera's hardware parameters to the preset hardware parameters includes: When the conditions for entering privacy mode are triggered, the focal length of the camera is adjusted to a preset focal length, wherein the preset focal length is different from the focal length of the camera in the focused state.

7. A camera monitoring device, characterized in that, The camera monitoring device includes: The parameter adjustment unit is used to adjust the hardware parameters of the camera to preset hardware parameters when the triggering condition for entering the privacy mode is triggered, wherein the image captured by the camera under the preset hardware parameters has a lower resolution than the image captured by the camera in focus mode. The monitoring unit is used to monitor the target scene in privacy mode using a camera with the hardware parameters adjusted. The preset hardware parameters include the target position of the lens. When the trigger condition for entering privacy mode is triggered, adjusting the camera's hardware parameters to the preset hardware parameters includes: When the conditions for entering privacy mode are triggered, the camera lens is moved to the target position, wherein the target position is different from the lens position of the camera in focus state; The camera monitoring device also includes: Acquire a fourth scene image captured by the camera at the target location; Perform human contour recognition on the fourth scene image to obtain the number of human contours in the fourth scene image; Human contour recognition is performed on the first scene image to obtain the number of human contours in the first scene image, where the first scene image is captured by the camera in a focused state; Determine whether the number of human body outlines in the fourth scene image is the same as the number of human body outlines in the first scene image; If the number of human figures in the fourth scene image is different from the number of human figures in the first scene image, then exit privacy mode.

8. A smart device, characterized in that, The intelligent device includes: One or more processors; Memory; and One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement the camera monitoring method of any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, It stores a computer program, which is loaded by a processor to execute the steps of the camera monitoring method according to any one of claims 1 to 6.

Images