Privacy protection camera intelligent monitoring method and device, equipment and medium

Abstract

The invention belongs to the technical field of video monitoring, and provides a privacy protection camera intelligent monitoring method and device, equipment and a medium. The method comprises the following steps: acquiring a monitoring video stream of a camera in real time; anonymizing the video stream, detecting a human body frame by frame for each frame of image based on the recognition model during anonymization, if no human body is detected in the current frame of image, updating the current frame of image as a background image, and if the human body is detected in the current frame of image, adding a human body bounding box in the current frame of image and positioning a posture key point; on the basis of the posture key points, skeleton representation of the human body is formed, the joint angle of the skeleton representation is calculated so as to adjust the joint angle of the 2D virtual human body, and the joint angle is superposed in the bounding box; based on a tumble detection algorithm, actions and postures represented by the bones are analyzed in real time to confirm whether tumble exists or not, and the health condition of the human body is obtained; uploading the anonymized video stream and health condition to a cloud end, and providing the anonymized video stream and health condition to an administrator through a smart phone program; according to the invention, the problem that privacy is easily exposed in falling detection based on video monitoring in the prior art can be solved.

Description

Technical Field

[0001] The present invention relates to the field of video surveillance technology, and in particular to a privacy-protected camera intelligent monitoring method, device, equipment, and medium. Background Art

[0002] With the development of intelligent monitoring technology, fall detection has become an important research area in elderly care and safety monitoring. Falls are a common and serious health risk among the elderly. Promptly detecting and responding to fall events can significantly reduce the risk of injury. Traditional fall detection methods rely on sensors such as accelerometers and gyroscopes, but these methods may have limitations in some complex environments, such as the inability to accurately detect non-vertical falls or misjudging the elderly's activities.

[0003] In recent years, visual detection-based video surveillance has become a hot topic of research. It can be used to monitor falls in videos. However, like conventional video surveillance, setting up surveillance in the home presents privacy challenges. Once surveillance videos are uploaded to a server, there's a risk of privacy exposure online. This is particularly true in elderly care settings, where personal privacy is a sensitive issue. While visual detection-based video surveillance can effectively identify falls, without proper privacy protection measures, the images captured in the surveillance videos could reveal the daily lives of elderly individuals, leading to privacy breaches. Therefore, effective fall detection without infringing privacy has become a pressing technical challenge in this field. Summary of the Invention

[0004] In view of the above technical problems, the present invention provides a privacy-protected camera intelligent monitoring method, device, equipment and medium to solve the problem that fall detection based on video monitoring in the prior art is prone to privacy exposure.

[0005] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by practice of the present disclosure.

[0006] According to one aspect of the present invention, a privacy-preserving camera intelligent monitoring method is disclosed, the method comprising:

[0007] Get the camera's surveillance video stream in real time;

[0008] In a local network, the video stream is anonymized. During the anonymization, an image filled with black pixels is used as a background image. A human body is detected frame by frame for each frame based on a recognition model. If no human body is detected in a current frame, the current frame is updated to the background image. If a human body is detected in the current frame, a human body bounding box is added to the current frame and posture key points are located. An area corresponding to the bounding box in the background image is used to update an area within the bounding box, and an area outside the bounding box in the current frame is used to update a corresponding area portion of the background image. A skeleton representation of the human body is formed based on the posture key points. Joint angles of the skeleton representation are calculated to adjust the joint angles of a 2D virtual human body, and the virtual human body is superimposed within the bounding box.

[0009] Based on the fall detection algorithm, the movement and posture of the skeleton representation are analyzed in real time to confirm whether there is a fall phenomenon and obtain the health status of the human body;

[0010] The anonymized video stream and the health status are uploaded to the cloud and provided to an administrator via a smartphone program.

[0011] Furthermore, the recognition model is one of YOLOv8, OpenPose, MediaPipe, PoseNet, and AlphaPose.

[0012] Furthermore, when calculating the joint angles of the skeleton representation, it includes:

[0013] Get the joint B whose angle is to be calculated, get the two adjacent joints A and C of joint B, and calculate the following formula:

[0014] θ=arctan2(u y ,u x )-arctan2(v y ,v x );

[0015] Among them, u y and u x are the y-axis component and x-axis component of the vector from point B to A, v y and v x are the y-axis component and x-axis component of the vector from point B to point C, θ is the angle of joint B, and arctan2 is the inverse tangent function.

[0016] Furthermore, after calculating the angle of joint B, the angle of joint B is smoothed based on a third-order polynomial fitting. The formula for the third-order polynomial fitting is:

[0017]

[0018] Among them, a0, a1, a2, and a3 are coefficients determined by multi-factor fitting of historical data of joint angles using the least squares method. represents the joint angle θ changing with time t.

[0019] Furthermore, the real-time analysis of the skeletal representation of motion and posture based on the fall detection algorithm includes:

[0020] calculating an average velocity and body center of the skeleton representation, a center and an aspect ratio of the bounding box of multiple frames in the video stream to obtain a motion feature;

[0021] Calculating the change of the motion feature over time to obtain a time change vector;

[0022] The time change vector is input into a one-dimensional convolutional neural network with residual connection, and a time series analysis is performed on the time change vector to confirm whether there is a fall phenomenon.

[0023] Furthermore, the one-dimensional convolutional neural network includes a plurality of residual blocks connected in sequence and a fully connected layer, the residual block includes a dilated causal convolution, a first activation function, a causal convolution, and a second activation function connected in sequence, the dilated causal convolution is used to input data, and the causal convolution is used to ensure that the analysis result of each of the time change vectors depends only on the current and previous time change vectors.

[0024] Furthermore, the average speed is the position change of each of the posture key points in two consecutive frames, and the average speed is divided by the time between the two frames;

[0025] The body center is determined by the midpoint of the shoulders and hips, the midpoint of the shoulders is determined by the positions of the left and right shoulders in the posture key points, and the midpoint of the hips is determined by the positions of the left and right hips in the posture key points;

[0026] The center of the bounding box is calculated by the coordinates and average of its four corners;

[0027] The aspect ratio of the bounding box is determined by the ratio of its width to its height;

[0028] The time change vector is a series of degrees of change of the motion features over time within a sliding window.

[0029] According to the second aspect of the present disclosure, a privacy-protected camera intelligent monitoring device is provided, comprising: an acquisition module for acquiring a camera's monitoring video stream in real time; a privacy-protected anonymity module for anonymizing the video stream in a local network; during anonymization, an image filled with black pixels is used as a background image, and a human body is detected frame by frame for each frame based on a recognition model; if a human body is not detected in the current frame image, the current frame image is updated to the background image; if a human body is detected in the current frame image, a human body boundary box is added to the current frame image and posture key points are located; and the image in the background image that is aligned with the boundary box is used to identify the human body. The system updates the area within the bounding box with the area corresponding to the frame, and uses the area outside the bounding box in the current frame image to update the corresponding area of ​​the background image. Based on the posture key points, a skeleton representation of the human body is formed, and the joint angles of the skeleton representation are calculated to adjust the joint angles of the 2D virtual human body and superimpose them in the bounding box; an analysis module is used to analyze the movements and postures of the skeleton representation in real time based on the fall detection algorithm to confirm whether there is a fall phenomenon and obtain the health status of the human body; a transmission module is used to upload the anonymized video stream and the health status to the cloud and provide them to the administrator through a smartphone program.

[0030] According to a third aspect of the present disclosure, there is provided a privacy-protected camera intelligent monitoring device, comprising: a processor; and a memory arranged to store computer-executable instructions, wherein the executable instructions, when executed, cause the processor to implement the privacy-protected camera intelligent monitoring method described above;

[0031] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided, storing a computer program, which, when executed by a processor, implements the above-mentioned privacy-protected camera intelligent monitoring method.

[0032] The technical solution disclosed in this disclosure has the following beneficial effects:

[0033] By adopting anonymization technology, the present invention can effectively hide personal features in the video and only display the human skeleton model. This not only ensures the monitoring function of fall events, but also respects the privacy rights of the elderly, eliminates the privacy leakage problem in traditional video surveillance, and thus improves the acceptance of this technology among the elderly. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 This is a flowchart of a privacy protection camera intelligent monitoring method according to an embodiment of this specification;

[0035] Figure 2 This is a structural block diagram of a one-dimensional convolutional neural network in an embodiment of this specification.

[0036] Figure 3 This is a structural block diagram of a camera intelligent monitoring device with privacy protection according to an embodiment of this specification;

[0037] Figure 4 It is a terminal device of the camera intelligent monitoring method with privacy protection in the embodiments of this specification;

[0038] Figure 5 A computer-readable storage medium for the privacy-protected camera intelligent monitoring method in the embodiments of this specification. DETAILED DESCRIPTION

[0039] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the present disclosure will be more comprehensive and complete and will fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced while omitting one or more of the specific details, or that other methods, components, devices, steps, etc. may be employed. In other cases, well-known technical solutions are not shown or described in detail to avoid obscuring various aspects of the present disclosure.

[0040] The accompanying drawings are merely schematic illustrations of the present disclosure. Identical reference numerals in the drawings denote identical or similar components, and thus their repeated description will be omitted. Some of the blocks shown in the accompanying drawings represent functional entities that do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0041] like Figure 1 As shown, the embodiment of this specification provides a privacy protection camera intelligent monitoring method, and the execution subject of the method can be a computer. The method can specifically include the following steps S101 to S104:

[0042] In step S101, the surveillance video stream of the camera is acquired in real time.

[0043] The camera can specifically be an Internet Protocol camera that supports Wi-Fi. The camera can be connected to a router, and the router is connected to a computer. The computer performs the following anonymization processing and fall analysis, and the processed video is uploaded to the cloud server by the computer.

[0044] In step S102, the video stream is anonymized in the local network. During anonymization, an image filled with black pixels is used as the background image, and a human body is detected frame by frame for each frame based on a recognition model. If a human body is not detected in the current frame image, the current frame image is updated to the background image. If a human body is detected in the current frame image, a human body bounding box is added to the current frame image and posture key points are located. The area within the bounding box is updated using the area in the background image corresponding to the bounding box, and the area outside the bounding box in the current frame image is used to update the corresponding area of ​​the background image. Based on the posture key points, a skeleton representation of the human body is formed, and the joint angles of the skeleton representation are calculated to adjust the joint angles of the 2D virtual human body and superimpose them in the bounding box.

[0045] Therein, the video stream is analyzed in the local network.

[0046] Initially, a background image filled with black pixels is provided to the video stream, creating an empty buffer for the background of subsequent frames. As each frame of the video stream is processed, a detection model is used to detect bounding boxes of people within the frame, identifying areas within the frame that require anonymization or privacy protection. If no bounding box is detected, indicating that no person is present in the current frame, the current frame is set as the new background image, placing the current frame into the background buffer. This dynamic update mechanism ensures that the background image remains up to date. When a bounding box appears, indicating that a person has been identified in the current frame, the area corresponding to the bounding box in the background image is captured and replaced within the bounding box of the current frame, hiding the person and ensuring that the person is separated from the actual image and preventing the disclosure of true identity information. The area outside the bounding box is then updated, using the corresponding portion of the background image in the current frame's image. This helps maintain background consistency in areas without a person. In cases where a person is hidden within the bounding box, posture keypoints, such as the shoulders, elbows, and knees, are simultaneously identified, and the angles of each joint are calculated. These joint angles help understand the person's movement and posture changes, which are particularly important when analyzing actions such as falls. Based on the calculated joint angles, the virtual character's posture is adjusted to ensure that the virtual character's movements are consistent with the detected actual movements. The adjusted 2D virtual character is superimposed within the previously detected human bounding box for subsequent analysis. After processing each frame, the system checks whether the video stream has ended. If so, processing stops and the process ends. If not, processing continues with the next frame.

[0047] Additionally, the recognition model is one of YOLOv8, OpenPose, MediaPipe, PoseNet, and AlphaPose. YOLOv8 adopts a lightweight design and supports a wider range of applications. It can detect human body bounding boxes and perform posture estimation at the same time. OpenPose is a posture estimation tool based on deep learning, which can detect multiple key points of the human body (such as joints, head, shoulders, elbows, knees, etc.) and provide a complete skeleton model of the human body. OpenPose can perform multi-person posture estimation at the same time and return the coordinates of each key point. MediaPipe is an open source cross-platform framework dedicated to efficient multimodal processing (for example, video streams, audio, sensor data, etc.). The MediaPipe Pose model can be used to detect human posture and key points. PoseNet is a lightweight model suitable for real-time human posture estimation. It can perform human posture estimation on various devices and support single-person and multi-person detection. AlphaPose is a multi-person pose estimation model that supports real-time multi-person pose estimation. It can effectively detect the human body bounding box in images or videos and provide accurate joint point coordinates. It is suitable for real-time processing on mobile devices and embedded devices and can provide high-precision pose estimation and key point detection.

[0048] In step S103, based on a fall detection algorithm, the movement and posture represented by the skeleton are analyzed in real time to confirm whether a fall occurs and obtain the health status of the human body.

[0049] Among them, based on the fall detection algorithm, the human skeleton representation (i.e., the position information of the key points of the posture) is analyzed in real time to identify the human body's movements and postures. By calculating the joint angles, position changes, and other motion characteristics of the skeleton, it is possible to determine whether the person has fallen. For example, when a person's body suddenly tilts significantly or makes abnormal movements, these motion characteristics can be identified and it can be determined whether it is a fall. Ultimately, the health status of the human body can be output based on these analysis results, and timely identification and alarm can be issued to ensure the safety of the elderly.

[0050] In step S104, the anonymized video stream and the health status are uploaded to the cloud and provided to the administrator via a smartphone program.

[0051] Among them, in the previous step, the video stream has been anonymized, that is, the human body image in the video is replaced with a skeleton representation or a 2D virtual character to protect personal privacy. At this point, the video stream no longer contains any personal identity information (such as facial features, etc.), and only retains sufficient action and posture features for analysis. In addition to the video stream, real-time monitored health status data (such as the detection results of fall events) are also uploaded to the cloud. Health status data includes whether a fall has occurred, the severity of the fall, and possible other health risk indicators (such as abnormal posture, etc.).

[0052] In one embodiment, when calculating the joint angles of the skeleton representation, the method includes:

[0053] Get the joint B whose angle is to be calculated, get the two adjacent joints A and C of joint B, and calculate the following formula:

[0054] θ=arctan2(u y ,u x )-arctan2(v y ,v x );

[0055] Among them, u y and u x are the y-axis component and x-axis component of the vector from point B to A, v y and v x where θ is the y-axis component and x-axis component of the vector from point B to point C, respectively. θ is the angle of joint B, and arctan2 is the inverse tangent function. Accurately calculating vector direction significantly improves the performance of 2D virtual humans, which is crucial for accurately replicating human motion. Arctan2 can be used to evaluate both vector magnitude and direction, ensuring measurement of angles across the entire 360° range, from 180° to -180°. This allows for capturing motion in all four quadrants, a crucial feature for creating complex motion animations with realistic accuracy.

[0056] As a supplement, in order to make the movement of the 2D virtual human body natural, after calculating the angle of joint B, the angle of joint B is calculated and smoothed based on the third-order polynomial fitting. The formula of the third-order polynomial fitting is:

[0057]

[0058] Among them, a0, a1, a2, and a3 are coefficients determined by multi-factor fitting of historical data of joint angles using the least squares method. represents the joint angle θ changing with time t.

[0059] In one embodiment, the real-time analysis of the motion and posture of the skeletal representation based on the fall detection algorithm includes:

[0060] The average speed and body center of the skeleton representation of multiple frames in the video stream, the center and aspect ratio of the bounding box are calculated to obtain motion features; the change of the motion features over time is calculated to obtain a time change vector; the time change vector is input into a one-dimensional convolutional neural network with residual connection, and a time series analysis is performed on the time change vector to confirm whether there is a fall phenomenon.

[0061] Wherein, the average speed is the position change of each posture key point in two consecutive frames, divided by the time between the two frames. i,t and P i,t+1 Represent the positions of the posture key points at time t and t+1 respectively, then the velocity V of the posture key point i i It can be expressed as:

[0062]

[0063] Then, the average velocity of N pose keypoints can be expressed as:

[0064]

[0065] The body center C body Confirmed by the midpoint between the shoulder and hip, the midpoint of the shoulder M shoulder The left shoulder P in the pose key point LS and right shoulder P RS The position of the hip is determined by the midpoint M of the hip hip The left hip P in the posture key point LH and right hip P RH The specific calculation is as follows;

[0066]

[0067] The center C of the bounding box bbox Calculated from the x and y coordinates of its four corners and the average value, if B tl and B br Represent the upper left corner and lower right corner of the bounding box respectively, then the center of the bounding box is represented as;

[0068]

[0069] The aspect ratio of the bounding box is determined by the ratio of its width to its height. The width can be calculated by taking the top right corner B tl and upper right corner B tr The height can be obtained by calculating the upper left corner B tl and lower left corner B blThe aspect ratio can be expressed as:

[0070]

[0071] The time variation vector σ is a series of degrees of change of the motion features over time in a sliding window. The time variation vector σ of the feature is in the sequence of feature values ​​[F t ,F t+1 ,F t+2 ,...,F t+n ] is calculated in a sliding window with time t. Within this window, the time change can be expressed using the formula:

[0072]

[0073] Among them, F i represents the feature value of each frame in the window, μ is the mean of these values, and k is the total number of frames in the window. This quantifies the variability of the feature values ​​around the mean, improving the measure of temporal change across multiple video frames.

[0074] In order to analyze the above-mentioned time-changing vector, a one-dimensional convolutional neural network is adopted, which includes a plurality of residual blocks connected in sequence and a fully connected layer. The residual block includes a dilated causal convolution, a first activation function, a causal convolution, and a second activation function connected in sequence. The dilated causal convolution is used to input data, and the causal convolution is used to ensure that the analysis result of each of the time-changing vectors depends only on the current and previous time-changing vectors.

[0075] like Figure 2The structure of a one-dimensional convolutional neural network model is shown. Considering that motion features and bounding box attributes have already been extracted and their temporal variations are used for training, the model architecture is simplified compared to the existing state-of-the-art. The model consists of three residual blocks followed by a fully connected layer with 32 neurons and a sigmoid activation function at the end. The output node consists of a single neuron with a sigmoid activation function for binary classification of "fall" and "non-fall" scenarios. Each residual block in the model begins with a dilated causal convolutional layer using a specific number of filters and dilation parameters, followed by a scaled exponential linear unit (SELU) activation layer. This initially expands the network's receptive field, enabling it to capture a wider range of nonlinear temporal patterns without significantly increasing computational complexity. Another causal convolutional layer is then applied, using the same number of filters but without dilation, also followed by a SELU activation layer. To integrate learned features while preserving information from earlier layers, each block includes a skip connection and is processed through a 1×1 convolution. This convolution is used to match the number of filters to the final layer of the block, which is then added to the block's output. The model consists of three residual blocks, each with a progressively increasing number of filters (16, 32, and 64) and dilation parameters (1, 2, and 3). This progressive design allows the model to extract and refine features from simple to complex, while the progressive increase in the number of filters and dilation rate ensures that the model can learn information from different motion features, thereby distinguishing normal activities from falls based on temporal patterns in the data.

[0076] Based on the same idea, Figure 3As shown, the exemplary embodiment of the present disclosure also provides a privacy-protected camera intelligent monitoring device, including: an acquisition module 201, used to obtain the camera's monitoring video stream in real time; a privacy-protection anonymization module 202, used to anonymize the video stream in a local network. During anonymization, an image filled with black pixels is used as a background image, and a human body is detected frame by frame for each frame based on a recognition model. If a human body is not detected in the current frame image, the current frame image is updated to the background image. If a human body is detected in the current frame image, a human body bounding box is added to the current frame image and posture key points are located. The human body is detected using the background image. The area corresponding to the bounding box updates the area within the bounding box, and uses the area outside the bounding box in the current frame image to update the corresponding area of ​​the background image. Based on the posture key points, a skeleton representation of the human body is formed, and the joint angles of the skeleton representation are calculated to adjust the joint angles of the 2D virtual human body and superimpose them within the bounding box; the analysis module 203 is used to analyze the movements and postures of the skeleton representation in real time based on the fall detection algorithm to confirm whether there is a fall phenomenon and obtain the health status of the human body; the transmission module 204 is used to upload the anonymized video stream and the health status to the cloud and provide them to the administrator through the smartphone program.

[0077] By adopting anonymization technology, the above-mentioned device can effectively hide personal features in the video and only display the human skeleton model. This not only ensures the monitoring function of fall incidents, but also respects the privacy rights of the elderly, eliminates the privacy leakage problem in traditional video surveillance, and thus improves the acceptance of this technology among the elderly.

[0078] The specific details of each module in the above device have been described in detail in the implementation method part. The undisclosed details can be found in the implementation method part, so they will not be repeated here.

[0079] Based on the same idea, the embodiment of this specification also provides a privacy protection camera intelligent monitoring device, such as Figure 4 shown.

[0080] The privacy-protected camera intelligent monitoring device may be the terminal device or server provided in the above embodiment.

[0081] Privacy-protected camera smart monitoring devices may vary significantly due to different configurations or performance, and may include one or more processors 301 and memory 302. Memory 302 may store one or more applications or data. Memory 502 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) and / or a cache memory unit, and may further include a read-only storage unit. Applications stored in memory 302 may include one or more program modules (not shown). Such program modules include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each of these examples, or some combination thereof, may include an implementation of a network environment. Furthermore, processor 301 may be configured to communicate with memory 302 to execute a series of computer-executable instructions in memory 302 on the privacy-protected camera smart monitoring device. The privacy-protected camera smart surveillance device may also include one or more power supplies 303, one or more wired or wireless network interfaces 304, one or more I / O interfaces (input and output interfaces) 305, and one or more external devices 306 (e.g., keyboards, pointing devices, Bluetooth devices, etc.). It may also communicate with one or more devices that enable a receiving end to interact with the device, and / or with any device that enables the device to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via the I / O interface 305. Furthermore, the device may also communicate with one or more networks (e.g., a local area network (LAN)) via the wired or wireless interface 304.

[0082] Specifically in this embodiment, the privacy protection camera intelligent monitoring device includes a memory and one or more programs, wherein the one or more programs are stored in the memory, and the one or more programs may include one or more modules, and each module may include a series of computer-executable instructions for the privacy protection camera intelligent monitoring device, and the one or more programs are configured to be executed by one or more processors, including computer-executable instructions for performing the following:

[0083] Get the camera's surveillance video stream in real time;

[0084] In a local network, the video stream is anonymized. During the anonymization, an image filled with black pixels is used as a background image. A human body is detected frame by frame for each frame based on a recognition model. If no human body is detected in a current frame, the current frame is updated to the background image. If a human body is detected in the current frame, a human body bounding box is added to the current frame and posture key points are located. An area corresponding to the bounding box in the background image is used to update an area within the bounding box, and an area outside the bounding box in the current frame is used to update a corresponding area portion of the background image. A skeleton representation of the human body is formed based on the posture key points. Joint angles of the skeleton representation are calculated to adjust the joint angles of a 2D virtual human body, and the virtual human body is superimposed within the bounding box.

[0085] Based on the fall detection algorithm, the movement and posture of the skeleton representation are analyzed in real time to confirm whether there is a fall phenomenon and obtain the health status of the human body;

[0086] The anonymized video stream and the health status are uploaded to the cloud and provided to an administrator via a smartphone program.

[0087] Based on the same idea, the exemplary embodiments of the present disclosure also provide a computer-readable storage medium on which is stored a program product capable of implementing the methods described above in this specification. In some possible implementations, various aspects of the present disclosure may also be implemented in the form of a program product, which includes program code. When the program product is executed on a terminal device, the program code is used to cause the terminal device to execute the steps described in the "Exemplary Methods" section above according to various exemplary embodiments of the present disclosure.

[0088] refer to Figure 5 As shown, a program product 400 for implementing the above method according to an exemplary embodiment of the present disclosure is described. The program product 400 may be a portable compact disc read-only memory (CD-ROM) and include program code, and may be run on a terminal device, such as a personal computer. However, the program product of the present disclosure is not limited thereto. In this document, a readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0089] The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0090] A computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium that can transmit, propagate, or transfer a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0091] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0092] The program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, and the like, as well as conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the receiving computing device, partially on the receiving device, as a stand-alone software package, partially on the receiving computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the receiving computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0093] Through the description of the above embodiments, it is easy for those skilled in the art to understand that the example embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the exemplary embodiment of the present disclosure.

[0094] Furthermore, the figures above are merely illustrative of the processes included in the methods according to exemplary embodiments of the present disclosure and are not intended to be limiting. It is readily understood that the processes illustrated in the figures above do not indicate or limit the temporal order of these processes. Furthermore, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0095] It should be noted that although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the exemplary embodiments of the present disclosure, the features and functions of two or more modules or units described above can be concretized in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into multiple modules or units to be concretized.

[0096] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and embodiments are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the claims.

Claims

1. A privacy-protected camera intelligent monitoring method, characterized in that: The method comprises: Get the camera's surveillance video stream in real time; In a local network, the video stream is anonymized. During the anonymization, an image filled with black pixels is used as a background image. A human body is detected frame by frame for each frame based on a recognition model. If no human body is detected in a current frame, the current frame is updated to the background image. If a human body is detected in the current frame, a human body bounding box is added to the current frame and posture key points are located. An area corresponding to the bounding box in the background image is used to update an area within the bounding box, and an area outside the bounding box in the current frame is used to update a corresponding area portion of the background image. A skeleton representation of the human body is formed based on the posture key points. Joint angles of the skeleton representation are calculated to adjust the joint angles of a 2D virtual human body, and the virtual human body is superimposed within the bounding box. Based on the fall detection algorithm, the movement and posture of the skeleton representation are analyzed in real time to confirm whether there is a fall phenomenon and obtain the health status of the human body; The anonymized video stream and the health status are uploaded to the cloud and provided to an administrator via a smartphone program.

2. The method for intelligent camera monitoring with privacy protection according to claim 1, characterized in that: The recognition model is one of YOLOv8, OpenPose, MediaPipe, PoseNet, and AlphaPose.

3. The method for intelligent camera monitoring with privacy protection according to claim 1, characterized in that: When calculating the joint angles of the skeleton representation, including: Get the joint B whose angle is to be calculated, get the two adjacent joints A and C of joint B, and calculate the following formula: θ=arctan2(u y ,u x )-arctan2(v y ,v x ); Among them, u y and u x are the y-axis component and x-axis component of the vector from point B to A, u y and u x are the y-axis component and x-axis component of the vector from point B to point C, θ is the angle of joint B, and arctan2 is the inverse tangent function.

4. The method for intelligent camera monitoring with privacy protection according to claim 3, characterized in that: After calculating the angle of joint B, the angle of joint B is smoothed based on a third-order polynomial fitting. The formula for the third-order polynomial fitting is: Among them, a0, a1, a2, and a3 are coefficients determined by multi-factor fitting of historical data of joint angles using the least squares method. represents the joint angle θ changing with time t.

5. The method for intelligent camera monitoring with privacy protection according to claim 1, characterized in that: The real-time analysis of the motion and posture of the skeleton representation based on the fall detection algorithm includes: calculating an average velocity and body center of the skeleton representation, a center and an aspect ratio of the bounding box of multiple frames in the video stream to obtain a motion feature; Calculating the change of the motion feature over time to obtain a time change vector; The time change vector is input into a one-dimensional convolutional neural network with residual connection, and a time series analysis is performed on the time change vector to confirm whether there is a fall phenomenon.

6. The method for intelligent camera monitoring with privacy protection according to claim 5, characterized in that: The one-dimensional convolutional neural network includes multiple residual blocks connected in sequence and a fully connected layer. The residual block includes a dilated causal convolution, a first activation function, a causal convolution, and a second activation function connected in sequence. The dilated causal convolution is used to input data, and the causal convolution is used to ensure that the analysis result of each time change vector depends only on the current and previous time change vectors.

7. The method for intelligent camera monitoring with privacy protection according to claim 5, characterized in that: The average speed is the position change of each posture key point in two consecutive frames, and is divided by the time between the two frames; The body center is determined by the midpoint of the shoulders and hips, the midpoint of the shoulders is determined by the positions of the left and right shoulders in the posture key points, and the midpoint of the hips is determined by the positions of the left and right hips in the posture key points; The center of the bounding box is calculated by the coordinates and average of its four corners; The aspect ratio of the bounding box is determined by the ratio of its width to its height; The time change vector is a series of degrees of change of the motion features over time within a sliding window.

8. A privacy-protected camera intelligent monitoring device, characterized in that: include: Acquisition module, used to obtain the monitoring video stream of the camera in real time; a privacy protection anonymity module, configured to anonymize the video stream in a local network; during anonymization, using an image filled with black pixels as a background image, detecting a human body frame by frame for each frame based on a recognition model; updating the current frame image to the background image if no human body is detected; and adding a human body bounding box to the current frame image and locating posture key points if a human body is detected; updating an area within the bounding box using an area in the background image corresponding to the bounding box, and updating a corresponding area portion of the background image using an area outside the bounding box in the current frame image; forming a skeleton representation of the human body based on the posture key points, calculating joint angles of the skeleton representation, adjusting the joint angles of a 2D virtual human body, and superimposing the human body within the bounding box; An analysis module is used to analyze the motion and posture of the skeleton representation in real time based on a fall detection algorithm to confirm whether there is a fall phenomenon and obtain the health status of the human body; The transmission module is used to upload the anonymized video stream and the health status to the cloud and provide them to the administrator through a smartphone program.

9. A privacy-protected camera intelligent monitoring device, characterized in that: include: processor; And a memory arranged to store computer-executable instructions, which, when executed, cause the processor to execute the privacy-protected camera intelligent monitoring method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the privacy-protected camera intelligent monitoring method according to any one of claims 1 to 7 is implemented.

Images