A swimming pool video processing method and system that provides personal privacy-protecting videos and selectively excludes protected videos depending on the state of drowning

Abstract

The present invention relates to computer vision technology for processing images captured in an underwater environment such as a swimming pool, and to an image processing method and system that selectively excludes images for protecting personal privacy according to a drowning state by combining a preprocessing pipeline that comprehensively considers temporal characteristics, optical characteristics, and spatial structure, joint information generation control based on area thresholds, skeleton-based state analysis, and drowning determination logic based on temporal continuity verification.

Description

Technology Field

[0001] The present invention relates to computer vision technology for processing images captured in an underwater environment such as a swimming pool, and more specifically, to an image processing method and system for detecting human objects from images captured in a public facility environment such as a swimming pool, providing a privacy-protected image for protecting personal privacy, and selectively excluding the privacy-protected image depending on the state of being submerged.

[0002] In particular, the present invention is technically characterized by combining a preprocessing pipeline that comprehensively considers temporal, optical, and spatial characteristics, joint information generation control based on area thresholds, skeleton-based state analysis, and drowning determination logic based on temporal continuity verification, in order to solve the problem where image noise caused by water spray, surface reflections, and wave ripples unique to swimming pool environments significantly degrades the accuracy of human object detection and joint estimation. Background Technology

[0003] Swimming pools, which are typically facilities for enjoying water activities and swimming, are highly beneficial to modern life, providing not only physical fitness through exercise but also health, leisure activities, and relaxation; as a result, the number of facilities and the number of users are on the rise.

[0004] A key point regarding the use or management of such swimming pools is the need to pay particular attention to safety accidents. Generally, safety accidents in swimming pools include falls caused by slipping on wet surfaces, drowning accidents resulting from inexperienced swimmers or sudden heart or brain-related conditions, and drowning accidents that occur when a person gets trapped in a drain during the drainage process.

[0005] Although swimming pool operators station lifeguards in the pool to prepare for drowning accidents in such pools, it is natural that it is difficult to monitor the underwater conditions inside the pool, which are not easily visible to the human eye, especially when a drowning occurs and the person exhibits abnormal behavior or remains motionless and submerged.

[0006] Accordingly, most swimming pools install cameras inside the pool to monitor and store footage; however, as managers constantly check the situation inside the pool via camera footage, swimmers' faces and bodies can be identified, raising concerns about privacy issues and the infringement of personal information and portrait rights.

[0007] Conventional video-based safety management systems proposed as a solution to this often applied processing such as face blurring or masking to protect personal privacy. However, this method made it difficult to identify the victim's posture, location, and surrounding environment information in the event of a drowning accident, thereby limiting rescue judgment and post-accident analysis.

[0008] Furthermore, in swimming pool environments, the continuous occurrence of characteristic noise such as water spray, surface reflections, and waves degrades the accuracy of human object detection and state analysis. Accordingly, conditional image processing technology is required to simultaneously satisfy the protection of personal privacy and the response to safety incidents. Prior art literature

[0009] Republic of Korea Patent Registration No. 10-2432675 (Registration Date: August 10, 2022) The problem to be solved

[0010] The present invention, devised in response to the problems and necessities of the aforementioned prior art, aims to solve the following problems.

[0011] (1) Suppressing noise generated in the swimming pool environment to improve the reliability of human object detection and state analysis.

[0012] (2) Providing a privacy-protected image by default for a person object in a normal state.

[0013] (3) For objects found to be in a state of drowning, the privacy protection footage is selectively excluded to maintain the original footage, thereby enabling a rapid response.

[0014] (4) Reducing false alarms by verifying time continuity rather than immediately determining suspected drowning as a drowning state. means of solving the problem

[0015] The present invention for achieving the above objective presents the following means of solution.

[0016] The image processing method of the present invention is characterized by comprising: a swimming pool noise removal preprocessing step for processing an image obtained from a swimming pool camera to suppress water spray, surface reflection, and wave noise; a human object detection and tracking step for detecting and tracking a human object from the image; a step for determining whether to perform joint information generation based on an area threshold, which generates a joint probability map corresponding to a plurality of human joint positions for an area of ​​a human object; a joint candidate point estimation and joint information generation step for estimating joint candidate points using the local maximum value of the joint probability map and generating joint information by selecting only joint candidate points whose reliability is greater than or equal to a threshold value; a skeleton-based representation generation step through joint information linkage for generating a skeleton-based representation by linking joint information; a drowning state determination step for analyzing the state of a human object to determine whether it is in a drowning state; a step for providing a skeleton-based personal privacy protection image when it is not in a drowning state; and a protection image selective exclusion control step (h) for selectively excluding the personal privacy protection image when it is in a drowning state.

[0017] The image processing system of the present invention is characterized by comprising: an image receiving unit for receiving real-time images; a preprocessing unit for suppressing noise; an object detection unit for detecting human objects; an object tracking unit for assigning a tracking ID to a human object and tracking it; a joint information generation unit for generating a joint probability map, estimating joint candidate points, and generating joint information; a joint information linking unit for generating a skeleton-based representation by linking joint information; a state analysis unit for determining whether a drowning state exists and verifying time continuity; a protection image generation unit for providing a personal privacy protection image; and a control unit for controlling the operation of the protection image generation unit according to the drowning state to selectively exclude the personal privacy protection image. Effects of the invention

[0018] According to the above-mentioned solution, the following effects can be obtained.

[0019] (1) The reliability of human object detection and state analysis is improved by suppressing swimming pool-specific noise (splash / reflection / waves).

[0020] (2) Under normal conditions, a privacy-protected image is provided by default to reduce the exposure of external identification information.

[0021] (3) In the event of drowning, the protected video is selectively excluded to maintain the original video, thereby enabling quick and accurate detection and response to the accident.

[0022] (4) Reduce false alarms by performing time continuity verification instead of immediately determining suspected drowning.

[0023] (5) Control of joint information generation based on bounding box area threshold improves computational efficiency and reduces unnecessary processing. Brief explanation of the drawing

[0024] FIG. 1 is an exemplary configuration diagram of an image processing system according to the present invention. FIG. 2 is an operation flowchart of an image processing method according to the present invention. FIG. 3 is an example diagram of the human object detection / tracking process according to the present invention. FIG. 4 is an example diagram of the process for area threshold-based joint information generation control, joint candidate point estimation, and drowning state determination (temporal continuity verification) according to the present invention. FIG. 5 is an example diagram of privacy protection image provision and selective exclusion control of protection images according to drowning status according to the present invention. Specific details for implementing the invention

[0025] Hereinafter, the configuration of the present invention is to be described in detail. This description is based on embodiments for implementing the present invention and is not intended to limit the technical concept pursued by the present invention to the form described in the embodiments described in this document, and should be understood to include various modifications, equivalents, and / or alternatives of the embodiments of the present invention.

[0027] Each step of the image processing method according to the present invention is,

[0028] A real-time video reception step (a) for receiving video captured from an underwater camera in a swimming pool;

[0029] A swimming pool noise removal preprocessing step (b) that performs preprocessing to suppress water spray, surface reflections, and wave noise occurring in a swimming pool environment;

[0030] Person object detection and tracking step (c) for detecting and tracking person objects from an image;

[0031] A step (d) for determining whether to perform area threshold-based joint information generation, which generates a joint probability map corresponding to multiple human body joint positions for the area of ​​the above-mentioned person object;

[0032] A joint candidate point estimation and joint information generation step (e) that estimates joint candidate points using the local maximum value of the joint probability map above, and generates joint information by selecting only joint candidate points whose confidence level is greater than or equal to a threshold value;

[0033] A step (f) for generating a skeleton-based representation through joint information connection, which generates a skeleton-based representation by connecting the joint information according to a predefined human body structure;

[0034] A drowning state determination step (g) through binary classification processing and time continuity verification that analyzes the state of the above-mentioned person object to determine whether it is in a drowning state;

[0035] Step (h) of providing a privacy protection image according to a state of intoxication, which provides a privacy protection image including the above-mentioned skeleton-based representation when the state of intoxication is not a state of intoxication;

[0036] It consists of a protection video selective exclusion control step (i) that selectively excludes the above personal privacy protection video in the case of drowning.

[0038] The image processing system according to the present invention is characterized by comprising: an image receiving unit (100) that receives real-time images from an underwater camera of a swimming pool; a preprocessing unit (200) that performs preprocessing to suppress water spray, surface reflections, and wave noise; an object detection unit (300) that detects human objects; an object tracking unit (400) that assigns a tracking ID to a human object and tracks it; a joint information generation unit (500) that performs joint probability map generation, joint candidate point estimation, and joint information generation; a joint information linking unit (600) that connects joint information to generate a skeleton-based representation; a state analysis unit (700) that determines whether a person is in a drowning state and performs time continuity verification; a protection image generation unit (800) that provides a personal privacy protection image; and a control unit (900) that controls the operation of the protection image generation unit according to the drowning state to selectively exclude the personal privacy protection image.

[0040] The operation of the present invention as described above is explained in detail as follows in the order of execution.

[0041] 1. Real-time video reception

[0042] A video receiving unit (100) receives real-time video from a video equipment (underwater camera; 1) in a swimming pool. The underwater camera may be installed on the bottom, side, or under the water surface of the swimming pool, and may be a fixed camera or a variable camera including pan, tilt, and zoom functions.

[0043] 2. Swimming pool noise removal preprocessing

[0044] In the preprocessing unit (200), noise generated in a swimming pool environment is characterized by (i) water spray (high-frequency components with short lifespans), (ii) water surface reflection (high-intensity sparkle / shimmering), and (iii) water wave (low-frequency deformation / shaking of the background). The present invention applies a preprocessing pipeline including temporal filtering + reflection suppression + background modeling + structure-preserving smoothing to suppress these noises.

[0045] 2.1 Spray Suppression (Time-based)

[0046] It detects instantaneously occurring components using the difference in consecutive frames.

[0047] Since the splash component has a short duration, pixels / areas existing for less than a certain number of frames at the same location (or vicinity) are identified as splashes and suppressed.

[0048] 2.2 Water Surface Reflection / Shimmer Suppression (Brightness Normalization + Reflection Mask)

[0049] Surface reflections often appear as high-intensity spikes in specific areas. They are normalized using brightness components.

[0050] Reflection candidates are masked by setting a high brightness threshold.

[0051] The reflection mask cleans up point noise using morphological operations (dilation / erosion / opening / closing) and performs local corrections (inpainting or marginal interpolation) on the reflection area.

[0052] 2.3 Wave Suppression (Background Modeling Based)

[0053] Background changes caused by waves can appear as low-frequency distortions throughout the frame, so a time-averaged background model is constructed.

[0054] Suppresses wave-like change components by utilizing the difference from the background model (Composition example: background stabilization).

[0055] 3. Human Object Detection and Object Tracking

[0056] The object detection unit (300) calculates a set of human object bounding boxes from the preprocessed image.

[0057] This is for identifying human objects from an input image, and the bounding box of each object is extracted using YOLO or a similar deep learning-based object detection model.

[0058] Object tracking of the object tracking unit (400) can be performed by combining inter-frame box similarity (e.g., IoU-based distance) and prediction-based tracking.

[0059] Each object is assigned a unique tracking ID, and time information at the time of initial creation is recorded.

[0060] 4. Determining whether to perform joint information generation based on area threshold

[0061] Calculate the bounding box area of ​​the person object.

[0062] Joint candidate point estimation and joint information generation are performed only when the area exceeds a threshold. This prevents unnecessary joint estimation for distant objects or noisy objects within the image that lack sufficient resolution.

[0063] 5. Joint Candidate Point Estimation and Joint Information Generation

[0064] The steps in the joint information generation unit (500) are not dependent on a specific model name and are performed as follows: (1) feature extraction → (2) joint probability map generation → (3) local maximum / non-maximum suppression → (4) subpixel correction → (5) connection consistency verification → (6) final joint information determination.

[0065] 5.1 Region of Interest (ROI) Normalization

[0066] 5.2 Feature Extraction Unit

[0067] 5.3 Joint Probability Map (Heatmap) Generation Section

[0068] 5.4 Joint Candidate Point Extraction and Subpixel Correction

[0069] 5.5 Verification of Joint Connection Consistency (Connection Strength Map)

[0070] 5.6 Confirmation of Joint Information (Reliability Threshold)

[0071] 6. The joint information connection unit (600) connects the joint information determined according to a predefined human body structure model, connects the joint information based on the skeleton, and generates a representation.

[0072] 7. The state analysis unit (700) determines the suspicion of drowning (binary classification processing).

[0073] 8. Only when the binary classification processing result is maintained continuously for more than a preset time for the same tracking ID, time continuity is verified to determine the drowning state.

[0074] 9. If it is determined to be drowning, the protection video generation unit (800) provides a personal privacy protection video.

[0075] 10. Control unit (900): Selectively excludes protection images according to the drowning state.

[0076] Through the system and control steps described above, it is possible to protect swimmers' privacy and minimize elements of infringement on portrait rights and personal information, while shortening the response time for drowning accidents and strengthening the ability to prevent safety accidents, and in particular, improving trust between users and operators.

[0078] Even if the operation and effects according to the configuration of the present invention were not explicitly described while explaining the embodiments of the present invention as described above, it is natural that the effects predictable by said configuration should also be acknowledged. Explanation of the symbols

[0079] 1: Underwater camera 100: Video receiver 200: Preprocessing section 300: Object detection unit 400: Object tracking section 500: Joint Information Generation Unit 600: Joint Information Connection 700: State Analysis Department 800: Protected Image Generation Unit 900: Control unit

Claims

Claim 1 A video processing method that protects the privacy of normal users regarding video captured by an underwater camera installed within a swimming pool tank while providing the original video in the event of a drowning situation, comprising: a real-time video reception step (a) for receiving video captured by an underwater camera installed on the bottom, side, or below the water surface within the swimming pool tank; a swimming pool noise removal preprocessing step (b) for performing preprocessing to suppress water spray, water surface reflection, and wave noise occurring in the swimming pool environment by comprehensively considering temporal characteristics, optical characteristics, and spatial structure from the received video; a human object detection and tracking step (c) for calculating a human object bounding box from the preprocessed video; a step for determining whether to perform area threshold-based joint information generation, which generates a joint probability map corresponding to a plurality of human joint positions only when the bounding box area of ​​the human object is greater than or equal to a preset threshold; a joint candidate point estimation and joint information generation step (e) for estimating joint candidate points using the local maximum value of the joint probability map, and generating joint information by selecting only joint candidate points whose reliability is greater than or equal to the threshold; and a skeleton-based representation formed by connecting the joint information according to the human body structure defined in step (e). A step (f) for generating a skeleton-based representation through the connection of generated joint information; a step (g) for determining whether the person object is in a drowning state by analyzing the state of the person object and determining the drowning state through binary classification processing and time continuity verification if the suspected drowning state for the same Tracking ID is maintained for a preset time or longer; and a step (h) for providing a privacy-protected video according to the drowning state, which provides a privacy-protected video including the skeleton-based representation if the person is not in a drowning state.A swimming pool image processing method comprising a protection image selective exclusion control step (i) that selectively excludes the protection image only when the protection image is in a state of drowning, characterized by providing a protection image that protects personal privacy and selectively excludes the protection image according to the state of drowning. Claim 2 delete Claim 3 delete Claim 4 A drowning detection and personal privacy protection system installed within a swimming pool tank to protect the privacy of normal users while providing original video in the event of a drowning situation, comprising: an underwater camera that films the interior of the swimming tank; an image preprocessing unit that removes water spray, surface reflections, and wave noise by comprehensively considering temporal characteristics, optical characteristics, and spatial structure of the video input from the underwater camera; an object detection unit that calculates and tracks a bounding box of a human object from the preprocessed video; a joint information generation unit that generates joint information by creating a joint probability map only when the bounding box area of ​​the human object detected by the object detection unit is greater than or equal to a preset threshold; a skeleton generation unit that generates a skeleton-based representation using the joint information; a state analysis unit that analyzes the state of the human object using the skeleton-based representation and determines a drowning state only when a suspected drowning state is continuously maintained for a preset time or longer for the same Tracking ID; a protection video generation unit that generates a personal privacy protection video for human objects that are not in a drowning state according to the determination result of the state analysis unit; and connected to the state analysis unit, and in the case of not being in a drowning state, the protection video A swimming pool image processing system that provides a privacy-protected image and selectively excludes the protection image according to the drowning state, characterized by including a control unit that outputs a privacy-protected image generated by a generation unit and controls the output of the original image by stopping the output of the privacy-protected image for the person object in the event that it is determined to be in a drowning state.

Images