Computer vision anti-drowning early warning method based on splicing of multiple cameras

Abstract

The invention relates to the technical field of computer vision, in particular to a computer vision anti-drowning early warning method based on multi-camera splicing. Acquiring swimming data of swimmers in real time by adopting a multi-path camera; video frame pictures in the video stream are subjected to image preprocessing, and frames collected by multiple cameras at the same time are spliced; detecting frames and key points of swimmers and tracking id of each swimmer in the spliced video frames are recognized in sequence; inputting the detected data into a detection model to judge whether the swimmer is drowning or not, triggering an alarm when judging that the swimmer is drowning, and confirming that the swimmer is a drowning person; and acquiring a screenshot when the drowning person drowns in the spliced video frame, triggering the alarm time, the position of the drowning person and the tracking id, and uploading the screenshot and the tracking id to a cloud end in real time, so that multiple paths of videos are changed into one path, the image processing efficiency is improved, and drowning of the swimmer can be found in time.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision, and in particular to a computer vision drowning prevention warning method based on multi-channel camera splicing. Background Art

[0002] Multi-channel video stream detection is widely used in the security industry, and swimming pools are a practical application scenario. Swimming pools have a large number of drowning incidents every year due to insufficient manual inspection of dangers. With the advancement of computer vision technology and the popularity of surveillance cameras, real-time monitoring can detect drowning incidents, issue alarm signals in advance, and reduce casualties.

[0003] In general scenarios, a single camera cannot cover the entire scene, and multiple cameras are required to work together. Generally, multi-threading technology is used to process each video stream in turn. This technology is more practical in scenarios with simpler business scenarios, and the screen display does not show obvious lags or rotation. In addition, the actual scene of the swimming pool is relatively complex, and the entire business code volume is large. When using multi-threading, there will be obvious lags and rotation of the screen, and only two cameras can be barely run; at the same time, manual investigation is required after the screen is displayed, which is difficult and not conducive to timely detection of drowning swimmers.

[0004] In summary, it is very necessary to propose a computer vision anti-drowning warning method based on multi-channel camera splicing, which can transform multiple video streams into one, improve the image processing efficiency, and facilitate timely detection of drowning swimmers. Summary of the invention

[0005] The purpose of the present invention is to provide a computer vision anti-drowning warning method based on multi-channel camera splicing, which can transform multiple video streams into one, improve the image processing efficiency, and facilitate timely detection of drowning of swimmers.

[0006] To achieve the above object, the present invention adopts a computer vision drowning prevention warning method based on multi-channel camera splicing, which includes the following steps:

[0007] Adjust the camera distortion, use multiple cameras to collect swimming data of swimmers in real time, and save the collected video streams;

[0008] Perform image preprocessing on the video frames in the video stream and stitch the frames captured by multiple cameras at the same time;

[0009] The video stream is processed by dividing it into frames, and the key points and frames of the spliced ​​video frames are annotated. The detection model is trained using the annotated video frame data.

[0010] Identify the detection frame and key points of the swimmers in the spliced ​​video frames and each person tracking ID in turn;

[0011] The detected data is input into the detection model to determine whether the swimmer is drowning, and an alarm is triggered when the swimmer is drowning, confirming that the swimmer is drowning;

[0012] Get a screenshot of the drowning person in the stitched video frame, the location of the drowning person at the time the alarm is triggered, and the tracking ID, and upload them to the cloud in real time.

[0013] Among them, in the step of performing image preprocessing on the video frame pictures in the video stream and splicing the frames collected by multiple cameras at the same time:

[0014] Enter the rtsp stream address in the network stream of the VLC software, and pull the video stream from each camera;

[0015] Create a video capture object for each video stream;

[0016] Read the first frame of any video and store the width and height of this frame;

[0017] Read the first frame of each video in turn, and convert the width and height to the stored frame width and height;

[0018] Use the concatenation functions concatenate, vstack, and hstack in the numpy library of Python to stitch images together;

[0019] Transform the spliced ​​video frames to the width and height of the initial storage frames, and save the successfully spliced ​​frames.

[0020] Among them, in the steps of processing the video stream into frames, annotating the key points and frames of the spliced ​​video frames, and training the detection model using the annotated video frame data:

[0021] Obtaining pool surveillance videos under different environments; the different environments include the same lighting conditions in different pools and different lighting conditions in the same pool;

[0022] Save video frames at certain intervals;

[0023] Each video frame after segmentation is labeled and the data is labeled using a four-point labeling method; the four points in the four-point labeling include the left shoulder, the right shoulder, the left hip, and the right hip;

[0024] Label the key points and frames of the swimmers in the spliced ​​video frames;

[0025] Convert the exported format of the annotated video frame data into the txt format required by yolo training, and use the yolov8pose training program to train the detection model.

[0026] Among them, in the step of inputting the detected data into the detection model to determine whether the swimmer is drowning, and triggering an alarm when it is determined that the swimmer is drowning, and confirming that the swimmer is a drowning person:

[0027] Save the target detection, target tracking, and human key point detection results of each frame into a table created by Pandas;

[0028] Use the timestamp stored in the table to remove the swimmer IDs that did not appear within a fixed time, as well as the swimmer detection frames and human body key points;

[0029] According to the timestamp stored in the table and the ID of the swimmers who have been appearing, the swimmers who have been present for more than the set time are filtered out, and the list information database of the above swimmers is retained as the swimmers to be judged;

[0030] Filter out the swimmers that need to be judged from the table created by Pandas, make drowning judgments, and trigger an alarm when a swimmer is found to be drowning.

[0031] Among them, in the steps of drowning judgment:

[0032] The drowning judgment conditions include: calculating the angles between the vectors of the left shoulder and left hip and the right shoulder and right hip formed by the key points of the human body and the vectors parallel to the horizontal plane, judging whether the angle between the human body and the horizontal plane is close to 90 degrees. When it is close to 90 degrees, it is judged to be a normal state.

[0033] Among them, in the steps of drowning judgment:

[0034] The drowning judgment condition also includes: calculating the aspect ratio of the detection frame of each swimmer to be detected, and judging it as a normal state when the aspect ratio is less than 1.

[0035] Among them, in the step of drowning judgment, the judgment logic sequence is:

[0036] First determine whether the angle between the human body and the horizontal plane is close to 90 degrees, and then calculate the aspect ratio of the detection frame.

[0037] Among them, after first determining that the angle between the human body and the horizontal plane is close to 90 degrees, in the step of calculating the width and height of the detection frame:

[0038] If the aspect ratio is less than 1, it is judged as normal swimming;

[0039] If the aspect ratio is greater than 1, it is considered to be a drowning state;

[0040] If the angle between the human body and the horizontal plane is determined to be between 0 and 60 degrees, it is directly determined as drowning.

[0041] The computer vision drowning prevention and early warning method based on multi-channel camera splicing of the present invention adjusts camera distortion, uses multi-channel cameras to collect swimming data of swimmers in real time, and saves the collected video stream; performs image preprocessing on video frame pictures in the video stream, and splices frames collected by multi-channel cameras at the same time; performs frame processing on the video stream, performs data annotation of key points and frames on the spliced ​​video frames, and uses the video frame data after data annotation to train a detection model; sequentially identifies the detection frame and key points of the swimmer in the spliced ​​video frame and each person tracking ID; inputs the detected data into the detection model to determine whether the swimmer is drowning, and triggers an alarm when it is determined that the swimmer is drowning, confirming that the swimmer is a drowning person; obtains a screenshot of the drowning person when drowning in the spliced ​​video frame, the position of the drowning person at the time of triggering the alarm and the tracking ID, and uploads them to the cloud in real time; realizes that multi-channel video streams are converted into one, improves image processing efficiency, and is conducive to timely detection of drowning of swimmers. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0043] Figure 1 It is a flow chart of the computer vision drowning prevention and early warning method based on multi-channel camera splicing of the present invention.

[0044] Figure 2 It is a schematic diagram of the process of camera frame splicing of the present invention.

[0045] Figure 3 The present invention is a flowchart of the steps of the computer vision drowning prevention and early warning method based on multi-channel camera splicing.

[0046] Figure 4 It is a step flow chart of S200 of the present invention.

[0047] Figure 5 It is a step flow chart of S300 of the present invention.

[0048] Figure 6 It is a step flow chart of S500 of the present invention. DETAILED DESCRIPTION

[0049] Here, exemplary embodiments are described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application.

[0050] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0051] It should be understood that although the terms first, second, third, etc. may be used in the present application to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0052] See also Figure 1 to Figure 6 The present invention provides a computer vision drowning prevention warning method based on multi-channel camera splicing, the method comprising the following steps:

[0053] S100: adjusting the camera distortion, using multiple cameras to collect swimming data of the swimmer in real time, and saving the collected video streams.

[0054] In this embodiment, the camera distortion is adjusted, and multiple cameras are used to collect swimming data of swimmers in real time, and the collected video streams are saved in the local host memory space of the Ubuntu system.

[0055] S200: performing image preprocessing on video frame images in the video stream, and splicing frames captured by multiple cameras at the same time.

[0056] In this embodiment, image preprocessing is performed on the video frame images in the video stream, and the frames captured by multiple cameras at the same time are spliced ​​to convert multiple video streams into one, thereby improving the image processing efficiency and facilitating the timely detection of drowning swimmers.

[0057] Specifically, Figure 4 As shown, this step can be divided into the following sub-steps:

[0058] S201: Enter the rtsp stream address in the network stream of the VLC software, and each camera pulls the video stream;

[0059] S202: Create a video capture object for each video stream respectively;

[0060] S203: Read the first frame of any video and store the width and height of this frame;

[0061] S204: Read the first frame of each video in turn, and convert the width and height to the stored frame width and height;

[0062] S205: Use the concatenation functions concatenate, vstack, and hstack in the numpy library of the Python language to stitch images together;

[0063] S206: Convert the spliced ​​video frames to the width and height of the initially stored frames, and save the successfully spliced ​​frames.

[0064] In this embodiment, firstly, the rtsp stream address is input in the network stream of the VLC software, and each camera pulls the video stream; then a video capture object is created for each video stream; then the first frame of any video is read, and the width and height of this frame are stored; then the first frame of each video is read in turn, and the width and height are converted to the stored frame width and height for easy splicing; then the splicing functions concatenate, vstack, and hstack in the numpy library of the python language are used to stitch the pictures; finally, the spliced ​​video frames are converted to the width and height of the frames initially stored, and the successfully spliced ​​frames are saved.

[0065] In the above steps, the camera network segment information is configured according to the camera configuration software provided by the manufacturer, and all cameras are configured in the same LAN environment. After receiving each frame of each video stream, image preprocessing is performed. Check that each camera successfully pulls the video stream, and obtain each rtsp stream address according to the rtsp video stream address format based on the camera account and password information. Enter the rtsp stream address in the network streaming option of the VLC software. The normal display of the video surveillance screen indicates that the stream is pulled successfully.

[0066] In the above steps, if Figure 2 As shown in the figure, the entire video frame splicing process is implemented in Python. Create a video capture object for each video stream. Read the first frame of any video and store the width and height of this frame. Read the first frame of each video in turn, and convert the width and height to the stored frame width and height for easy splicing. Splice all the video frames that need to be spliced ​​in the appropriate position by splicing left and right and up and down. Convert the spliced ​​video frame to the width and height of the frame that was stored at the beginning, and save the successfully spliced ​​frame.

[0067] In the above steps, a normal swimming pool uses two cameras for a small pool, and a standard pool uses four cameras. For two-way stitching, we generally stitch left and right, use the Python language numpy library function concatenate to do the image stitching parameter axis set to 1 or use hstask for left and right stitching. For four-way cameras, use top, bottom, left and right stitching, use concatenate, set its parameter axis to 0 to do top and bottom stitching of two of them, or use vstask to stack vertically, and the other two are the same process, save the two stitched frames, and then use concatenate, set its parameter to 1, and stitch the two stitched video frames above to get four-way stitching frames. Whether it is two-way or four-way, the last step is to convert it to the original frame size and store it.

[0068] S300: Process the video stream by dividing it into frames, annotate the spliced ​​video frames with key points and frames, and train the detection model using the annotated video frame data.

[0069] In this embodiment, the video stream is frame-processed, key points and frames are annotated for the spliced ​​video frames, and the detection model is trained using the annotated video frame data.

[0070] Specifically, Figure 5 As shown, this step can be divided into the following sub-steps:

[0071] S301: Acquire swimming pool surveillance videos under different environments; wherein the different environments include the same lighting conditions in different swimming pools and different lighting conditions in the same swimming pool;

[0072] S302: Save video frames at regular intervals;

[0073] S303: labeling each segmented video frame image, using a four-point labeling method to label data; wherein the four points in the four-point labeling include a left shoulder, a right shoulder, a left hip, and a right hip;

[0074] S304: labeling key points and frames of the swimmer target in the spliced ​​video frame;

[0075] S305: Convert the exported format of the annotated video frame data into the txt format required by yolo training, and use the yolov8pose training program to train the detection model.

[0076] In this embodiment, first, obtain the swimming pool monitoring video under different environments; the different environments include the same lighting conditions in different swimming pools and different lighting conditions in the same swimming pool; then save the video frames at a certain interval, if the interval is too short, the image content between the two frames is not much different, and the effect on training a good model file is not obvious; then mark each video frame image after segmentation, and use the four-point marking method for data marking; the four points in the four-point marking include the left shoulder, right shoulder, left hip, and right hip; then make data marking of the key points and frames of the swimmer target in the spliced ​​video frame; finally, convert the exported format of the marked video frame data into the txt format required by yolo training, and use the yolov8pose training program to train the detection model.

[0077] In the above steps, the real-time video frames of each camera are read cyclically during the acquisition process, and the multi-channel stitching images of each stitching are saved after the image stitching program. About 5,000 video frames are collected in various environments, including morning, noon and evening time periods, with or without lights, etc., and data are annotated using coco-Annotator software. The four-point annotation method is used instead of the seventeen-point annotation, and only the left shoulder, right shoulder, left hip, and right hip are annotated. Since only four key points are used to meet the project requirements, the annotation process is simplified, and the model can detect the key points more accurately. It is faster than the seventeen-point annotation, and the optimized annotation method can more accurately identify the key points that need to be detected in complex scenes such as swimming pools.

[0078] S400: sequentially identifying the detection frame and key points of the swimmer in the spliced ​​video frame and each person tracking ID.

[0079] In this embodiment, the detection model is used to sequentially identify the detection frame and key points of the swimmer in the spliced ​​video frame and each person tracking ID.

[0080] In the above steps, the pre-trained yolov8 framework model yolov8pose.pt is added to the built algorithm program. The detection frame and key points of the swimmers in the spliced ​​video frames and the tracking ID of each person are identified in turn through the yolov8 detection program and bytetrack program.

[0081] S500: inputting the detected data into a detection model to determine whether the swimmer is drowning, and triggering an alarm when it is determined that the swimmer is drowning, confirming that the swimmer is a drowning person;

[0082] In this embodiment, a detection model is input based on the detected data to determine whether the swimmer is drowning, and an alarm is triggered when it is determined that the swimmer is drowning, confirming that the swimmer is the drowning person.

[0083] Specifically, Figure 6 As shown, this step can be divided into the following sub-steps:

[0084] S501: Save the target detection, target tracking, and human key point detection results of each frame into a table created by Pandas;

[0085] S502: using the timestamp stored in the table to remove the swimmer IDs that have not appeared within a fixed time, as well as the swimmer detection frames and human body key points;

[0086] S503: Filter out swimmers who have been there for more than a set time according to the timestamp stored in the table and the ID of the swimmers who have been there, and retain the list information database of the above swimmers as the swimmers to be judged;

[0087] S504: Filter out the swimmers that need to be judged from the table created by Pandas, make a drowning judgment, and trigger an alarm when it is judged that the swimmer is drowning.

[0088] In this embodiment, the target detection, target tracking, and human key point detection results of each frame are first saved in a table created by Pandas; then the timestamp stored in the table is used to remove the swimmer ID, the swimmer detection frame, and the human key points that have not appeared within a fixed time; the fixed time is set according to the actual situation of the specific swimming pool; then, according to the timestamp stored in the table and the swimmer ID that has always appeared, the swimmers who have been there for more than the set time are screened out, and the list information library of the above swimmers is retained as the swimmers who need to be judged; wherein the set time is set according to the actual situation of the specific swimming pool host configuration and the size of the swimming pool; finally, the swimmers who need to be judged are screened out from the table created by Pandas, and drowning judgment is performed, and an alarm is triggered when the swimmer is judged to be drowning.

[0089] The drowning judgment conditions include: calculating the angle between the vectors of the left shoulder and left hip and the right shoulder and right hip formed by the key points of the human body and the vectors parallel to the horizontal plane, and judging whether the angle between the human body and the horizontal plane is close to 90 degrees; for example, if 17 frames or more in 20 consecutive frames are close to 90 degrees, it is judged to be in a normal state. The drowning judgment conditions also include: calculating the aspect ratio of the detection frame of each swimmer to be detected; for example, if 17 frames or more in 20 consecutive frames have a width and height less than 1, it is judged to be in a normal state.

[0090] In the above two types of drowning judgment, the judgment logic order is: if it is determined that the angle between the human body and the horizontal plane is close to 90 degrees, then calculate the aspect ratio of the detection frame; if the aspect ratio is less than 1, it is judged as normal swimming; if the aspect ratio is greater than 1, it is judged as drowning; if it is determined that the angle between the human body and the horizontal plane is between 0 degrees and 60 degrees, it is directly judged as drowning.

[0091] The drowning identification algorithm determines whether someone is drowning. If it is, it will be marked with a red frame on the large display screen, and relevant data such as the drowning screenshot and drowning time will be saved. The bracelet worn by the lifeguard and the alarm sound will issue a prompt to remind the lifeguard to observe the situation.

[0092] The drowning detection algorithm determines whether someone is drowning. If the drowning judgment result is normal swimming, the monitoring screen will be displayed normally, and swimmers who are swimming normally will be marked with a green frame.

[0093] S600: Obtain a screenshot of the drowning person in the spliced ​​video frame, the location of the drowning person at the time of triggering the alarm, and the tracking ID, and upload them to the cloud in real time.

[0094] In this embodiment, after determining that someone is drowning, a screenshot of the drowning person in the spliced ​​video frame, the location of the drowning person at the time of triggering the alarm, and the tracking ID are obtained and uploaded to the cloud in real time.

[0095] In the present invention, first check whether the existing cameras in the swimming pool can monitor the entire swimming pool, and if it can, check whether each camera can pull the video stream normally. If the conditions are met, there is no need to reinstall a new camera, and the existing camera can be used to pull the stream for program processing. If the conditions are not met and a new camera needs to be installed, first, install a camera with a focal length of 4mm on the walls or pillars around the swimming pool at a distance of 3 to 4 meters. The lens angle is between 35 degrees and 45 degrees. The number of lenses and the installation position depend on the size of the swimming pool to ensure that the swimming pool is not covered at all four corners and that the images of each lens are crossed. Secondly, the shooting device is connected to the POE network switch through a POE network cable, and the high-performance host is also connected to the local area network to ensure that the host can access the camera, and the real-time streaming media transmission image can be obtained through the H.264 solution based on the RTSP protocol, and the image resolution is 108 0p, frame rate is 25fps; then the real-time image is first processed by the image distortion correction algorithm on the high-performance host. The distortion of the image taken by the camera will cause errors and affect the effect of image recognition. Therefore, it is necessary to obtain some parameters of the camera, including: intrinsic parameter matrix, extrinsic parameter matrix, distortion coefficient. The parameters can be obtained from the camera manufacturer, or converted into a camera coordinate system through the intrinsic parameter matrix, and the camera coordinates of the image are corrected by the distortion coefficient. After correction, the camera coordinate system is converted into an image pixel coordinate system through the intrinsic parameter matrix, and the pixel values ​​of the source image coordinates are assigned to the new image coordinates to form a new image.

[0096] Write a python script to read the multi-channel splicing of video streams and save each splicing frame. Use the script to collect enough video frame splicing pictures of the swimming pool. Perform data cleaning to remove irregular pictures, pictures of no one swimming, and relatively blurred pictures. Use the annotation tool to annotate the pictures. It takes about 5,000 pictures in different scenes to annotate each venue from 0. After the annotation is completed, it is converted to the txt format required for yolov8 training. After about 600 rounds of training, the accuracy can reach 0.98. Take the best weight best.pt for the detection model.

[0097] Use yolov8pose framework, best.pt and bytetrack tracking algorithm to get the detection frame, key points, and target tracking id information. Use pandas to create a table with 18 columns. Each column stores the tracking id, the person's label person_id in the list, the timestamp timestamp for entering the list, the drowning status status, the lane and section for judging the middle value of the drowning status, the box information x, y, w, h, and the horizontal and vertical coordinates of the four points of the left shoulder, right shoulder, left hip, and right hip for the key point information of the skeleton detection. Use pandas to operate on the timestamp and id of the list to filter out the person_id that needs to be detected and save it. Get each column of information corresponding to the person_id in the list and input it into the drowning detection program to determine whether it is drowning.

[0098] There are two main conditions for drowning judgment: First, the angle between the vectors of the left shoulder and left hip and the right shoulder and right hip formed by the key points of the human body and the vectors parallel to the horizontal plane is calculated to determine whether the angle between the human body and the horizontal plane is close to 90 degrees. The range is between 60 degrees and 90 degrees. If more than 17 frames in 20 consecutive frames are close to 90 degrees, it is determined to be in a normal state. Second, the aspect ratio of the detection frame of each swimmer to be detected is calculated. If more than 17 frames in 20 consecutive frames have a width and height less than 1, it is determined to be in a normal state. This condition mainly assists the first condition. After practical verification, when encountering external factors such as different lighting or low data annotation accuracy, this judgment criterion can effectively supplement the shortcomings of the first condition to prevent false alarms and missed alarms. Among them, 20 frames and 17 frames can be adjusted according to the actual situation of the swimming pool; judgment logic order: according to condition one, if it is determined that the angle between the human body and the horizontal plane is close to 90 degrees, then enter condition two to judge. If the aspect ratio is less than 1, it is determined to be normal swimming. If the aspect ratio is greater than 1, it is determined to be a drowning state; according to condition 1, if the angle between the human body and the horizontal plane is between 0 and 60 degrees, it is directly determined to be drowning. The above judgment logic is the result obtained by comprehensively analyzing the judgment situations between 20 consecutive frames and 60 frames, to prevent misjudgment due to one or two special frames, such as the limb changes of a person swimming in a pool.

[0099] When no drowning occurs, the status value is 1. When drowning occurs, the status value is set to 0 and stored in the Redis database. The drowning person is marked with a red frame on the large screen. Peripheral alarm devices such as bracelets and speakers detect changes in the status in the Redis database and issue an alarm in time. Screenshots of drowning, drowning time, name of drowning pool, etc. are uploaded to the cloud for storage.

[0100] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the contents disclosed herein. The present application is intended to cover any variations, uses or adaptations of the present application, which follow the general principles of the present application and include common knowledge or customary technical means in the art that are not disclosed in the present application.

[0101] It should be understood that the present application is not limited to the exact construction that has been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof.

Claims

1. A computer vision drowning prevention warning method based on multi-channel camera splicing, characterized in that: The steps include: Adjust the camera distortion, use multiple cameras to collect swimming data of swimmers in real time, and save the collected video streams; Perform image preprocessing on the video frames in the video stream and stitch the frames captured by multiple cameras at the same time; The video stream is processed by dividing it into frames, and the key points and frames of the spliced ​​video frames are annotated. The detection model is trained using the annotated video frame data. Identify the detection frame and key points of the swimmers in the spliced ​​video frames and each person tracking ID in turn; The detected data is input into the detection model to determine whether the swimmer is drowning, and an alarm is triggered when the swimmer is drowning, confirming that the swimmer is drowning; Get a screenshot of the drowning person in the stitched video frame, the location of the drowning person at the time the alarm is triggered, and the tracking ID, and upload them to the cloud in real time.

2. The computer vision drowning prevention warning method based on multi-channel camera splicing as claimed in claim 1, characterized in that: In the step of preprocessing the video frames in the video stream and stitching the frames collected by multiple cameras at the same time: Enter the rtsp stream address in the network stream of the VLC software, and pull the video stream from each camera; Create a video capture object for each video stream; Read the first frame of any video and store the width and height of this frame; Read the first frame of each video in turn, and convert the width and height to the stored frame width and height; Use the concatenation functions concatenate, vstack, and hstack in the numpy library of Python to stitch images together; Transform the spliced ​​video frames to the width and height of the initial storage frames, and save the successfully spliced ​​frames.

3. The computer vision drowning prevention and warning method based on multi-channel camera splicing as claimed in claim 1 is characterized in that: In the steps of dividing the video stream into frames, annotating the key points and frames of the spliced ​​video frames, and training the detection model using the annotated video frame data: Obtaining pool surveillance videos under different environments; the different environments include the same lighting conditions in different pools and different lighting conditions in the same pool; Save video frames at certain intervals; Each video frame after segmentation is labeled and the data is labeled using a four-point labeling method; the four points in the four-point labeling include the left shoulder, the right shoulder, the left hip, and the right hip; Label the key points and frames of the swimmers in the spliced ​​video frames; Convert the exported format of the annotated video frame data into the txt format required by yolo training, and use the yolov8pose training program to train the detection model.

4. The computer vision drowning prevention warning method based on multi-channel camera splicing as claimed in claim 3 is characterized in that: In the step of inputting the detected data into the detection model to determine whether the swimmer is drowning, and triggering an alarm when determining that the swimmer is drowning, and confirming that the swimmer is a drowning person: Save the target detection, target tracking, and human key point detection results of each frame into a table created by Pandas; Use the timestamp stored in the table to remove the swimmer IDs that did not appear within a fixed time, as well as the swimmer detection frames and human body key points; According to the timestamp stored in the table and the ID of the swimmers who have been appearing, the swimmers who have been present for more than the set time are filtered out, and the list information database of the above swimmers is retained as the swimmers to be judged; Filter out the swimmers that need to be judged from the table created by Pandas, make drowning judgments, and trigger an alarm when a swimmer is found to be drowning.

5. The computer vision drowning prevention and warning method based on multi-channel camera splicing as claimed in claim 4 is characterized in that: In the steps of drowning assessment: The drowning judgment conditions include: calculating the angles between the vectors of the left shoulder and left hip and the right shoulder and right hip formed by the key points of the human body and the vectors parallel to the horizontal plane, judging whether the angle between the human body and the horizontal plane is close to 90 degrees. When it is close to 90 degrees, it is judged to be a normal state.

6. The computer vision drowning prevention and warning method based on multi-channel camera splicing as claimed in claim 5, characterized in that: In the steps of drowning assessment: The drowning judgment condition also includes: calculating the aspect ratio of the detection frame of each swimmer to be detected, and when the aspect ratio is less than 1, it is determined to be a normal state.

7. The computer vision drowning prevention and warning method based on multi-channel camera splicing as claimed in claim 6 is characterized in that: In the steps of drowning judgment, the logical order of judgment is: First determine whether the angle between the human body and the horizontal plane is close to 90 degrees, and then calculate the aspect ratio of the detection frame.

8. The computer vision drowning prevention and early warning method based on multi-channel camera splicing as claimed in claim 7, characterized in that: After first determining that the angle between the human body and the horizontal plane is close to 90 degrees, in the step of calculating the width and height of the detection frame: If the aspect ratio is less than 1, it is judged as normal swimming; If the aspect ratio is greater than 1, it is considered to be a drowning state; If the angle between the human body and the horizontal plane is determined to be between 0 and 60 degrees, it is directly determined as drowning.

Images