Knowledge distillation-based drowning identification method and system of low-frame-rate SlowFast model

Abstract

The invention discloses a drowning identification method and system of a low frame rate SlowFast model based on knowledge distillation. The method comprises the following steps: acquiring a to-be-recognized image; inputting the to-be-recognized image into a drowning recognition model for drowning behavior recognition to obtain a recognition result; wherein the drowning identification model comprises a SlowFast model, and the drowning identification model is a model obtained by training a Fast branch through a knowledge distillation technology, combining the trained Fast branch with a Slow branch parameter of an open source SlowFast pre-training model to form a network, and carrying out transfer learning through a swimming pool environment behavior data set; and outputting the identification result. By implementing the method provided by the invention, on the premise of keeping a good recognition effect, a lower frame rate condition can be adapted, and the drowning recognition accuracy is improved.

Description

Technical Field

[0001] The present invention relates to a drowning recognition method, and more specifically to a drowning recognition method and system based on a low frame rate SlowFast model of knowledge distillation. Background Art

[0002] In indoor swimming pools, the pool environment presents a variety of complex characteristics, which bring many challenges to target detection and tracking in video surveillance systems. The main background of the pool is the water body. The movements of swimmers will cause the water surface to fluctuate, further increasing the difficulty of target detection and tracking. When swimmers dive underwater, the refraction, turbidity and wave effects of the water make it difficult to clearly observe most body parts. Whether it is natural light or artificial lighting, reflections will be generated on the water surface. Although it can be eliminated to a certain extent through preprocessing technology, the location of the reflection will change due to the fluctuation of the water surface, which increases the difficulty of visual recognition of the target. In addition, swimmers come from different age groups and they show various postures and behaviors in the water or on the shore, which puts higher requirements on the detection algorithm. In addition to the swimmers themselves, there are many fixed or moving objects in the pool and surrounding environment, such as stands, life-saving equipment, training equipment and personal items, which may become interference sources.

[0003] Swimming is not only a popular sport, it is also one of the main causes of drowning accidents. In order to ensure the safety of swimming pools, regulations require that each swimming pool be equipped with a certain number of professional lifeguards, but this part of the cost accounts for a large proportion of the operating expenses. During peak hours, the number of lifeguards needs to be increased to achieve full coverage monitoring. Despite the sufficient number of lifeguards, problems such as blind spots and lack of concentration may still occur in actual operations. Especially in emergency situations, missing the best time for rescue may threaten life safety. Currently, most drowning detection systems rely on visible light images captured by RGB cameras for analysis. Although this method can meet the needs to a certain extent, its effect will be greatly reduced under special conditions such as low light and high reflection. In addition, in order to improve the effect of motion recognition, some advanced algorithms usually require a higher video frame rate input, which poses a considerable challenge to actual application scenarios because many places cannot provide stable and sufficiently high frame rate support.

[0004] Therefore, it is necessary to design a new method to adapt to lower frame rate conditions and improve the accuracy of drowning recognition while maintaining good recognition effect. Summary of the invention

[0005] The purpose of the present invention is to overcome the defects of the prior art and provide a drowning recognition method and system based on a low frame rate SlowFast model of knowledge distillation.

[0006] To achieve the above object, the present invention adopts the following technical solution: a drowning recognition method based on a low frame rate SlowFast model of knowledge distillation, comprising: Obtain an image to be recognized; Inputting the image to be identified into a drowning identification model to perform drowning behavior identification to obtain an identification result; Outputting the recognition result; Wherein, the drowning recognition model includes: Train the Fast branch through knowledge distillation technology; Combine the trained Fast branch with the Slow branch parameters of the open source SlowFast pre-trained model to form a network to obtain a new SlowFast model; Get the swimming pool environment behavior dataset; The new SlowFast model is trained using the swimming pool environment behavior dataset and label smoothing technology to obtain a drowning recognition model.

[0007] The further technical solution is: the training of the Fast branch by the knowledge distillation technology includes: Using the open source SlowFast model trained with the Kinetics-700 dataset as a basis, pre-training the Fast branch of the SlowFast model at a high frame rate, and using the existing pre-trained weights of the Fast branch of the SlowFast model as a starting point to obtain a high frame rate version of the Fast branch; Build a low frame rate version of the Fast branch to simulate the full frame input format by copying part of the frame image; The high frame rate version of the Fast branch is used as the teacher model, and the low frame rate version of the Fast branch is used as the student model. The information between the teacher model and the student model is transferred through knowledge distillation to obtain the trained Fast branch.

[0008] A further technical solution is as follows: the high frame rate pre-training is to use a video frame sequence consisting of 32 frames as input to perform pre-training of the Fast branch.

[0009] A further technical solution is: the step of simulating the input format of the full frame by copying a partial frame image includes: By selecting 4 frames of images and replicating them 8 times, a video frame sequence containing 32 frames of images is formed as input for training the Fast branch.

[0010] Its further technical solution is: in the process of training the Fast branch by the knowledge distillation technology, a mean square error loss function is used for processing so that the low frame rate version of the Fast branch approaches the high frame rate version of the Fast branch.

[0011] A further technical solution is: using the swimming pool environment behavior data set to apply label smoothing technology to train the new SlowFast model to obtain a drowning recognition model, including: Every 4 frames of images in the swimming pool environment behavior dataset are copied 8 times to form a 32-frame video input to obtain a sample set; The new SlowFast model is trained using the sample set and label smoothing technology to obtain a drowning recognition model.

[0012] A further technical solution is: using the sample set to apply label smoothing technology to train the new SlowFast model to obtain a drowning recognition model, including: During the training process, the labels in the sample set are To convert, represents the total number of categories, represents the smoothing parameter, , Indicates the current actual behavior category; is the label after smoothing.

[0013] The present invention also provides a drowning recognition system based on a low frame rate SlowFast model of knowledge distillation, comprising: An image acquisition unit, used for acquiring an image to be recognized; A recognition unit, used for inputting the image to be recognized into a drowning recognition model to perform drowning behavior recognition to obtain a recognition result; wherein the drowning recognition model includes a SlowFast model, which is a model obtained by performing transfer learning on a swimming pool environment behavior dataset by training a Fast branch through knowledge distillation technology and combining the trained Fast branch with the Slow branch parameters of an open source SlowFast pre-trained model to form a network; An output unit is used to output the recognition result.

[0014] Its further technical solution is: it also includes a training unit for: The Fast branch is trained through knowledge distillation technology; the trained Fast branch is combined with the Slow branch parameters of the open source SlowFast pre-trained model to form a network to obtain a new SlowFast model; a swimming pool environment behavior dataset is obtained; and the new SlowFast model is trained using the label smoothing technology using the swimming pool environment behavior dataset to obtain a drowning recognition model.

[0015] The beneficial effects of the present invention compared with the prior art are: The present invention inputs the image to be identified into the SlowFast model and uses its dual-branch structure to process information at different frame rates. The Fast branch is trained through knowledge distillation technology so that it can effectively obtain timing information from the Slow branch, thereby improving the performance under low frame rate conditions; the trained Fast branch is merged with the Slow branch parameters of the open source SlowFast pre-trained model to form an efficient network structure that adapts to low frame rates; the swimming pool environment behavior dataset is used for transfer learning so that the model can accurately identify drowning behavior in specific swimming pool scenes and improve the model's adaptability to low frame rate images; finally, the drowning behavior is identified through the model and the identification results are output, so that the drowning detection capability with high accuracy can still be maintained under low frame rate conditions.

[0016] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.

[0018] Figure 1 A schematic flow chart of a drowning recognition method based on a low frame rate SlowFast model of knowledge distillation provided in an embodiment of the present invention; Figure 2 Sub-process diagram of a drowning recognition method based on a low frame rate SlowFast model of knowledge distillation provided in an embodiment of the present invention Figure 1 ; Figure 3 Sub-process diagram of a drowning recognition method based on a low frame rate SlowFast model of knowledge distillation provided in an embodiment of the present invention Figure 2 ; Figure 4 Sub-process diagram of a drowning recognition method based on a low frame rate SlowFast model of knowledge distillation provided in an embodiment of the present invention Figure 3 ; Figure 5 A schematic diagram of the training process of the Fast branch provided in an embodiment of the present invention; Figure 6 A schematic diagram of a swimming pool behavior data set provided by an embodiment of the present invention; Figure 7 A schematic block diagram of a drowning recognition system based on a low frame rate SlowFast model of knowledge distillation provided in an embodiment of the present invention; Figure 8A schematic block diagram of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0019] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0020] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0021] It should also be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0022] It should be further understood that the term "and / or" used in the present description and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0023] See also Figure 1 , Figure 1 A schematic flow chart of a drowning recognition method based on a low frame rate SlowFast model with knowledge distillation provided in an embodiment of the present invention. The drowning recognition method based on a low frame rate SlowFast model with knowledge distillation is applied to a server. The server interacts with the terminal and the camera for data, uses the high frame rate version of the Fast branch as a teacher model, and transfers its knowledge to the low frame rate version of the Fast branch, thereby improving the performance of the low frame rate model; by copying part of the frame image, the full frame input format is simulated to ensure that the low frame rate Fast branch can process input information similar to the high frame rate; the swimming pool environment behavior data set is used for transfer learning, and the label smoothing technology is applied to reduce overfitting and improve the model's recognition ability for unknown behaviors; the mean square error loss function is used in the knowledge distillation process to make the low frame rate Fast branch approach the output of the high frame rate version and optimize the performance; the swimming pool data set is enhanced (such as copying frame images to form a longer video sequence) to enhance the model's adaptability to low frame rate input, thereby improving the accuracy of drowning recognition.

[0024] Figure 1 is a flow chart of a drowning recognition method based on a low frame rate SlowFast model of knowledge distillation provided by an embodiment of the present invention. Figure 1 As shown, the method includes the following steps S110 to S130.

[0025] S110: Obtain an image to be recognized.

[0026] In this embodiment, the image to be identified refers to image data used for drowning behavior identification. These images usually come from surveillance cameras, drones, mobile phones and other devices, and represent real-time video frames or static images that need to be detected to determine whether drowning behavior has occurred.

[0027] S120. Input the image to be identified into a drowning recognition model to perform drowning behavior recognition to obtain a recognition result; wherein, the drowning recognition model includes a SlowFast model, and the drowning recognition model is a model obtained by transfer learning from a swimming pool environment behavior dataset by training a Fast branch through knowledge distillation technology, and combining the trained Fast branch with the Slow branch parameters of an open source SlowFast pre-trained model to form a network.

[0028] In this embodiment, the recognition result refers to whether there is drowning behavior in the image to be recognized.

[0029] The method of this embodiment uses the SlowFast model to identify swimmers' behaviors. The model processes video data through two parallel paths: a low-frame-rate Slow path and a high-frame-rate Fast path. This design enables the model to capture both slow-motion and fast-motion features in the video, thereby improving the performance of motion recognition and video analysis. However, in actual applications, the frame rate of the swimmer drowning alarm system cannot meet the frame rate requirements of the Fast path. Therefore, the method of this embodiment is based on the knowledge distillation method to train a low-frame-rate SlowFast model with performance comparable to that of the high-frame-rate SlowFast model.

[0030] The SlowFast model is a deep learning model for video understanding tasks, specifically designed to process spatiotemporal information and capture motion details in videos. The model performs well in tasks such as video classification and action recognition, and is particularly suitable for processing videos containing fast and slow movements.

[0031] The core idea of ​​the SlowFast model is to use two different network branches to process information of different frame rates respectively: Slow branch: Responsible for capturing low frame rate video information, usually using a lower sampling frequency (such as 3 frames per second). This branch focuses on the global scene information of the video, processes dynamic changes on a longer time scale, and helps understand the overall structure of the video.

[0032] Fast branch: Responsible for capturing high frame rate video information, usually using a higher sampling frequency (such as 30 frames per second or higher). This branch focuses on fast motion and details in the video, capturing dynamic changes in local short time scales.

[0033] Through the low frame rate Slow branch and the high frame rate Fast branch, the model can efficiently capture the spatiotemporal features in the video. Combining the features of the two time scales, it can better handle complex actions and video scenes, and improve the robustness of the model in various tasks. It can achieve high accuracy in multiple video understanding tasks, especially in action recognition tasks that require fast response.

[0034] In one embodiment, see Figure 2 The above-mentioned drowning recognition model is a model obtained by transfer learning from a swimming pool environment behavior dataset by training the Fast branch through knowledge distillation technology, and combining the trained Fast branch with the Slow branch parameters of the open source SlowFast pre-trained model to form a network, including: steps S121~S124.

[0035] S121. Train the Fast branch through knowledge distillation technology.

[0036] In one embodiment, see Figure 3 , the above-mentioned step S121 may include steps S1211~S1213.

[0037] S1211. Using the open source SlowFast model trained with the Kinetics-700 dataset as a basis, pre-train the Fast branch of the SlowFast model at a high frame rate, and use the existing pre-trained weights of the Fast branch of the SlowFast model as a starting point to obtain a high frame rate version of the Fast branch.

[0038] The high frame rate pre-training is to use a video frame sequence consisting of 32 frames as input to perform pre-training of the Fast branch.

[0039] In this embodiment, the open source SlowFast model that has been trained on the large-scale video dataset Kinetics-700 is selected as the starting point. This model is widely used due to its superior performance in various action recognition tasks. Since the Fast branch of the SlowFast model is designed to capture fast motion, it has high requirements for frame rate, such as the set 32 ​​frames per second. This makes it very suitable for use as a pre-trained model under high frame rate conditions.

[0040] Through further high frame rate pre-training, the goal is to enhance the Fast branch's ability to capture rapidly changing actions (such as drowning), thereby obtaining a high frame rate Fast branch with better performance.

[0041] Specifically, we used the pre-trained weights of the existing SlowFast model Fast branch as initialization parameters to build a new network structure FastOnly, which only retains part of the original Fast branch. Then, we used a continuous video frame sequence of 32 frames as input for training to ensure that the FastOnly model can fully learn the representation method of video features at high frame rates.

[0042] S1212. Build a low frame rate version of the Fast branch, and simulate the input format of the full frame by copying part of the frame image.

[0043] Specifically, 4 frames of images are selected and replicated 8 times respectively to form a video frame sequence containing 32 frames of images as input for training the Fast branch.

[0044] In this embodiment, considering that the actual application environment may not provide a sufficiently high frame rate, for example, some surveillance cameras can only output video streams with lower frame rates. In order to adapt the model to these scenarios, a new Fast branch must be developed that can run under low frame rate conditions but still maintain good performance.

[0045] In response to the above challenges, this embodiment proposes a method for simulating high frame rate input. Specifically, 4 frames of images are selected from the original video, and then each frame is copied 8 times to form a hypothetical video frame sequence containing 32 frames as input. This approach not only meets the requirements of the Fast branch for the input format, but also simplifies the processing flow of low frame rate videos.

[0046] In this way, the newly constructed low frame rate version Fast branch can learn how to effectively process low frame rate videos while receiving the same input format as the high frame rate pre-training, laying a solid foundation for the subsequent knowledge distillation step.

[0047] S1213. Use the high frame rate version of the Fast branch as a teacher model and the low frame rate version of the Fast branch as a student model, and transfer information between the teacher model and the student model through knowledge distillation to obtain a trained Fast branch.

[0048] During the training process, the mean square error loss function is used to make the low frame rate version of the Fast branch approach the high frame rate version of the Fast branch.

[0049] In this embodiment, if Figure 5 As shown in the figure, the knowledge distillation method is used to optimize the Fast branch so that it can still maintain high-precision behavior recognition capabilities under low frame rate conditions; specifically, knowledge distillation is a model compression technology that allows a more complex "teacher" model to guide the learning process of a simpler or more efficient "student" model. In this case, the Fast branch pre-trained at a high frame rate plays the role of "teacher", while the newly constructed low frame rate version of the Fast branch is the "student".

[0050] In order to quantify the difference between the teacher model and the student model and guide the latter to better imitate the behavior of the former, the mean square error (MSE) is introduced as a loss function. MSE is used to evaluate the similarity between the output feature map of the teacher model and the output feature map of the student model. The formula is as follows: ; , where L represents the MSE loss between two features and N represents the number of pixels in the feature map. and They correspond to the eigenvalues ​​of the teacher model and the student model at the same position respectively; Output feature map for the teacher model; Output feature maps for the student model.

[0051] Based on the calculated MSE loss, the back-propagation algorithm can be used to automatically adjust the parameters of the student model so that it gradually approaches the performance of the teacher model. The key to this process is to find a suitable optimization strategy to minimize the gap between the two while ensuring that the student model does not overfit the data distribution of the teacher model. The ultimate goal is to enable the low-frame-rate version of the Fast branch to accurately identify emergency situations such as drowning in actual applications.

[0052] In summary, through the carefully designed high frame rate pre-training, low frame rate version construction and knowledge distillation process, the method of this embodiment successfully reduces the frame rate requirement of the Fast branch and realizes efficient drowning behavior recognition in a resource-constrained environment.

[0053] S122. Combining the trained Fast branch with the Slow branch parameters of the open source SlowFast pre-trained model to form a network to obtain a new SlowFast model.

[0054] In this embodiment, the new SlowFast model refers to a network formed by combining the trained Fast branch with the Slow branch parameters of the open source SlowFast pre-trained model.

[0055] S123, obtaining a swimming pool environment behavior data set; S124. Using the swimming pool environment behavior dataset and applying label smoothing technology to train the new SlowFast model to obtain a drowning recognition model.

[0056] In one embodiment, see Figure 4 , the above-mentioned step S124 may include steps S1241~S1242.

[0057] S1241, copying every 4 frames of images in the swimming pool environment behavior dataset 8 times to form a 32-frame video input to obtain a sample set; S1242: Using the sample set and applying label smoothing technology to train the new SlowFast model to obtain a drowning recognition model.

[0058] Specifically, during the training process, the labels in the sample set are To convert, represents the total number of categories, represents the smoothing parameter, , Indicates the current actual behavior category; is the label after smoothing.

[0059] In this example, after completing the Fast branch pre-training optimized for high frame rate and low frame rate conditions, the next key step is to apply these results to a specific scenario, namely human behavior recognition in a swimming pool environment. To achieve this goal, the transfer learning method is used, specifically to customize the open source SlowFast model to adapt to the data characteristics of the swimming pool scene. The following is a detailed implementation process: First, use Figure 6 The behavior dataset shown is designed for swimming pool environments. This dataset contains a variety of behavior samples that may occur in a swimming pool, which is essential for training a model that can accurately identify emergency situations such as drowning.

[0060] Slow branch: Since the Slow branch is mainly responsible for capturing long-term motion trends in videos, its pre-trained parameters are directly derived from the open source SlowFast model trained on the Kinetics-700 dataset. This step ensures that the model can fully utilize the advantages of large-scale datasets while maintaining the ability to understand slower motion changes.

[0061] Fast branch: For the Fast branch, the parameters of the low-frame-rate FastOnly model obtained through high-frame-rate pre-training and knowledge distillation are introduced. The purpose of this is to enable the Fast branch to effectively capture fast action features under low frame rate conditions, especially those behaviors that require timely response in the swimming pool environment.

[0062] Considering the particularity of the Fast branch, the method described above is continued at the input level: 4 frames are selected from the original video, and each frame is copied 8 times to form a 32-frame sequence as input. This method not only simplifies the data preprocessing process, but also ensures consistency with the previous pre-training stage, which helps to improve the learning efficiency of the model.

[0063] Finally, after the above preparations are completed, the entire SlowFast model is trained on the swimming pool behavior dataset. In this process, the Slow branch and the Fast branch use the pre-trained weights of their respective fields as the starting point, combined with the unique behavior patterns of the swimming pool, and gradually adjust the model parameters, and finally obtain a customized SlowFast model that can run under low frame rate conditions and can efficiently identify key behaviors in the swimming pool.

[0064] In summary, through a carefully designed transfer learning strategy, the general SlowFast model is transformed into a version specifically suitable for the swimming pool environment, which significantly improves its practicality and accuracy in specific application scenarios. This method not only demonstrates how to effectively use existing resources to solve specific problems, but also provides a valuable reference case for other similar fields.

[0065] In addition, during the training process, in order to enhance the model's ability to identify behaviors (especially drowning situations) in a swimming pool environment and avoid possible overfitting problems during training, the Label Smoothing technology was introduced during the training phase. This technology adjusts the original one-hot encoded labels to make them "smooth", thereby guiding the model to learn more robust and generalized feature representations.

[0066] In this way, the original clear binary classification label (i.e. 0 or 1) is transformed into a series of values ​​in between: for the correct category, its label is no longer an absolute 1, but is slightly reduced to , while the error class is assigned a small positive value This change helps reduce the model's sensitivity to noise or outliers in the training dataset and forces the model to generate more uniform probability distributions with better generalization capabilities.

[0067] Furthermore, during training, To calculate the cross-entropy loss (Cross-Entropy Loss) to evaluate the difference between the model prediction results and the actual labels after smoothing, where is the target probability after applying label smoothing; is the probability distribution of the model output, representing the model's estimate of the attribution to the i-th category. Represents the total number of categories.

[0068] The design of this loss function ensures that the model can not only accurately distinguish between categories, but also maintain a good sense of uncertainty and diversity, which is particularly important for complex and changeable application scenarios such as swimming pools. Through the above measures, a SlowFast model was finally built that can adapt to low frame rate video input and effectively identify potentially dangerous behaviors (such as drowning), greatly improving the practicality and reliability of the system.

[0069] S130: Output the recognition result.

[0070] The above recognition results are output to the terminal for display.

[0071] In this embodiment, the method of this embodiment conducts a detailed analysis of possible drowning situations in a swimming pool environment, and specially designs a SlowFast model that can adapt to low frame rate video input. Through knowledge distillation technology, an efficient model dedicated to the swimming pool environment is trained, which can more accurately judge three specific behavior categories - struggling by holding the waterline, struggling in the middle of the lane, and safe state. This improvement not only improves the accuracy of the system, but also effectively reduces false alarms.

[0072] Equipped with advanced camera stream processing capabilities, it can capture and analyze swimmers' behavior patterns in real time. When potential drowning behavior is detected, the system automatically starts a warning countdown and sends an alarm message to lifeguards after the countdown ends. This feature enables pool managers to take action immediately, greatly shortening response time and improving rescue efficiency.

[0073] Special waterproof and fog-proof cameras are installed at key locations in the swimming pool to ensure stable operation even in humid environments. It uses advanced technologies such as server-side behavior analysis and AI pattern recognition to achieve intelligent alarm and accident video storage. The hardware system on which the entire method relies consists of multiple components such as high-performance computers, waterproof spherical cameras, on-site monitoring screens, alarm devices, etc., forming a complete automated monitoring network. All equipment adopts a fully waterproof design, which can provide accurate location guidance at the moment of danger, helping lifeguards to quickly locate and rescue drowning people. In addition, the system also integrates various sensors and sound and light alarms to further enhance its safety and reliability.

[0074] Without changing the existing facilities, the system can be seamlessly integrated by simply installing a high-resolution water camera on the top of the pool. This method avoids the complexity and cost increase caused by traditional construction wiring, while ensuring the high accuracy of the algorithm in identifying drowning behavior.

[0075] It can be seen that the method of this embodiment not only has significant advantages in technology and function, but also its design and deployment method fully considers the convenience and economy in practical applications.

[0076] The above-mentioned drowning recognition method based on the low-frame-rate SlowFast model of knowledge distillation inputs the image to be recognized into the SlowFast model and uses its dual-branch structure to process information at different frame rates. The Fast branch is trained through knowledge distillation technology so that it can effectively obtain timing information from the Slow branch, thereby improving the performance under low frame rate conditions; the trained Fast branch is fused with the Slow branch parameters of the open source SlowFast pre-trained model to form an efficient network structure that adapts to low frame rates; the swimming pool environment behavior dataset is used for transfer learning so that the model can accurately identify drowning behavior in specific swimming pool scenes and improve the model's adaptability to low frame rate images; finally, the drowning behavior is recognized through the model and the recognition result is output, so that the drowning detection capability with high accuracy can still be maintained under low frame rate conditions.

[0077] Figure 7 is a schematic block diagram of a drowning recognition system 300 based on a low frame rate SlowFast model of knowledge distillation provided by an embodiment of the present invention. Figure 7As shown, corresponding to the above drowning recognition method based on the low frame rate SlowFast model of knowledge distillation, the present invention also provides a drowning recognition system 300 based on the low frame rate SlowFast model of knowledge distillation. The drowning recognition system 300 based on the low frame rate SlowFast model of knowledge distillation includes a unit for executing the above drowning recognition method based on the low frame rate SlowFast model of knowledge distillation, and the system can be configured in a server. Specifically, please refer to Figure 7 The drowning recognition system 300 based on the low frame rate SlowFast model of knowledge distillation includes an image acquisition unit 301, a recognition unit 302 and an output unit 303.

[0078] An image acquisition unit 301 is used to acquire an image to be identified; a recognition unit 302 is used to input the image to be identified into a drowning recognition model to perform drowning behavior recognition to obtain a recognition result; wherein the drowning recognition model includes a SlowFast model, wherein the drowning recognition model is a model obtained by transfer learning of a swimming pool environment behavior dataset by training a Fast branch through knowledge distillation technology and combining the trained Fast branch with the Slow branch parameters of an open source SlowFast pre-trained model to form a network; an output unit 303 is used to output the recognition result.

[0079] In one embodiment, the drowning recognition system 300 based on the knowledge distillation low frame rate SlowFast model further includes a training unit for: The Fast branch is trained through knowledge distillation technology; the trained Fast branch is combined with the Slow branch parameters of the open source SlowFast pre-trained model to form a network to obtain a new SlowFast model; a swimming pool environment behavior dataset is obtained; and the new SlowFast model is trained using the label smoothing technology using the swimming pool environment behavior dataset to obtain a drowning recognition model.

[0080] In one embodiment, the training unit is further used to: use the open source SlowFast model trained with the Kinetics-700 data set as a basis, pre-train the Fast branch of the SlowFast model at a high frame rate, and use the existing pre-trained weights of the Fast branch of the SlowFast model as a starting point to obtain a high frame rate version of the Fast branch; construct a low frame rate version of the Fast branch to simulate the input format of the full frame by copying part of the frame image; use the high frame rate version of the Fast branch as a teacher model and the low frame rate version of the Fast branch as a student model, and transfer information between the teacher model and the student model through knowledge distillation to obtain a trained Fast branch.

[0081] In one embodiment, the training unit is further configured to: Every 4 frames of images in the swimming pool environment behavior dataset are replicated 8 times to form 32 frames of video input to obtain a sample set; the sample set is used to apply label smoothing technology to train the new SlowFast model to obtain a drowning recognition model.

[0082] In one embodiment, the training unit is further configured to: During the training process, the labels in the sample set are To convert, represents the total number of categories, represents the smoothing parameter, , Indicates the current actual behavior category; is the label after smoothing.

[0083] It should be noted that technicians in the relevant field can clearly understand that the specific implementation process of the drowning recognition system 300 and each unit of the low frame rate SlowFast model based on knowledge distillation can refer to the corresponding description in the aforementioned method embodiment, and for the convenience and conciseness of the description, it will not be repeated here.

[0084] The drowning recognition system 300 based on the knowledge distillation low frame rate SlowFast model can be implemented in the form of a computer program. Figure 8 Runs on the computer device shown.

[0085] See also Figure 8 , Figure 8 5 is a schematic block diagram of a computer device provided in an embodiment of the present application. The computer device 500 may be a server, wherein the server may be an independent server or a server cluster composed of multiple servers.

[0086] See also Figure 8 The computer device 500 includes a processor 502 , a memory and a network interface 505 connected via a system bus 501 , wherein the memory may include a non-volatile storage medium 503 and an internal memory 504 .

[0087] The non-volatile storage medium 503 may store an operating system 5031 and a computer program 5032. The computer program 5032 includes program instructions, and when the program instructions are executed, the processor 502 may execute a drowning recognition method based on a low frame rate SlowFast model of knowledge distillation.

[0088] The processor 502 is used to provide computing and control capabilities to support the operation of the entire computer device 500 .

[0089] The internal memory 504 provides an environment for the operation of the computer program 5032 in the non-volatile storage medium 503. When the computer program 5032 is executed by the processor 502, the processor 502 can execute a drowning recognition method based on a low frame rate SlowFast model of knowledge distillation.

[0090] The network interface 505 is used to communicate with other devices over the network. Figure 8 The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present application, and does not constitute a limitation on the computer device 500 to which the solution of the present application is applied. The specific computer device 500 may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0091] The processor 502 is used to run the computer program 5032 stored in the memory to implement the following steps: Acquire an image to be identified; input the image to be identified into a drowning recognition model to perform drowning behavior recognition to obtain a recognition result; wherein the drowning recognition model includes a SlowFast model, wherein the drowning recognition model is a model obtained by transfer learning of a swimming pool environment behavior dataset by training a Fast branch through knowledge distillation technology and combining the trained Fast branch with the Slow branch parameters of an open source SlowFast pre-trained model to form a network; and output the recognition result.

[0092] In one embodiment, the processor 502 implements the drowning recognition model by training the Fast branch through the knowledge distillation technology, and combining the trained Fast branch with the Slow branch parameters of the open source SlowFast pre-trained model to form a network, and the model step obtained by transfer learning from the swimming pool environment behavior dataset, specifically implementing the following steps: The Fast branch is trained through knowledge distillation technology; the trained Fast branch is combined with the Slow branch parameters of the open source SlowFast pre-trained model to form a network to obtain a new SlowFast model; a swimming pool environment behavior dataset is obtained; and the new SlowFast model is trained using the label smoothing technology using the swimming pool environment behavior dataset to obtain a drowning recognition model.

[0093] In the process of training the Fast branch by the knowledge distillation technology, a mean square error loss function is used for processing so that the low frame rate version of the Fast branch approaches the high frame rate version of the Fast branch.

[0094] In one embodiment, when the processor 502 implements the step of training the Fast branch by using the knowledge distillation technology, it specifically implements the following steps: The open source SlowFast model trained with the Kinetics-700 dataset is used as a basis, the Fast branch of the SlowFast model is pre-trained at a high frame rate, and the existing pre-trained weights of the Fast branch of the SlowFast model are used as a starting point to obtain a high frame rate version of the Fast branch; a low frame rate version of the Fast branch is constructed, and the input format of the full frame is simulated by copying part of the frame image; the high frame rate version of the Fast branch is used as a teacher model, and the low frame rate version of the Fast branch is used as a student model, and the information between the teacher model and the student model is transferred through knowledge distillation to obtain the trained Fast branch.

[0095] The high frame rate pre-training is to use a video frame sequence consisting of 32 frames as input to perform pre-training of the Fast branch.

[0096] In one embodiment, when the processor 502 implements the step of simulating the input format of the full frame by copying a partial frame image, the processor 502 specifically implements the following steps: By selecting 4 frames of images and replicating them 8 times, a video frame sequence containing 32 frames of images is formed as input for training the Fast branch.

[0097] In one embodiment, when the processor 502 implements the step of using the swimming pool environment behavior data set to apply the label smoothing technology to train the new SlowFast model to obtain the drowning recognition model, the processor 502 specifically implements the following steps: Every 4 frames of images in the swimming pool environment behavior dataset are replicated 8 times to form 32 frames of video input to obtain a sample set; the sample set is used to apply label smoothing technology to train the new SlowFast model to obtain a drowning recognition model.

[0098] In one embodiment, when the processor 502 implements the step of using the sample set to apply the label smoothing technology to train the new SlowFast model to obtain the drowning recognition model, the processor 502 specifically implements the following steps: During the training process, the labels in the sample set are To convert, represents the total number of categories, represents the smoothing parameter, , Indicates the current actual behavior category; is the label after smoothing.

[0099] It should be understood that in the embodiment of the present application, the processor 502 may be a central processing unit (CPU), and the processor 502 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0100] It can be understood by those skilled in the art that all or part of the processes in the method for implementing the above embodiment can be completed by instructing the relevant hardware through a computer program. The computer program includes program instructions, and the computer program can be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in the computer system to implement the process steps of the embodiment of the above method.

[0101] Therefore, the present invention also provides a storage medium. The storage medium may be a computer-readable storage medium. The storage medium stores a computer program, wherein when the computer program is executed by a processor, the processor executes the following steps: Acquire an image to be identified; input the image to be identified into a drowning recognition model to perform drowning behavior recognition to obtain a recognition result; wherein the drowning recognition model includes a SlowFast model, wherein the drowning recognition model is a model obtained by transfer learning of a swimming pool environment behavior dataset by training a Fast branch through knowledge distillation technology and combining the trained Fast branch with the Slow branch parameters of an open source SlowFast pre-trained model to form a network; and output the recognition result.

[0102] In one embodiment, when the processor executes the computer program to implement the drowning recognition model, the following steps are specifically implemented: training the Fast branch through knowledge distillation technology, and combining the trained Fast branch with the Slow branch parameters of the open source SlowFast pre-trained model to form a network, and performing transfer learning on the swimming pool environment behavior dataset to obtain the model step: The Fast branch is trained through knowledge distillation technology; the trained Fast branch is combined with the Slow branch parameters of the open source SlowFast pre-trained model to form a network to obtain a new SlowFast model; a swimming pool environment behavior dataset is obtained; and the new SlowFast model is trained using the label smoothing technology using the swimming pool environment behavior dataset to obtain a drowning recognition model.

[0103] In the process of training the Fast branch by the knowledge distillation technology, a mean square error loss function is used for processing so that the low frame rate version of the Fast branch approaches the high frame rate version of the Fast branch.

[0104] In one embodiment, when the processor executes the computer program to implement the step of training the Fast branch by the knowledge distillation technology, the processor specifically implements the following steps: The open source SlowFast model trained with the Kinetics-700 dataset is used as a basis, the Fast branch of the SlowFast model is pre-trained at a high frame rate, and the existing pre-trained weights of the Fast branch of the SlowFast model are used as a starting point to obtain a high frame rate version of the Fast branch; a low frame rate version of the Fast branch is constructed, and the input format of the full frame is simulated by copying part of the frame image; the high frame rate version of the Fast branch is used as a teacher model, and the low frame rate version of the Fast branch is used as a student model, and the information between the teacher model and the student model is transferred through knowledge distillation to obtain the trained Fast branch.

[0105] The high frame rate pre-training is to use a video frame sequence consisting of 32 frames as input to perform pre-training of the Fast branch.

[0106] In one embodiment, when the processor executes the computer program to implement the step of simulating the input format of the full frame by copying a partial frame image, the processor specifically implements the following steps: By selecting 4 frames of images and replicating them 8 times, a video frame sequence containing 32 frames of images is formed as input for training the Fast branch.

[0107] In one embodiment, when the processor executes the computer program to implement the step of using the swimming pool environment behavior data set to apply the label smoothing technology to train the new SlowFast model to obtain the drowning recognition model, the processor specifically implements the following steps: Every 4 frames of images in the swimming pool environment behavior dataset are replicated 8 times to form 32 frames of video input to obtain a sample set; the sample set is used to apply label smoothing technology to train the new SlowFast model to obtain a drowning recognition model.

[0108] In one embodiment, when the processor executes the computer program to implement the step of using the sample set to apply the label smoothing technology to train the new SlowFast model to obtain the drowning recognition model, the processor specifically implements the following steps: During the training process, the labels in the sample set are To convert, represents the total number of categories, represents the smoothing parameter, , Indicates the current actual behavior category; is the label after smoothing.

[0109] The storage medium may be a USB flash drive, a mobile hard disk, a read-only memory (ROM), a magnetic disk, or an optical disk, etc., which are computer-readable storage media that can store program codes.

[0110] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0111] In the several embodiments provided by the present invention, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are only schematic. For example, the division of each unit is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0112] The steps in the method of the embodiment of the present invention can be adjusted in order, combined and deleted according to actual needs. The units in the system of the embodiment of the present invention can be combined, divided and deleted according to actual needs. In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0113] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, terminal, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention.

[0114] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A drowning recognition method based on a low frame rate SlowFast model with knowledge distillation, characterized in that: include: Obtain an image to be recognized; Inputting the image to be identified into a drowning identification model to perform drowning behavior identification to obtain an identification result; Outputting the recognition result; Wherein, the drowning recognition model includes: Train the Fast branch through knowledge distillation technology; Combine the trained Fast branch with the Slow branch parameters of the open source SlowFast pre-trained model to form a network to obtain a new SlowFast model; Get the swimming pool environment behavior dataset; The new SlowFast model is trained using the swimming pool environment behavior dataset and label smoothing technology to obtain a drowning recognition model.

2. The drowning recognition method based on the low frame rate SlowFast model of knowledge distillation according to claim 1 is characterized in that: The Fast branch is trained by the knowledge distillation technology, including: Using the open source SlowFast model trained with the Kinetics-700 dataset as a basis, pre-training the Fast branch of the SlowFast model at a high frame rate, and using the existing pre-trained weights of the Fast branch of the SlowFast model as a starting point to obtain a high frame rate version of the Fast branch; Build a low frame rate version of the Fast branch to simulate the full frame input format by copying part of the frame image; The high frame rate version of the Fast branch is used as the teacher model, and the low frame rate version of the Fast branch is used as the student model. The information between the teacher model and the student model is transferred through knowledge distillation to obtain the trained Fast branch.

3. The drowning recognition method based on the low frame rate SlowFast model of knowledge distillation according to claim 2 is characterized in that: The high frame rate pre-training is to use a video frame sequence consisting of 32 frames as input to perform pre-training of the Fast branch.

4. The drowning recognition method based on the low frame rate SlowFast model of knowledge distillation according to claim 3 is characterized in that: The method of simulating the input format of the full frame by copying a partial frame image includes: By selecting 4 frames of images and replicating them 8 times, a video frame sequence containing 32 frames of images is formed as input for training the Fast branch.

5. The drowning recognition method based on the low frame rate SlowFast model of knowledge distillation according to claim 3 is characterized in that: In the process of training the Fast branch by the knowledge distillation technology, a mean square error loss function is used for processing so that the low frame rate version of the Fast branch approaches the high frame rate version of the Fast branch.

6. The drowning recognition method based on the low frame rate SlowFast model of knowledge distillation according to claim 1 is characterized in that: The method of using the swimming pool environment behavior dataset to apply label smoothing technology to train the new SlowFast model to obtain a drowning recognition model includes: Every 4 frames of images in the swimming pool environment behavior dataset are copied 8 times to form a 32-frame video input to obtain a sample set; The new SlowFast model is trained using the sample set and label smoothing technology to obtain a drowning recognition model.

7. The drowning recognition method based on the low frame rate SlowFast model of knowledge distillation according to claim 6 is characterized in that: The method of using the sample set to apply label smoothing technology to train the new SlowFast model to obtain a drowning recognition model includes: During the training process, the labels in the sample set are To convert, represents the total number of categories, represents the smoothing parameter, , Indicates the current actual behavior category; is the label after smoothing.

8. A drowning recognition system based on the low frame rate SlowFast model of knowledge distillation, characterized in that: include: An image acquisition unit, used for acquiring an image to be recognized; A recognition unit, used for inputting the image to be recognized into a drowning recognition model to perform drowning behavior recognition to obtain a recognition result; wherein the drowning recognition model includes a SlowFast model, which is a model obtained by performing transfer learning on a swimming pool environment behavior dataset by training a Fast branch through knowledge distillation technology and combining the trained Fast branch with the Slow branch parameters of an open source SlowFast pre-trained model to form a network; An output unit is used to output the recognition result.

9. The drowning recognition system based on the low frame rate SlowFast model of knowledge distillation according to claim 8, characterized in that: Also included are training units for: The Fast branch is trained through knowledge distillation technology; the trained Fast branch is combined with the Slow branch parameters of the open source SlowFast pre-trained model to form a network to obtain a new SlowFast model; a swimming pool environment behavior dataset is obtained; and the new SlowFast model is trained using the label smoothing technology using the swimming pool environment behavior dataset to obtain a drowning recognition model.

Images