A multifunctional intelligent monitoring method and device for outdoor swimming

Abstract

This invention discloses a multifunctional intelligent monitoring method and device for outdoor swimming. The method includes: acquiring dual fisheye images; expanding and stitching the dual fisheye images to obtain a panoramic video stream; constructing a classification model and training the classification model using the labeled image sequence from the panoramic video stream as a training set, so that the trained classification model can classify the input image sequence and output a fixed-size classified image sequence; constructing a motion posture detection model and inputting the swimmer image from the classified image sequence into the motion posture detection model, so that the swimmer's posture can be detected by the motion posture detection model, and an alarm can be triggered when drowning is detected; finally, judging whether the swimmer is close to a dangerous object based on the surrounding environment image of the swimmer in the classified image sequence, and triggering an alarm when the swimmer is close to a dangerous object; thus, the swimmer and the surrounding environment can be monitored simultaneously, thereby improving the monitoring effect.

Description

Technical Field

[0001] This invention relates to the field of safety monitoring technology, and in particular to a multifunctional intelligent monitoring method and a multifunctional intelligent monitoring device for outdoor swimming. Background Technology

[0002] In related technologies, open water swimming is an outdoor activity involving swimming in the sea, lakes, or rivers. It is both a mass activity and a competitive sport. In particular, my country has abundant water resources, and open water swimming activities cover a large area with a high degree of freedom, making outdoor swimming very popular. Current swimming monitoring algorithms mainly use wearable devices for monitoring. However, wearable devices not only restrict swimmers and are prone to falling off, but also have poor monitoring effects and are not suitable for monitoring outdoor swimming. Summary of the Invention

[0003] This invention aims to at least partially solve one of the technical problems in the aforementioned technologies. Therefore, one objective of this invention is to propose a multifunctional intelligent monitoring method for outdoor swimming, which combines a dual fisheye camera on a following float with a constructed classification model to simultaneously monitor swimmers and the surrounding environment; and, considering the characteristics of turbid outdoor water and continuous swimming movements, to achieve real-time monitoring of outdoor swimmers' swimming postures by constructing a motion posture detection model; thereby improving monitoring effectiveness.

[0004] The second objective of this invention is to provide a multifunctional intelligent monitoring device for outdoor swimming.

[0005] To achieve the above objectives, a first aspect of the present invention proposes a multifunctional intelligent monitoring method for outdoor swimming, which employs a follow-up float connected to the swimmer. The outdoor swimming monitoring method includes the following steps: acquiring dual fisheye images using a dual fisheye camera mounted on the follow-up float; expanding and stitching the dual fisheye images to obtain a panoramic video stream, wherein the panoramic video stream includes the swimmer's swimming state and the swimmer's surrounding environment; constructing a classification model and training the classification model using the labeled image sequence from the panoramic video stream as a training set, so as to classify the input image sequence through the trained classification model and output a fixed-size classified image sequence; constructing a motion posture detection model and inputting the swimmer image from the classified image sequence into the motion posture detection model, so as to detect the swimmer's posture through the motion posture detection model and issue an alarm when drowning is detected; determining whether the swimmer is close to a dangerous object based on the swimmer's surrounding environment image from the classified image sequence, and issuing an alarm when the swimmer is close to a dangerous object.

[0006] According to an embodiment of the present invention, a multifunctional outdoor swimming intelligent monitoring method firstly acquires dual fisheye images using a dual fisheye camera mounted on a following float; then, the dual fisheye images are expanded and stitched together to obtain a panoramic video stream, wherein the panoramic video stream includes the swimmer's swimming state and the swimmer's surrounding environment; next, a classification model is constructed, and the image sequence in the labeled panoramic video stream is used as a training set to train the classification model, so that the trained classification model can classify the input image sequence to output a fixed-size classification image sequence; next, a motion posture detection model is constructed, and the swimmer image in the classification image sequence is input into the motion posture detection model, so that the motion posture detection model can detect the swimmer's posture and issue an alarm when drowning is detected; finally, the surrounding environment image of the swimmer in the classification image sequence is used to determine whether the swimmer is close to a dangerous object, and an alarm is issued when the swimmer is close to a dangerous object; thus, by combining the dual fisheye camera on the following float with the constructed classification model, simultaneous monitoring of the swimmer and the surrounding environment is achieved; and considering the characteristics of turbid outdoor water and continuous swimming movement, the construction of the motion posture detection model enables real-time monitoring of the swimming posture of outdoor swimmers, thereby improving the monitoring effect.

[0007] In addition, the multifunctional outdoor swimming intelligent monitoring method proposed in the above embodiments of the present invention may also have the following additional technical features:

[0008] Optionally, the double fisheye image is expanded, including: presetting the size of the expanded image to h*w; mapping the fisheye image onto a unit sphere and establishing a spherical rectangular coordinate system; using the plane formed by the x-axis and y-axis as the starting position of longitude (longitude is 0), establishing w uniformly spaced meridians with sizes ranging from -π to +π, thereby calculating the angular interval of longitude. Similarly, taking the plane formed by the x-axis and z-axis as the starting position of latitude (latitude 0), establish h evenly spaced parallels of latitude, ranging in size from -π / 2 to +π / 2, where the positive direction of the y-axis is the direction of increasing latitude. From this, the angular interval of latitude can be calculated as follows: The image is unfolded based on the principle that pixels at the same latitude on the sphere are located in the same row in the unfolded image, and pixels at the same longitude on the sphere are located in the same column in the unfolded image, thus obtaining the unfolded image.

[0009] Optionally, the double fisheye images are stitched together, including: performing feature-based matching and homography transformation on the unfolded image obtained after unfolding the double fisheye images to obtain a homography matrix; performing a projective transformation on the homography matrix, and performing weighted image fusion on the two transformed images to obtain a panoramic planar image.

[0010] Optionally, the motion posture detection model is a POSE_LSTM network. By modifying the key points originally set in the Openpose network and selecting 12 locations, including the back of the head, neck, left ear, right ear, both shoulders, both elbows, both wrists, and both hips, as new key points for detection; and by increasing the color contrast between the human body and the water body through a contrast-limited adaptive histogram equalization algorithm, the key points of N consecutive frames of images are passed through the LSTM network to supplement the swimmer's motion features in the time series, and finally output the judgment result of the swimmer's posture.

[0011] Optionally, determining whether the swimmer is approaching a dangerous object based on the surrounding environment image of the swimmer in the classified image sequence, and issuing an alarm when approaching a dangerous object, includes: inputting the surrounding environment image of the swimmer in the classified image sequence into the environment detection module on the following buoy to calculate the area of ​​the dangerous object in the classified image sequence; when the area of ​​the dangerous object in the next frame is greater than or equal to that in the previous frame, it is considered that the swimmer is approaching a dangerous object; let the width and height of the image be img_width and img_height. Then, the proportion of the dangerous object in the image, pro, is calculated using the formula area / (img_width*img_height); when the proportion pro exceeds a set threshold TH, it is considered that the swimmer has entered a dangerous area, and an alarm is broadcast to the swimmer indicating that a dangerous object is approaching.

[0012] At the same time, when a swimmer enters a pre-recorded dangerous area or when the wave level on the water surface is detected to exceed a specified threshold, the swimmer will be informed of the current danger of the water area.

[0013] Optionally, the follow float is also equipped with a voice recognition module, which takes a photo or video when it detects a specific voice command from the swimmer.

[0014] To achieve the above objectives, a second aspect of the present invention provides a multifunctional outdoor swimming intelligent monitoring device, which stores a multifunctional outdoor swimming intelligent monitoring program. When the multifunctional outdoor swimming intelligent monitoring program is executed by a processor, it implements the multifunctional outdoor swimming intelligent monitoring method as described above.

[0015] To achieve the above objectives, a third aspect of the present invention provides a computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the multifunctional outdoor swimming intelligent monitoring device as described above. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating the multifunctional outdoor swimming intelligent monitoring method according to an embodiment of the present invention;

[0017] Figure 2 This is a flowchart illustrating a multifunctional outdoor swimming intelligent monitoring method according to an embodiment of the present invention;

[0018] Figure 3 This is a schematic diagram of the structure of a multifunctional outdoor swimming intelligent monitoring device according to an embodiment of the present invention;

[0019] Figure 4 This is a schematic diagram of image unfolding and stitching according to an embodiment of the present invention, wherein Figure (a) is an unprocessed double fisheye image and Figure (b) is the stitching result of the processed fisheye image;

[0020] Figure 5 This is a schematic diagram of image classification and cropping according to an embodiment of the present invention;

[0021] Figure 6 This is a schematic diagram of the POSE_LSTM network structure according to an embodiment of the present invention;

[0022] Figure 7 This is a schematic diagram of environmental detection according to an embodiment of the present invention, wherein Figure (a) represents the safe distance and Figure (b) represents the danger distance. Detailed Implementation

[0023] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0024] To better understand the above technical solutions, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present invention and to fully convey the scope of the invention to those skilled in the art.

[0025] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.

[0026] like Figure 1 As shown, this multifunctional outdoor swimming intelligent monitoring method includes the following steps:

[0027] S101 uses a dual fisheye camera set on the following float to acquire dual fisheye images.

[0028] In other words, a dual-fisheye camera is installed on the float, through which dual-fisheye images can be acquired. The acquired dual-fisheye images are as follows: Figure 4 As shown in Figure (a).

[0029] It should be noted that, as Figure 3 The following float shown includes a main body 1, a camera module 2, a computing module, and a power supply module. The main body 1 includes a rescue airbag and clothing storage space, and is strapped to the swimmer's waist to follow the swimmer's movements and ensure the swimmer's safety. The camera module 2 includes dual fisheye cameras, mainly used to capture and record the swimmer's current swimming movements and the surrounding environment. The computing module consists of an embedded device that deploys posture detection algorithms, environment detection algorithms, gesture and voice recognition algorithms, and voice communication algorithms to achieve various functions. The power supply module is mainly used to provide power to the above modules.

[0030] S102, after unfolding and stitching the dual fisheye images, a panoramic video stream is obtained, which includes the swimmer's swimming state and the swimmer's surrounding environment.

[0031] In other words, the follow float follows behind the swimmer so that the dual fisheye cameras can capture a panoramic video stream that includes the swimmer's swimming status and the swimmer's surrounding environment.

[0032] As an example, the process of unfolding a double fisheye image includes: presetting the size of the unfolded image to h*w; mapping the fisheye image onto a unit sphere and establishing a spherical rectangular coordinate system; using the plane formed by the x-axis and y-axis as the starting position of longitude (longitude is 0), establishing w uniformly spaced meridians with sizes ranging from -π to +π, thereby calculating the angular interval of longitude. Similarly, taking the plane formed by the x-axis and z-axis as the starting position of latitude (latitude 0), establish h evenly spaced parallels of latitude, ranging in size from -π / 2 to +π / 2, where the positive direction of the y-axis is the direction of increasing latitude. From this, the angular interval of latitude can be calculated as follows: The image is unfolded based on the principle that pixels at the same latitude on the sphere are located in the same row in the unfolded image, and pixels at the same longitude on the sphere are located in the same column in the unfolded image, thus obtaining the unfolded image.

[0033] As an example, the process of stitching two fisheye images includes: performing feature-based matching and homography transformation on the unfolded image obtained after unfolding the two fisheye images to obtain a homography matrix; performing a projective transformation on the homography matrix; and performing weighted image fusion on the two transformed images to obtain a panoramic planar image.

[0034] As a specific embodiment, matching feature point pairs are obtained by selecting an appropriate corner detector; the homography matrix H is calculated based on the obtained feature point pairs; and then, points on one projective plane are mapped to another projective plane using the projective transformation formula.

[0035] in, The mapping transformation formula is

[0036] The two transformed images are weighted and fused to obtain a panoramic planar image, achieving corrected stitching of the two fisheye images. Combined with the gyroscope parameters in the following float, the unfolded video is de-shaken to obtain a more stable panoramic video stream.

[0037] It should be noted that the stitched image is as follows: Figure 4 As shown in Figure (b).

[0038] S103, construct a classification model, and use the image sequence in the labeled panoramic video stream as the training set to train the classification model so that the trained classification model can classify the input image sequence and output a fixed-size classified image sequence.

[0039] As a specific implementation, a portion of the acquired panoramic images are used as a training set to label swimmers and surrounding dangerous objects such as boats and fish cages, and then a YOLOv5 model is trained. This model performs real-time detection on the input image, obtaining coordinates (x, y) and category labels. The image is then cropped at a set ratio centered on the detection box center point. Maintaining the image height, a horizontal range of (x±960) is cropped based on the obtained x-coordinate, resulting in a labeled image with a length of 1920 pixels and a width of 1080 pixels. The image is then sent to the corresponding processing module according to the labels. A schematic diagram of the cropped image is shown below. Figure 5 As shown.

[0040] In other words, the images are divided into two categories with a certain size: one is the image of the swimmer, and the other is the image of the surrounding environment.

[0041] S104, Construct a motion posture detection model and input swimmer images from the classified image sequence into the motion posture detection model so as to detect swimmer posture through the motion posture detection model and issue an alarm when drowning is detected.

[0042] As an example, such as Figure 6As shown, the motion posture detection model is a POSE_LSTM network. By modifying the key points originally set in the Openpose network and selecting 12 new key points, namely the back of the head, neck, left ear, right ear, both shoulders, both elbows, both wrists and both hips, the model detects the motion posture. In addition, the color contrast between the human body and the water is increased by limiting the contrast adaptive histogram equalization algorithm. The key points of N consecutive frames of images are passed through the LSTM network to supplement the swimmer's motion features in the time series, and finally output the judgment result of the swimmer's posture.

[0043] As a specific implementation, the key points originally set in the Openpose network were modified to improve the shooting effect of the dual fisheye cameras following the float. Twelve parts, including the back of the head, neck, left and right ears, shoulders, elbows, wrists, and hips, were selected as new key points for detection. Considering the impact of water turbidity on detection, the contrast-limited adaptive histogram equalization (CLAHE) algorithm was used to increase the color contrast between the human body and the water to improve the detection effect. Considering the continuity of swimming movements and the lack of key point detection due to water turbidity, the key points of N consecutive frames of images were passed through an LSTM network to supplement the swimmer's movement features in the time series, and finally the swimming posture judgment result was output.

[0044] In other words, starting from the first frame, N consecutive frames of images labeled as swimmers are read in and input into the POSE_LSTM network to obtain the swimming posture monitoring results; the first frame of the sequence is removed by a sliding window and a new frame labeled as swimmer is read in to achieve real-time monitoring of the swimmer's swimming posture; when the detection result indicates drowning, the float emits an alarm sound and flashing lights to call for help from the surrounding area, and at the same time sends a drowning rescue signal to the pre-set emergency contacts.

[0045] S105, determine whether the swimmer is close to a dangerous object based on the surrounding environment image of the swimmer in the classified image sequence, and issue an alarm when the swimmer is close to a dangerous object.

[0046] As an example, such as Figure 7As shown, the system determines whether a swimmer is approaching a dangerous object based on the surrounding environment image of the swimmer in the classified image sequence, and issues an alarm when the swimmer approaches a dangerous object. This includes: inputting the surrounding environment image of the swimmer in the classified image sequence into the environment detection module on the following buoy to calculate the area (area) of the dangerous object in the classified image sequence; if the area of ​​the dangerous object in the next frame is greater than or equal to that in the previous frame, the swimmer is considered to be approaching a dangerous object; let the image width and height be img_width and img_height. Then, the proportion (pro) of the dangerous object in the image is calculated using the formula area / (img_width*img_height); when the proportion (pro) exceeds a set threshold TH, the swimmer is considered to have entered a dangerous area, and an alarm is broadcast to the swimmer indicating that they are approaching a dangerous object.

[0047] At the same time, when a swimmer enters a pre-recorded dangerous area or when the wave level on the water surface is detected to exceed a specified threshold, the swimmer will be informed of the current danger of the water area.

[0048] in, Figure 7 Figure (a) accounts for 12.13% of the image, while Figure (b) accounts for 50.17%. Assuming a threshold of 40%, Figure (a) indicates that the swimmer is in a safe zone, while Figure (b) indicates that the swimmer is in a danger zone.

[0049] As an example, the float is also equipped with a voice recognition module, which takes a picture or video when it detects a swimmer's specific voice command.

[0050] In other words, the system collects the swimmer's voice information, recognizes the voice through a voice recognition chip, and when the system recognizes a specific voice command, it performs the corresponding photo or video recording function according to the command.

[0051] As a specific example, such as Figure 2 As shown, a swimming monitoring device based on a following float is first designed, and the images obtained by the device's double fisheye lens are unfolded and stitched together to obtain a panoramic video stream. Then, a YOLOv5 network is used for image recognition and classification, and the images are cropped to a fixed size. When detecting human bodies, the POSE_LSTM network is used to detect the swimmer's posture, and a danger alarm is triggered if drowning is detected. When detecting dangerous objects, the distance between the dangerous object and the swimmer is judged, and a danger alarm is triggered if the swimmer is too close to the dangerous object. In addition, the swimmer's voice is also recognized during monitoring so that photos or videos can be taken according to instructions.

[0052] In summary, the multifunctional outdoor swimming intelligent monitoring method according to embodiments of the present invention firstly acquires dual fisheye images using a dual fisheye camera mounted on a following float; then, the dual fisheye images are unfolded and stitched together to obtain a panoramic video stream, wherein the panoramic video stream includes the swimmer's swimming state and the swimmer's surrounding environment; next, a classification model is constructed, and the image sequence in the labeled panoramic video stream is used as a training set to train the classification model, so that the trained classification model can classify the input image sequence to output a fixed-size classified image sequence; then, a motion posture detection model is constructed, and the classified image sequence is used as a training set to classify the input image sequence. Images of swimmers in a sequence are input into a motion posture detection model to detect their posture and trigger an alarm when drowning is detected. Finally, the model uses images of the swimmer's surrounding environment in a classified image sequence to determine if the swimmer is near a dangerous object and triggers an alarm if so. Thus, by combining a dual fisheye camera on a buoy with a classification model, simultaneous monitoring of swimmers and their surroundings is achieved. Furthermore, considering the turbidity of outdoor water and the continuous nature of swimming, a motion posture detection model enables real-time monitoring of outdoor swimmers' postures, thereby improving monitoring effectiveness.

[0053] In addition, this invention also proposes a multifunctional outdoor swimming intelligent monitoring device, which stores a multifunctional outdoor swimming intelligent monitoring program. When the multifunctional outdoor swimming intelligent monitoring program is executed by a processor, it implements the multifunctional outdoor swimming intelligent monitoring method as described above.

[0054] According to an embodiment of the present invention, the multifunctional outdoor swimming intelligent monitoring device implements the above-mentioned multifunctional outdoor swimming intelligent monitoring method by means of a multifunctional outdoor swimming intelligent monitoring program, which is executed by a processor. Thus, by combining the dual fisheye cameras on the following float with a constructed classification model, simultaneous monitoring of the swimmer and the surrounding environment is achieved. Furthermore, considering the characteristics of turbid outdoor water and continuous swimming movements, a motion posture detection model is constructed to achieve real-time monitoring of the swimming posture of outdoor swimmers, thereby improving the monitoring effect.

[0055] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0056] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0057] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0058] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0059] It should be noted that any reference signs placed between parentheses in the claims should not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claims. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The invention can be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

[0060] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0061] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

[0062] In the description of this invention, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0063] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0064] In this invention, unless otherwise explicitly specified and limited, "above" or "below" the second feature can mean that the first feature is in direct contact with the second feature, or that the first feature is in indirect contact with the second feature through an intermediate medium. Furthermore, "above," "over," and "on top" of the second feature can mean that the first feature is directly above or diagonally above the second feature, or simply that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature can mean that the first feature is directly below or diagonally below the second feature, or simply that the first feature is at a lower horizontal level than the second feature.

[0065] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms should not be construed as necessarily referring to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0066] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A multifunctional intelligent monitoring method for outdoor swimming, characterized in that, The method of connecting a follow-up float to the swimmer includes the following steps: A dual fisheye camera mounted on a following float is used to acquire dual fisheye images; After the dual fisheye images are expanded and stitched together, a panoramic video stream is obtained, wherein the panoramic video stream includes the swimmer's swimming state and the swimmer's surrounding environment; A classification model is constructed, and the image sequence in the labeled panoramic video stream is used as the training set to train the classification model so that the trained classification model can classify the input image sequence and output a fixed-size classified image sequence. A motion posture detection model is constructed, and swimmer images from the classified image sequence are input into the motion posture detection model so as to detect swimmer posture through the motion posture detection model and issue an alarm when drowning is detected. Based on the surrounding environment images of the swimmer in the classified image sequence, determine whether the swimmer is close to a dangerous object, and issue an alarm when the swimmer is close to a dangerous object; The process of expanding the dual fisheye image includes: The default size of the unfolded image is h*w; The fisheye image is mapped onto a unit sphere, and a spherical rectangular coordinate system is established. Using the plane formed by the x-axis and y-axis as the starting point for longitude, establish w evenly spaced meridians ranging from -π to +π. From this, the angular intervals of longitude can be calculated as follows: Similarly, using the plane formed by the x-axis and z-axis as the starting point of latitude, establish h evenly spaced parallels of latitude, ranging in size from -π / 2 to +π / 2, where the positive y-axis direction represents the direction of increasing latitude. From this, the angular interval of latitude can be calculated as follows: The image is unfolded based on the principle that pixels at the same latitude on the sphere are located in the same row in the unfolded image, and pixels at the same longitude on the sphere are located in the same column in the unfolded image, thus obtaining the unfolded image; The motion posture detection model is a POSE_LSTM network. It modifies the key points originally set in the Openpose network and selects 12 locations, including the back of the head, neck, left ear, right ear, both shoulders, both elbows, both wrists, and both hips, as new key points for detection. It also increases the color contrast between the human body and the water by limiting the contrast adaptive histogram equalization algorithm. The key points of N consecutive frames of images are passed through the LSTM network to supplement the swimmer's motion features in the time series, and finally outputs the judgment result of the swimmer's posture.

2. The multifunctional outdoor swimming intelligent monitoring method as described in claim 1, characterized in that, The process of stitching the two fisheye images includes: The unfolded image obtained after unfolding the double fisheye image is subjected to feature-based matching and homography transformation to obtain the homography matrix; A projective transformation is performed on the homography matrix, and the two transformed images are then weighted and fused to obtain a panoramic planar image.

3. The multifunctional outdoor swimming intelligent monitoring method as described in claim 1, characterized in that, Based on the surrounding environment image of the swimmer in the classified image sequence, determine whether the swimmer is near a dangerous object, and issue an alarm when the swimmer is near a dangerous object, including: The surrounding environment image of the swimmer in the classified image sequence is input into the environmental detection module on the following float in order to calculate the area occupied by dangerous objects in the classified image sequence. When the area occupied by the next frame is greater than or equal to that of the previous frame, it is considered that the swimmer is approaching a dangerous object; let the width and height of the image be img_width and img_height, then the proportion of the dangerous object in the image, pro, is calculated using the formula area / (img_width*img_height); When the percentage of pro exceeds the set threshold TH, it is considered that the swimmer has entered the danger zone, and a warning is broadcast to the swimmer that a dangerous object is approaching. At the same time, when a swimmer enters a pre-recorded dangerous area or when the wave level on the water surface is detected to exceed a specified threshold, the swimmer will be informed of the current danger of the water area.

4. The multifunctional outdoor swimming intelligent monitoring method as described in claim 1, characterized in that, The follower float is also equipped with a voice recognition module, which takes a photo or video when it detects a specific voice command from the swimmer.

5. A multifunctional outdoor swimming intelligent monitoring device, characterized in that, It stores a multi-functional outdoor swimming intelligent monitoring program, which, when executed by the processor, implements the multi-functional outdoor swimming intelligent monitoring method as described in any one of claims 1-4.

Images