Swimming pool head position monitoring method, device, computer equipment and storage medium

By combining visible light and heat map images, a heat map model is established, and the drowning situation is judged by the difference in human head temperature and water temperature, the accuracy problem under the influence of environmental factors in the existing technology is solved, and the accuracy and generalization of drowning monitoring are improved.

CN118196717BActive Publication Date: 2024-11-19巨岩智能科技(杭州)有限公司
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410609167.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-05-16
Publication Date
2024-11-19
Estimated Expiration
2044-05-16

AI Technical Summary

Technical Problem

Existing anti-drowning monitoring technologies rely on visible light images, are susceptible to environmental factors, have low accuracy, and do not fully utilize infrared technology, especially when judging the position of human heads on and underwater.

Method used

Using a method combining visible light images and heat map images, a heat map model is established through object detection and tracking, and the tracking results of visible light images are mapped into the heat map model. The head temperature and surrounding water temperature are used to determine the head position, and the heat map model is updated to improve the accuracy.

Benefits of technology

It improves the accuracy and generalization of drowning judgments, reduces the impact of environmental factors on monitoring, and enhances the ability to identify the location of human heads on water and underwater.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118196717B_ABST
    Figure CN118196717B_ABST
Patent Text Reader

Abstract

The embodiments of the present invention disclose a method, device, computer equipment and storage medium for monitoring the position of a person's head in a swimming pool. The method comprises: obtaining a visible light image and a thermal image of a swimming pool; performing target detection and target tracking on the visible light image; performing thermal map modeling on the thermal image to obtain a thermal map model; mapping the tracking result of the visible light image to the thermal map model to obtain the temperature of the person's head and the temperature of the surrounding water; and determining the position of the person's head according to the temperature of the person's head and the temperature of the surrounding water. By implementing the method of the embodiments of the present invention, the temperature can be used to assist in determining the above-water and below-water position of the person's head, thereby improving the accuracy and generalization of drowning judgment.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to computer vision, and more specifically to a method, device, computer equipment and storage medium for monitoring the position of a person's head in a swimming pool. Background Art

[0002] Sports activities play an important role in people's daily lives. Swimming, as a sport that allows people to forget their worries, relieve stress, and keep fit, has attracted many people. However, despite its popularity, swimming faces the challenge of the highest risk of accidents. Especially in drowning accidents, children and adolescents become a high-risk group, who may have accidents due to lack of sufficient swimming experience or incorrect behavior. Therefore, drowning prevention monitoring has become an important safety measure at present, and it is crucial to improve its accuracy and effectiveness.

[0003] At present, there are some anti-drowning monitoring technologies, but these methods still have some shortcomings. Existing technologies mainly use visible light images for judgment and processing. However, relying solely on visible light images is easily affected by environmental factors, such as reflections, splashes, waves, lane lines, and scene switching, which will reduce the accuracy of monitoring. There are also a few methods that use infrared technology, but they are limited to detecting the specific position coordinates of the human head and do not fully utilize the advantages of infrared technology, especially judging from the perspective of swimming pool water temperature and human body temperature.

[0004] Therefore, it is necessary to design a new method to use temperature to assist in determining the above-water and below-water positions of a person's head, thereby improving the accuracy and generalization of drowning situation judgments. Summary of the invention

[0005] The purpose of the present invention is to overcome the defects of the prior art and provide a method, device, computer equipment and storage medium for monitoring the position of a person's head in a swimming pool.

[0006] To achieve the above object, the present invention adopts the following technical solution: a method for monitoring the position of a person's head in a swimming pool, comprising:

[0007] Obtain visible light images and thermal map images of the swimming pool;

[0008] Performing target detection and target tracking on the visible light image;

[0009] Performing heat map modeling on the heat map image to obtain a heat map model;

[0010] Mapping the tracking result of the visible light image into the thermal map model to obtain the head temperature and the surrounding water temperature;

[0011] The position of the human head is determined according to the temperature of the human head and the surrounding water temperature.

[0012] A further technical solution thereof is: after determining the position of the human head according to the temperature of the human head and the surrounding water temperature, it also includes:

[0013] The heat map model is updated using the newly acquired heat map data and the mask obtained after the human body frame is removed.

[0014] A further technical solution is: the target detection and target tracking of the visible light image includes:

[0015] Using the YOLOv5 target detection algorithm to perform target detection on the visible light image to obtain a detection result;

[0016] The ByteTrack multi-target tracking algorithm is used to perform real-time multi-target tracking on the detection results.

[0017] A further technical solution is: performing heat map modeling on the heat map image to obtain a heat map model includes:

[0018] Drawing an initial mask of the swimming pool on the heat map image;

[0019] The coordinate transformation matrix between visible light and infrared is calculated by four pairs of calibration points corresponding to the visible light image and the thermal image;

[0020] Map the coordinates of the human frame in the target tracking result to the initial mask, and remove the human frame on the mapped mask;

[0021] Accumulate several frames of thermal image and the corresponding masks after removing the human body frame;

[0022] Accumulate several frames of thermal map images to obtain the total temperature of several frames at each pixel;

[0023] Set the elements that are 0 in the accumulated mask to 1, and set the element temperature in the corresponding heat map matrix to the average temperature of the current heat map;

[0024] Divide several frames of heat map images and the corresponding masks to obtain a heat map model.

[0025] A further technical solution is: the coordinate conversion matrix of visible light and infrared is calculated by four pairs of calibration points corresponding to the visible light image and the thermal image, including:

[0026] The getPerspectiveTransform() method is used in OpenCV to calculate the coordinate transformation matrix of visible light and infrared through the four pairs of calibration points corresponding to the visible light image and the thermal map image.

[0027] A further technical solution is: mapping the coordinates of the human frame in the target tracking result to the initial mask, and removing the human frame on the mapped mask, including:

[0028] The human frame coordinates in the target tracking result are mapped to the initial mask and set to 0.

[0029] A further technical solution is: the method of updating the heat map model by using the acquired new heat map data and the mask obtained after the human body frame is removed, comprises:

[0030] Multiply the acquired new heat map data with the mask obtained after removing the human body frame to obtain the pixel temperature of the water surface without people;

[0031] The temperature of the unmanned water surface pixel is added to the heat map model with a certain weight to update the heat map model.

[0032] The present invention also provides a swimming pool head position monitoring device, comprising:

[0033] An acquisition unit, used for acquiring a visible light image and a thermal image of the swimming pool;

[0034] A detection and tracking unit, used for performing target detection and target tracking on the visible light image;

[0035] A modeling unit, configured to perform thermal map modeling on the thermal map image to obtain a thermal map model;

[0036] A mapping unit, used for mapping the tracking result of the visible light image into the thermal map model to obtain the temperature of the human head and the surrounding water temperature;

[0037] The position determination unit is used to determine the position of the human head according to the temperature of the human head and the surrounding water temperature.

[0038] The present invention further provides a computer device, which includes a memory and a processor. The memory stores a computer program, and the processor implements the above method when executing the computer program.

[0039] The present invention also provides a storage medium, wherein the storage medium stores a computer program, and the computer program implements the above method when executed by a processor.

[0040] Compared with the prior art, the beneficial effects of the present invention are as follows: the present invention performs target detection and tracking on visible light images, performs thermal map modeling on thermal map images, and then maps the tracking results of the visible light images to the modeled thermal map model, thereby determining the temperature of the human head and the surrounding water temperature, and utilizing the temperature difference between the two to determine the position of the human head, thereby utilizing temperature to assist in determining the above-water and underwater positions of the human head, thereby improving the accuracy and generalization of drowning situation judgments.

[0041] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.

[0043] Figure 1 A schematic diagram of an application scenario of the method for monitoring the position of a person's head in a swimming pool provided by an embodiment of the present invention;

[0044] Figure 2 A schematic diagram of a flow chart of a method for monitoring the position of a person's head in a swimming pool provided by an embodiment of the present invention;

[0045] Figure 3 A schematic diagram of a sub-process of a method for monitoring the position of a person's head in a swimming pool provided by an embodiment of the present invention;

[0046] Figure 4 A schematic diagram of a sub-process of a method for monitoring the position of a person's head in a swimming pool provided by an embodiment of the present invention;

[0047] Figure 5 A schematic flow chart of a method for monitoring the head position of a person in a swimming pool provided by another embodiment of the present invention;

[0048] Figure 6 A schematic diagram of a sub-process of a method for monitoring the position of a person's head in a swimming pool provided by another embodiment of the present invention;

[0049] Figure 7 A schematic block diagram of a swimming pool head position monitoring device provided by an embodiment of the present invention;

[0050] Figure 8 A schematic block diagram of a detection and tracking unit of a swimming pool head position monitoring device provided by an embodiment of the present invention;

[0051] Fig. 9 A schematic block diagram of a modeling unit of a swimming pool head position monitoring device provided by an embodiment of the present invention;

[0052] Fig.10A schematic block diagram of a swimming pool head position monitoring device provided by another embodiment of the present invention;

[0053] Fig.11 A schematic block diagram of an updating unit of a swimming pool head position monitoring device provided by another embodiment of the present invention;

[0054] Fig.12 A schematic block diagram of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0055] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0056] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0057] It should also be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0058] It should be further understood that the term "and / or" used in the present description and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0059] See also Figure 1 and Figure 2 , Figure 1 A schematic diagram of an application scenario of the method for monitoring the head position of a person in a swimming pool provided by an embodiment of the present invention. Figure 2 A schematic flow chart of a method for monitoring the position of a person's head in a swimming pool provided by an embodiment of the present invention. The method for monitoring the position of a person's head in a swimming pool is applied to a server. The server exchanges data with a terminal and an infrared camera, uses an infrared camera to monitor the water surface temperature in real time, and uses the modeling of the water surface temperature and the temperature of the person's head position to determine the position of the person's head above or below the water, thereby providing a more effective basis for drowning prevention, improving the accuracy of drowning judgment, and thus reducing the occurrence of drowning.

[0060] Figure 2 FIG. 1 is a flow chart of a method for monitoring the position of a person's head in a swimming pool provided by an embodiment of the present invention. Figure 2 As shown, the method includes the following steps S110 to S150.

[0061] S110 : Acquire a visible light image and a thermal image of the swimming pool.

[0062] In this embodiment, the visible light image is an image of a swimming pool scene captured by a camera. The visible light image shows the swimming pool scene visible to the human eye, including the outline of the swimming pool, the water surface, the surrounding environment, etc. This image is usually presented in color or black and white.

[0063] A heat map image is an image of a swimming pool scene captured by an infrared camera. A heat map image shows the heat distribution of objects rather than their visible appearance. In a swimming pool scene, a heat map image can show the temperature distribution of the water surface and surrounding objects, as well as the heat radiation of any human body in or on the water.

[0064] These two types of images can be captured by different sensors or cameras and used in image processing for different analyses and applications, such as swimming pool water temperature monitoring, human detection, security monitoring, etc.

[0065] S120: Perform target detection and target tracking on the visible light image.

[0066] In this embodiment, target detection refers to detecting the position of a person in an image, and target tracking refers to tracking the dynamic movement of a target in real time.

[0067] In one embodiment, see Figure 3 , the above-mentioned step S120 may include steps S121~S122.

[0068] S121. Use the YOLOv5 target detection algorithm to perform target detection on the visible light image to obtain a detection result.

[0069] In this embodiment, the detection result refers to the position of the person in the swimming pool. Through the above steps, the coordinates of the upper left corner and the lower right corner of the human body frame and the head frame of the swimmer in the swimming pool can be obtained.

[0070] Specifically, it is necessary to prepare a visible light image dataset for training and testing, which should include various situations in the swimming pool scene, including people. Select a YOLOv5 model suitable for the target detection task and train the model based on the prepared dataset. The training process includes inputting the image into the model, calculating the loss function, and then updating the model parameters through back propagation so that the model can better predict the location of people in the swimming pool. The trained model is tuned to improve its accuracy and robustness in detecting the location of people in the swimming pool scene. Use the trained YOLOv5 model to perform target detection on the visible light image of the swimming pool scene to obtain the detection results of the location of people.

[0071] S122, using ByteTrack multi-target tracking algorithm to perform real-time multi-target tracking on the detection results.

[0072] Specifically, the detection results obtained by the YOLOv5 object detection algorithm are used as input, including the location information of each detected person. The ByteTrack multi-object tracking algorithm is used, which is able to achieve real-time multi-object tracking in video sequences. This module can continuously track people in the pool based on the detection results and context information, and identify and track them. The algorithm needs to run on a real-time video stream, so efficient implementation and optimization are required to ensure accurate multi-object tracking under real-time requirements.

[0073] Through the above steps, real-time target detection and multi-target tracking of people in the swimming pool scene can be achieved to monitor and track the positions of people in the swimming pool.

[0074] The ByteTrack algorithm is used to ensure that each person detected has a corresponding ID, and if the same object is detected in subsequent frames, the same ID needs to be assigned to it.

[0075] S130, performing thermal map modeling on the thermal map image to obtain a thermal map model.

[0076] In this embodiment, the heat map model refers to the content consisting of the average temperature data of the swimming pool water surface within a certain period of time.

[0077] Specifically, this embodiment accepts the first 300 frames of heat map images by default for heat map modeling initialization. Of course, other numbers of heat map images can also be used for modeling.

[0078] In one embodiment, see Figure 4 , the above-mentioned step S130 may include steps S131~S137.

[0079] S131, drawing an initial mask of the swimming pool on the thermal map image.

[0080] In this embodiment, an initial mask is drawn for the thermal map image to mark the location of the swimming pool. This mask will help us to accurately extract the temperature information of the swimming pool water surface in the subsequent steps.

[0081] Specifically, the initial mask of the swimming pool is drawn by setting a matrix (256*192) that is consistent with the width and height of the heat map, where the matrix element 1 represents the water surface and 0 represents the non-water surface. Only the surface of the swimming pool is retained to model the surface temperature of the swimming pool.

[0082] S132, calculating the coordinate conversion matrix of visible light and infrared through four pairs of calibration points corresponding to the visible light image and the thermal image.

[0083] In this embodiment, the coordinate conversion matrix between visible light and infrared is calculated by corresponding four pairs of calibration points on the visible light image and the thermal image. This matrix will be used to map the coordinates on the visible light image to the thermal image, thereby realizing the coordinate conversion between the two.

[0084] Specifically, the coordinate transformation matrix of visible light and infrared is calculated using the getPerspectiveTransform() method in OpenCV by corresponding to four pairs of calibration points of the visible light image and the thermal map image.

[0085] Through the four pairs of calibration points corresponding to the visible light image and the thermal map image, the getPerspectiveTransform() method in OpenCV is used to calculate the coordinate transformation matrix of visible light and infrared. After obtaining the coordinate transformation matrix, the obtained human body frame coordinates are multiplied by the coordinate transformation matrix to obtain the coordinates of the human body frame corresponding to the human body in the thermal map image, so that the coordinates on the visible light can be mapped to the infrared thermal map.

[0086] S133, mapping the coordinates of the human frame in the target tracking result to the initial mask, and removing the human frame on the mapped mask.

[0087] In this embodiment, the coordinates of the human body frame in the target tracking result are mapped to the initial mask and set to 0.

[0088] Specifically, the coordinates of the human frame in the target tracking result are mapped to the initial mask. In this process, the area corresponding to the human frame is set to 0 on the mapped mask so that the influence of the human frame can be removed in the subsequent steps, that is, average water temperature modeling is required to eliminate the influence of human body temperature on water temperature modeling.

[0089] S134, accumulating a plurality of frames of thermal image and corresponding masks after removing the human body frame.

[0090] In this embodiment, 300 frames of thermal map data (300 256*192 matrices) and corresponding 300 masks after removing the human body frame, that is, 300 256*192 matrices, are accumulated, where matrix element 1 represents the water surface and 0 represents the non-water surface. In this way, enough data can be effectively accumulated to accurately calculate the average temperature of the swimming pool water surface in the subsequent steps.

[0091] S135 , accumulating several frames of thermal image to obtain the total temperature of each pixel point at the several frames.

[0092] In this embodiment, 300 frames of thermal map images are accumulated to obtain the total temperature of 300 frames at each pixel point, and 300 corresponding masks are accumulated to obtain how many frames of water temperature are in each pixel point, which can more accurately reflect the temperature distribution of the swimming pool water surface.

[0093] S136, setting the elements that are 0 in the accumulated mask to 1, and setting the element temperature in the corresponding heat map matrix to the average temperature of the current heat map.

[0094] In this embodiment, the elements that are 0 in the accumulated mask are set to 1, and the element temperature in the corresponding heat map matrix is ​​set to the average temperature of the current heat map, which can avoid the situation where the divisor is 0 in subsequent steps and ensure the accuracy of the data.

[0095] S137. Divide several frames of thermal map images and corresponding masks to obtain a thermal map model.

[0096] In this embodiment, the 300 frames of thermal map images are divided by the corresponding mask after the human body frame is removed, so that the average temperature of the water surface in 300 frames is preliminarily obtained. This thermal map model will reflect the average temperature of the swimming pool water surface within a certain period of time, providing an important basis for subsequent analysis.

[0097] S140: Map the tracking result of the visible light image into the thermal map model to obtain the temperature of the human head and the surrounding water temperature.

[0098] In this embodiment, after the water temperature model is obtained, the tracking result of the human head in the visible light image is mapped to the infrared corresponding area of ​​the thermal map model to obtain the maximum temperature value of the human head in the area.

[0099] First, a thermal map model has been obtained, which reflects the average temperature distribution of the swimming pool surface within a certain period of time. Next, the tracking result of the human head in the visible light image is mapped to the infrared corresponding area of ​​the thermal map model. This mapping process uses the visible light and infrared coordinate conversion matrix calculated in step S132 to map the position of the human head in the visible light image to the corresponding area on the thermal map model. After the mapping is completed, the maximum temperature value of the human head in the area can be obtained in the thermal map model. This maximum temperature value reflects the high and low water temperature around the human head, thereby providing an estimate of the water temperature around the human head.

[0100] S150: Determine the position of the human head according to the temperature of the human head and the surrounding water temperature.

[0101] In this embodiment, the temperature of the human head is subtracted from the average water temperature obtained by the heat map model. When the temperature difference between the maximum temperature of the human head and the average water temperature is greater than the set threshold value h, the human head is considered to be above the water surface, otherwise it is considered to be below the water surface. After obtaining the position information of the human head above and below the water, this part of information can be used to improve the accuracy of drowning judgment.

[0102] Specifically, the temperature difference between the maximum temperature of the human head and the average water temperature obtained by the heat map model is calculated; then, a threshold h is set to determine the position of the human head. If the temperature difference between the maximum temperature of the human head and the average water temperature is greater than the threshold h, the human head is considered to be above the water surface; otherwise, the human head is considered to be below the water surface; based on the above judgment, the location information of the human head (above or below the water) is determined. This part of information can be used to improve the accuracy of drowning judgment, such as timely issuing an alarm in the monitoring system or triggering rescue measures.

[0103] Through this process, the location of the human head can be effectively determined based on the difference between the temperature of the human head and the surrounding water temperature, thereby improving the ability to identify and respond to drowning incidents.

[0104] The above-mentioned method for monitoring the position of a person's head in a swimming pool performs target detection and tracking on visible light images, performs thermal map modeling on thermal map images, and then maps the tracking results of visible light images to the modeled thermal map model to determine the temperature of the person's head and the surrounding water temperature. The temperature difference between the two is used to determine the position of the person's head, thereby realizing the use of temperature to assist in determining the above-water and underwater positions of the person's head, thereby improving the accuracy and generalization of drowning situation judgments.

[0105] Figure 5 FIG. 2 is a flow chart of a method for monitoring the position of a person's head in a swimming pool provided by another embodiment of the present invention. Figure 5As shown, the swimming pool head position monitoring method of this embodiment includes steps S210-S260. Steps S210-S250 are similar to steps S110-S150 in the above embodiment, and will not be repeated here. The following is a detailed description of step S260 added in this embodiment.

[0106] S260: Update the heat map model using the acquired new heat map data and the mask obtained after the human body frame is removed.

[0107] Updating the heat map model means that after obtaining a new heat map image and the corresponding mask for removing the human body frame (wherein the matrix element 1 represents the water surface and 0 represents the human body), the heat map image and the mask are multiplied to obtain the pixel temperature of the unmanned water surface, and the temperature and the corresponding average water temperature modeled above are added with a certain weight, that is, the modeled water surface temperature * weight + current frame water surface temperature * (1-weight) is the new average water surface temperature. The default weight value is 0.8. Of course, it can also be adjusted according to actual conditions.

[0108] In one embodiment, see Figure 6 , the above-mentioned step S260 may include steps S261~S262.

[0109] S261, multiplying the acquired new thermal map data and the corresponding mask obtained after removing the human body frame to obtain the pixel temperature of the water surface without people.

[0110] In this embodiment, new thermal map data is obtained from the sensor, which reflects the temperature distribution of the current water surface. Then, the mask obtained after removing the human body frame is used to exclude the part of the thermal map data that overlaps with the human body frame, and the pixel temperature of the water surface without people is obtained. This can eliminate the interference of the human body, so that the temperature data can more accurately reflect the actual situation of the water surface.

[0111] S262: Add the temperature of the unmanned water surface pixel to the thermal map model with a certain weight to update the thermal map model.

[0112] Specifically, the unmanned water surface pixel temperature obtained in step S261 is added to the original thermal map model, and the newly acquired temperature information can be integrated into the original model. During the addition process, a certain weight can be used to adjust the influence of new and old data to balance the adaptability of the updated model to historical data and new data.

[0113] Through the implementation of these two steps, the thermal map model can be updated in real time, so that it can better adapt to the changes in water surface temperature in different time periods and reduce the impact of human interference on temperature data.

[0114] Figure 7FIG. 3 is a schematic block diagram of a swimming pool head position monitoring device 300 provided by an embodiment of the present invention. Figure 7 As shown, corresponding to the above swimming pool head position monitoring method, the present invention also provides a swimming pool head position monitoring device 300. The swimming pool head position monitoring device 300 includes a unit for executing the above swimming pool head position monitoring method, and the device can be configured in a server. Figure 7 The swimming pool head position monitoring device 300 includes an acquisition unit 301, a detection and tracking unit 302, a modeling unit 303, a mapping unit 304 and a position determination unit 305.

[0115] The acquisition unit 301 is used to acquire the visible light image and the thermal map image of the swimming pool; the detection and tracking unit 302 is used to perform target detection and target tracking on the visible light image; the modeling unit 303 is used to perform thermal map modeling on the thermal map image to obtain a thermal map model; the mapping unit 304 is used to map the tracking result of the visible light image to the thermal map model to obtain the human head temperature and the surrounding water temperature; the position determination unit 305 is used to determine the position of the human head according to the human head temperature and the surrounding water temperature.

[0116] In one embodiment, if Figure 8 As shown, the detection and tracking unit 302 includes a target detection subunit 3021 and a tracking subunit 3022 .

[0117] The target detection subunit 3021 is used to perform target detection on the visible light image using the YOLOv5 target detection algorithm to obtain a detection result; the tracking subunit 3022 is used to perform real-time multi-target tracking on the detection result using the ByteTrack multi-target tracking algorithm.

[0118] In one embodiment, if Fig. 9 As shown, the modeling unit 303 includes a drawing subunit 3031, a matrix calculation subunit 3032, a coordinate mapping subunit 3033, an accumulation subunit 3034, an accumulation subunit 3035, a setting subunit 3036 and a division subunit 3037.

[0119] The drawing subunit 3031 is used to draw the initial mask of the swimming pool for the thermal image; the matrix calculation subunit 3032 is used to calculate the coordinate conversion matrix of visible light and infrared through four pairs of calibration points corresponding to the visible light image and the thermal image; the coordinate mapping subunit 3033 is used to map the coordinates of the human body frame in the result obtained by target tracking to the initial mask, and remove the human body frame on the mapped mask; the accumulation subunit 3034 is used to accumulate several frames of thermal image and the corresponding masks after removing the human body frame; the accumulation subunit 3035 is used to accumulate several frames of thermal image to obtain the total temperature of several frames at each pixel; the setting subunit 3036 is used to set the elements of 0 in the accumulated mask to 1, and set the element temperature in the corresponding thermal map matrix to the average temperature of the current thermal map; the division subunit 3037 is used to divide several frames of thermal image and the corresponding mask to obtain a thermal map model.

[0120] In one embodiment, the matrix calculation subunit 3032 is used to calculate the coordinate transformation matrix of visible light and infrared by using the getPerspectiveTransform() method in OpenCV through four pairs of calibration points corresponding to the visible light image and the thermal image.

[0121] In one embodiment, the coordinate mapping subunit 3033 is used to map the coordinates of the human frame in the result obtained by target tracking to the initial mask and set them to 0.

[0122] Fig.10 FIG. 3 is a schematic block diagram of a swimming pool head position monitoring device 300 provided by another embodiment of the present invention. Fig.10 As shown, the swimming pool head position monitoring device 300 of this embodiment is based on the above embodiment and adds an updating unit 306.

[0123] The updating unit 306 is used to update the heat map model by using the acquired new heat map data and the mask obtained after the human body frame is removed.

[0124] In one embodiment, if Fig.11 As shown, the updating unit 306 includes a multiplication subunit 3061 and an addition subunit 3062 .

[0125] The multiplication subunit 3061 is used to multiply the newly acquired thermal map data and the corresponding mask obtained after removing the human body frame to obtain the unmanned water surface pixel temperature; the addition subunit 3062 is used to add the unmanned water surface pixel temperature and the thermal map model with a certain weight to update the thermal map model.

[0126] It should be noted that those skilled in the art can clearly understand that the specific implementation process of the above-mentioned swimming pool head position monitoring device 300 and each unit can refer to the corresponding description in the aforementioned method embodiment, and for the convenience and brevity of description, it will not be repeated here.

[0127] The above swimming pool head position monitoring device 300 can be implemented in the form of a computer program. The computer program can be used in Fig.12 Runs on the computer device shown.

[0128] See also Fig.12 , Fig.12 5 is a schematic block diagram of a computer device provided in an embodiment of the present application. The computer device 500 may be a server, wherein the server may be an independent server or a server cluster composed of multiple servers.

[0129] See also Fig.12 The computer device 500 includes a processor 502 , a memory and a network interface 505 connected via a system bus 501 , wherein the memory may include a non-volatile storage medium 503 and an internal memory 504 .

[0130] The non-volatile storage medium 503 can store an operating system 5031 and a computer program 5032. The computer program 5032 includes program instructions, and when the program instructions are executed, the processor 502 can execute a method for monitoring the position of a person's head in a swimming pool.

[0131] The processor 502 is used to provide computing and control capabilities to support the operation of the entire computer device 500 .

[0132] The internal memory 504 provides an environment for the operation of the computer program 5032 in the non-volatile storage medium 503. When the computer program 5032 is executed by the processor 502, the processor 502 can execute a method for monitoring the position of a person's head in a swimming pool.

[0133] The network interface 505 is used to communicate with other devices over the network. Fig.12 The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present application, and does not constitute a limitation on the computer device 500 to which the solution of the present application is applied. The specific computer device 500 may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0134] The processor 502 is used to run the computer program 5032 stored in the memory to implement the following steps:

[0135] Obtain a visible light image and a thermal map image of a swimming pool; perform target detection and target tracking on the visible light image; perform thermal map modeling on the thermal map image to obtain a thermal map model; map the tracking result of the visible light image to the thermal map model to obtain a human head temperature and a surrounding water temperature; and determine a position of a human head according to the human head temperature and the surrounding water temperature.

[0136] In one embodiment, after implementing the step of determining the position of the human head according to the temperature of the human head and the surrounding water temperature, the processor 502 further implements the following steps:

[0137] The heat map model is updated using the newly acquired heat map data and the mask obtained after the human body frame is removed.

[0138] In one embodiment, when the processor 502 implements the step of performing target detection and target tracking on the visible light image, the processor 502 specifically implements the following steps:

[0139] The YOLOv5 target detection algorithm is used to perform target detection on the visible light image to obtain a detection result; and the ByteTrack multi-target tracking algorithm is used to perform real-time multi-target tracking on the detection result.

[0140] In one embodiment, when the processor 502 implements the step of performing heat map modeling on the heat map image to obtain a heat map model, the processor 502 specifically implements the following steps:

[0141] Draw an initial mask of the swimming pool for the thermal image; calculate the coordinate conversion matrix of visible light and infrared through four pairs of calibration points corresponding to the visible light image and the thermal image; map the coordinates of the human body frame in the result of target tracking to the initial mask, and remove the human body frame on the mapped mask; accumulate several frames of thermal image and the corresponding masks after removing the human body frame; accumulate several frames of thermal image to obtain the total temperature of several frames at each pixel; set the elements of 0 in the accumulated mask to 1, and set the element temperature in the corresponding thermal map matrix to the average temperature of the current thermal map; divide several frames of thermal image and the corresponding mask to obtain a thermal map model.

[0142] In one embodiment, when the processor 502 implements the step of calculating the coordinate conversion matrix of visible light and infrared by using four pairs of calibration points corresponding to the visible light image and the thermal image, the processor 502 specifically implements the following steps:

[0143] The getPerspectiveTransform() method is used in OpenCV to calculate the coordinate transformation matrix of visible light and infrared through the four pairs of calibration points corresponding to the visible light image and the thermal map image.

[0144] In one embodiment, when the processor 502 implements the step of mapping the human frame coordinates in the target tracking result to the initial mask and removing the human frame on the mapped mask, the processor 502 specifically implements the following steps:

[0145] The human frame coordinates in the target tracking result are mapped to the initial mask and set to 0.

[0146] In one embodiment, when the processor 502 implements the step of updating the heat map model using the acquired new heat map data and the mask obtained after the human body frame is removed, the processor 502 specifically implements the following steps:

[0147] The newly acquired thermal map data is multiplied by the mask obtained after the human body frame is removed to obtain the pixel temperature of the water surface without people; the pixel temperature of the water surface without people is added to the thermal map model with a certain weight to update the thermal map model.

[0148] It should be understood that in the embodiment of the present application, the processor 502 may be a central processing unit (CPU), and the processor 502 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0149] It can be understood by those skilled in the art that all or part of the processes in the method for implementing the above embodiment can be completed by instructing the relevant hardware through a computer program. The computer program includes program instructions, and the computer program can be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in the computer system to implement the process steps of the embodiment of the above method.

[0150] Therefore, the present invention also provides a storage medium. The storage medium may be a computer-readable storage medium. The storage medium stores a computer program, wherein when the computer program is executed by a processor, the processor executes the following steps:

[0151] Obtain a visible light image and a thermal map image of a swimming pool; perform target detection and target tracking on the visible light image; perform thermal map modeling on the thermal map image to obtain a thermal map model; map the tracking result of the visible light image to the thermal map model to obtain a human head temperature and a surrounding water temperature; and determine a position of a human head according to the human head temperature and the surrounding water temperature.

[0152] In one embodiment, after executing the computer program to implement the step of determining the position of the human head according to the temperature of the human head and the surrounding water temperature, the processor further implements the following steps:

[0153] The heat map model is updated using the newly acquired heat map data and the mask obtained after the human body frame is removed.

[0154] In one embodiment, when the processor executes the computer program to implement the steps of performing target detection and target tracking on the visible light image, the processor specifically implements the following steps:

[0155] The YOLOv5 target detection algorithm is used to perform target detection on the visible light image to obtain a detection result; and the ByteTrack multi-target tracking algorithm is used to perform real-time multi-target tracking on the detection result.

[0156] In one embodiment, when the processor executes the computer program to implement the step of performing thermal map modeling on the thermal map image to obtain a thermal map model, the processor specifically implements the following steps:

[0157] Draw an initial mask of the swimming pool for the thermal image; calculate the coordinate conversion matrix of visible light and infrared through four pairs of calibration points corresponding to the visible light image and the thermal image; map the coordinates of the human body frame in the result of target tracking to the initial mask, and remove the human body frame on the mapped mask; accumulate several frames of thermal image and the corresponding masks after removing the human body frame; accumulate several frames of thermal image to obtain the total temperature of several frames at each pixel; set the elements of 0 in the accumulated mask to 1, and set the element temperature in the corresponding thermal map matrix to the average temperature of the current thermal map; divide several frames of thermal image and the corresponding mask to obtain a thermal map model.

[0158] In one embodiment, when the processor executes the computer program to implement the step of calculating the coordinate transformation matrix of visible light and infrared by using four pairs of calibration points corresponding to the visible light image and the thermal image, the processor specifically implements the following steps:

[0159] The getPerspectiveTransform() method is used in OpenCV to calculate the coordinate transformation matrix of visible light and infrared through the four pairs of calibration points corresponding to the visible light image and the thermal map image.

[0160] In one embodiment, when the processor executes the computer program to implement the steps of mapping the human frame coordinates in the target tracking result to the initial mask and removing the human frame on the mapped mask, the processor specifically implements the following steps:

[0161] Map the coordinates of the human frame in the target tracking result to the initial mask, and remove the human frame on the mapped mask

[0162] In one embodiment, when the processor executes the computer program to implement the step of updating the heat map model using the acquired new heat map data and the mask obtained after the human body frame is removed, the processor specifically implements the following steps:

[0163] The newly acquired thermal map data is multiplied by the mask obtained after the human body frame is removed to obtain the pixel temperature of the water surface without people; the pixel temperature of the water surface without people is added to the thermal map model with a certain weight to update the thermal map model.

[0164] The storage medium may be a USB flash drive, a mobile hard disk, a read-only memory (ROM), a magnetic disk, or an optical disk, etc., which are computer-readable storage media that can store program codes.

[0165] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0166] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of each unit is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0167] The steps in the method of the embodiment of the present invention can be adjusted in order, combined and deleted according to actual needs. The units in the device of the embodiment of the present invention can be combined, divided and deleted according to actual needs. In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0168] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, terminal, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention.

[0169] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for monitoring the head position of a person in a swimming pool, characterized in that: include: Obtain visible light images and thermal imagery of the swimming pool; Performing target detection and target tracking on the visible light image; Performing heat map modeling on the heat map image to obtain a heat map model; Mapping the tracking result of the visible light image into the thermal map model to obtain the head temperature and the surrounding water temperature; Determine the position of the human head according to the temperature of the human head and the surrounding water temperature; After determining the position of the human head according to the temperature of the human head and the surrounding water temperature, the method further includes: The heat map model is updated using the acquired new heat map data and the mask obtained after the human body frame is removed; The method of updating the heat map model by using the acquired new heat map data and the mask obtained after the human body frame is removed includes: Multiply the acquired new heat map data with the mask obtained after removing the human body frame to obtain the pixel temperature of the water surface without people; Adding the temperature of the unmanned water surface pixel to the heat map model with a certain weight to update the heat map model; Updating the thermal map model means that after obtaining a new thermal map image and a corresponding mask for removing the human body frame, the thermal map image and the mask are multiplied to obtain the pixel temperature of the water surface without people, and the temperature and the corresponding average water temperature completed by the above modeling are added with a certain weight. The modeled water surface temperature*weight+current frame water surface temperature*(1-weight) is the new average water surface temperature.

2. The method for monitoring the head position of a person in a swimming pool according to claim 1, characterized in that: The performing target detection and target tracking on the visible light image includes: Using the YOLOv5 target detection algorithm to perform target detection on the visible light image to obtain a detection result; The ByteTrack multi-target tracking algorithm is used to perform real-time multi-target tracking on the detection results.

3. The method for monitoring the head position of a person in a swimming pool according to claim 1, characterized in that: The performing heat map modeling on the heat map image to obtain a heat map model includes: Drawing an initial mask of the swimming pool on the heat map image; The coordinate transformation matrix between visible light and infrared is calculated by four pairs of calibration points corresponding to the visible light image and the thermal image; Map the coordinates of the human frame in the target tracking result to the initial mask, and remove the human frame on the mapped mask; Accumulate several frames of thermal image and the corresponding masks after removing the human body frame; Accumulate several frames of thermal map images to obtain the total temperature of several frames at each pixel; Set the elements that are 0 in the accumulated mask to 1, and set the element temperature in the corresponding heat map matrix to the average temperature of the current heat map; Divide several frames of heat map images and the corresponding masks to obtain a heat map model.

4. The method for monitoring the head position of a person in a swimming pool according to claim 3, characterized in that: The coordinate conversion matrix of visible light and infrared is calculated by four pairs of calibration points corresponding to the visible light image and the thermal image, including: The getPerspectiveTransform() method is used in OpenCV to calculate the coordinate transformation matrix of visible light and infrared through the four pairs of calibration points corresponding to the visible light image and the thermal map image.

5. The method for monitoring the head position of a person in a swimming pool according to claim 3, characterized in that: The step of mapping the coordinates of the human frame in the target tracking result to the initial mask and removing the human frame on the mapped mask includes: The human frame coordinates in the target tracking result are mapped to the initial mask and set to 0.

6. A swimming pool head position monitoring device, characterized in that: include: An acquisition unit, used for acquiring a visible light image and a thermal image of the swimming pool; A detection and tracking unit, used for performing target detection and target tracking on the visible light image; A modeling unit, configured to perform thermal map modeling on the thermal map image to obtain a thermal map model; A mapping unit, used for mapping the tracking result of the visible light image into the thermal map model to obtain the temperature of the human head and the surrounding water temperature; A position determination unit, used to determine the position of the human head according to the temperature of the human head and the surrounding water temperature; An updating unit, used to update the heat map model by using the acquired new heat map data and the mask obtained after the human body frame is removed; The updating unit comprises a multiplication subunit and an addition subunit; A multiplication subunit is used to multiply the acquired new thermal map data and the mask obtained after the human body frame is removed to obtain the pixel temperature of the water surface without people; An adding subunit, used for adding the temperature of the unmanned water surface pixel to the thermal map model with a certain weight to update the thermal map model; Updating the thermal map model means that after obtaining a new thermal map image and a corresponding mask for removing the human body frame, the thermal map image and the mask are multiplied to obtain the pixel temperature of the water surface without people, and the temperature and the corresponding average water temperature completed by the above modeling are added with a certain weight. The modeled water surface temperature*weight+current frame water surface temperature*(1-weight) is the new average water surface temperature.

7. A computer device, characterized in that: The computer device comprises a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 5 when executing the computer program.

8. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.