Multi-strategy fusion swimming pool small target detection tracking enhancement method and system

Abstract

The invention discloses a multi-strategy fusion swimming pool small target detection tracking enhancement method and system. The method comprises the following steps: acquiring an image to be detected and tracked to obtain an initial image; inputting the initial image into a target detection model for target detection to obtain an ROI region; wherein the target detection model is obtained by obtaining an image with a target in the swimming pool, performing cutting and data enhancement on the image, and training the pre-training detection model by using the image as a sample set; cutting and amplifying the ROI region, and inputting the processed ROI region into the target detection model again for target detection to obtain a detection result; mapping the detection result back to a coordinate system corresponding to the initial image to obtain a mapping result; performing multi-threshold detection and compensation tracking on the mapping result to obtain a tracking result; and outputting a tracking result. By implementing the method provided by the invention, the detection precision of the small target can be improved, and more stable target tracking can be realized.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and more specifically to a multi-strategy fusion small target detection and tracking enhancement method and system for a swimming pool. Background Art

[0002] In recent years, computer vision technology has made significant progress in many application fields, especially in target detection and tracking, where deep learning algorithms have been widely adopted. However, in practical applications, the detection and tracking of small targets still face many challenges, especially when the target size is small, the background is complex, or the target is dense, there are often problems of missed detection or false detection. These problems are particularly prominent in indoor swimming venues.

[0003] Swimming pools have some unique characteristics that increase the difficulty of detecting and tracking small targets. Specifically, the surveillance cameras in swimming pools usually cover the entire pool area, which makes the targets in the pool, such as the swimmer's head and body, relatively small in size in the image. Traditional target detection algorithms have difficulty in effectively extracting the features of such small targets, which affects the detection effect. Background factors in the swimming pool environment, such as water surface reflections, water ripples, lane lines and buoys, are all dynamically changing, which can easily interfere with the target detection algorithm's recognition of small targets. Swimmers move at a fast speed and have irregular trajectories, which puts higher real-time and robustness requirements on the target tracking algorithm, especially in the case of high-speed movement, the tracking accuracy may be affected. During the peak swimming period, there may be a large number of swimmers in the pool, and the targets may block or overlap each other, which increases the difficulty of detecting and tracking small targets.

[0004] In response to these problems, the main challenges facing current monitoring needs include: difficulty in extracting features of small targets, insufficient integration of detection and tracking, and easy loss of tracking in dynamic scenes.

[0005] Therefore, it is necessary to design a new method that can improve the detection accuracy of small targets and achieve more stable target tracking. Summary of the invention

[0006] The purpose of the present invention is to overcome the defects of the prior art and provide a multi-strategy fusion small target detection and tracking enhancement method and system for swimming pools.

[0007] To achieve the above purpose, the present invention adopts the following technical solution: a multi-strategy fusion swimming pool small target detection and tracking enhancement method, comprising: Acquire the image to be detected and tracked to obtain an initial image; Input the initial image into the target detection model for target detection to obtain the ROI area; wherein the target detection model is obtained by obtaining images with targets in the swimming pool, cropping them, and performing data enhancement to train the pre-trained detection model as a sample set; The ROI region is cropped and enlarged, and the processed ROI region is input into the target detection model again for target detection to obtain a detection result; Mapping the detection result back to the coordinate system corresponding to the initial image to obtain a mapping result; Using multi-threshold detection and compensation tracking on the mapping result to obtain a tracking result; The tracking result is outputted.

[0008] The further technical solution is: the target detection model is obtained by obtaining images with targets in the swimming pool, cropping them, and enhancing the data to train the pre-trained detection model as a sample set, including: Acquire images with targets in the swimming pool; Cropping the image using a dynamic sliding window to obtain a cropping result; Labeling the cropping results to generate a training set; Performing data enhancement on the training set to obtain a sample set; Build a pre-trained detection model and loss function; The pre-trained detection model is trained using the sample set, and the target detection model is determined by combining the loss function and the trained model.

[0009] A further technical solution is: the image is cropped using a dynamic sliding window to obtain a cropping result, including: The image is cropped using different sliding windows according to the lane area and the non-critical area to obtain a cropping result.

[0010] A further technical solution is: labeling the cropping results to generate a training set, including: The cropping results are labeled with objects, and the positions of the objects are recorded to obtain a training set.

[0011] A further technical solution is: performing data enhancement on the training set to obtain a sample set includes: Identifying and labeling the head and body features of the swimmers in the training set, and adjusting the scaling to magnify the region extracted from the training set to obtain an enlarged image; Using a target tiling algorithm to stitch the enlarged images according to the distribution of targets in the actual scene, and updating corresponding annotation information to obtain a synthesis result; Performing weighted processing on the small targets in the synthesis result to obtain a weighted processing result; The pre-trained model is used to filter out small target instances whose confidence meets the requirements from the weighted processing results to obtain a sample set.

[0012] A further technical solution is: the mapping result is subjected to multi-threshold detection and compensation tracking to obtain a tracking result, including: Setting a first threshold and a second threshold, wherein the first threshold is greater than the second threshold, the first threshold is used for multi-target tracking; and the second threshold is used to compensate for lost target frames; Multi-target tracking is performed on the results in the mapping results that are greater than the first threshold, and when the tracking is currently lost in the current frame, a compensation mechanism is started, and a matching target frame is found from the results in the multi-target tracking results that are lower than the second threshold by using IoU matching and the Hungarian algorithm to compensate for the lost human tracking frame to obtain a tracking result.

[0013] A further technical solution is: performing multi-target tracking on the results greater than the first threshold in the mapping results, and when the tracking is currently lost in the current frame, starting a compensation mechanism, using IoU matching and the Hungarian algorithm to find a matching target frame from the results less than the second threshold in the multi-target tracking results, to compensate for the lost human tracking frame, so as to obtain a tracking result, including: Performing multi-target tracking on the results greater than the first threshold in the mapping results using a multi-target tracking algorithm to obtain a multi-target tracking result, wherein the multi-target tracking result includes a list of human body tracking frames of tracking IDs; The status of each tracked object is checked frame by frame. When it is found that the tracking ID of a swimmer disappears in the current frame and a swimmer has not swam out of the range, the IoU matching and Hungarian algorithm are used to find the matching target frame from the results below the second threshold in the multi-target tracking results to compensate for the lost human tracking frame and obtain the tracking result.

[0014] A further technical solution is as follows: the state of each tracked object is checked frame by frame, and when it is found that the tracking ID of a swimmer disappears in the current frame and the swimmer has not swam out of the range, the IoU matching and the Hungarian algorithm are used to find a matching target frame from the results below the second threshold in the multi-target tracking results to compensate for the lost human tracking frame, so as to obtain the tracking result, including: Check the status of each tracked object frame by frame. When it is found that the tracking ID of a swimmer disappears in the current frame and the swimmer has not swam out of the range, compare the corresponding human tracking frame in the multi-target tracking result corresponding to the disappeared human tracking frame with all human tracking frames in the current frame that are lower than the second threshold, and calculate the intersection over union ratio between the two human tracking frames to obtain the IoU matrix. Construct a cost matrix based on the IoU matrix; Applying the Hungarian algorithm to the cost matrix to match all human tracking frames below the second threshold and disappeared human tracking frames in the current frame to obtain a matching result; The matching result is added to the multi-target tracking result of the current frame to obtain the tracking result.

[0015] The present invention also provides a multi-strategy fusion swimming pool small target detection and tracking enhancement system, including: An image acquisition unit, used to acquire an image to be detected and tracked to obtain an initial image; A target detection unit, used for inputting the initial image into a target detection model for target detection to obtain a ROI area; wherein the target detection model is obtained by obtaining an image with a target in a swimming pool, cropping it, and performing data enhancement to train a pre-trained detection model as a sample set; A re-detection unit, used for cropping and enlarging the ROI area, and inputting the processed ROI area into the target detection model again for target detection to obtain a detection result; A mapping unit, used for mapping the detection result back to the coordinate system corresponding to the initial image to obtain a mapping result; A tracking unit, used for applying multi-threshold detection and compensation tracking to the mapping result to obtain a tracking result; An output unit is used to output the tracking result.

[0016] Its further technical solution is: it also includes a training unit for: Acquire images with targets in the swimming pool; Cropping the image using a dynamic sliding window to obtain a cropping result; Labeling the cropping results to generate a training set; Performing data enhancement on the training set to obtain a sample set; Build a pre-trained detection model and loss function; The pre-trained detection model is trained using the sample set, and the target detection model is determined by combining the loss function and the trained model.

[0017] The beneficial effects of the present invention compared with the prior art are as follows: the present invention obtains the initial image and inputs it into the target detection model to identify the potential small target area, and improves the sensitivity and accuracy of the target detection model to small targets by cropping and data enhancement of the training data set, and crops and enlarges the detected ROI area to further enhance the detail features of the small target and improve the accuracy of subsequent detection. The detection results are mapped back to the coordinate system of the initial image to ensure the accuracy of the target position, and the target is stably tracked by combining multi-threshold detection and compensation tracking strategies to reduce misidentification and loss, thereby improving the stability and accuracy of the overall tracking.

[0018] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.

[0020] Figure 1 A schematic diagram of the flow of a multi-strategy fusion method for enhancing detection and tracking of small targets in swimming pools provided by an embodiment of the present invention; Figure 2 Schematic diagram of the sub-process of the multi-strategy fusion swimming pool small target detection and tracking enhancement method provided by the embodiment of the present invention Figure 1 ; Figure 3 Schematic diagram of the sub-process of the multi-strategy fusion swimming pool small target detection and tracking enhancement method provided by the embodiment of the present invention Figure 2 ; Figure 4 Schematic diagram of the sub-process of the multi-strategy fusion swimming pool small target detection and tracking enhancement method provided by the embodiment of the present invention Figure 3 ; Figure 5 Schematic diagram of the sub-process of the multi-strategy fusion swimming pool small target detection and tracking enhancement method provided by the embodiment of the present invention Figure 4 ; Figure 6 Schematic diagram of the sub-process of the multi-strategy fusion swimming pool small target detection and tracking enhancement method provided by the embodiment of the present invention Figure 5 ; Figure 7 A schematic diagram of a multi-threshold detection and compensation tracking process of a multi-strategy fusion swimming pool small target detection and tracking enhancement method provided in an embodiment of the present invention; Figure 8A schematic block diagram of a multi-strategy fusion swimming pool small target detection and tracking enhancement system provided by an embodiment of the present invention; Fig. 9 A schematic block diagram of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0021] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0022] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0023] It should also be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0024] It should be further understood that the term "and / or" used in the present description and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0025] See also Figure 1 , Figure 1 A schematic flow chart of a multi-strategy fusion method for detecting, tracking and enhancing small targets in swimming pools provided in an embodiment of the present invention. The multi-strategy fusion method for detecting, tracking and enhancing small targets in swimming pools is applied to a server, which interacts with a terminal and a camera for data, and generates diversified training samples by cropping and data enhancing images in the swimming pool, thereby improving the target detection model's ability to recognize small targets. The ROI area in the initial image is cropped and enlarged to enhance the details of the small target and further improve the detection accuracy. The detection results are mapped back to the initial image coordinate system to ensure the accuracy of the target position. Combined with multi-threshold detection and compensation tracking methods, stable tracking of small targets is ensured to reduce loss and false detection phenomena.

[0026] Figure 1FIG. 1 is a flow chart of a method for enhancing detection, tracking and enhancement of small targets in swimming pools by integrating multiple strategies provided by an embodiment of the present invention. Figure 1 As shown, the method includes the following steps S110 to S160.

[0027] S110: Acquire an image to be detected and tracked to obtain an initial image.

[0028] In this embodiment, the initial image refers to the original image first obtained from the swimming pool monitoring system or other image acquisition equipment in the target detection and tracking task. These images contain the targets to be detected and tracked (such as swimmers) and serve as the basis for subsequent processing. The initial image is usually the original input image without any processing or enhancement. Before the target is recognized by the target detection model, it is mainly used to extract the potential region of interest (ROI) of the target and the subsequent processing steps.

[0029] S120, input the initial image into the target detection model for target detection to obtain the ROI area; wherein the target detection model is obtained by obtaining images with targets in the swimming pool, cropping them, and performing data enhancement, and then using them as sample sets to train the pre-trained detection model.

[0030] In this embodiment, the ROI region refers to the region with the target of the swimmer.

[0031] Specifically, the target detection model is used to identify small target areas in the image and extract the corresponding ROI area. The image with an original resolution of 1920×1080 is segmented through a pre-made mask to determine specific areas in the swimming pool and filter out areas that do not require attention, such as the shore.

[0032] In one embodiment, see Figure 2 The above-mentioned target detection model is obtained by obtaining images with targets in a swimming pool, cropping them, and performing data enhancement as a sample set to train a pre-trained detection model, including steps S121 to S126.

[0033] S121. Acquire an image with a target in a swimming pool.

[0034] In this embodiment, video streams or image libraries from the swimming pool monitoring system are collected to ensure that the images cover various scenes and lighting conditions in the swimming pool area. These images will serve as the basis for subsequent processing.

[0035] S122: Crop the image using a dynamic sliding window to obtain a cropping result.

[0036] In this embodiment, the cropping result refers to the result obtained by cropping the image according to different sliding windows.

[0037] Specifically, the image is cropped using different sliding windows according to the lane area and the non-critical area to obtain the cropping result. The size and step size of the sliding window are dynamically adjusted according to the density and distribution of the target in the image, rather than using a fixed window size and overlap rate. This ensures that small targets are covered more effectively when the target density in the swimming pool area is high.

[0038] According to the characteristics of the swimming pool environment (such as lane layout) and the distribution of targets in the image, a sliding window algorithm with dynamic adjustment of size and step size is used to crop the input image. The initial window size is set to 256×256 pixels, and the default overlap rate is 50%, but the window size and overlap rate will be adaptively adjusted according to the actual target density. In particular, the window size is reduced in the lane area to increase the coverage of small targets, while the window is enlarged between lanes to reduce irrelevant background interference. The position information of each cropping window is recorded for subsequent processing. The specific cropping formula is: .in Indicates the step size of each sliding of the sliding window, Indicates the window size, It represents the ratio of the overlapping area between adjacent windows, which determines the degree of overlap between two consecutive windows when sliding.

[0039] Considering the lane layout of the swimming pool and the typical movement trajectory of the swimmer, the window cropping strategy is optimized. Specifically, a smaller window is used in the lane area to improve the coverage of small targets, while a larger window is used at the lane interval to reduce the cropping of irrelevant background.

[0040] S123: label the cropping results to generate a training set.

[0041] In this embodiment, the cropping results are labeled with objects, and the positions of the objects are recorded to obtain a training set.

[0042] Specifically, each cropped image segment is accurately annotated with the object, especially the position of the swimmer’s head and body, and the position of each cropped window relative to the original image is marked. This step creates a new and rich training set of small object instances.

[0043] Each cropped small image is carefully labeled, and the position information of each cropped window relative to the original image is recorded. This step ensures that all small object instances are correctly labeled, thus providing a high-quality data foundation for subsequent steps. After cropping and labeling, these data are integrated to form a new training set. This training set pays special attention to covering more small object instances, which enhances the model's ability to recognize small objects.

[0044] S124: Perform data enhancement on the training set to obtain a sample set.

[0045] In this embodiment, the sample set refers to a data set used to train and verify the performance of the trained model.

[0046] Data augmentation: In order to improve the generalization and robustness of the model, various methods are used to expand the training data. For example: Zoom and translate: Enlarge the image by a certain ratio (for example, 2 times) and re-mark the position of small objects.

[0047] Tiled stitching: Based on the actual layout of the swimming pool, multiple images containing small objects are randomly selected and stitched into a larger synthetic image.

[0048] Target insertion: Simulate the real motion trajectory, insert additional small targets into the new image, and update the corresponding annotation information.

[0049] Weighted annotation: Give higher weights to objects that meet the definition of small objects, especially at the head position, to emphasize their importance during training.

[0050] In one embodiment, see Figure 3 The above-mentioned step S124 may include steps S1241~S1244.

[0051] S1241. Identify and mark the head and body features of the swimmers in the training set, and adjust the zoom ratio to enlarge the area extracted from the training set to obtain an enlarged image.

[0052] In this embodiment, the enlarged image refers to the image in which the head and body features of the swimmer are identified and marked and then the proportions are adjusted again to enlarge the area where the head and body features are located. Combined with the characteristics of the swimming pool scene, the pre-trained swimming pool detection model is used to identify and focus on marking the head and body positions of the swimmer. The typical posture of the swimmer is used for feature extraction to improve the model's ability to understand specific postures.

[0053] Considering the small size of small targets in the original image, the bilinear interpolation algorithm is used to scale up the original image by a factor of 2 by default. The positions of small targets are re-marked in the enlarged image and used in the training phase to better capture the details of small targets and improve the detection rate. The following formula is used to implement this, where Represents the transformed horizontal axis coordinate, Represents the transformed vertical axis coordinate, Indicates the width of the converted target box. Indicates the height of the converted target frame. Indicates the image scaling factor. , x, y, w, and h correspond to the horizontal axis coordinate, vertical axis coordinate, width of the target frame, and height of the target frame before transformation, respectively.

[0054] S1242: Using a target tiling algorithm to stitch the enlarged images according to the distribution of targets in the actual scene, and updating corresponding annotation information to obtain a synthesis result.

[0055] In this embodiment, the synthesis result refers to a result of stitching together a plurality of images with small objects selected from the above-mentioned enlarged image.

[0056] Specifically, considering the actual layout of the swimming pool, such as the lane distribution and edge position, 4 images containing small targets are randomly selected from the existing training set, specifically the enlarged images, and spliced ​​into a new image according to the 2×2 layout. This method ensures the realism of the synthesized image and avoids unreasonable target arrangement. The updated annotation information ensures that the annotation box of the original image can be correctly mapped to the new image, increasing the diversity of training samples. The mapping formula is: ,in Represents the horizontal axis coordinate after mapping, represents the vertical axis after mapping, Indicates the horizontal axis coordinate before mapping, Represents the horizontal axis coordinate after mapping, , Respectively represent the offset of the horizontal axis coordinate and the vertical axis coordinate.

[0057] Based on the movement trajectory of swimmers and the layout of the pool, a target insertion algorithm is designed. The algorithm first crops out the small target object and then inserts it into the new background image while resizing it to match the proportions of the current image. Finally, the annotation information is updated to ensure that the information of the inserted target is accurately added to the annotation file of the new image.

[0058] S1243. Perform weighted processing on the small targets in the synthesis result to obtain a weighted processing result.

[0059] In this embodiment, a standard is set to define which objects are "small objects". For example, at a resolution of 1920x1080, objects with a width and height less than 40 pixels are considered small objects; or more generally, the small object threshold can be set to 5% of the width and height of the image resolution.

[0060] In order to make the model pay more attention to small targets, especially the swimmer's head, these targets are given higher weights. By adjusting the weight parameters in the loss function, the small target detection task occupies a larger proportion in the training process, thereby strengthening the model's attention to small targets and detection accuracy.

[0061] In summary, the above measures work together in the data preparation and model training process to improve the detection performance of small and medium-sized targets (such as distant or small swimmers) in the swimming venue monitoring system, ensuring that targets can be efficiently and accurately identified and tracked even in complex environments.

[0062] S1244. Filter out small target instances whose confidences meet the requirements from the weighted processing results using a pre-trained model to obtain a sample set.

[0063] In this embodiment, a pre-trained detection model is used to perform inference in a large amount of swimming pool monitoring data to filter out high-confidence results containing small objects. The filtered images are manually verified to ensure the accuracy of the annotations, thereby building a dedicated small object training set to further optimize the model performance.

[0064] S125. Build a pre-trained detection model and loss function.

[0065] In this embodiment, a deep learning architecture suitable for processing small targets is developed or selected as the basis of a pre-trained detection model, such as a deep learning network, and an appropriate loss function is designed or selected, which can identify and give more attention to small targets, and this is achieved by setting weight parameters.

[0066] S126: Use the sample set to train the pre-trained detection model, and determine the target detection model in combination with the loss function and the trained model.

[0067] In this embodiment, the sample set after data enhancement is used to train the pre-trained detection model constructed above. The model parameters are continuously optimized during the training process until satisfactory performance indicators are achieved. Ultimately, the fully trained model will have a stronger small target detection capability and can be used in actual application environments to accurately identify and track swimmers, especially those small targets that may be difficult to capture.

[0068] In summary, this series of steps aims to improve the detection accuracy and reliability of small targets (such as distant swimmers) in the swimming venue monitoring system through sophisticated data preprocessing, effective data enhancement strategies, and targeted model training.

[0069] S130, cropping and enlarging the ROI area, and inputting the processed ROI area into the target detection model again to perform target detection to obtain a detection result.

[0070] In this embodiment, the detection result refers to the area where the small target is located.

[0071] Specifically, first, the original 1920×1080 resolution image is segmented using a pre-made mask. This mask is used to accurately locate the interior of the swimming pool, thereby effectively filtering out the shore or other irrelevant background parts. This step ensures that subsequent processing only focuses on the activities in the swimming pool.

[0072] In the area determined by the above mask, the target detection model is used to identify and locate all small target regions of interest (ROI). For each detected ROI, a cropping operation is performed and these areas are moderately enlarged, by default to twice the size. This process not only helps to enhance the feature expression of small targets, but also reduces the impact of background information on the detection results.

[0073] The cropped and enlarged images are fed into a YOLOv5 detection model that has been pre-trained using a dataset tailored for private swimming pool scenarios. This scenario-specific pre-training improves the model’s generalization and accuracy in real-world applications, especially for small object recognition.

[0074] In order to ensure that the model can recognize global targets in a large range and accurately capture small local targets, the cropped image is combined with the original image. Specifically, the ROI that has been enlarged and further processed is remapped back to the original image position to form a new image with rich details. In addition, an effective algorithm is used to deduplicate the results of multi-target detection to avoid repeated counting of the same target and ensure the accuracy of the detection results.

[0075] S140: Map the detection result back to the coordinate system corresponding to the initial image to obtain a mapping result.

[0076] In this embodiment, the mapping result refers to the result obtained by mapping all the coordinates of the detection results to the coordinate system of the original initial image.

[0077] Finally, for all the detection results obtained in the cropped image, their coordinate positions are accurately mapped back to the corresponding positions in the original image. According to the coordinate mapping formula mentioned above, the conversion from the cropped image to the original image coordinates can be achieved. This step is crucial because it ensures that the final output detection box can correctly reflect the true position of the target in the original scene.

[0078] In summary, through effective preprocessing of swimming pool scene images, targeted target detection model training and reasonable image fusion strategy, the accuracy and efficiency of small target detection are significantly improved, providing strong technical support for swimming pool safety management.

[0079] S150: Using multi-threshold detection and compensation tracking on the mapping result to obtain a tracking result.

[0080] In this embodiment, the tracking result refers to the tracking trajectory of the small target, etc.

[0081] In one embodiment, see Figure 4 , the above-mentioned step S150 may include steps S151~S152.

[0082] S151. Set a first threshold and a second threshold, wherein the first threshold is greater than the second threshold, the first threshold is used for multi-target tracking, and the second threshold is used to compensate for lost target frames.

[0083] In this embodiment, first, two thresholds of different levels are set for detection, including: a first threshold (high threshold) and a second threshold (low threshold). The high threshold is used to ensure high-quality target detection results and is suitable for multi-target tracking; while the low threshold relaxes the detection standard to capture more small targets or blurred targets that may be ignored, which serves as the basis for compensation tracking.

[0084] S152, performing multi-target tracking on the results greater than the first threshold in the mapping results, and when the tracking is currently lost in the current frame, starting a compensation mechanism, using IoU matching and the Hungarian algorithm to find a matching target frame from the results below the second threshold in the multi-target tracking results to compensate for the lost human body tracking frame to obtain a tracking result.

[0085] In this embodiment, then, for all detection results above the first threshold, an advanced multi-target tracking algorithm (such as ByteTrack) is used to track and generate a list of human tracking frames containing tracking IDs. Whenever a swimmer's tracking ID disappears in the current frame, a compensation mechanism is activated, that is, a matching candidate human tracking frame is found from the results below the second threshold to compensate for the lost human tracking frame.

[0086] In one embodiment, see Figure 5 , the above-mentioned step S152 may include steps S1521~S1522.

[0087] S1521. Perform multi-target tracking on the results in the mapping results that are greater than the first threshold using a multi-target tracking algorithm to obtain a multi-target tracking result, wherein the multi-target tracking result includes a list of human body tracking frames of tracking IDs.

[0088] In this embodiment, the multi-target tracking result includes the tracking ID of each swimmer and its corresponding human tracking frame list. These human tracking frames represent the position information of each successfully tracked target in the current frame.

[0089] S1522. Check the status of each tracked object frame by frame. When it is found that the tracking ID of a swimmer disappears in the current frame and the swimmer has not swam out of the range, use IoU matching and the Hungarian algorithm to find a matching target frame from the results below the second threshold in the multi-target tracking results to compensate for the lost human tracking frame to obtain the tracking result.

[0090] In this embodiment, a multi-target tracking algorithm is used to process all detection results exceeding the first threshold value to obtain a multi-target tracking result including the tracking ID of each swimmer. The state of each tracked object in each frame image is checked. If it is found that the tracking ID of a swimmer disappears in the current frame, and it is judged that the swimmer has not swam out of the camera coverage according to the mask corresponding to the preset swimming pool range, it is considered that tracking failure has occurred, and the compensation mechanism needs to be activated at this time.

[0091] Specifically, a high threshold is set to filter out more reliable detection results, and these results are passed to the multi-target tracking module based on the ByteTrack algorithm. ByteTrack is an efficient online multi-target tracking algorithm that can handle target association problems in real-time video streams, thereby generating a stable and accurate list of human tracking boxes.

[0092] In one embodiment, see Figure 6 The above-mentioned step S1522 may include steps S15221~S15224.

[0093] S15221. Check the status of each tracked object frame by frame. When it is found that the tracking ID of a swimmer disappears in the current frame and the swimmer has not swam out of the range, compare the corresponding human tracking frame in the multi-target tracking result corresponding to the disappeared human tracking frame with all human tracking frames in the current frame that are lower than the second threshold, and calculate the intersection over union ratio between the two human tracking frames to obtain the IoU matrix.

[0094] In this embodiment, if Figure 7 As shown in the figure, for each frame, the algorithm checks whether the swimmer's tracking ID that existed in the previous frame still exists in the current frame. If a swimmer's tracking ID disappears, but the swimmer has not swum out of the camera coverage according to the mask corresponding to the preset pool range and the position information of the previous and next frames, it is inferred that the tracking has failed. This failure usually occurs when a small target at a long distance is difficult to track accurately, such as due to environmental factors such as reflections or water splashes.

[0095] Once tracking failure is confirmed, the compensation tracking mechanism is started. At this time, the IoU (intersection over union) of the stable human tracking frame previously generated by the high threshold corresponding to the disappeared human tracking frame is calculated with all human tracking frames below the low threshold in the current frame to construct an IoU matrix. IoU is a standard measure of the degree of overlap between two bounding boxes, which helps to identify potential best matching candidate frames.

[0096] Specifically, the disappeared human tracking frame corresponds to the stable human tracking frame previously generated by the high threshold Compare with all human detection frames below the low threshold in the current frame The incoming detection compensation tracking algorithm module first compensates the human frame that failed to be tracked. Process it and compare it with all human detection frames below the low threshold in the current frame ( Figure 7 middle to ) Perform IoU-based matching to obtain the IoU matrix The specific calculation formula is ,in The result representing the iou size between the high threshold human tracking frame and each low threshold human detection frame, Represents the iou size between the high threshold human tracking frame and each low threshold human detection frame, represents the area of ​​the high threshold human tracking box, Represents the area of ​​each low threshold human detection box.

[0097] S15222. Construct a cost matrix according to the IoU matrix.

[0098] In this embodiment, the IoU matrix is ​​converted into a cost matrix. In this process, a higher IoU value indicates a lower cost (i.e., a better match). This step is to prepare for the subsequent application of the Hungarian algorithm. The specific conversion formula is: ,in Indicates that the human tracking frame is lost. Represents a low-threshold human detection box.

[0099] S15223. Apply the Hungarian algorithm to the cost matrix to match all human tracking frames below the second threshold and disappeared human tracking frames in the current frame to obtain a matching result.

[0100] In this embodiment, the matching result refers to the human body tracking frame that matches the disappeared human body tracking frame among all human body tracking frames whose values ​​are lower than the second threshold in the current frame.

[0101] Specifically, the Hungarian algorithm is used to perform the optimal matching search on the cost matrix. The algorithm is designed to solve the assignment problem on the bipartite graph, and its goal is to minimize the total cost. Therefore, the Hungarian algorithm can find the best matching solution that minimizes the total cost, that is, find a low-threshold detection box that is most likely to correspond to the missing human box. .

[0102] S15224, adding the matching result to the multi-target tracking result of the current frame (represented as a set in the figure) to ) to obtain the tracking results.

[0103] In this embodiment, finally, the best matching relationship found by the Hungarian algorithm is applied to the multi-target tracking result of the current frame to supplement the tracking frames lost due to various reasons. This step not only restores continuity, but also improves the robustness and accuracy of the entire tracking system, especially for small targets that are prone to tracking loss.

[0104] In summary, the multi-threshold detection and compensation tracking algorithm effectively solves the problem of small target tracking in complex environments by introducing a high and low dual threshold strategy, combining advanced multi-target tracking technology and an effective compensation mechanism, and provides a strong technical guarantee for monitoring in specific scenarios such as swimming pools.

[0105] The human tracking frame mentioned above is actually a target detection frame.

[0106] S160: Output the tracking result.

[0107] The tracking result is output to the terminal for display.

[0108] The method of this embodiment makes full use of cutting-edge technologies in the field of computer vision and image processing, aiming to solve the detection and tracking problems of small targets (such as swimmers) in swimming pools caused by blurring caused by distance, water surface reflections, and splashes. Through the application of this innovative solution, the recall rate for small target detection is significantly improved, and the common ID switching (id_switch) phenomenon in the tracking process is effectively reduced.

[0109] This method enhances the recognition of small targets, ensuring that the location information of swimmers can be accurately captured even in adverse conditions. The optimized algorithm can more stably maintain continuous tracking of each individual, avoiding misidentification caused by environmental interference or target occlusion. It provides reliable data input for the anti-drowning intelligent system, thereby assisting in making more accurate safety judgments and emergency responses.

[0110] It can be used in the construction of smart swimming pools. As part of the intelligent infrastructure, this technology helps to build a more efficient and safe service system and promote the digital transformation of swimming pools. Through effective monitoring of swimmers' activities, managers can promptly identify potential risks and take preventive measures to reduce the probability of drowning accidents. Stable and reliable detection and tracking not only ensures safety, but also provides users with a more comfortable and secure sports environment.

[0111] In summary, the multi-strategy fusion small target detection and tracking enhancement method plays an irreplaceable role in promoting the modernization of swimming venues and improving the level of public safety protection. It not only improves the operational efficiency and service quality of facilities, but also brings users a higher sense of security and satisfaction. It effectively improves the detection accuracy and tracking stability of small targets. This method can accurately identify the position of swimmers in complex swimming pool environments and continuously track their movement trajectories, providing technical support for safety monitoring, intelligent management and drowning risk analysis of swimming venues.

[0112] The above-mentioned multi-strategy fusion small target detection and tracking enhancement method for swimming venues obtains the initial image and inputs it into the target detection model to identify potential small target areas. By cropping and enhancing the training data set, the sensitivity and accuracy of the target detection model for small targets are improved. The detected ROI area is cropped and enlarged to further enhance the detailed features of the small target and improve the accuracy of subsequent detection. The detection results are mapped back to the coordinate system of the initial image to ensure the accuracy of the target position. Combined with multi-threshold detection and compensation tracking strategies, the target is stably tracked to reduce misidentification and loss, thereby improving the stability and accuracy of the overall tracking.

[0113] Figure 8 FIG. 3 is a schematic block diagram of a multi-strategy fusion swimming pool small target detection and tracking enhancement system 300 provided by an embodiment of the present invention. Figure 8 As shown, corresponding to the above multi-strategy fusion swimming pool small target detection and tracking enhancement method, the present invention also provides a multi-strategy fusion swimming pool small target detection and tracking enhancement system 300. The multi-strategy fusion swimming pool small target detection and tracking enhancement system 300 includes a unit for executing the above multi-strategy fusion swimming pool small target detection and tracking enhancement method, and the system can be configured in a server. Specifically, please refer to Figure 8 The multi-strategy fusion swimming venue small target detection, tracking and enhancement system 300 includes an image acquisition unit 301, a target detection unit 302, a re-detection unit 303, a mapping unit 304, a tracking unit 305 and an output unit 306.

[0114] The image acquisition unit 301 is used to acquire the image to be detected and tracked to obtain the initial image; the target detection unit 302 is used to input the initial image into the target detection model for target detection to obtain the ROI area; wherein the target detection model is obtained by acquiring the image with the target in the swimming pool, cropping it, and data enhancing it as a sample set to train the pre-trained detection model; the re-detection unit 303 is used to crop and enlarge the ROI area, and input the processed ROI area into the target detection model again for target detection to obtain the detection result; the mapping unit 304 is used to map the detection result back to the coordinate system corresponding to the initial image to obtain the mapping result; the tracking unit 305 is used to use multi-threshold detection and compensation tracking on the mapping result to obtain the tracking result; the output unit 306 is used to output the tracking result.

[0115] In one embodiment, the multi-strategy fusion swimming pool small target detection and tracking enhancement system 300 further includes a training unit for: Acquire an image with a target in a swimming pool; crop the image using a dynamic sliding window to obtain a cropped result; label the cropped result to generate a training set; perform data enhancement on the training set to obtain a sample set; construct a pre-trained detection model and a loss function; use the sample set to train the pre-trained detection model, and determine the target detection model in combination with the loss function and the trained model.

[0116] In one embodiment, the training unit is further used to: crop the image according to the lane area and the non-critical area using different sliding windows to obtain a cropping result.

[0117] In one embodiment, the training unit is further used to: perform target labeling on the cropping result and record the position of the target to obtain a training set.

[0118] In one embodiment, the training unit is further used to: identify and annotate the head and body features of the swimmers in the training set, and adjust the scaling ratio to enlarge the area extracted from the training set to obtain an enlarged image; use a target tiling algorithm to splice the enlarged image according to the distribution of targets in the actual scene, and update the corresponding annotation information to obtain a synthetic result; perform weighted processing on the small targets in the synthetic result to obtain a weighted processing result; and use a pre-trained model to filter out small target instances whose confidence meets the requirements from the weighted processing result to obtain a sample set.

[0119] In one embodiment, the tracking unit 305 includes a setting subunit and a compensation subunit.

[0120] A setting subunit is used to set a first threshold and a second threshold, wherein the first threshold is greater than the second threshold, the first threshold is used for multi-target tracking; the second threshold is used to compensate for the lost target frame; a compensation subunit is used to perform multi-target tracking on the results greater than the first threshold in the mapping results, and when the tracking is currently lost in the current frame, a compensation mechanism is started, and the IoU matching and Hungarian algorithm are used to find a matching target frame from the results below the second threshold in the multi-target tracking results to compensate for the lost human tracking frame, so as to obtain a tracking result.

[0121] In one embodiment, the compensation subunit comprises: A multi-target tracking module is used to perform multi-target tracking on the results greater than the first threshold in the mapping results using a multi-target tracking algorithm to obtain a multi-target tracking result, wherein the multi-target tracking result includes a list of human tracking frames of the tracking ID; a compensation tracking module is used to check the state of each tracked object frame by frame, and when it is found that the tracking ID of a swimmer disappears in the current frame and the swimmer has not swam out of the range, use IoU matching and Hungarian algorithm to find a matching target frame from the results below the second threshold in the multi-target tracking results to compensate for the lost human tracking frame to obtain a tracking result.

[0122] In one embodiment, the compensation tracking module includes: The intersection-over-union (IoU) calculation submodule is used to check the status of each tracked object frame by frame. When it is found that the tracking ID of a swimmer disappears in the current frame and the swimmer has not swam out of the range, the human tracking frame corresponding to the disappeared human tracking frame in the multi-target tracking result is compared with all human tracking frames in the current frame that are lower than the second threshold, and the intersection-over-union (IoU) between the two human tracking frames is calculated to obtain an IoU matrix; the cost matrix construction submodule is used to construct a cost matrix according to the IoU matrix; the matching submodule is used to apply the Hungarian algorithm to the cost matrix to match all human tracking frames in the current frame that are lower than the second threshold and the disappeared human tracking frame to obtain a matching result; the matching addition submodule is used to add the matching result to the multi-target tracking result of the current frame to obtain a tracking result.

[0123] It should be noted that technical personnel in the relevant field can clearly understand that the specific implementation process of the above-mentioned multi-strategy fusion swimming venue small target detection and tracking enhancement system 300 and each unit can refer to the corresponding description in the aforementioned method embodiment, and for the convenience and conciseness of the description, it will not be repeated here.

[0124] The above-mentioned multi-strategy fusion swimming pool small target detection and tracking enhancement system 300 can be implemented in the form of a computer program. The computer program can be used in Fig. 9Runs on the computer device shown.

[0125] See also Fig. 9 , Fig. 9 5 is a schematic block diagram of a computer device provided in an embodiment of the present application. The computer device 500 may be a server, wherein the server may be an independent server or a server cluster composed of multiple servers.

[0126] See also Fig. 9 The computer device 500 includes a processor 502 , a memory and a network interface 505 connected via a system bus 501 , wherein the memory may include a non-volatile storage medium 503 and an internal memory 504 .

[0127] The non-volatile storage medium 503 can store an operating system 5031 and a computer program 5032. The computer program 5032 includes program instructions, and when the program instructions are executed, the processor 502 can execute a multi-strategy fusion swimming venue small target detection and tracking enhancement method.

[0128] The processor 502 is used to provide computing and control capabilities to support the operation of the entire computer device 500 .

[0129] The internal memory 504 provides an environment for the operation of the computer program 5032 in the non-volatile storage medium 503. When the computer program 5032 is executed by the processor 502, the processor 502 can execute a multi-strategy fusion swimming venue small target detection and tracking enhancement method.

[0130] The network interface 505 is used to communicate with other devices over the network. Fig. 9 The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present application, and does not constitute a limitation on the computer device 500 to which the solution of the present application is applied. The specific computer device 500 may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0131] The processor 502 is used to run the computer program 5032 stored in the memory to implement the following steps: Acquire an image to be detected and tracked to obtain an initial image; input the initial image into a target detection model for target detection to obtain a ROI area; wherein the target detection model is obtained by acquiring an image with a target in a swimming pool, cropping it, and performing data enhancement to train a pre-trained detection model as a sample set; cropping and enlarging the ROI area, and inputting the processed ROI area into the target detection model again for target detection to obtain a detection result; mapping the detection result back to the coordinate system corresponding to the initial image to obtain a mapping result; applying multi-threshold detection and compensation tracking to the mapping result to obtain a tracking result; and outputting the tracking result.

[0132] In one embodiment, when the processor 502 implements the step of obtaining the target detection model by cropping and data enhancing images with targets in the swimming pool as a sample set to train the pre-trained detection model, the processor 502 specifically implements the following steps: Acquire an image with a target in a swimming pool; crop the image using a dynamic sliding window to obtain a cropped result; label the cropped result to generate a training set; perform data enhancement on the training set to obtain a sample set; construct a pre-trained detection model and a loss function; use the sample set to train the pre-trained detection model, and determine the target detection model in combination with the loss function and the trained model.

[0133] In one embodiment, when the processor 502 implements the step of cropping the image using a dynamic sliding window to obtain a cropping result, the processor 502 specifically implements the following steps: The image is cropped using different sliding windows according to the lane area and the non-critical area to obtain a cropping result.

[0134] In one embodiment, when the processor 502 implements the step of labeling the cropping result to generate a training set, the processor 502 specifically implements the following steps: The cropping results are labeled with objects, and the positions of the objects are recorded to obtain a training set.

[0135] In one embodiment, when the processor 502 implements the step of performing data enhancement on the training set to obtain a sample set, the processor 502 specifically implements the following steps: The head and body features of the swimmers in the training set are identified and labeled, and the scaling ratio is adjusted to enlarge the area extracted from the training set to obtain an enlarged image; the enlarged image is spliced ​​according to the distribution of targets in the actual scene using a target tiling algorithm, and the corresponding labeling information is updated to obtain a synthesis result; the small targets in the synthesis result are weighted to obtain a weighted processing result; and the pre-trained model is used to filter out small target instances with confidence levels that meet the requirements from the weighted processing results to obtain a sample set.

[0136] In one embodiment, when the processor 502 implements the step of using multi-threshold detection and compensation tracking on the mapping result to obtain the tracking result, the processor 502 specifically implements the following steps: A first threshold and a second threshold are set, wherein the first threshold is greater than the second threshold, the first threshold is used for multi-target tracking, and the second threshold is used to compensate for the lost target frame; multi-target tracking is performed on the results greater than the first threshold in the mapping results, and when the tracking is currently lost in the current frame, the compensation mechanism is started, and the IoU matching and the Hungarian algorithm are used to find a matching target frame from the results lower than the second threshold in the multi-target tracking results to compensate for the lost human tracking frame, so as to obtain a tracking result.

[0137] In one embodiment, the processor 502 implements the step of performing multi-target tracking on the results greater than the first threshold in the mapping results, and when the tracking is currently lost in the current frame, starting the compensation mechanism, using IoU matching and the Hungarian algorithm to find a matching target frame from the results lower than the second threshold in the multi-target tracking results to compensate for the lost human body tracking frame to obtain the tracking result, specifically implementing the following steps: A multi-target tracking algorithm is used to perform multi-target tracking on the results greater than the first threshold in the mapping results to obtain a multi-target tracking result, wherein the multi-target tracking result includes a list of human tracking frames of the tracking ID; the state of each tracked object is checked frame by frame, and when it is found that the tracking ID of a swimmer disappears in the current frame and the swimmer has not swam out of the range, the IoU matching and the Hungarian algorithm are used to find a matching target frame from the results less than the second threshold in the multi-target tracking results to compensate for the lost human tracking frame to obtain the tracking result.

[0138] In one embodiment, when the processor 502 implements the step of checking the state of each tracked object frame by frame, and when it is found that the tracking ID of a swimmer disappears in the current frame and the swimmer has not swam out of the range, the processor 502 uses IoU matching and the Hungarian algorithm to find a matching target frame from the results below the second threshold in the multi-target tracking results to compensate for the lost human tracking frame to obtain the tracking result, the following steps are specifically implemented: The state of each tracked object is checked frame by frame. When it is found that the tracking ID of a swimmer disappears in the current frame and the swimmer has not swam out of the range, the corresponding human tracking frame in the multi-target tracking result corresponding to the disappeared human tracking frame is compared with all human tracking frames below the second threshold in the current frame, and the intersection over union ratio between the two human tracking frames is calculated to obtain an IoU matrix; a cost matrix is ​​constructed according to the IoU matrix; the Hungarian algorithm is applied to the cost matrix to match all human tracking frames below the second threshold in the current frame and the disappeared human tracking frame to obtain a matching result; the matching result is added to the multi-target tracking result of the current frame to obtain a tracking result.

[0139] It should be understood that in the embodiment of the present application, the processor 502 may be a central processing unit (CPU), and the processor 502 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0140] It can be understood by those skilled in the art that all or part of the processes in the method for implementing the above embodiment can be completed by instructing the relevant hardware through a computer program. The computer program includes program instructions, and the computer program can be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in the computer system to implement the process steps of the embodiment of the above method.

[0141] Therefore, the present invention also provides a storage medium. The storage medium may be a computer-readable storage medium. The storage medium stores a computer program, wherein when the computer program is executed by a processor, the processor executes the following steps: Acquire an image to be detected and tracked to obtain an initial image; input the initial image into a target detection model for target detection to obtain a ROI area; wherein the target detection model is obtained by acquiring an image with a target in a swimming pool, cropping it, and performing data enhancement to train a pre-trained detection model as a sample set; cropping and enlarging the ROI area, and inputting the processed ROI area into the target detection model again for target detection to obtain a detection result; mapping the detection result back to the coordinate system corresponding to the initial image to obtain a mapping result; applying multi-threshold detection and compensation tracking to the mapping result to obtain a tracking result; and outputting the tracking result.

[0142] In one embodiment, when the processor executes the computer program to implement the target detection model by obtaining images with targets in a swimming pool, cropping them, and performing data enhancement as sample sets to train a pre-trained detection model, the processor specifically implements the following steps: Acquire an image with a target in a swimming pool; crop the image using a dynamic sliding window to obtain a cropped result; label the cropped result to generate a training set; perform data enhancement on the training set to obtain a sample set; construct a pre-trained detection model and a loss function; use the sample set to train the pre-trained detection model, and determine the target detection model in combination with the loss function and the trained model.

[0143] In one embodiment, when the processor executes the computer program to implement the step of cropping the image using a dynamic sliding window to obtain a cropping result, the processor specifically implements the following steps: The image is cropped using different sliding windows according to the lane area and the non-critical area to obtain a cropping result.

[0144] In one embodiment, when the processor executes the computer program to implement the step of labeling the cropping results to generate a training set, the processor specifically implements the following steps: The cropping results are labeled with objects, and the positions of the objects are recorded to obtain a training set.

[0145] In one embodiment, when the processor executes the computer program to implement the step of performing data enhancement on the training set to obtain a sample set, the processor specifically implements the following steps: The head and body features of the swimmers in the training set are identified and labeled, and the scaling ratio is adjusted to enlarge the area extracted from the training set to obtain an enlarged image; the enlarged image is spliced ​​according to the distribution of targets in the actual scene using a target tiling algorithm, and the corresponding labeling information is updated to obtain a synthesis result; the small targets in the synthesis result are weighted to obtain a weighted processing result; and the pre-trained model is used to filter out small target instances with confidence levels that meet the requirements from the weighted processing results to obtain a sample set.

[0146] In one embodiment, when the processor executes the computer program to implement the step of using multi-threshold detection and compensation tracking on the mapping result to obtain the tracking result, the processor specifically implements the following steps: A first threshold and a second threshold are set, wherein the first threshold is greater than the second threshold, the first threshold is used for multi-target tracking, and the second threshold is used to compensate for the lost target frame; multi-target tracking is performed on the results greater than the first threshold in the mapping results, and when the tracking is currently lost in the current frame, the compensation mechanism is started, and the IoU matching and the Hungarian algorithm are used to find a matching target frame from the results lower than the second threshold in the multi-target tracking results to compensate for the lost human tracking frame, so as to obtain a tracking result.

[0147] In one embodiment, when the processor executes the computer program to implement the step of performing multi-target tracking on the results greater than the first threshold in the mapping results, and when the tracking is currently lost in the current frame, starting the compensation mechanism, using IoU matching and the Hungarian algorithm to find a matching target frame from the results below the second threshold in the multi-target tracking results to compensate for the lost human body tracking frame to obtain the tracking result, the following steps are specifically implemented: A multi-target tracking algorithm is used to perform multi-target tracking on the results greater than the first threshold in the mapping results to obtain a multi-target tracking result, wherein the multi-target tracking result includes a list of human tracking frames of the tracking ID; the state of each tracked object is checked frame by frame, and when it is found that the tracking ID of a swimmer disappears in the current frame and the swimmer has not swam out of the range, the IoU matching and the Hungarian algorithm are used to find a matching target frame from the results less than the second threshold in the multi-target tracking results to compensate for the lost human tracking frame to obtain the tracking result.

[0148] In one embodiment, the processor executes the computer program to implement the step of checking the state of each tracked object frame by frame, and when it is found that the tracking ID of a swimmer disappears in the current frame and the swimmer has not swam out of the range, uses IoU matching and the Hungarian algorithm to find a matching target frame from the results below the second threshold in the multi-target tracking results to compensate for the lost human tracking frame to obtain the tracking result, specifically implementing the following steps: The state of each tracked object is checked frame by frame. When it is found that the tracking ID of a swimmer disappears in the current frame and the swimmer has not swam out of the range, the corresponding human tracking frame in the multi-target tracking result corresponding to the disappeared human tracking frame is compared with all human tracking frames below the second threshold in the current frame, and the intersection over union ratio between the two human tracking frames is calculated to obtain an IoU matrix; a cost matrix is ​​constructed according to the IoU matrix; the Hungarian algorithm is applied to the cost matrix to match all human tracking frames below the second threshold in the current frame and the disappeared human tracking frame to obtain a matching result; the matching result is added to the multi-target tracking result of the current frame to obtain a tracking result.

[0149] The storage medium may be a USB flash drive, a mobile hard disk, a read-only memory (ROM), a magnetic disk, or an optical disk, etc., which are computer-readable storage media that can store program codes.

[0150] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0151] In the several embodiments provided by the present invention, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are only schematic. For example, the division of each unit is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0152] The steps in the method of the embodiment of the present invention can be adjusted in order, combined and deleted according to actual needs. The units in the system of the embodiment of the present invention can be combined, divided and deleted according to actual needs. In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0153] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, terminal, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention.

[0154] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A multi-strategy fusion method for small target detection and tracking enhancement in swimming pools, characterized in that: include: Acquire the image to be detected and tracked to obtain an initial image; Input the initial image into the target detection model for target detection to obtain the ROI area; wherein the target detection model is obtained by obtaining images with targets in the swimming pool, cropping them, and performing data enhancement to train the pre-trained detection model as a sample set; The ROI region is cropped and enlarged, and the processed ROI region is input into the target detection model again for target detection to obtain a detection result; Mapping the detection result back to the coordinate system corresponding to the initial image to obtain a mapping result; Using multi-threshold detection and compensation tracking on the mapping result to obtain a tracking result; The tracking result is outputted.

2. According to the multi-strategy fusion swimming pool small target detection and tracking enhancement method of claim 1, it is characterized in that: The target detection model is obtained by obtaining images with targets in the swimming pool, cropping them, and performing data enhancement as a sample set to train a pre-trained detection model, including: Acquire images with targets in the swimming pool; Cropping the image using a dynamic sliding window to obtain a cropping result; Labeling the cropping results to generate a training set; Performing data enhancement on the training set to obtain a sample set; Build a pre-trained detection model and loss function; The pre-trained detection model is trained using the sample set, and the target detection model is determined by combining the loss function and the trained model.

3. The multi-strategy fusion swimming pool small target detection and tracking enhancement method according to claim 2 is characterized in that: The step of cropping the image using a dynamic sliding window to obtain a cropping result includes: The image is cropped using different sliding windows according to the lane area and the non-critical area to obtain a cropping result.

4. The multi-strategy fusion swimming pool small target detection and tracking enhancement method according to claim 2 is characterized in that: The step of labeling the cropping results to generate a training set includes: The cropping results are labeled with objects, and the positions of the objects are recorded to obtain a training set.

5. The multi-strategy fusion swimming pool small target detection and tracking enhancement method according to claim 2 is characterized in that: The data enhancement of the training set to obtain a sample set includes: Identifying and labeling the head and body features of the swimmers in the training set, and adjusting the scaling to magnify the region extracted from the training set to obtain an enlarged image; Using a target tiling algorithm to stitch the enlarged images according to the distribution of targets in the actual scene, and updating corresponding annotation information to obtain a synthesis result; Performing weighted processing on the small targets in the synthesis result to obtain a weighted processing result; The pre-trained model is used to filter out small target instances whose confidence meets the requirements from the weighted processing results to obtain a sample set.

6. The multi-strategy fusion swimming pool small target detection and tracking enhancement method according to claim 1 is characterized in that: The mapping result is subjected to multi-threshold detection and compensation tracking to obtain a tracking result, including: Setting a first threshold and a second threshold, wherein the first threshold is greater than the second threshold, the first threshold is used for multi-target tracking; and the second threshold is used to compensate for lost target frames; Multi-target tracking is performed on the results in the mapping results that are greater than the first threshold, and when the tracking is currently lost in the current frame, a compensation mechanism is started, and a matching target frame is found from the results in the multi-target tracking results that are lower than the second threshold by using IoU matching and the Hungarian algorithm to compensate for the lost human tracking frame to obtain a tracking result.

7. The multi-strategy fusion swimming pool small target detection and tracking enhancement method according to claim 6 is characterized in that: The multi-target tracking is performed on the results greater than the first threshold in the mapping results, and when the tracking is currently lost in the current frame, a compensation mechanism is started to use IoU matching and the Hungarian algorithm to find a matching target frame from the results less than the second threshold in the multi-target tracking results to compensate for the lost human tracking frame to obtain a tracking result, including: Performing multi-target tracking on the results greater than the first threshold in the mapping results using a multi-target tracking algorithm to obtain a multi-target tracking result, wherein the multi-target tracking result includes a list of human body tracking frames of tracking IDs; The status of each tracked object is checked frame by frame. When it is found that the tracking ID of a swimmer disappears in the current frame and a swimmer has not swam out of the range, the IoU matching and Hungarian algorithm are used to find the matching target frame from the results below the second threshold in the multi-target tracking results to compensate for the lost human tracking frame and obtain the tracking result.

8. The multi-strategy fusion swimming pool small target detection and tracking enhancement method according to claim 7 is characterized in that: The state of each tracked object is checked frame by frame. When it is found that the tracking ID of a swimmer disappears in the current frame and the swimmer has not swam out of the range, the matching target frame is found from the results below the second threshold in the multi-target tracking results by using IoU matching and the Hungarian algorithm to compensate for the lost human tracking frame, so as to obtain the tracking result, including: Check the status of each tracked object frame by frame. When it is found that the tracking ID of a swimmer disappears in the current frame and the swimmer has not swam out of the range, compare the corresponding human tracking frame in the multi-target tracking result corresponding to the disappeared human tracking frame with all human tracking frames in the current frame that are lower than the second threshold, and calculate the intersection over union ratio between the two human tracking frames to obtain the IoU matrix. Construct a cost matrix based on the IoU matrix; Applying the Hungarian algorithm to the cost matrix to match all human tracking frames below the second threshold and disappeared human tracking frames in the current frame to obtain a matching result; The matching result is added to the multi-target tracking result of the current frame to obtain the tracking result.

9. The multi-strategy fusion small target detection and tracking enhancement system for swimming pools is characterized by: include: An image acquisition unit, used to acquire an image to be detected and tracked to obtain an initial image; A target detection unit, used for inputting the initial image into a target detection model for target detection to obtain a ROI area; wherein the target detection model is obtained by obtaining an image with a target in a swimming pool, cropping it, and performing data enhancement to train a pre-trained detection model as a sample set; A re-detection unit, used for cropping and enlarging the ROI area, and inputting the processed ROI area into the target detection model again for target detection to obtain a detection result; A mapping unit, used for mapping the detection result back to the coordinate system corresponding to the initial image to obtain a mapping result; A tracking unit, used for applying multi-threshold detection and compensation tracking to the mapping result to obtain a tracking result; An output unit is used to output the tracking result.

10. The multi-strategy fusion swimming pool small target detection and tracking enhancement system according to claim 9 is characterized in that: Also included are training units for: Acquire images with targets in the swimming pool; Cropping the image using a dynamic sliding window to obtain a cropping result; Labeling the cropping results to generate a training set; Performing data enhancement on the training set to obtain a sample set; Build a pre-trained detection model and loss function; The pre-trained detection model is trained using the sample set, and the target detection model is determined by combining the loss function and the trained model.

Images