A method, device, and storage medium for detecting a drowning person

Abstract

The application discloses a drowning person detection method and device, equipment and storage medium. Including: obtaining a sea surface image sample, establishing a target detection model according to the sea surface image sample, wherein the target detection model includes the corresponding relationship between the sea surface image sample and the human body related information; obtaining the to-be-detected image of the specified sea surface area, determining the human body related information of the to-be-detected image according to the target detection model; target tracking is performed on the human body related information to determine the drowning person detection result. The target detection model is established through the obtained sea surface image sample, then the to-be-detected image is input into the target detection model to determine the human body related information, finally the drowning person detection result can be determined through target tracking on the human body related information, the collection difficulty is reduced, the drowning person can be accurately identified without relying on the hardware sensor through the tracking track, the identification accuracy is high, the lifesaver can be prompted in time, and the safety of the swimming person is better ensured.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of machine vision detection, and in particular to a drowning person detection method, device, equipment and storage medium. BACKGROUND

[0002] With the increasing demand for leisure and entertainment, beaches have become a popular tourist and leisure destination, but drowning accidents in seawater have also increased, posing a serious threat to people's life safety. Although some preventive measures such as lifeguards and warning signs have been implemented, these measures are not always effective.

[0003] The prior art determines by installing hardware sensing devices in swimming pools, but the areas such as riversides and seashores have problems such as large area, low visibility in water, high tide and low tide, which make it difficult to install devices in the water bottom, making it difficult to collect and unable to improve the accuracy of drowning prevention supervision. SUMMARY

[0004] The present application provides a drowning person detection method, device, equipment and storage medium to detect drowning persons in sea surface areas.

[0005] According to an aspect of the present application, a drowning person detection method is provided, which comprises:

[0006] Obtaining a sea surface image sample, and establishing a target detection model according to the sea surface image sample, wherein the target detection model includes a corresponding relationship between the sea surface image sample and human body related information;

[0007] Obtaining a to-be-detected image of a specified sea surface area, and determining human body related information of the to-be-detected image according to the target detection model;

[0008] Target tracking of the human body related information to determine a drowning person detection result.

[0009] Optionally, obtaining a sea surface image sample comprises: collecting historical sea surface images, obtaining label boxes and labels of each historical sea surface image input by a user, wherein the labels include detection types and human body postures; cutting each historical sea surface image corresponding to each label box to generate each cut image; performing rotation, flipping and translation processing on each cut image to obtain each enhanced image; cutting each enhanced image according to a specified number and a specified mixed image proportion to generate sea surface image samples.

[0010] Optionally, the target detection model is established according to the sea surface image sample, and the method comprises the following steps: building a network structure of a target detection algorithm and a classification network, and determining initial model parameters of the network structure; determining final model parameters according to the sea surface image sample and the initial model parameters, and taking a network structure corresponding to the final model parameters as the target detection model, wherein the target detection model comprises a target frame detection model and a classification model.

[0011] Optionally, the final model parameters are determined according to the sea surface image sample and the initial model parameters, and the method comprises the following steps: inputting the sea surface image sample into the network structure to obtain an output sample label; determining a true label of the sea surface image sample, and determining an accuracy rate based on the sample label and the true label; determining whether the accuracy rate is less than a preset threshold, if yes, taking the initial model parameters as the final model parameters; otherwise, adjusting the initial model parameters based on the accuracy rate to obtain adjusted model parameters, and taking the adjusted model parameters as the final model parameters.

[0012] Optionally, the human body related information of the to-be-detected image is determined according to the target detection model, and the method comprises the following steps: inputting the to-be-detected image into the target frame detection model to determine each target detection frame in the to-be-detected image and a corresponding detection type, wherein the detection type comprises a human body, a human head and a floating object; inputting each target detection frame with the detection type of human body into the classification model to determine a human body posture corresponding to each human body, wherein the human body posture comprises a normal human body, a half human body and a human body in water; and taking the target detection frame, the detection type and the human body posture as the human body related information.

[0013] Optionally, the human body related information is subjected to target tracking to determine a drowning person detection result, and the method comprises the following steps: determining a moving speed corresponding to the target detection frame, and determining a tracking speed according to the moving speed; determining a tracking trajectory corresponding to the target detection frame based on the tracking speed by using a target tracking algorithm; obtaining a drowning detection rule, and determining whether the tracking trajectory conforms to the drowning detection rule, if yes, determining that the drowning person detection result is that there is a drowning person; otherwise, determining that the drowning person detection result is that there is no drowning person.

[0014] Optionally, after the human body related information is subjected to target tracking to determine the drowning person detection result, the method further comprises the following steps: when the drowning person detection result is that there is a drowning person, generating prompt information according to the human body related information; and alarming in a specified manner according to the prompt information.

[0015] According to another aspect of the present application, a drowning person detection device is provided, and the device comprises:

[0016] A target detection model establishing module is configured to obtain a sea surface image sample, and establish a target detection model according to the sea surface image sample, wherein the target detection model comprises a corresponding relationship between the sea surface image sample and human body related information.

[0017] The human body related information determination module is configured to acquire a to-be-tested image of a specified sea surface area, and determine human body related information of the to-be-tested image according to a target detection model.

[0018] The drowning person detection result determination module is configured to perform target tracking on the human body related information to determine a drowning person detection result.

[0019] According to another aspect of the present application, an electronic device is provided, which comprises:

[0020] at least one processor; and

[0021] a memory connected to the at least one processor in communication; wherein,

[0022] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform a drowning person detection method according to any one of the embodiments of the present application.

[0023] According to another aspect of the present application, a computer readable storage medium is provided, which stores computer instructions for enabling a processor to perform a drowning person detection method according to any one of the embodiments of the present application when executed.

[0024] The technical solution of the embodiment of the present application establishes a target detection model through the acquired sea surface image sample, then inputs the to-be-tested image into the target detection model to determine human body related information, and finally determines a drowning person detection result through target tracking on the human body related information, thereby reducing the collection difficulty, accurately identifying the drowning person through tracking trajectory without relying on a hardware sensor, having high identification accuracy, and being capable of timely prompting the lifesaver, and better ensuring the safety of the swimmer.

[0025] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present application, nor is it used to limit the scope of the present application. Other features of the present application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0026] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0027] Figure 1 is a flowchart of a drowning person detection method according to the first embodiment of the present application;

[0028] Figure 2 is a flow chart of another method for detecting a drowning person according to an embodiment of the present application;

[0029] Figure 3 is a flow chart of another method for detecting a drowning person according to an embodiment of the present application;

[0030] Figure 4 is a structural schematic diagram of a device for detecting a drowning person according to an embodiment of the present application;

[0031] Figure 5 is a structural schematic diagram of an electronic device for implementing a method for detecting a drowning person according to an embodiment of the present application. DETAILED DESCRIPTION

[0032] In order to make the personnel in the technical field better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all the other embodiments obtained by the personnel of ordinary skill in the art without creative labor should belong to the protection scope of the present application.

[0033] It should be noted that the terms "first", "second", and the like in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or a chronological sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0034] Embodiment one

[0035] Figure 1 A flow chart of a method for detecting a drowning person is provided for an embodiment of the present application. The present embodiment can be applicable to the case of detecting a drowning person in a sea area. The method can be executed by a drowning person detection device, which can be realized in the form of hardware and / or software, and can be configured in a computer. As shown in the figure, the method comprises: Figure 1

[0036] ​S110, acquire a sea surface image sample, and establish a target detection model according to the sea surface image sample, wherein the target detection model comprises a corresponding relationship between the sea surface image sample and human body related information.

[0037] The sea surface image sample refers to an image sample containing a sea surface region for model training, and the sea surface image sample can be divided into a training set and a test set, which are respectively used for training and testing of the target detection model. The target detection model refers to a model for detecting each swimmer in the sea surface region.

[0038] Figure 2 A flowchart of a drowning person detection method is provided for the first embodiment of the present application, and step S110 mainly comprises the following steps S111 to S113:

[0039] S111, acquire a sea surface image sample.

[0040] Optionally, the sea surface image sample is acquired, including: collecting historical sea surface images, acquiring the label frame and label of each historical sea surface image input by a user, wherein the label includes a detection type and a human body posture; cutting each historical sea surface image based on each label frame to generate each cut image; performing rotation, flipping and translation processing on each cut image to obtain each enhanced image; cutting each enhanced image according to a specified number and a specified mixed image proportion to generate the sea surface image sample.

[0041] Specifically, the present scheme includes a plurality of cameras arranged on the beach, and the coordinates of the coastline can be obtained using an online map service, and then the world coordinates are converted into camera coordinates, and the sea surface region is divided in the plurality of cameras according to the camera coordinates, so as to obtain the historical sea surface images collected by the cameras. It should be noted that the historical sea surface images can be visible light images, and since the visible light images collected by the cameras in a relatively specific area will result in unclear images, the present scheme further provides a thermal imaging camera to ensure the clarity of the images. The user can label the historical sea surface images, label all human bodies, heads and floating areas in the images in the form of a bounding box, i.e. a label frame. Then the controller can cut the human body, head and floating area regions corresponding to each label frame based on each label frame, as a data set for target classification, and set a label for the historical sea surface images, and the label includes a detection type, and the detection type includes a human body, a head and a float, and the human body detection type is further subdivided into a normal human body, a half-body human body and a water human body.

[0042] It should be noted that the normal human body refers to a human body with more than half of the body exposed to the sea surface region, which has no risk of drowning at the moment), the half-body human body refers to a human body with only the upper body exposed to the water surface, which has a potential risk of drowning, and the water human body refers to a swimmer in the water.

[0043] Further, in view of the problems of too many swimmers and too small targets on the beach, the embodiment uses a data enhancement technique to generate more training data, that is, each cropped image is rotated, flipped and translated to obtain each enhanced image, thereby increasing the number of small targets in the training data and achieving the effect of data augmentation. In addition, the seaside camera is usually erected high and has a wide field of view. In order to further increase the detection accuracy, the large picture can be cut into small pictures according to a specified number, but direct image cutting may cut off the target, so it is necessary to first set the "fusion" area, that is, the proportion of the mixed picture. For example, the specified number can be 4 and the proportion of the mixed picture can be 0.1. At this time, the controller will cut the enhanced image according to 2*2 and the proportion of the mixed picture 0.1. It should be noted that after cutting into small pictures, the label is also processed. The label file is read to judge the region of the annotation box, the coordinates corresponding to the small picture are converted and labeled to generate the sea surface image sample.

[0044] S112, build a network structure of a target detection algorithm and a classification network, and determine initial model parameters of the network structure.

[0045] The network structure of the target detection algorithm refers to a yolov5 algorithm, a network input size, and an adaptive function affecting the convergence speed of the algorithm. For example, the network input size can be set to 640*640, and the adaptive function can be defined as: mAP_0.5 accounts for 10% of the weight; mAP_0.5:0.95 accounts for 80% of the weight, and Precision and Recall each account for 5%. The network structure of the classification network refers to a convolution+full connection structure. Specifically, a repvgg network model is used. Repvgg is a lightweight model that can combine convolution operation and ReLU activation function together to reduce computational complexity, and can be quickly deployed and run on mobile devices with high computing accuracy.

[0046] S113, determine final model parameters according to the sea surface image sample and the initial model parameters, and use the network structure corresponding to the final model parameters as a target detection model, wherein the target detection model includes a target frame detection model and a classification model.

[0047] Optionally, determining the final model parameters according to the sea surface image sample and the initial model parameters includes: inputting the sea surface image sample into the network structure to obtain an output sample label; determining a true label of the sea surface image sample, and determining an accuracy rate based on the sample label and the true label; determining whether the accuracy rate is less than a preset threshold, if yes, using the initial model parameters as the final model parameters; otherwise, adjusting the initial model parameters based on the accuracy rate to obtain an adjusted model parameter, and using the adjusted model parameter as the final model parameter.

[0048] Specifically, the sea surface image sample can be input into the network structure to obtain an output sample label, and the real label of the sea surface image sample is determined, and then whether the sample label and the real label are consistent is further determined to determine the accuracy of the model. When the accuracy is less than a preset threshold, it indicates that the model parameters meet the user's demand, at this time, the initial model parameters can be directly used as the final model parameters, otherwise, the initial model parameters need to be adjusted, and the adjusted model parameters are used as the final model parameters. In the embodiment, the SAM optimizer can be used to solve the problem of the conventional optimizer in optimizing highly non-convex functions. It uses gradient information to adaptively adjust the learning rate, so as to optimize the loss function faster and better avoid falling into a local optimal solution.

[0049] S120, obtaining a to-be-tested image of a specified sea surface area, and determining human body related information of the to-be-tested image according to a target detection model.

[0050] Optionally, the human body related information of the to-be-tested image according to the target detection model includes: inputting the to-be-tested image into a target frame detection model to determine each target detection frame in the to-be-tested image and a corresponding detection type, wherein the detection type includes a human body, a human head and a float; inputting each target detection frame with the detection type of a human body into a classification model to determine a human body posture corresponding to each human body, wherein the human body posture includes a normal human body, a half-body human body and a human body in water; and taking the target detection frame, the detection type and the human body posture as the human body related information.

[0051] Specifically, after the target detection model is trained, the recognition of the to-be-tested image is completed, the controller can obtain a to-be-tested image of a specified sea surface area, and then input the to-be-tested image into a target frame detection model to determine each target detection frame in the to-be-tested image and a corresponding detection type, wherein the detection type includes a human body, a human head and a float, and then input each target detection frame with the detection type of a human body into a lightweight classification model to classify a normal human body, a half-body human body and a human body in water. It should be noted that the controller will also score each target detection frame according to the different human body posture types, and the score represents the safety of the state of the human body. The score range is between 0 and 1, and after the model output result, the image needs to be spliced according to the proportion of the cutting image to generate the final detection result.

[0052] S130, target tracking on the human body related information to determine a drowning person detection result.

[0053] Specifically, the human-related information detected can be tracked using the ByteTrack target tracking algorithm, and the trajectories of multiple targets are calculated, that is, the drowning personnel detection result can be determined, wherein the drowning personnel detection result includes the presence of drowning personnel and the absence of drowning personnel. ByteTrack is a tracking method based on the tracking-by-detection paradigm, which uses the similarity between detection boxes and tracking trajectories to remove the background from low-score detection results while retaining high-score detection results, and to mine real objects, thereby reducing missed detection and improving the continuity of the trajectory. The tracking effect depends on the effect of target detection, and the speed is very fast, wherein the default sampling frequency of target tracking is 5FPS / S.

[0054] Optionally, after target tracking of the human-related information to determine the drowning personnel detection result, the method further includes: when the drowning personnel detection result is the presence of drowning personnel, generating prompt information according to the human-related information; and alarming in a specified manner according to the prompt information.

[0055] Further, when the drowning personnel detection result is the presence of drowning personnel, the controller can generate prompt information according to the human-related information, and will alarm in a specified manner according to the prompt information. The alarm is to prompt the lifesaver, so as to help the lifesaver timely grasp the situation of the presence of drowning personnel, so as to carry out rescue work and ensure the safety of the drowning personnel. The specified manner includes voice or image. The voice can be broadcast through the loudspeaker connected to the controller, for example, the voice content can be: there is a person drowning at position A. The image can be displayed through the user terminal connected to the controller to prompt the user.

[0056] The technical scheme of the embodiment of the present application establishes a target detection model through the obtained sea surface image sample, then inputs the to-be-detected image into the target detection model to determine the human-related information, and finally determines the drowning personnel detection result through target tracking of the human-related information. The collection difficulty is reduced, the drowning personnel can be accurately identified through tracking trajectory judgment without relying on hardware sensors, the identification accuracy is high, the lifesaver can be prompted in time, and the safety of the swimming personnel is better ensured.

[0057] Embodiment Two

[0058] Figure 3 A flowchart of a drowning personnel detection method provided by Embodiment Two of the present application is shown in the figure. The embodiment adds specific description of the process of target tracking of the human-related information to determine the drowning personnel detection result on the basis of Embodiment One described above. The specific content of steps S210-S220 is substantially the same as that of steps S110-S120 in Embodiment One, and therefore will not be described again in this embodiment. As shown in the figure, the method comprises: Figure 3 ​

[0059] S210, acquire a sea surface image sample, and establish a target detection model according to the sea surface image sample, wherein the target detection model comprises a corresponding relationship between the sea surface image sample and human body related information.

[0060] Optionally, the sea surface image sample is acquired, including: collecting historical sea surface images, acquiring a label and a bounding box of each historical sea surface image input by a user, wherein the label comprises a detection type and a human body posture; cutting each historical sea surface image based on each bounding box to generate each cut image; performing rotation, flipping and translation processing on each cut image to obtain each enhanced image; cutting each enhanced image according to a specified number and a specified mixed image proportion to generate the sea surface image sample.

[0061] Optionally, the target detection model is established according to the sea surface image sample, including: building a network structure of a target detection algorithm and a classification network, and determining initial model parameters of the network structure; determining final model parameters according to the sea surface image sample and the initial model parameters, and taking a network structure corresponding to the final model parameters as the target detection model, wherein the target detection model comprises a target bounding box detection model and a classification model.

[0062] Optionally, the final model parameters are determined according to the sea surface image sample and the initial model parameters, including: inputting the sea surface image sample into the network structure to obtain an output sample label; determining a true label of the sea surface image sample, and determining an accuracy rate based on the sample label and the true label; determining whether the accuracy rate is less than a preset threshold, if yes, taking the initial model parameters as the final model parameters; otherwise, adjusting the initial model parameters based on the accuracy rate to obtain adjusted model parameters, and taking the adjusted model parameters as the final model parameters.

[0063] S220, acquire a to-be-tested image of a specified sea surface area, and determine human body related information of the to-be-tested image according to the target detection model.

[0064] Optionally, the human body related information of the to-be-tested image is determined according to the target detection model, including: inputting the to-be-tested image into the target bounding box detection model to determine each target detection bounding box in the to-be-tested image and a corresponding detection type, wherein the detection type comprises a human body, a human head and a floating object; inputting each target detection bounding box with the detection type of human body into the classification model to determine a human body posture corresponding to each human body, wherein the human body posture comprises a normal human body, a half human body and a human body in water; and taking the target detection bounding box, the detection type and the human body posture as the human body related information.

[0065] S230, determine a moving speed corresponding to the target detection bounding box, and determine a tracking speed according to the moving speed.

[0066] Specifically, the moving speed of the target in the target detection frame can be calculated according to the distance and the time difference between two adjacent positions, and then the tracking speed is determined according to the moving speed, so that the motion trajectory of the target can be accurately reflected.

[0067] In S240, a target tracking algorithm is used to determine a tracking trajectory corresponding to the target detection frame based on the tracking speed.

[0068] The target tracking algorithm is a Bytetrack algorithm, and the tracking trajectory is a trajectory created from the first frame of the code, including all the trajectories of continuous tracking and interrupted tracking. When tracking the target, the first frame detected has no trajectory, and the box of target detection is used. At this time, the algorithm creates a trajectory object for all target detection frames and stores it for subsequent construction of the tracking trajectory. From the second frame, the algorithm gradually constructs the trajectory. First, the tracking trajectory and the target detection frame are classified, and all tracking trajectories are divided into active and inactive categories. Active refers to tracking more than two frames of target detection frames, and inactive refers to new trajectories that have not appeared before. Then, according to the score threshold, the active target detection frame is divided into high-score frames and low-score frames, for example, the score threshold can be 0.5. Then, the high-score detection frame is tracked first.

[0069] It should be noted that the tracked trajectory and the lost trajectory can be combined into a preliminary tracking trajectory during tracking, and then the Kalman filter algorithm is used to predict the possible position and size of the next frame of the preliminary tracking trajectory, and the IOU value between the preliminary tracking trajectory and the high-score frame of the current target detection frame is calculated to obtain the IOU relationship loss matrix between each other. Then, according to the IOU relationship loss matrix, the Hungarian algorithm is used to match the preliminary tracking trajectory and the high-score target detection frame of the current frame to obtain the matched trajectory and target detection frame, the unmatched trajectory, and the current frame target detection frame that is not successfully matched. Finally, the preliminary tracking trajectory can be updated using the successfully matched current frame boundary box. Then, the second tracking is performed. For the trajectories that are not matched in the first tracking, the tracked trajectories are selected, the IOU between the tracked trajectories and the low-score boundary frame of the current frame is calculated, the Hungarian algorithm is used to match the above tracking trajectories and the low-score boundary frame of the current frame, and the above tracking trajectories are updated using the successfully matched current frame boundary box. Finally, the trajectories that are not successfully tracked at this time are marked as lost tracking trajectories.

[0070] Further, the unactivated target detection frame is tracked, that is, the track that is not matched in the first time is screened, the IOU between the track and the bounding box of the current frame is calculated, the Hungarian algorithm is used to match the track and the bounding box, the successfully matched bounding box of the current frame is used to update the track, and the unactivated track that is not successfully tracked is directly marked as a deleted track. At this time, the high-score target detection frame that is still not successfully matched is determined as a new target, and a new track and a new ID are assigned. At this time, the target tracking is completed, and the final tracking track is returned.

[0071] In S250, a drowning detection rule is acquired, and it is determined whether the tracking track conforms to the drowning detection rule. If yes, S260 is executed; otherwise, S270 is executed.

[0072] In S260, it is determined that the drowning person detection result is that there is a drowning person.

[0073] In S270, it is determined that the drowning person detection result is that there is no drowning person.

[0074] Specifically, when the tracking track conforms to the drowning detection rule, it is indicated that there is a drowning person; otherwise, it is indicated that there is no drowning person. The drowning detection rule can be set by a user. The drowning detection rule can be a swimming speed, for example, the swimming speed is unchanged for 40 consecutive frames, or a detection of an abnormal human body target, for example, an abnormal human body disappears for more than 40 frames, or a specified swimming range, for example, a region in which a person can swim and play in the sea is divided in the camera, and clear signs and warning signs are provided. If a human body, a head, a float, or the like is detected in the swimable region, it is determined to be abnormal. When an abnormal situation of a swimmer is monitored, for example, the swimmer is swept away by a sea wave, fainting, and drowning, and the like, an alarm device connected with a controller immediately issues an alarm, and automatically informs a rescue personnel, thereby protecting the safety of the swimmer on the beach.

[0075] The technical scheme of the embodiment of the application establishes a target detection model through the acquired sea surface image sample, then inputs a to-be-detected image into the target detection model to determine human body related information, and finally determines a drowning person detection result through target tracking on the human body related information, thereby reducing the collection difficulty, accurately identifying the drowning person through the tracking track without relying on a hardware sensor, having high identification accuracy, and timely prompting a lifesaver, and better protecting the safety of the swimmer.

[0076] Embodiment three

[0077] Figure 4 A structure schematic diagram of a drowning person detection device provided by the embodiment three of the application is shown in FIG. 3. Figure 4As shown, the apparatus comprises: a target detection model establishing module 310, configured to acquire a sea surface image sample, and establish a target detection model according to the sea surface image sample, wherein the target detection model comprises a corresponding relationship between the sea surface image sample and human body related information; a human body related information determining module 320, configured to acquire a to-be-detected image of a specified sea surface region, and determine human body related information of the to-be-detected image according to the target detection model; and a drowning person detection result determining module 330, configured to perform target tracking on the human body related information to determine a drowning person detection result.

[0078] Optionally, the target detection model establishing module 310 specifically comprises: a sea surface image sample acquiring unit, configured to: collect historical sea surface images, acquire a label and a bounding box of each historical sea surface image input by a user, wherein the label comprises a detection type and a human body posture; crop each historical sea surface image based on each bounding box to generate each cropped image; perform rotation, flipping and translation processing on each cropped image to acquire each enhanced image; and cut each enhanced image according to a specified number and a specified mixed image proportion to generate the sea surface image sample.

[0079] Optionally, the target detection model establishing module 310 specifically comprises: a target detection model establishing unit, configured to: build a network structure of a target detection algorithm and a classification network, and determine initial model parameters of the network structure; determine final model parameters according to the sea surface image sample and the initial model parameters, and take the network structure corresponding to the final model parameters as the target detection model, wherein the target detection model comprises a target bounding box detection model and a classification model.

[0080] Optionally, the target detection model establishing unit specifically comprises: a final model parameter determining subunit, configured to: input the sea surface image sample into the network structure to acquire an output sample label; determine a true label of the sea surface image sample, and determine an accuracy rate based on the sample label and the true label; determine whether the accuracy rate is less than a preset threshold, if yes, take the initial model parameters as the final model parameters; otherwise, adjust the initial model parameters based on the accuracy rate to acquire adjusted model parameters, and take the adjusted model parameters as the final model parameters.

[0081] Optionally, the human body related information determining module 320 is specifically configured to: input the to-be-detected image into the target bounding box detection model to determine each target detection bounding box in the to-be-detected image and a corresponding detection type, wherein the detection type comprises a human body, a human head and a float; input each target detection bounding box with the detection type of human body into the classification model to determine a human body posture corresponding to each human body, wherein the human body posture comprises a normal human body, a half human body and a human body in water; and take the target detection bounding box, the detection type and the human body posture as the human body related information.

[0082] Optionally, the drowning person detection result determination module 330 is specifically configured to: determine a moving speed corresponding to the target detection frame, and determine a tracking speed according to the moving speed; determine a tracking trajectory corresponding to the target detection frame based on the tracking speed by using a target tracking algorithm; acquire a drowning detection rule, and determine whether the tracking trajectory meets the drowning detection rule; if yes, determine that the drowning person detection result is that there is a drowning person; otherwise, determine that the drowning person detection result is that there is no drowning person.

[0083] Optionally, the device further comprises a drowning alarm module configured to, after the target tracking on the human body related information to determine the drowning person detection result, generate prompt information according to the human body related information when the drowning person detection result is that there is a drowning person; and alarm in a specified manner according to the prompt information.

[0084] The technical scheme of the embodiment of the present application establishes a target detection model through the acquired sea surface image sample, then inputs the to-be-detected image into the target detection model to determine human body related information, and finally determines the drowning person detection result by target tracking on the human body related information, thereby reducing the collection difficulty, accurately identifying the drowning person through the tracking trajectory without relying on the hardware sensor, having high identification accuracy, and being capable of timely prompting the lifesaver, and better guaranteeing the safety of the swimmer.

[0085] The drowning person detection device provided in the embodiment of the present application can execute the drowning person detection method provided in any embodiment of the present application, and has the corresponding function modules and beneficial effects of the execution method.

[0086] Embodiment four

[0087] Figure 5 A structural schematic diagram of an electronic device 10 that can be used to implement embodiments of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular telephones, smart phones, wearable devices (e.g., headsets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not intended to limit implementations of the present application described and / or claimed in this document.

[0088] As Figure 5As shown, the electronic device 10 includes at least one processor 11, and a memory, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., which is communicatively connected to the at least one processor 11, wherein the memory stores a computer program that can be executed by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or loaded from the storage unit 18 into the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0089] A plurality of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, a loudspeaker, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunication networks.

[0090] The processor 11 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 performs various methods and processes described above, such as a method for detecting a drowning person. That is: obtaining a sea surface image sample, establishing a target detection model according to the sea surface image sample, wherein the target detection model includes a corresponding relationship between the sea surface image sample and human-related information; obtaining a to-be-detected image of a specified sea surface region, determining human-related information of the to-be-detected image according to the target detection model; and performing target tracking on the human-related information to determine a drowning person detection result.

[0091] In some embodiments, a method of detecting a drowning person can be implemented as a computer program tangibly embodied in a computer readable storage medium, e.g., storage unit 18. In some embodiments, parts or all of the computer program can be loaded and / or installed onto electronic device 10 via, e.g., ROM 12 and / or communication unit 19. When the computer program is loaded onto RAM 13 and executed by processor 11, one or more steps of a method of detecting a drowning person as described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to perform a method of detecting a drowning person by other means, e.g., with the aid of firmware.

[0092] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a programmable logic device (PLD), a computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0093] Computer programs used to implement the methods of the present application can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the computer program, when executed by the processor, implements the functions / acts specified in the flowcharts and / or block diagrams. The computer program can be executed entirely on a machine, partially on a machine, partially on a machine as part of a standalone software package, partially on a machine and partially on a remote machine or entirely on a remote machine or server.

[0094] In the context of the present application, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. A computer-readable storage medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of a machine-readable storage medium will include one or more lines of a program of instructions in a transitory signal, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0095] To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the electronic device. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0096] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0097] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system, to solve the defects of large management difficulty and weak business scalability in traditional physical host and VPS service.

[0098] It should be understood that the various forms of flow shown above can be used to reorder, add or delete steps. For example, each step described in the present application can be executed in parallel, sequentially or in a different order, as long as the desired results of the technical solutions of the present application can be achieved, which is not limited herein.

[0099] The above detailed description does not constitute a limitation on the scope of protection of the present application. Those skilled in the art should understand that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A method for detecting drowning victims, characterized in that, include: Obtain sea surface image samples, and establish a target detection model based on the sea surface image samples, wherein the target detection model includes the correspondence between sea surface image samples and human body related information; Acquire a test image of a specified sea surface area, and determine human-related information in the test image based on the target detection model; Target tracking is performed on the aforementioned human information to determine the results of drowning detection; The step of establishing a target detection model based on the sea surface image samples includes: Build the network structure of the object detection algorithm and classification network, and determine the initial model parameters of the network structure; The final model parameters are determined based on the sea surface image samples and the initial model parameters, and the network structure corresponding to the final model parameters is used as the target detection model, wherein the target detection model includes a bounding box detection model and a classification model; The step of determining the human body-related information of the image to be tested based on the target detection model includes: The image to be tested is input into the target bounding box detection model to determine each target bounding box in the image to be tested and the corresponding detection type, wherein the detection type includes human body, human head and floating object; Each of the target detection boxes with the detection type of human body is input into the classification model to determine the human body posture corresponding to each human body, wherein the human body posture includes normal human body, half-body human body and underwater human body; The target detection box, the detection type, and the human pose are used as the human body-related information. The method further includes: scoring each target detection box according to the different human posture types, with the score representing the safety status of the human body's state, and the score range being between 0 and 1.

2. The method according to claim 1, characterized in that, The acquisition of sea surface image samples includes: Collect historical sea surface images, and obtain the bounding boxes and labels of each historical sea surface image input by the user, wherein the labels include detection type and human posture; Each cropped image is generated by cropping the corresponding historical sea surface image based on each of the aforementioned bounding boxes; Each cropped image is rotated, flipped, and translated to obtain an enhanced image. The enhanced images are cut according to a specified number and a specified blending ratio to generate the sea surface image samples.

3. The method according to claim 1, characterized in that, The step of determining the final model parameters based on the sea surface image samples and the initial model parameters includes: The sea surface image samples are input into the network structure to obtain the output sample labels; Determine the true labels of the sea surface image samples, and determine the accuracy rate based on the sample labels and the true labels; Determine whether the accuracy is less than a preset threshold; if so, use the initial model parameters as the final model parameters. Otherwise, the initial model parameters are adjusted based on the accuracy to obtain adjusted model parameters, and the adjusted model parameters are used as the final model parameters.

4. The method according to claim 1, characterized in that, The target tracking of the human body-related information to determine the drowning victim detection results includes: Determine the movement speed corresponding to the target detection box, and determine the tracking speed based on the movement speed; Based on the tracking speed, a target tracking algorithm is used to determine the tracking trajectory corresponding to the target detection box; Obtain the drowning detection rules, determine whether the tracking trajectory conforms to the drowning detection rules, and if so, determine that the drowning person detection result indicates that a drowning person exists; Otherwise, the test results for the drowning victims are determined to be negative.

5. The method according to claim 4, characterized in that, After performing target tracking on the human body-related information to determine the drowning victim detection results, the method further includes: When the drowning person detection result indicates that a drowning person exists, a prompt message is generated based on the relevant human information. An alarm will be triggered in the specified manner according to the provided prompt.

6. A drowning detection device, characterized in that, include: The target detection model building module is used to acquire sea surface image samples and build a target detection model based on the sea surface image samples. The target detection model includes the correspondence between the sea surface image samples and human body related information. The human body related information determination module is used to acquire a test image of a specified sea surface area and determine human body related information of the test image based on the target detection model. The drowning victim detection result determination module is used to perform target tracking on the human body-related information to determine the drowning victim detection result; The target detection model building module specifically includes: a target detection model building unit, used to: build the network structure of the target detection algorithm and the classification network, and determine the initial model parameters of the network structure; The final model parameters are determined based on the sea surface image samples and the initial model parameters, and the network structure corresponding to the final model parameters is used as the target detection model, wherein the target detection model includes a bounding box detection model and a classification model; The human body related information determination module is specifically used to: input the image to be tested into the target box detection model to determine each target detection box in the image to be tested and the corresponding detection type, wherein the detection type includes human body, human head and floating object; Each of the target detection boxes with the detection type of human body is input into the classification model to determine the human body posture corresponding to each human body, wherein the human body posture includes normal human body, half-body human body and underwater human body; The target detection box, the detection type, and the human pose are used as the human body-related information. The human body related information determination module is also used to: score each target detection box according to the different human posture types, the score represents the safety status of the human body, and the score range is between 0 and 1.

7. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-5.

8. A computer storage medium, characterized in that, The computer storage medium stores computer instructions that are used to cause a processor to execute the method of any one of claims 1-5.

Images