Protective clothing wearing and taking-off detection model training method and device, electronic equipment and storage medium

Abstract

The invention discloses a training method and device for a protective clothing wearing and taking-off detection model, electronic equipment and a storage medium. The method comprises the following steps: acquiring a protective clothing putting-on and taking-off standard operation video and a protective clothing putting-on and taking-off examination video; the protective clothing putting-on and taking-off standard operation video and the protective clothing putting-on and taking-off examination video are preprocessed, a protective clothing putting-on and taking-off image set is obtained, and preprocessing comprises frame extraction, slicing and clipping; in response to a labeling operation on the protective clothing putting-on and taking-off image set, obtaining a labeled protective clothing putting-on and taking-off image set; and training an initial target detection model based on the labeled protective clothing wearing and taking-off image set to obtain a trained protective clothing wearing and taking-off detection model. According to the technical scheme, the protective clothing wearing and taking-off detection model capable of accurately detecting the wearing and taking-off process of the protective clothing can be obtained, and compared with a manual detection mode, the protective clothing wearing and taking-off detection standard is unified, and the protective clothing wearing and taking-off detection efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a training method, device, electronic equipment and storage medium for a protective clothing donning and doffing detection model. Background Art

[0002] In order to ensure the work safety of medical workers, learning to put on and take off personal protective equipment (PPE) quickly and correctly is crucial to preventing the spread of pathogens and protecting the health and safety of medical workers and patients.

[0003] At present, the training on putting on and taking off protective clothing mainly relies on the traditional face-to-face teaching method, that is, experienced medical workers demonstrate and explain, and then nursing students imitate and practice, and then experts test the nursing students' putting on and taking off protective clothing process.

[0004] In the process of realizing the present invention, the inventors found that there are at least the following technical problems in the prior art: the prior art solution has the problem of low efficiency in detecting the process of putting on and taking off protective clothing. Summary of the invention

[0005] The present invention provides a training method, device, electronic device and storage medium for a protective clothing putting on and taking off detection model, so as to obtain a protective clothing putting on and taking off detection model that can accurately detect the protective clothing putting on and taking off process. Compared with the manual detection method, the detection standards are unified and the efficiency of protective clothing putting on and taking off detection is improved.

[0006] According to one aspect of the present invention, a training method for a protective clothing putting on and taking off detection model is provided, comprising:

[0007] Obtain standard operation videos on putting on and taking off protective clothing and videos on the test of putting on and taking off protective clothing;

[0008] Preprocessing the protective clothing putting on and taking off standard operation video and the protective clothing putting on and taking off test video to obtain a protective clothing putting on and taking off image set, wherein the preprocessing includes frame extraction, slicing and cropping;

[0009] In response to the labeling operation on the protective clothing putting on and taking off image set, a labeled protective clothing putting on and taking off image set is obtained;

[0010] The initial target detection model is trained based on the labeled protective clothing putting on and taking off image set to obtain a trained protective clothing putting on and taking off detection model.

[0011] According to another aspect of the present invention, a training device for a protective clothing donning and doffing detection model is provided, comprising:

[0012] Protective clothing donning and doffing video acquisition module, used to obtain protective clothing donning and doffing standard operation videos and protective clothing donning and doffing test videos;

[0013] A protective clothing putting on and taking off video preprocessing module, used to preprocess the protective clothing putting on and taking off standard operation video and the protective clothing putting on and taking off test video to obtain a protective clothing putting on and taking off image set, wherein the preprocessing includes frame extraction, slicing and cropping;

[0014] A protective clothing putting on and taking off image set annotation module, used to obtain an annotated protective clothing putting on and taking off image set in response to an annotation operation on the protective clothing putting on and taking off image set;

[0015] The protective clothing putting on and taking off detection model training module is used to train the initial target detection model based on the labeled protective clothing putting on and taking off image set to obtain a trained protective clothing putting on and taking off detection model.

[0016] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0017] at least one processor;

[0018] and a memory communicatively coupled to the at least one processor;

[0019] Among them, the memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the training method of the protective clothing putting on and taking off detection model described in any embodiment of the present invention.

[0020] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the training method of the protective clothing putting on and taking off detection model described in any embodiment of the present invention when executed.

[0021] The technical solution of the embodiment of the present invention can obtain a protective clothing putting on and taking off detection model that can accurately detect the protective clothing putting on and taking off process through the above-mentioned training scheme. Compared with the manual detection method, it unifies the protective clothing putting on and taking off detection standards and improves the efficiency of protective clothing putting on and taking off detection.

[0022] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0024] Figure 1 is a flow chart of a training method for a protective clothing putting on and taking off detection model provided according to Embodiment 1 of the present invention;

[0025] Figure 2 is a flow chart of a training method for a protective clothing putting on and taking off detection model provided according to Embodiment 2 of the present invention;

[0026] Figure 3 is a flow chart of a method for training a protective clothing putting on and taking off detection model provided according to Embodiment 3 of the present invention;

[0027] Figure 4 is a flow chart of a training method for a protective clothing putting on and taking off detection model provided according to a fourth embodiment of the present invention;

[0028] Figure 5 2 is a schematic diagram of the structure of a training device for a protective clothing putting on and taking off detection model provided according to Embodiment 5 of the present invention;

[0029] Figure 6 It is a structural schematic diagram of an electronic device for implementing the training method of the protective clothing putting on and taking off detection model according to an embodiment of the present invention. DETAILED DESCRIPTION

[0030] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0031] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices. The acquisition, storage, use, processing, etc. of data in the technical solution of this application comply with the relevant provisions of national laws and regulations.

[0032] Embodiment 1

[0033] Figure 1 This is a flowchart of a training method for a protective clothing donning and taking off detection model provided in the first embodiment of the present invention. This embodiment is applicable to the training of artificial intelligence models. The method can be executed by a training device for a protective clothing donning and taking off detection model. The training device for the protective clothing donning and taking off detection model can be implemented in the form of hardware and / or software. The training device for the protective clothing donning and taking off detection model can be configured in a terminal and / or a server. Figure 1 As shown, the method includes:

[0034] S110. Obtain the standard operation video of putting on and taking off protective clothing and the test video of putting on and taking off protective clothing.

[0035] In the embodiment of the present invention, the protective clothing donning and taking off standard operation video refers to video data containing protective clothing donning and taking off standard operation actions, the number of which can be one or more, which is not limited here, and the protective clothing donning and taking off standard operation actions refer to correct or qualified protective clothing donning and taking off actions. The protective clothing donning and taking off test video refers to video data collected on-site for the protective clothing donning and taking off test, which may include incorrect protective clothing donning and taking off actions, the number of which can be one or more, which is not limited here.

[0036] Exemplarily, 9 standard operation videos of putting on and taking off protective clothing and 14 test videos of putting on and taking off protective clothing can be obtained from the preset storage path of the electronic device.

[0037] S120, pre-processing the protective clothing putting on and taking off standard operation video and the protective clothing putting on and taking off test video to obtain a protective clothing putting on and taking off image set, wherein the pre-processing includes frame extraction, slicing and cropping.

[0038] Specifically, preprocessing operations such as frame extraction, slicing and cropping are performed on the standard operation video of putting on and taking off protective clothing and the test video of putting on and taking off protective clothing, so as to obtain a set of protective clothing putting on and taking off images consisting of multiple protective clothing putting on and taking off images.

[0039] S130: In response to the labeling operation on the protective clothing putting on and taking off image set, a labeled protective clothing putting on and taking off image set is obtained.

[0040] In an embodiment of the present invention, the marked content may include but is not limited to protective clothing, human body, clothing, hand sanitizer, goggles, gloves, masks, hats, hands, trash cans, hands with masks, hands crossed, hand washing, taking off gloves and boot covers, etc.

[0041] It should be noted that the embodiment of the present invention can annotate all protective clothing putting on and taking off images in the protective clothing putting on and taking off image set one by one, thereby obtaining an annotated protective clothing putting on and taking off image set, that is, the annotated protective clothing putting on and taking off image set has images that correspond one to one with the protective clothing putting on and taking off image set.

[0042] S140: training an initial target detection model based on the labeled protective clothing putting on and taking off image set to obtain a trained protective clothing putting on and taking off detection model.

[0043] Specifically, the protective clothing putting on and taking off images in the annotated protective clothing putting on and taking off image set can be input into the initial target detection model to obtain the predicted target detection result of the initial target detection model, and then the predicted target detection result is compared with the actual target, and the model loss value is calculated. Then, the model parameters are updated by back propagation based on the model loss value until the training stop condition is met, and a trained protective clothing putting on and taking off detection model is obtained.

[0044] The technical solution of the embodiment of the present invention can obtain a protective clothing putting on and taking off detection model that can accurately detect the protective clothing putting on and taking off process through the above-mentioned training scheme. Compared with the manual detection method, it unifies the detection standards and improves the efficiency of protective clothing putting on and taking off detection.

[0045] Embodiment 2

[0046] Figure 2 A flowchart of a method for training a protective clothing putting on and taking off detection model provided in Embodiment 2 of the present invention. The method of this embodiment can be combined with the various optional schemes in the method for training a protective clothing putting on and taking off detection model provided in the above embodiments. The training method for the protective clothing putting on and taking off detection model provided in this embodiment is further optimized. Optionally, the response to the annotation operation of the protective clothing putting on and taking off image set includes: responding to the annotation operation of the protective clothing putting on and taking off process and the protective clothing taking off process in the protective clothing putting on and taking off image set.

[0047] like Figure 2 As shown, the method includes:

[0048] S210. Obtain the standard operation video of putting on and taking off protective clothing and the test video of putting on and taking off protective clothing.

[0049] S220, pre-processing the protective clothing putting on and taking off standard operation video and the protective clothing putting on and taking off test video to obtain a protective clothing putting on and taking off image set, wherein the pre-processing includes frame extraction, slicing and cropping.

[0050] S230: responding to the labeling operation of the protective clothing putting on process and the protective clothing taking off process in the protective clothing putting on and taking off image set.

[0051] S240: training an initial target detection model based on the labeled protective clothing putting on and taking off image set to obtain a trained protective clothing putting on and taking off detection model.

[0052] In an embodiment of the present invention, in response to the annotation operation of the protective clothing wearing and removing process and the protective clothing removing process in the protective clothing wearing and removing image set, the operation includes: responding to the annotation operation of the hand pressing hand sanitizer action in the protective clothing wearing and removing image set; responding to the annotation operation of the two-hand crossing action in the protective clothing wearing and removing image set; responding to the annotation operation of the two-hand spreading and showing action in the protective clothing wearing and removing image set; responding to the annotation operation of the hat at each angle on the head in the protective clothing wearing and removing image set; responding to the annotation operation of the mask at each angle on the face in the protective clothing wearing and removing image set; responding to the annotation operation of the open protective clothing in the protective clothing wearing and removing image set; responding to the annotation operation of the hands without gloves at each angle in the protective clothing wearing and removing image set; responding to the annotation operation of the hands with gloves at each angle in the protective clothing wearing and removing image set; responding to the annotation operation of the gloves not worn on the table in the protective clothing wearing and removing image set; responding to the annotation operation of the hands holding gloves in the protective clothing wearing and removing image set; responding to the annotation of the goggles at each angle in the protective clothing wearing and removing image set. Operation; in response to the marking operation of the unworn gloves on the table in the protective clothing putting on and taking off image set; in response to the marking operation of the hands wearing gloves at various angles in the protective clothing putting on and taking off image set; in response to the marking operation of the crossed hands action in the protective clothing putting on and taking off image set; in response to the marking operation of the gloves placed on the table in various states in the protective clothing putting on and taking off image set; in response to the marking operation of the goggles taken off and placed on the table at various angles in the protective clothing putting on and taking off image set; in response to the marking operation of the protective clothing putting on and taking off image set in the state of putting on the table after taking off; in response to the marking operation of the gloves on the table at various angles after taking off in the protective clothing putting on and taking off image set; in response to the marking operation of the masks on the table at various angles after taking off in the protective clothing putting on and taking off image set; in response to the marking operation of the hats on the table at various angles after taking off in the protective clothing putting on and taking off image set; in response to the marking operation of the hands in front of the masks in the protective clothing putting on and taking off image set; in response to the marking operation of the table in the protective clothing putting on and taking off image set; in response to the marking operation of the head in the protective clothing putting on and taking off image set.

[0053] It should be noted that, through the above-mentioned labeling operations, the labeling of various detection items in the process of wearing protective clothing, such as hand hygiene, wearing a hat, wearing a mask and protective clothing, wearing inner gloves, wearing goggles and wearing outer gloves, as well as the labeling of various detection items in the process of taking off protective clothing, such as taking off outer gloves, taking off goggles, taking off protective clothing, taking off inner gloves, taking off masks and taking off hats, are completed; and the labeling of erroneous action detection of hands appearing in front of the mask is completed. Compared with the existing detection methods, the labeling or detection points are more refined, so the initial target detection model can be trained using the labeled protective clothing wearing and taking off image set, which can improve the prediction accuracy of the protective clothing wearing and taking off detection model.

[0054] Embodiment 3

[0055] Figure 3 A flowchart of a training method for a protective clothing putting on and taking off detection model provided in the third embodiment of the present invention, the method of this embodiment can be combined with the various optional schemes in the training method for the protective clothing putting on and taking off detection model provided in the above embodiments. The training method for the protective clothing putting on and taking off detection model provided in this embodiment is further optimized. Optionally, the initial target detection model includes a backbone network, a feature pyramid network, a path aggregation network and a detection head; accordingly, the initial target detection model is trained based on the annotated protective clothing putting on and taking off image set to obtain a trained protective clothing putting on and taking off detection model, including: for any protective clothing putting on and taking off image in the annotated protective clothing putting on and taking off image set, the protective clothing putting on and taking off image is input into the backbone network to obtain feature information corresponding to the protective clothing putting on and taking off image; the feature information corresponding to the protective clothing putting on and taking off image is respectively input into the feature pyramid network to obtain the first semantic information corresponding to the protective clothing putting on and taking off image; The first semantic information corresponding to the protective clothing putting on and taking off image is input into the path aggregation network to obtain the second semantic information corresponding to the protective clothing putting on and taking off image; the second semantic information corresponding to the protective clothing putting on and taking off image is input into the detection head to obtain the predicted target information of putting on and taking off protective clothing corresponding to the protective clothing putting on and taking off image; the model loss value is determined based on the predicted target information of putting on and taking off protective clothing corresponding to the protective clothing putting on and taking off image and the actual target information of putting on and taking off protective clothing corresponding to the protective clothing putting on and taking off image; the parameters of the initial target detection model are updated based on the model loss value until the training stop condition is met, so as to obtain a trained protective clothing putting on and taking off detection model.

[0056] like Figure 3 As shown, the method includes:

[0057] S310. Obtain the standard operation video of putting on and taking off protective clothing and the test video of putting on and taking off protective clothing.

[0058] S320, pre-processing the protective clothing putting on and taking off standard operation video and the protective clothing putting on and taking off test video to obtain a protective clothing putting on and taking off image set, wherein the pre-processing includes frame extraction, slicing and cropping.

[0059] S330: In response to the labeling operation on the protective clothing putting on and taking off image set, a labeled protective clothing putting on and taking off image set is obtained.

[0060] S340: for any protective clothing putting on and taking off image in the annotated protective clothing putting on and taking off image set, input the protective clothing putting on and taking off image into the backbone network to obtain feature information corresponding to the protective clothing putting on and taking off image.

[0061] S350, inputting feature information corresponding to the protective clothing putting on and taking off image into the feature pyramid network respectively to obtain first semantic information corresponding to the protective clothing putting on and taking off image.

[0062] S360: Input the first semantic information corresponding to the protective clothing putting on and taking off image into the path aggregation network to obtain the second semantic information corresponding to the protective clothing putting on and taking off image.

[0063] S370: Input the second semantic information corresponding to the protective clothing putting on and taking off image into the detection head to obtain the protective clothing putting on and taking off predicted target information corresponding to the protective clothing putting on and taking off image.

[0064] S380, determining a model loss value based on the predicted target information of putting on and taking off the protective clothing corresponding to the protective clothing putting on and taking off image and the actual target information of putting on and taking off the protective clothing corresponding to the protective clothing putting on and taking off image.

[0065] S390. Update the parameters of the initial target detection model based on the model loss value until the training stop condition is met, thereby obtaining a trained protective clothing putting on and taking off detection model.

[0066] In some optional embodiments, the initial target detection model may further include an anchor frame unit, which is used to predict the position of the target.

[0067] In an embodiment of the present invention, the model loss value may include classification loss, positioning loss and confidence loss, wherein the positioning loss is used to measure the position difference between the predicted bounding box and the true bounding box; the confidence loss is used to determine whether the anchor box contains the target; and the classification loss is used to determine which category the target in the anchor box belongs to.

[0068] In an embodiment of the present invention, the semantic information of the high-level and low-level layers of the protective clothing putting on and taking off images can be obtained by using a feature pyramid network and a path aggregation network, thereby improving the accuracy of the prediction results of the protective clothing putting on and taking off detection model.

[0069] Embodiment 4

[0070] Figure 4 A flowchart of a training method for a protective clothing putting on and taking off detection model provided in Embodiment 4 of the present invention, the method of this embodiment can be combined with the various optional schemes in the training method for the protective clothing putting on and taking off detection model provided in the above embodiments. The training method for the protective clothing putting on and taking off detection model provided in this embodiment is further optimized. Optionally, after the initial target detection model is trained based on the annotated protective clothing putting on and taking off image set to obtain the trained protective clothing putting on and taking off detection model, it also includes: obtaining a protective clothing putting on and taking off image test sample; inputting the protective clothing putting on and taking off image test sample into the trained protective clothing putting on and taking off detection model to obtain a protective clothing putting on and taking off target detection result.

[0071] like Figure 4 As shown, the method includes:

[0072] S410. Obtain the standard operation video of putting on and taking off protective clothing and the test video of putting on and taking off protective clothing.

[0073] S420, pre-processing the protective clothing putting on and taking off standard operation video and the protective clothing putting on and taking off test video to obtain a protective clothing putting on and taking off image set, wherein the pre-processing includes frame extraction, slicing and cropping.

[0074] S430: In response to the labeling operation on the protective clothing putting on and taking off image set, a labeled protective clothing putting on and taking off image set is obtained.

[0075] S440: training an initial target detection model based on the labeled protective clothing putting on and taking off image set to obtain a trained protective clothing putting on and taking off detection model.

[0076] S450, obtaining a protective clothing putting on and taking off image test sample; inputting the protective clothing putting on and taking off image test sample into a trained protective clothing putting on and taking off detection model to obtain a protective clothing putting on and taking off target detection result.

[0077] Among them, the detection results of the protective clothing wearing and taking off targets include one or more of the actions of hands pressing hand sanitizer, a hat on the head, a mask on the face, an open protective clothing, a gloved hand, a pair of gloves not worn on the table, goggles on the head, no gloves on the table, the action of crossing hands, taking off goggles and placing them on the table, the protective clothing being taken off and placed on the table, the state of hands without gloves, the state of gloves being taken off and placed on the table, the mask on the table after being taken off, and the hat on the table after being taken off.

[0078] In an embodiment of the present invention, after the protective clothing putting on and taking off image test sample is input into the trained protective clothing putting on and taking off detection model to obtain the protective clothing putting on and taking off target detection result, it also includes: in response to the selection operation of the protective clothing putting on and taking off sample image, a protective clothing putting on and taking off secondary training sample image set is obtained; based on the protective clothing putting on and taking off target detection result, the protective clothing putting on and taking off secondary training sample image set is annotated to obtain the annotated protective clothing putting on and taking off secondary training sample image set; based on the annotated protective clothing putting on and taking off image set and the annotated protective clothing putting on and taking off secondary training sample image set, the protective clothing putting on and taking off detection model is trained to obtain an optimized protective clothing putting on and taking off detection model.

[0079] Among them, the secondary training sample image set of protective clothing putting on and taking off refers to the image sample data selected for the second round of model training.

[0080] Exemplarily, based on the protective clothing putting on and taking off target detection results of the first round of model training, an embodiment of the present invention performs a second round of targeted labeling work. For example, the hand sanitizer detection rate reaches 92%, but the accuracy of hand washing actions is only 71%. The second round of model training can focus on labeling hand washing actions to increase the sample diversity and accuracy of the data set, thereby improving the prediction accuracy of the protective clothing putting on and taking off detection model.

[0081] The technical solution of the embodiment of the present invention can improve the prediction accuracy of the protective clothing putting on and taking off detection model through secondary targeted training.

[0082] Embodiment 5

[0083] Figure 5 This is a schematic diagram of the structure of a training device for a protective clothing donning and doffing detection model provided in Embodiment 5 of the present invention. Figure 5 As shown, the device comprises:

[0084] The protective clothing putting on and taking off video acquisition module 510 is used to acquire the protective clothing putting on and taking off standard operation video and protective clothing putting on and taking off test video;

[0085] The protective clothing donning and taking-off video preprocessing module 520 is used to preprocess the protective clothing donning and taking-off standard operation video and the protective clothing donning and taking-off test video to obtain a protective clothing donning and taking-off image set, wherein the preprocessing includes frame extraction, slicing and cropping;

[0086] The protective clothing putting on and taking off image set annotation module 530 is used to obtain an annotated protective clothing putting on and taking off image set in response to an annotation operation on the protective clothing putting on and taking off image set;

[0087] The protective clothing putting on and taking off detection model training module 540 is used to train the initial target detection model based on the labeled protective clothing putting on and taking off image set to obtain a trained protective clothing putting on and taking off detection model.

[0088] The technical solution of the embodiment of the present invention can obtain a protective clothing putting on and taking off detection model that can accurately detect the protective clothing putting on and taking off process through the above-mentioned training scheme. Compared with the manual detection method, it unifies the detection standards and improves the efficiency of protective clothing putting on and taking off detection.

[0089] In some optional implementations, the protective clothing putting on and taking off image set annotation module 530 includes:

[0090] The protective clothing putting on and taking off process labeling unit is used to respond to the labeling operation of the protective clothing putting on process and the protective clothing taking off process in the protective clothing putting on and taking off image set.

[0091] In some optional embodiments, the protective clothing putting on and taking off process labeling unit is specifically used for:

[0092] In response to a labeling operation of a hand pressing hand sanitizer action in a protective clothing putting on and taking off image collection;

[0093] In response to a labeling operation of a hands-crossing action in a protective clothing putting-on-and-taking-off image collection;

[0094] In response to a labeling operation of an action of spreading out both hands in a collection of protective clothing putting on and taking off images;

[0095] In response to the labeling operation of hats at various angles on the head in the protective clothing putting on and taking off image set;

[0096] In response to the labeling operation of masks at various angles on the face in the protective clothing putting on and taking off image set;

[0097] In response to a labeling operation of an open protective clothing in a protective clothing donning and doffing image collection;

[0098] In response to the labeling operation of the ungloved hands at various angles in the protective clothing putting on and taking off image set;

[0099] In response to the labeling operation of the gloved hands at various angles in the protective clothing putting on and taking off image set;

[0100] In response to a labeling operation of an unworn glove on a table in a protective clothing donning and doffing image collection;

[0101] In response to a labeling operation of a hand holding a glove in a protective clothing putting on and taking off image collection;

[0102] In response to the labeling operation of wearing goggles at each angle in the protective clothing putting on and taking off image set;

[0103] In response to a labeling operation of an unworn glove on a table in a protective clothing donning and doffing image collection;

[0104] In response to the labeling operation of gloves worn at each angle in the protective clothing putting on and taking off image set;

[0105] In response to a labeling operation of a hands-crossing action in a protective clothing putting-on-and-taking-off image collection;

[0106] In response to a labeling operation of each state of gloves placed on a table in a collection of protective clothing putting on and taking off images;

[0107] In response to the labeling operation of taking off the goggles and placing them on the table at various angles in the protective clothing putting on and taking off image set;

[0108] In response to a labeling operation of a state in which the protective clothing is taken off and placed on a table in the protective clothing putting on and taking off image set;

[0109] In response to the labeling operation of gloves at various angles on the table after taking off the protective clothing in the protective clothing putting on and taking off image set;

[0110] In response to the labeling operation of masks at various angles on the table after they are taken off in the protective clothing putting on and taking off image set;

[0111] In response to the labeling operation of the hats at various angles on the table after taking off the protective clothing in the protective clothing putting on and taking off image collection;

[0112] In response to a labeling operation of a hand movement in front of a mask in a collection of protective clothing putting on and taking off images;

[0113] In response to a labeling operation on a table in a protective clothing donning and doffing image collection;

[0114] In response to a labeling operation on a head in a protective clothing putting on and taking off image collection.

[0115] In some optional embodiments, the initial target detection model includes a backbone network, a feature pyramid network, a path aggregation network and a detection head;

[0116] Accordingly, the protective clothing putting on and taking off detection model training module 540 can be specifically used for:

[0117] For any protective clothing putting on and taking off image in the annotated protective clothing putting on and taking off image set, input the protective clothing putting on and taking off image into the backbone network to obtain feature information corresponding to the protective clothing putting on and taking off image;

[0118] Inputting feature information corresponding to the protective clothing putting on and taking off image into the feature pyramid network respectively to obtain first semantic information corresponding to the protective clothing putting on and taking off image;

[0119] Inputting the first semantic information corresponding to the protective clothing putting on and taking off image into the path aggregation network to obtain the second semantic information corresponding to the protective clothing putting on and taking off image;

[0120] Inputting the second semantic information corresponding to the protective clothing putting on and taking off image into the detection head to obtain the protective clothing putting on and taking off predicted target information corresponding to the protective clothing putting on and taking off image;

[0121] Determine a model loss value based on the predicted target information of putting on and taking off the protective clothing corresponding to the protective clothing putting on and taking off image and the actual target information of putting on and taking off the protective clothing corresponding to the protective clothing putting on and taking off image;

[0122] The parameters of the initial target detection model are updated based on the model loss value until the training stop condition is met, thereby obtaining a trained protective clothing putting on and taking off detection model.

[0123] In some optional implementations, the initial target detection model further includes an anchor frame unit, and the anchor frame unit is used to predict the position of the target.

[0124] In some optional embodiments, the training device for the protective clothing putting on and taking off detection model further includes:

[0125] A test sample acquisition module is used to acquire test samples of protective clothing putting on and taking off images;

[0126] A protective clothing putting on and taking off detection model application module is used to input the protective clothing putting on and taking off image test sample into the trained protective clothing putting on and taking off detection model to obtain the protective clothing putting on and taking off target detection result;

[0127] Among them, the target detection results of putting on and taking off protective clothing include one or more of the actions of pressing hand sanitizer with hands, a hat on the head, a mask on the face, an open protective clothing, a gloved hand, a pair of gloves not worn on the table, goggles on the head, no gloves on the table, the action of crossing both hands, taking off goggles and placing them on the table, the state of putting protective clothing on the table after taking it off, the state of not wearing gloves on hands, the state of putting gloves on the table after taking them off, the mask on the table after taking them off, and the hat on the table after taking them off.

[0128] In some optional embodiments, the training device for the protective clothing putting on and taking off detection model further includes:

[0129] The secondary training module is used to obtain a set of secondary training sample images of putting on and taking off protective clothing in response to the selection operation of the sample images of putting on and taking off protective clothing; based on the target detection result of putting on and taking off protective clothing, the set of secondary training sample images of putting on and taking off protective clothing is annotated to obtain a set of annotated secondary training sample images of putting on and taking off protective clothing; based on the annotated set of protective clothing putting on and taking off images and the annotated set of secondary training sample images of putting on and taking off protective clothing, the protective clothing putting on and taking off detection model is trained to obtain an optimized protective clothing putting on and taking off detection model.

[0130] The training device for the protective clothing putting on and taking off detection model provided in the embodiment of the present invention can execute the training method for the protective clothing putting on and taking off detection model provided in any embodiment of the present invention, and has the functional modules and beneficial effects corresponding to the execution method.

[0131] Embodiment 6

[0132] Figure 6 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0133] like Figure 6 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. The I / O interface 15 is also connected to the bus 14.

[0134] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0135] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The processor 11 executes the various methods and processes described above, such as a training method for a protective clothing donning and undressing detection model, which includes:

[0136] Obtain standard operation videos on putting on and taking off protective clothing and videos on the test of putting on and taking off protective clothing;

[0137] Preprocessing the protective clothing putting on and taking off standard operation video and the protective clothing putting on and taking off test video to obtain a protective clothing putting on and taking off image set, wherein the preprocessing includes frame extraction, slicing and cropping;

[0138] In response to the labeling operation on the protective clothing putting on and taking off image set, a labeled protective clothing putting on and taking off image set is obtained;

[0139] The initial target detection model is trained based on the labeled protective clothing putting on and taking off image set to obtain a trained protective clothing putting on and taking off detection model.

[0140] In some embodiments, the training method of the protective clothing donning and removing detection model can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the training method of the protective clothing donning and removing detection model described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to execute the training method of the protective clothing donning and removing detection model in any other appropriate manner (for example, by means of firmware).

[0141] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0142] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0143] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0144] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0145] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0146] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0147] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0148] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A training method for a protective clothing wearing and taking off detection model, characterized in that: include: Obtain standard operation videos on putting on and taking off protective clothing and videos on the test of putting on and taking off protective clothing; Preprocessing the protective clothing putting on and taking off standard operation video and the protective clothing putting on and taking off test video to obtain a protective clothing putting on and taking off image set, wherein the preprocessing includes frame extraction, slicing and cropping; In response to the labeling operation on the protective clothing putting on and taking off image set, a labeled protective clothing putting on and taking off image set is obtained; The initial target detection model is trained based on the labeled protective clothing putting on and taking off image set to obtain a trained protective clothing putting on and taking off detection model.

2. The method according to claim 1, characterized in that The response to the annotation operation on the protective clothing putting on and taking off image set includes: In response to the labeling operation of the protective clothing putting on process and the protective clothing taking off process in the protective clothing putting on and taking off image set.

3. The method according to claim 2, characterized in that The response to the labeling operation of the protective clothing putting on process and the protective clothing taking off process in the protective clothing putting on and taking off image set includes: In response to a labeling operation of a hand pressing hand sanitizer action in a protective clothing putting on and taking off image collection; In response to a labeling operation of a hands-crossing action in a protective clothing putting-on-and-taking-off image collection; In response to a labeling operation of an action of spreading out both hands in a collection of protective clothing putting on and taking off images; In response to the labeling operation of hats at various angles on the head in the protective clothing putting on and taking off image set; In response to the labeling operation of masks at various angles on the face in the protective clothing putting on and taking off image set; In response to a labeling operation of an open protective clothing in a protective clothing donning and doffing image collection; In response to the labeling operation of the ungloved hands at various angles in the protective clothing putting on and taking off image set; In response to the labeling operation of the gloved hands at various angles in the protective clothing putting on and taking off image set; In response to a labeling operation of an unworn glove on a table in a protective clothing donning and doffing image collection; In response to a labeling operation of a hand holding a glove in a protective clothing putting on and taking off image collection; In response to the labeling operation of wearing goggles at each angle in the protective clothing putting on and taking off image set; In response to a labeling operation of an unworn glove on a table in a protective clothing donning and doffing image collection; In response to the labeling operation of gloves worn at each angle in the protective clothing putting on and taking off image set; In response to a labeling operation of a hands-crossing action in a protective clothing putting-on-and-taking-off image collection; In response to a labeling operation of each state of gloves placed on a table in a collection of protective clothing putting on and taking off images; In response to the labeling operation of taking off the goggles and placing them on the table at various angles in the protective clothing putting on and taking off image set; In response to a labeling operation of a state in which the protective clothing is taken off and placed on a table in the protective clothing putting on and taking off image set; In response to the labeling operation of gloves at various angles on the table after taking off the protective clothing in the protective clothing putting on and taking off image set; In response to the labeling operation of masks at various angles on the table after they are taken off in the protective clothing putting on and taking off image set; In response to the labeling operation of the hats at various angles on the table after taking off the protective clothing in the protective clothing putting on and taking off image collection; In response to a labeling operation of a hand movement in front of a mask in a collection of protective clothing putting on and taking off images; In response to a labeling operation on a table in a protective clothing donning and doffing image collection; In response to a labeling operation on a head in a protective clothing putting on and taking off image collection.

4. The method according to claim 1, characterized in that The initial target detection model includes a backbone network, a feature pyramid network, a path aggregation network and a detection head; Accordingly, the initial target detection model is trained based on the annotated protective clothing putting on and taking off image set to obtain a trained protective clothing putting on and taking off detection model, including: For any protective clothing putting on and taking off image in the annotated protective clothing putting on and taking off image set, input the protective clothing putting on and taking off image into the backbone network to obtain feature information corresponding to the protective clothing putting on and taking off image; Inputting feature information corresponding to the protective clothing putting on and taking off image into the feature pyramid network respectively to obtain first semantic information corresponding to the protective clothing putting on and taking off image; Inputting the first semantic information corresponding to the protective clothing putting on and taking off image into the path aggregation network to obtain the second semantic information corresponding to the protective clothing putting on and taking off image; Inputting the second semantic information corresponding to the protective clothing putting on and taking off image into the detection head to obtain the protective clothing putting on and taking off predicted target information corresponding to the protective clothing putting on and taking off image; Determine a model loss value based on the predicted target information of putting on and taking off the protective clothing corresponding to the protective clothing putting on and taking off image and the actual target information of putting on and taking off the protective clothing corresponding to the protective clothing putting on and taking off image; The parameters of the initial target detection model are updated based on the model loss value until the training stop condition is met, thereby obtaining a trained protective clothing putting on and taking off detection model.

5. The method according to claim 4, characterized in that The initial target detection model also includes an anchor frame unit, which is used to predict the position of the target.

6. The method according to claim 1, characterized in that After the initial target detection model is trained based on the annotated protective clothing putting on and taking off image set to obtain the trained protective clothing putting on and taking off detection model, the method further includes: Obtain test samples of protective clothing wearing and taking off images; Input the protective clothing putting on and taking off image test sample into the trained protective clothing putting on and taking off detection model to obtain the protective clothing putting on and taking off target detection result; Among them, the target detection results of putting on and taking off protective clothing include one or more of the actions of pressing hand sanitizer with hands, a hat on the head, a mask on the face, an open protective clothing, a gloved hand, a pair of gloves not worn on the table, goggles on the head, no gloves on the table, the action of crossing both hands, taking off goggles and placing them on the table, the state of putting protective clothing on the table after taking it off, the state of not wearing gloves on hands, the state of putting gloves on the table after taking them off, the mask on the table after taking them off, and the hat on the table after taking them off.

7. The method according to claim 6, characterized in that After the protective clothing putting on and taking off image test sample is input into the trained protective clothing putting on and taking off detection model to obtain the protective clothing putting on and taking off target detection result, the method further includes: In response to the selection operation of the protective clothing putting on and taking off sample images, a set of protective clothing putting on and taking off secondary training sample images is obtained; Based on the protective clothing putting on and taking off target detection result, the protective clothing putting on and taking off secondary training sample image set is labeled to obtain a labeled protective clothing putting on and taking off secondary training sample image set; Based on the annotated protective clothing putting on and taking off image set and the annotated protective clothing putting on and taking off secondary training sample image set, the protective clothing putting on and taking off detection model is trained to obtain an optimized protective clothing putting on and taking off detection model.

8. A training device for a protective clothing donning and doffing detection model, characterized in that: include: Protective clothing donning and doffing video acquisition module, used to obtain protective clothing donning and doffing standard operation videos and protective clothing donning and doffing test videos; A protective clothing putting on and taking off video preprocessing module, used to preprocess the protective clothing putting on and taking off standard operation video and the protective clothing putting on and taking off test video to obtain a protective clothing putting on and taking off image set, wherein the preprocessing includes frame extraction, slicing and cropping; A protective clothing putting on and taking off image set annotation module, used to obtain an annotated protective clothing putting on and taking off image set in response to an annotation operation on the protective clothing putting on and taking off image set; The protective clothing putting on and taking off detection model training module is used to train the initial target detection model based on the labeled protective clothing putting on and taking off image set to obtain a trained protective clothing putting on and taking off detection model.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively coupled to the at least one processor; Wherein, the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the training method for the protective clothing putting on and taking off detection model described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the training method of the protective clothing putting on and taking off detection model according to any one of claims 1 to 7 when executed.

Images