Target detection model training method, device, target detection method, device, medium and equipment

Abstract

The present invention discloses a training method for a target detection model, comprising: obtaining a picture including multiple objects; preprocessing the picture; the preprocessing comprising: labeling the multiple objects to obtain head truth frames and body truth frames of the multiple objects; matching the head truth frames and the body truth frames to determine the head truth frames and the body truth frames belonging to the same object; based on the body truth frames, obtaining a torso truth frame that matches the body truth frame to complete the labeling of the picture; using the labeled picture as a sample picture to train an initial neural network model to obtain a target detection model. The target detection model of the present invention has a torso detection branch. When the torso detection branch outputs a human body detection frame (i.e., Sub1) and a head detection frame (i.e., Sub2), the recall rate of the human body is improved by more than 3 percentage points, and the recall rate of the head is improved by 0.5 percentage points.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a training method and device for a target detection model, a target detection method, device, medium and equipment. Background Art

[0002] In fully structured pedestrian detection, detection models are typically deployed at high camera positions, such as at major thoroughfares and building entrances, inevitably encountering dense crowds of pedestrians. Closely spaced pedestrians are often largely obscured, making human detection quite challenging.

[0003] In addition, the overlapping and occluding of pedestrians in crowded scenes increases the difficulty of associating the human body and the human head, resulting in the phenomenon of "putting the head on the wrong head" being associated with the body of pedestrian B.

[0004] Missed human detection and incorrect human-head association problems will have a significant impact on subsequent tracking and recognition; therefore, it is very important to ensure the detection rate of occluded human bodies and the stability of human-head association. Summary of the Invention

[0005] In view of the above-mentioned shortcomings of the prior art, the object of the present invention is to provide a target detection model training method, device, target detection method, device, medium and equipment to solve at least one defect in the prior art.

[0006] To achieve the above and other related objectives, the present invention provides a target tracking method, comprising:

[0007] Get a picture including multiple objects;

[0008] Preprocessing the image; the preprocessing includes:

[0009] Labeling multiple objects to obtain ground truth frames of heads and bodies of the multiple objects; matching the ground truth frames of heads and bodies to determine which ground truth frames of heads and bodies belong to the same object; and obtaining ground truth frames of torsos that match the ground truth frames of bodies based on the ground truth frames to complete the labeling of the image;

[0010] The labeled images are used as sample images to train the initial neural network model to obtain the target detection model.

[0011] Optionally, matching the head ground truth frame with the human body ground truth frame includes:

[0012] The head truth frame and the human body truth frame are matched based on their IOU and positional relationship.

[0013] Optionally, the target detection model includes a head detection branch, a human body detection branch and a torso detection branch; the number of channels of the torso detection branch is 12*A, and the number of channels of the head detection branch and the human body detection branch is 4*A, where A represents the number of anchor frames.

[0014] Optionally, the torso detection branch takes the area of ​​the torso truth frame, the area of ​​the head truth frame, and the area of ​​the human body truth frame as input, and outputs a human body detection frame, a torso detection frame, and a head detection frame.

[0015] Optionally, an NMS method is used to remove duplicates from the torso detection frame.

[0016] To achieve the above-mentioned and other related purposes, the present invention provides a training device for a target detection model, comprising:

[0017] A picture acquisition module, used to acquire a picture including multiple objects;

[0018] An image preprocessing module is used to preprocess the image; the preprocessing module includes:

[0019] The first labeling submodule is used to label multiple objects and obtain the true value frames of the heads and the true value frames of the human bodies of the multiple objects;

[0020] A matching submodule, configured to match the head truth frame and the body truth frame to determine whether the head truth frame and the body truth frame belong to the same object;

[0021] A second labeling submodule is used to obtain a torso truth frame that matches the human body truth frame based on the human body truth frame to complete the labeling of the image;

[0022] The training module is used to train the initial neural network model using the labeled images as sample images to obtain the target detection model.

[0023] Optionally, the matching submodule matches the head truth frame and the human body truth frame according to the IOU and positional relationship between the head truth frame and the human body truth frame.

[0024] Optionally, the target detection model includes a head detection branch, a human body detection branch and a torso detection branch; the number of channels of the torso detection branch is 12*A, and the number of channels of the head detection branch and the human body detection branch is 4*A, where A represents the number of anchor frames; the torso detection branch takes the area of ​​the torso truth frame, the area of ​​the head truth frame, and the area of ​​the human body truth frame as input, and outputs the human body detection frame, the torso detection frame and the head detection frame.

[0025] Optionally, the method further includes: a deduplication module, configured to perform deduplication on the torso detection frame using an NMS method.

[0026] To achieve the above-mentioned and other related objectives, the present invention provides a target detection method, comprising:

[0027] Get the image to be detected;

[0028] Extracting features from the image to be detected to obtain human biometric features, including head features, body features, and torso features;

[0029] Based on the biometric features, the detection image is detected using the target detection model to obtain the target object.

[0030] To achieve the above-mentioned and other related objectives, the present invention provides a target detection device, comprising:

[0031] Image acquisition module, used to obtain the image to be detected;

[0032] A feature extraction module is used to extract features from the image to be detected to obtain human biological features, wherein the human biological features include: head features, body features, and torso features;

[0033] The target detection module is used to detect the detection image based on the biometric features using the target detection model to obtain the target object.

[0034] To achieve the above-mentioned and other related objectives, the present invention provides an electronic device, comprising:

[0035] one or more processors; and

[0036] One or more machine-readable media having instructions stored thereon, which, when executed by the one or more processors, cause the apparatus to perform one or more of the described methods.

[0037] To achieve the above and other related objectives, the present invention provides one or more machine-readable media having instructions stored thereon, which, when executed by one or more processors, cause a device to perform one or more of the methods described above.

[0038] As described above, the target detection model training method, device, target detection method, device, medium and equipment of the present invention have the following beneficial effects:

[0039] A training method for a target detection model of the present invention includes: obtaining a picture including multiple objects; preprocessing the picture; the preprocessing includes: labeling the multiple objects to obtain head truth frames and body truth frames of the multiple objects; matching the head truth frames and the body truth frames to determine the head truth frames and body truth frames belonging to the same object; based on the body truth frames, obtaining a torso truth frame that matches the body truth frame to complete the labeling of the picture; using the labeled picture as a sample picture to train an initial neural network model to obtain a target detection model. The target detection model of the present invention has a torso detection branch. When the torso detection branch outputs a human body detection frame (i.e., Sub1) and a head detection frame (i.e., Sub2), the human body recall rate is increased by more than 3 percentage points, and the head recall rate is increased by 0.5 percentage points. At the same time, since the human body and the head are associated, the computational complexity and time required for the association task can be saved for the downstream tracking task.

[0040] Illustrations

[0041] Figure 1 This is a flowchart of a method for training a target detection model according to an embodiment of the present invention;

[0042] Figure 2 This is a flowchart of a method for training a target detection model according to another embodiment of the present invention;

[0043] Figure 3 This is a flow chart of a training device for a target detection model according to an embodiment of the present invention;

[0044] Figure 4 This is a flow chart of a target detection method according to an embodiment of the present invention;

[0045] Figure 5 This is a flow chart of a target detection method according to another embodiment of the present invention;

[0046] Figure 6 This is a schematic diagram of the hardware structure of a target detection device according to an embodiment of the present invention;

[0047] Figure 7 A schematic diagram of the hardware structure of a terminal device in one embodiment of the present invention;

[0048] Figure 8 FIG. 1 is a schematic diagram of the hardware structure of a terminal device in one embodiment of the present invention. DETAILED DESCRIPTION

[0049] The following describes the embodiments of the present invention through specific examples. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments. The details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other unless they conflict.

[0050] It should be noted that the illustrations provided in the following embodiments are merely schematic illustrations of the basic concept of the present invention. Therefore, the illustrations only show components related to the present invention and are not drawn according to the number, shape, and size of components in actual implementation. In actual implementation, the type, quantity, and proportion of each component may be changed arbitrarily, and the component layout may also be more complex.

[0051] In fully structured pedestrian detection, detection models are typically deployed at high camera positions, such as at major thoroughfares and building entrances, inevitably encountering dense crowds of pedestrians. Closely spaced pedestrians are often largely obscured, making human detection quite challenging.

[0052] In addition, the overlapping and occluding of pedestrians in crowded scenes increases the difficulty of associating the human body and the human head, resulting in the phenomenon of "putting the head on the wrong head" being associated with the body of pedestrian B.

[0053] Missed human detection and incorrect human-head association problems will have a significant impact on subsequent tracking and recognition; therefore, it is very important to ensure the detection rate of occluded human bodies and the stability of human-head association.

[0054] There are two common approaches to detecting a human body that is partially obscured by objects or other human bodies:

[0055] First, increase the proportion of occluded human bodies during training: The most common method is to use random occlusion data augmentation, that is, before inputting the training image into the network, randomly cover a part of the human body in the image to simulate the situation where the human body is occluded by other people or objects. However, this has a major drawback. Since the shape of the area covering the human body is often designed to be rectangular, and the pixels are often designed to be fixed values, it cannot simulate the real occlusion situation; in real life, the people or objects that occlude the human body may be strange or even discontinuous. Even though this method can enhance the network's learning ability to a certain extent, making it easier to predict the whole body from the visible part, it is often helpless in the face of various real occlusion situations.

[0056] Secondly, both the visible area and the entire body are predicted simultaneously: that is, two detection frames are predicted for each person, one for the visible area of ​​the person, and the other for the entire area of ​​the person (including the occluded parts). A major problem with this is that it greatly increases the difficulty of annotation, requiring two frames to be marked for each person when annotating the data. Moreover, for occluded people, the complete body frame is even more difficult to mark, which is undoubtedly a very costly solution. To solve the above problem, during the model training process, the head and body GroundTruth frames (true value frames) belonging to the same pedestrian in the training set can be associated together, and a torso GroundTruth frame is generated for each person (the torso frame includes the head, shoulders, and upper torso of the person, ending below the waist). The torso frame can be generated by training a small detector specifically for detecting the torso, or by manual annotation. To save costs, a small amount of annotation was chosen, and the rest was generated by detection.

[0057] Taking the structured three-category pedestrian detector as an example, the three detection heads (detection branches) are responsible for predicting the human body, face, and head, respectively. Therefore, the GroundTruth frames of each category are input into the corresponding detection branch for training. Now, a torso detection branch is added. For this detection branch, not only the GroundTruth frames of the torso are input for training, but also the human body and head frames associated with the torso (that is, belonging to the same person) are fed into it.

[0058] For one branch, each positive anchor box was previously responsible for predicting four coordinate offsets, which were used to calculate the final detection frame. Now, with the added torso detection branch, each positive anchor box is responsible for predicting 12 coordinate offsets: four for predicting the torso, four for predicting the body associated with the torso, and four for predicting the head associated with the torso. The final output is a torso, body, and head detection frame, all of which belong to the same person.

[0059] In the post-processing stage, the torso detection frame is usually not output, but each human detection frame and head detection frame associated with the torso detection frame will be output, and each pair of human-head detection frames are naturally associated because they are determined to belong to the same pedestrian when they are output.

[0060] The above method allows for simple association logic to pair the training set's human bodies and heads, and a small detector to generate torso annotations, requiring virtually no additional annotation costs. Furthermore, the method improves the accuracy of human-head association by associating the human body with the head. Furthermore, the addition of a torso detection branch only adds a small amount of parameters and computation. In summary, the aforementioned steps ultimately increase the detection rate of densely packed and occluded human bodies. On a test set of three pedestrian categories, with a torso branch outputting both human and head detection boxes, the recall rate for humans increased by over 3 percentage points, and the recall rate for heads increased by 0.5 percentage points. Furthermore, because human bodies and heads are self-associated, this method can also save the computational effort and time required for association tasks in downstream tracking tasks.

[0061] According to the above description, the embodiment of the present application provides a method for training a target detection model, such as Figure 1 As shown, specifically including:

[0062] S100 acquires a picture including a plurality of objects;

[0063] S200 pre-processes the image; the pre-processing includes:

[0064] Labeling multiple objects to obtain ground truth frames of heads and bodies of the multiple objects; matching the ground truth frames of heads and bodies to determine which ground truth frames of heads and bodies belong to the same object; and obtaining ground truth frames of torsos that match the ground truth frames of bodies based on the ground truth frames to complete the labeling of the image;

[0065] S300 uses the labeled pictures as sample pictures to train the initial neural network model to obtain a target detection model.

[0066] In step S200, when annotating the images in the training set, the ground-truth human and head frames in each image are first associated, matching the ground-truth human and head frames belonging to the same person. This can be determined using simple IoU logic, size relationships, and positional relationships. A small torso detector or manual annotation is then used to generate a ground-truth torso frame for each ground-truth human frame. This completes the integration of the torso, human, and head frames for each person.

[0067] In one embodiment, matching the human head ground truth frame with the human body ground truth frame includes:

[0068] The head truth frame and the human body truth frame are matched based on their IOU and positional relationship.

[0069] Specifically, 1. If they belong to the same object, the area of ​​intersection between the head ground truth frame and the body ground truth frame is greater than a preset threshold. If the area of ​​intersection is greater than the preset threshold, it can be considered that the head ground truth frame and the body ground truth frame belong to the same object, and the match is complete; if the area of ​​intersection is less than the preset threshold, it can be considered that the head ground truth frame and the body ground truth frame do not belong to the same object. 2. Generally speaking, if they belong to the same object, the center point of the head ground truth frame is above the body ground truth frame. Therefore, the position of the head ground truth frame and the body ground truth frame can be used as a factor in matching the same object.

[0070] In one embodiment, the object detection model includes a head detection branch, a human body detection branch, and a torso detection branch; the torso detection branch has a channel number of 12*A, and the head detection branch and the human body detection branch have a channel number of 4*A, where A represents the number of anchor boxes. The torso detection branch takes the area of ​​the torso ground truth box, the area of ​​the head ground truth box, and the area of ​​the human body ground truth box as input, and outputs a human detection box, a torso detection box, and a head detection box. The object detection model can employ an anchor-based single-stage detection model such as YOLO or RetinaNet. When employing an anchor-based single-stage detection model such as YOLO or RetinaNet, the branch that predicts the detection box has a channel number of 4*A, where A represents the number of anchor boxes and 4 represents the four predicted coordinate information. These four coordinate information can be converted with the anchor box coordinates to obtain the final detection box. In particular, the torso detection branch has a channel number of 12*A. 4 coordinate information is used to predict the torso detection frame, 4 coordinate information is used to predict the head detection frame, and 4 is used to predict the human body detection frame.

[0071] In one embodiment, the training method of the target detection model includes:

[0072] The images in the training set are labeled with the true value frames of the head, body, and torso respectively, and the true value frames of the head, body, and torso belonging to the same person are associated;

[0073] The output channel of the human detection branch calculation channel of the target detection model is set to 4*A, the output channel of the head detection branch calculation channel is set to 4*A, and the output channel of the torso detection branch calculation channel is set to 12*A

[0074] The images and human ground truth box annotations in the training set are input into the human detection branch of the target detection model for training. The human detection box is output and compared with the human ground truth box. The training parameters are adjusted based on the comparison results to improve the accuracy of the human detection branch of the target detection model.

[0075] The images and ground-truth head box annotations in the training set are input into the head detection branch of the object detection model for training. The head detection box is output and compared with the ground-truth head box. The training parameters are adjusted based on the comparison results to improve the accuracy of the head detection branch of the object detection model.

[0076] The images in the training set, the ground-truth frames of the human head, the ground-truth frames of the human body associated with the ground-truth frames of the human head, and the ground-truth frames of the torso are input into the torso detection branch of the object detection model for training. The torso detection frames are output and compared with the ground-truth frames. The training parameters are adjusted based on the comparison results to improve the accuracy of the torso detection branch of the object detection model.

[0077] Get the trained pedestrian detection model.

[0078] During the training process of the target detection model, each torso detection frame will be assigned to one or more anchor frames, and these anchor frames will be used for prediction. For the human body detection frame Sub1 and the head detection frame Sub2, the step of assigning anchor frames is omitted. Since the torso detection frame and the human body detection frame Sub1 and the human body detection frame Sub2 each have 4*A channels in the 12*A channels of the torso detection branch, and the anchor frames of the human body detection frame Sub1 and the head detection frame Sub2 are one-to-one corresponding to the anchor frames of the torso detection frame, the human body detection frame Sub1 and the human body detection frame Sub2 will also belong to the corresponding anchor frame(s) to whichever anchor frame(s) the torso detection frame belongs to.

[0079] In one embodiment, the NMS (Non-Maximum Suppression) method is used to deduplicate the torso detection frames. In this embodiment, the human body detection frame Sub1 and the head detection frame Sub2 are merely dependencies of the torso detection frame. That is, for torso detection frames that are retained after NMS, their human body detection frames Sub1 and head detection frames Sub2 are also retained. For torso detection frames that are filtered out, their human body detection frames Sub1 and head detection frames Sub2 are also filtered out.

[0080] like Figure 2 As shown, an embodiment of the present application provides a method for training a target detection model, comprising the following steps:

[0081] C101: Get the original images for training.

[0082] S101: Extract image features through convolutional neural networks, going through the Backbone and FPN stages.

[0083] C102: Obtain the feature map of the regression branch in the torso detection head. The number of channels of the torso detection branch is 12*A (A is the number of anchor frames), and the number of channels of the torso frame, human body detection frame Sub1, and head detection frame Sub2 are all 4*A.

[0084] C103: Anchor box. It is a priori information in common anchor-based detectors.

[0085] C104: Head detection box annotated in the image.

[0086] C105: Human body detection box annotated in the image.

[0087] S102: Pair the head detection frame with the human body detection frame based on information such as IoU and position relationship.

[0088] S103: Generate a torso detection frame including the torso area for each human body detection frame by using a small torso detector or manual labeling.

[0089] C106: The torso detection frame, human body detection frame Sub1, and head detection frame Sub2 are obtained in groups. Each group represents the torso detection frame, human body detection frame, and head detection frame of the same person.

[0090] S104: Assign one or more anchor frames to the torso detection frame. The anchor frames of the human body detection frame Sub1 and the head detection frame Sub2 are in one-to-one correspondence with the anchor frames of the torso detection frame.

[0091] C107: Based on the corresponding relationship between the anchor frames, the coordinate prediction values ​​in the feature map and the corresponding anchor frame coordinates are used to calculate the final prediction frame (detection frame, including the torso detection frame, human body detection frame Sub1, and head detection frame Sub2);

[0092] S104: Calculate the regression loss and perform training based on the torso detection frame, human detection frame Sub1, head detection frame Sub2, and the torso ground truth frame, human ground truth frame Sub1, and head ground truth frame Sub2. Finally, a trained object detection model is obtained.

[0093] like Figure 3 As shown, an embodiment of the present application provides a training device for a target detection model, comprising:

[0094] The picture acquisition module 100 is used to acquire a picture including multiple objects;

[0095] The image preprocessing module 200 is used to preprocess the image; the preprocessing module includes:

[0096] The first labeling submodule 201 is used to label multiple objects and obtain the true value frames of the heads and the true value frames of the human bodies of the multiple objects;

[0097] A matching submodule 202 is configured to match the head ground truth frame and the body ground truth frame to determine that the head ground truth frame and the body ground truth frame belong to the same object;

[0098] The second labeling submodule 203 is configured to obtain a torso ground truth frame that matches the human body ground truth frame based on the human body ground truth frame, so as to complete the labeling of the image;

[0099] The training module 300 is used to train the initial neural network model using the labeled pictures as sample pictures to obtain a target detection model.

[0100] In one embodiment, the matching submodule matches the head truth frame and the human body truth frame according to the IOU and positional relationship between the head truth frame and the human body truth frame.

[0101] When annotating images in the training set, we first associate the ground-truth human and head frames in each image, matching the ground-truth human and head frames belonging to the same person. This is done using simple IoU logic, size relationships, and positional relationships. We then generate a ground-truth torso frame for each ground-truth human frame using a small torso detector or manual annotation. This completes the trinity of torso, body, and head frames for each person.

[0102] Specifically, 1. If they belong to the same object, the area of ​​intersection between the head ground truth frame and the body ground truth frame is greater than a preset threshold. If the area of ​​intersection is greater than the preset threshold, it can be considered that the head ground truth frame and the body ground truth frame belong to the same object, and the match is complete; if the area of ​​intersection is less than the preset threshold, it can be considered that the head ground truth frame and the body ground truth frame do not belong to the same object. 2. Generally speaking, if they belong to the same object, the center point of the head ground truth frame is above the body ground truth frame. Therefore, the position of the head ground truth frame and the body ground truth frame can be used as a factor in matching the same object.

[0103] In one embodiment, the object detection model includes a head detection branch, a human body detection branch, and a torso detection branch; the torso detection branch has a channel number of 12*A, and the head detection branch and the human body detection branch have a channel number of 4*A, where A represents the number of anchor boxes. The torso detection branch takes the area of ​​the torso ground truth box, the area of ​​the head ground truth box, and the area of ​​the human body ground truth box as input, and outputs a human detection box, a torso detection box, and a head detection box. The object detection model can employ an anchor-based single-stage detection model such as YOLO or RetinaNet. When employing an anchor-based single-stage detection model such as YOLO or RetinaNet, the branch that predicts the detection box has a channel number of 4*A, where A represents the number of anchor boxes and 4 represents the four predicted coordinate information. These four coordinate information can be converted with the anchor box coordinates to obtain the final detection box. In particular, the torso detection branch has a channel number of 12*A. 4 coordinate information is used to predict the torso detection frame, 4 coordinate information is used to predict the head detection frame, and 4 is used to predict the human body detection frame.

[0104] During the training process of the target detection model, each torso detection frame will be assigned to one or more anchor frames, and these anchor frames will be used for prediction. For the human body detection frame Sub1 and the head detection frame Sub2, the step of assigning anchor frames is omitted. Since the torso detection frame and the human body detection frame Sub1 and the human body detection frame Sub2 each have 4*A channels in the 12*A channels of the torso detection branch, and the anchor frames of the human body detection frame Sub1 and the head detection frame Sub2 are one-to-one corresponding to the anchor frames of the torso detection frame, the human body detection frame Sub1 and the human body detection frame Sub2 will also belong to the corresponding anchor frame(s) to whichever anchor frame(s) the torso detection frame belongs to.

[0105] In one embodiment, the deduplication module uses the NMS (Non-Maximum Suppression) method to deduplicate the torso detection frames. In this embodiment, the human body detection frame Sub1 and the head detection frame Sub2 are merely dependencies of the torso detection frame. That is, for torso detection frames that are retained after NMS, their human body detection frames Sub1 and head detection frames Sub2 are also retained. For torso detection frames that are filtered out, their human body detection frames Sub1 and head detection frames Sub2 are also filtered out.

[0106] Since the above-mentioned device embodiment corresponds to the method embodiment, the functional implementation of each module in the device embodiment can refer to the implementation method of the method embodiment, and will not be repeated here.

[0107] like Figure 4As shown, an embodiment of the present application provides a target detection method, including:

[0108] S400 obtains the image to be detected;

[0109] S500 performs feature extraction on the image to be detected to obtain human biological features, wherein the human biological features include: head features, body features, and torso features;

[0110] S600 detects the detection image based on the biometric feature using the target detection model to obtain a target object.

[0111] Among them, when extracting features from the image to be detected, convolutional neural network can be used for feature extraction. The target detection model is based on Figure 1 The target detection model obtained by training the steps shown is obtained.

[0112] like Figure 5 As shown, an embodiment of the present application provides a target detection method, comprising the following steps:

[0113] C101: Get the image to be detected.

[0114] S101: The convolutional neural network extracts the features of the image to be detected. After the backbone and FPN stages of the convolutional neural network, human body features are obtained, including head features, body features, and torso features.

[0115] C102: Use the torso detection branch of the object detection model and human features to perform predictions, obtaining prediction results. The prediction results include the torso confidence score and the torso detection box, the human detection box Sub1, and the head detection box Sub2.

[0116] S102: Remove duplicate detection boxes in the detection task through the non-maximum suppression method.

[0117] C103: Final torso detection box.

[0118] C104: Sub1 frame (human body frame) included with the torso detection frame.

[0119] C105: Sub2 frame (head frame) included with the torso detection frame.

[0120] C106: Paired (i.e., belonging to the same person) output of human body detection frame and head detection frame.

[0121] like Figure 6 As shown, an embodiment of the present application provides a target detection device, comprising:

[0122] The image acquisition module 400 is used to acquire the image to be detected;

[0123] A feature extraction module 500 is used to extract features from the image to be detected to obtain human biometric features, wherein the human biometric features include: head features, body features, and torso features;

[0124] The target detection module 600 is used to detect the detection image based on the biometric features using the target detection model to obtain the target object.

[0125] Since the above-mentioned device embodiment corresponds to the method embodiment, the functional implementation of each module in the device embodiment can refer to the implementation method of the method embodiment, and will not be repeated here.

[0126] The embodiment of the present application also provides an electronic device, which may include: one or more processors; and one or more machine-readable media having instructions stored thereon, which, when executed by the one or more processors, enable the device to perform Figure 1 In practical applications, the device can be used as a terminal device or as a server. Examples of terminal devices may include: smartphones, tablet computers, e-book readers, MP3 (Moving Picture Experts Group Audio Layer III) players, MP4 (Moving Picture Experts Group Audio Layer IV) players, laptop computers, car computers, desktop computers, set-top boxes, smart TVs, wearable devices, etc. The embodiments of this application do not limit the specific devices.

[0127] The present application also provides a non-volatile readable storage medium, which stores one or more modules (programs). When the one or more modules are applied to a device, the device can execute the embodiment of the present application. Figure 1 Instructions for the steps in a method.

[0128] Figure 7 A schematic diagram of the hardware structure of a terminal device provided in one embodiment of the present application. As shown in the figure, the terminal device may include: an input device 1100, a first processor 1101, an output device 1102, a first memory 1103, and at least one communication bus 1104. Communication bus 1104 is used to achieve communication connections between components. First memory 1103 may include high-speed RAM memory, and may also include non-volatile storage NVM, such as at least one disk storage device. Various programs can be stored in first memory 1103 to perform various processing functions and implement the method steps of this embodiment.

[0129] Optionally, the first processor 1101 may be implemented as, for example, a central processing unit (CPU), an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a controller, a microcontroller, a microprocessor or other electronic components, and the first processor 1101 is coupled to the input device 1100 and the output device 1102 via a wired or wireless connection.

[0130] Optionally, the input device 1100 may include multiple input devices, such as at least one of a user interface for a user, a device interface for a device, a software programmable interface, a camera, and a sensor. Optionally, the device interface for a device may be a wired interface for data transmission between devices, or a hardware plug-in interface for data transmission between devices (such as a USB interface, a serial port, etc.); optionally, the user interface for a user may be, for example, a user-oriented control button, a voice input device for receiving voice input, and a touch sensing device for receiving user touch input (such as a touch screen or touchpad with touch sensing function); optionally, the software programmable interface may be, for example, an entry for a user to edit or modify a program, such as an input pin interface or input interface of a chip; the output device 1102 may include output devices such as a display and a speaker.

[0131] In this embodiment, the processor of the terminal device includes a processor for executing the functions of each module in each device. The specific functions and technical effects can be referred to the above embodiments and will not be repeated here.

[0132] Figure 8 A schematic diagram of the hardware structure of a terminal device provided for one embodiment of the present application. Figure 8 Yes Figure 7 A specific embodiment in the implementation process. As shown in the figure, the terminal device of this embodiment may include a second processor 1201 and a second memory 1202.

[0133] The second processor 1201 executes the computer program code stored in the second memory 1202 to implement the above embodiment. Figure 1 The method.

[0134] The second memory 1202 is configured to store various types of data to support operations on the terminal device. Examples of such data include instructions for any application or method operating on the terminal device, such as messages, images, videos, etc. The second memory 1202 may include random access memory (RAM) and may also include non-volatile memory (non-volatile memory), such as at least one disk storage device.

[0135] Optionally, the second processor 1201 is provided in the processing component 1200. The terminal device may further include: a communication component 1203, a power component 1204, a multimedia component 1205, a voice component 1206, an input / output interface 1207, and / or a sensor component 1208. The specific components included in the terminal device are set according to actual needs and are not limited in this embodiment.

[0136] The processing component 1200 generally controls the overall operation of the terminal device. The processing component 1200 may include one or more second processors 1201 to execute instructions to complete all or part of the steps in the above-described data processing method. In addition, the processing component 1200 may include one or more modules to facilitate interaction between the processing component 1200 and other components. For example, the processing component 1200 may include a multimedia module to facilitate interaction between the multimedia component 1205 and the processing component 1200.

[0137] The power supply component 1204 provides power to various components of the terminal device. The power supply component 1204 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the terminal device.

[0138] The multimedia component 1205 includes a display screen that provides an output interface between the terminal device and the user. In some embodiments, the display screen may include a liquid crystal display (LCD) and a touch panel (TP). If the display screen includes a touch panel, the display screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor can not only sense the boundaries of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation.

[0139] The voice component 1206 is configured to output and / or input voice signals. For example, the voice component 1206 includes a microphone (MIC), which is configured to receive external voice signals when the terminal device is in an operating mode, such as a voice recognition mode. The received voice signals can be further stored in the second memory 1202 or transmitted via the communication component 1203. In some embodiments, the voice component 1206 also includes a speaker for outputting voice signals.

[0140] The input / output interface 1207 provides an interface between the processing component 1200 and peripheral interface modules, which may be click wheels, buttons, etc. These buttons may include but are not limited to: volume buttons, start buttons, and lock buttons.

[0141] Sensor component 1208 includes one or more sensors for providing various status assessments for the terminal device. For example, sensor component 1208 can detect the open / closed state of the terminal device, the relative positioning of components, and the presence or absence of user contact with the terminal device. Sensor component 1208 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact, including detecting the distance between the user and the terminal device. In some embodiments, sensor component 1208 may also include a camera, etc.

[0142] The communication component 1203 is configured to facilitate wired or wireless communication between the terminal device and other devices. The terminal device can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In one embodiment, the terminal device may include a SIM card slot for inserting a SIM card, allowing the terminal device to log into a GPRS network and establish communication with a server via the Internet.

[0143] From the above, we can see that Figure 8 The communication component 1203, the voice component 1206, the input / output interface 1207, and the sensor component 1208 involved in the embodiment can all be used as Figure 7 Implementation of the input device in the embodiment.

[0144] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the present invention. Anyone skilled in the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by one of ordinary skill in the art without departing from the spirit and technical principles disclosed herein are intended to be covered by the claims of the present invention.

Claims

1. A method for training a target detection model, characterized in that: include: Get a picture including multiple objects; Preprocessing the image; The pretreatment includes: Labeling multiple objects to obtain ground truth frames of heads and bodies of the multiple objects; matching the ground truth frames of heads and bodies to determine which ground truth frames of heads and bodies belong to the same object; and obtaining ground truth frames of torsos that match the ground truth frames of bodies based on the ground truth frames to complete the labeling of the image; The labeled images are used as sample images to train the initial neural network model to obtain the target detection model; The target detection model includes a head detection branch, a human body detection branch, and a torso detection branch; the number of channels of the torso detection branch is 12*A, and the number of channels of the head detection branch and the human body detection branch is 4*A, where A represents the number of anchor frames; The torso detection branch takes the area of ​​the torso truth frame, the area of ​​the head truth frame, and the area of ​​the human body truth frame as input, and outputs the human body detection frame, the torso detection frame, and the head detection frame; the area of ​​the torso truth frame includes the head, shoulders, and the area from the upper body to below the waist.

2. The target detection model training method according to claim 1, characterized in that: The matching of the head truth frame and the human body truth frame includes: The head truth frame and the human body truth frame are matched based on their IOU and positional relationship.

3. The target detection model training method according to claim 1, characterized in that: The NMS method is used to remove duplicates from the torso detection frame.

4. A training device for a target detection model, characterized in that: include: A picture acquisition module, used to acquire a picture including multiple objects; An image preprocessing module, used for preprocessing the image; The pre-processing module comprises: The first labeling submodule is used to label multiple objects and obtain the true value frames of the heads and the true value frames of the human bodies of the multiple objects; A matching submodule, configured to match the head truth frame and the body truth frame to determine whether the head truth frame and the body truth frame belong to the same object; A second labeling submodule is used to obtain a torso truth frame that matches the human body truth frame based on the human body truth frame to complete the labeling of the image; The training module is used to train the initial neural network model using the labeled images as sample images to obtain the target detection model; The target detection model includes a head detection branch, a human body detection branch and a torso detection branch; the number of channels of the torso detection branch is 12*A, and the number of channels of the head detection branch and the human body detection branch is 4*A, where A represents the number of anchor frames; the torso detection branch takes the area of ​​the torso truth frame, the area of ​​the head truth frame and the area of ​​the human body truth frame as input, and outputs the human body detection frame, the torso detection frame and the head detection frame; the area of ​​the torso truth frame includes the head, shoulders and the area from the upper body to below the waist.

5. The target detection model training device according to claim 4, characterized in that: The matching submodule matches the head truth frame and the human body truth frame according to the IOU and position relationship between the head truth frame and the human body truth frame.

6. The target detection model training device according to claim 4, characterized in that: Also includes: The deduplication module is used to remove duplicates from the torso detection frame using the NMS method.

7. A target detection method, characterized in that: include: Get the image to be detected; Extracting features from the image to be detected to obtain human biometric features, including head features, body features, and torso features; Based on the biometric features, the detection image is detected using the target detection model described in any one of claims 1 to 3 to obtain a target object.

8. A target detection device, characterized in that: include: Image acquisition module, used to obtain the image to be detected; A feature extraction module is used to extract features from the image to be detected to obtain human biological features, wherein the human biological features include: head features, body features, and torso features; The target detection module is used to detect the detection image based on the biometric feature using the target detection model described in any one of claims 1 to 3 to obtain the target object.

9. An electronic device, characterized in that: include: one or more processors; and One or more machine-readable media having instructions stored thereon, which, when executed by the one or more processors, cause the apparatus to perform the method of one or more of claims 1-3 or 7.

10. One or more machine-readable media, characterized in that Instructions are stored thereon, which, when executed by one or more processors, cause the device to perform the method as claimed in one or more of claims 1-3 or 7.

Images