Target object detection method and apparatus

Abstract

Embodiments of the present specification provide a target object detection method and device, wherein the target object detection method comprises: inputting a picture carrying a target object into an object detection model to obtain an initial detection box of the target object in the picture; filtering the initial detection box based on a preset filtering method to obtain a target detection box of the target object; and rotating the target object in the target detection box to a preset angle to obtain the target object in the target detection box after rotation.

Description

Technical Field

[0001] This specification relates to the field of computer technology, and in particular to a method for detecting target objects. One or more embodiments of this specification also relate to a method for training an object detection model, a method for detecting key points, a method for training a key point detection model, a target object detection device, a key point detection device, an object detection model training device, a key point detection model training device, a computing device, and a computer-readable storage medium. Background Technology

[0002] In image-based object detection systems, situations often arise where the target object has a large rotation angle, such as the detection of faces, pedestrians, and cars. If the user-input image is flipped vertically, a system that only detects objects rotating between 0 and 90 degrees will struggle to detect all objects in the image. Taking faces as an example, in face detection systems, detecting faces rotated at any angle within a plane is sometimes essential, especially for cloud service APIs. These APIs have diverse users, and uploaded images may contain faces facing various directions, while the uploaded images may lack orientation information. Currently, there is no method that can achieve both high detection speed and strong robustness for face detection across different orientations. Summary of the Invention

[0003] In view of this, embodiments of this specification provide a target object detection method. One or more embodiments of this specification also relate to an object detection model training method, a keypoint detection method, a keypoint detection model training method, a target object detection device, a keypoint detection device, an object detection model training device, a keypoint detection model training device, a computing device, and a computer-readable storage medium, to address the technical deficiencies existing in the prior art.

[0004] According to a first aspect of the embodiments of this specification, a target object detection method is provided, comprising:

[0005] The image carrying the target object is input into the object detection model to obtain the initial detection box of the target object in the image;

[0006] The initial detection box is filtered based on a preset filtering method to obtain the target detection box of the target object;

[0007] Rotate the target object in the target detection box to a preset angle to obtain the rotated target object in the target detection box.

[0008] According to a second aspect of the embodiments of this specification, a method for training an object detection model is provided, comprising:

[0009] Acquire sample images and label detection boxes for initial objects in the sample images;

[0010] The sample image is input into the feature extraction model at an initial angle to obtain the initial feature map of the sample image at the initial angle and the initial detection box corresponding to the initial object in the initial feature map;

[0011] The initial feature map is rotated according to a preset rotation rule to obtain a rotated feature map and a rotation detection box corresponding to the rotated object in the rotated feature map;

[0012] The object detection model is trained based on the initial feature map, the rotated feature map, the initial detection box, and the rotated detection box to obtain the object detection model.

[0013] According to a third aspect of the embodiments of this specification, a key point detection method is provided, comprising:

[0014] The image carrying the target object is input into the object detection model to obtain the initial detection box of the target object in the image;

[0015] The initial detection box is filtered based on a preset filtering method to obtain the target detection box of the target object;

[0016] The target object in the target detection box is rotated to a preset angle, and the rotated target detection box is input into the key point detection model to obtain the key points of the target object in the target detection box.

[0017] The object detection model is trained using the object detection model training method described above.

[0018] According to a fourth aspect of the embodiments of this specification, a keypoint detection model training method is provided, comprising:

[0019] Acquire a sample image, input the sample image into the object detection model, and obtain the initial detection box of the target object in the sample image;

[0020] The initial detection box is filtered based on a preset filtering method to obtain the target detection box of the target object;

[0021] Rotate the target object in the target detection box to a preset angle and mark key points on the target object;

[0022] The keypoint detection model is obtained by training based on the target object and the key points corresponding to the target object.

[0023] The object detection model is trained using the object detection model training method described above.

[0024] According to a fifth aspect of the embodiments of this specification, a target object detection method is provided, comprising:

[0025] Based on the user's request, display the image input interface to the user;

[0026] Obtain the image containing the target object that the user inputs based on the image input interface;

[0027] The image is input into an object detection model to obtain the initial detection bounding box of the target object in the image;

[0028] The initial detection box is filtered based on a preset filtering method to obtain the target detection box of the target object;

[0029] The target object in the target detection box is rotated to a preset angle, and the rotated target object in the target detection box is obtained and returned to the user.

[0030] According to a sixth aspect of the embodiments of this specification, a target object detection method is provided, comprising:

[0031] Receive a call request sent by the user, wherein the call request carries an image containing the target object;

[0032] The image is input into an object detection model to obtain the initial detection bounding box of the target object in the image;

[0033] The initial detection box is filtered based on a preset filtering method to obtain the target detection box of the target object;

[0034] The target object in the target detection box is rotated to a preset angle, and the rotated target object in the target detection box is obtained and returned to the user.

[0035] According to a seventh aspect of the embodiments of this specification, a key point detection method is provided, comprising:

[0036] Based on the user's request, display an image input interface to the user and obtain the image containing the target object that the user inputs based on the image input interface;

[0037] The image is input into an object detection model to obtain the initial detection bounding box of the target object in the image;

[0038] The initial detection box is filtered based on a preset filtering method to obtain the target detection box of the target object;

[0039] The target object in the target detection box is rotated to a preset angle, and the rotated target detection box is input into the key point detection model to obtain the key points of the target object in the target detection box.

[0040] The object detection model is trained using the object detection model training method described above.

[0041] According to an eighth aspect of the embodiments of this specification, a key point detection method is provided, comprising:

[0042] Receive a call request sent by the user, wherein the call request carries an image containing the target object;

[0043] The image is input into an object detection model to obtain the initial detection bounding box of the target object in the image;

[0044] The initial detection box is filtered based on a preset filtering method to obtain the target detection box of the target object;

[0045] The target object in the target detection box is rotated to a preset angle, and the rotated target detection box is input into the key point detection model to obtain the key points of the target object in the target detection box.

[0046] The object detection model is trained using the object detection model training method described above.

[0047] According to a ninth aspect of the embodiments of this specification, a method for training an object detection model is provided, comprising:

[0048] Based on the user's request, display the image input interface to the user;

[0049] Obtain the sample image input by the user based on the image input interface, and label the initial objects in the sample image with detection boxes;

[0050] The sample image is input into the feature extraction model at an initial angle to obtain the initial feature map of the sample image at the initial angle and the initial detection box corresponding to the initial object in the initial feature map;

[0051] The initial feature map is rotated according to a preset rotation rule to obtain a rotated feature map and a rotation detection box corresponding to the rotated object in the rotated feature map;

[0052] The object detection model is trained based on the initial feature map, the rotated feature map, the initial detection box, and the rotated detection box to obtain the object detection model and return it to the user.

[0053] According to a tenth aspect of the embodiments of this specification, a method for training an object detection model is provided, comprising:

[0054] Receive a call request sent by the user, wherein the call request carries a sample image;

[0055] Label the initial objects in the sample image with detection boxes;

[0056] The sample image is input into the feature extraction model at an initial angle to obtain the initial feature map of the sample image at the initial angle and the initial detection box corresponding to the initial object in the initial feature map;

[0057] The initial feature map is rotated according to a preset rotation rule to obtain a rotated feature map and a rotation detection box corresponding to the rotated object in the rotated feature map;

[0058] The object detection model is trained based on the initial feature map, the rotated feature map, the initial detection box, and the rotated detection box to obtain the object detection model and return it to the user.

[0059] According to the eleventh aspect of the embodiments of this specification, a key point detection model training method is provided, including:

[0060] Based on the user's request, display the image input interface to the user;

[0061] Obtain the sample image input by the user based on the image input interface, input the sample image into the object detection model, and obtain the initial detection box of the target object in the sample image;

[0062] The initial detection box is filtered based on a preset filtering method to obtain the target detection box of the target object;

[0063] Rotate the target object in the target detection box to a preset angle and mark key points on the target object;

[0064] The keypoint detection model is obtained by training based on the target object and the key points corresponding to the target object.

[0065] The object detection model is trained using the object detection model training method described above and then returned to the user.

[0066] According to the twelfth aspect of the embodiments of this specification, a keypoint detection model training method is provided, comprising:

[0067] Receive a call request sent by the user, wherein the call request carries a sample image;

[0068] The sample image is input into the object detection model to obtain the initial detection bounding box of the target object in the sample image;

[0069] The initial detection box is filtered based on a preset filtering method to obtain the target detection box of the target object;

[0070] Rotate the target object in the target detection box to a preset angle and mark key points on the target object;

[0071] The keypoint detection model is obtained by training based on the target object and the key points corresponding to the target object.

[0072] The object detection model is trained using the object detection model training method described above and then returned to the user.

[0073] According to a thirteenth aspect of the embodiments of this specification, a target object detection apparatus is provided, comprising:

[0074] The detection box acquisition module is configured to input an image carrying a target object into an object detection model to obtain an initial detection box of the target object in the image.

[0075] The detection box filtering module is configured to filter the initial detection box based on a preset filtering method to obtain the target detection box of the target object;

[0076] The object acquisition module is configured to rotate the target object in the target detection box to a preset angle to obtain the target object in the target detection box after rotation.

[0077] According to a fourteenth aspect of the embodiments of this specification, a key point detection device is provided, comprising:

[0078] The detection box acquisition module is configured to input an image carrying a target object into an object detection model to obtain an initial detection box of the target object in the image.

[0079] The detection box filtering module is configured to filter the initial detection box based on a preset filtering method to obtain the target detection box of the target object;

[0080] The key point acquisition module is configured to rotate the target object in the target detection box to a preset angle, input the rotated target detection box into the key point detection model, and obtain the key points of the target object in the target detection box.

[0081] The object detection model is trained using the object detection model training method described above.

[0082] According to the fifteenth aspect of the embodiments of this specification, an object detection model training apparatus is provided, comprising:

[0083] The sample acquisition module is configured to acquire sample images and label detection boxes for initial objects in the sample images;

[0084] The feature map acquisition module is configured to input the sample image into the feature extraction model at an initial angle to obtain an initial feature map of the sample image at the initial angle and an initial detection box corresponding to the initial object in the initial feature map;

[0085] The rotation module is configured to rotate the initial feature map according to a preset rotation rule to obtain a rotated feature map and a rotation detection box corresponding to the rotated object in the rotated feature map;

[0086] The model training module is configured to train the object detection model based on the initial feature map, the rotated feature map, the initial detection box, and the rotated detection box to obtain the object detection model.

[0087] According to a sixteenth aspect of the embodiments of this specification, a key point detection model training apparatus is provided, comprising:

[0088] The detection bounding box acquisition module is configured to acquire sample images, input the sample images into the object detection model, and obtain the initial detection bounding boxes of the target objects in the sample images.

[0089] The detection box filtering module is configured to filter the initial detection box based on a preset filtering method to obtain the target detection box of the target object;

[0090] The object rotation module is configured to rotate the target object in the target detection box to a preset angle and mark key points for the target object;

[0091] The model training module is configured to train the key point detection model based on the target object and the key points corresponding to the target object.

[0092] The object detection model is trained using the object detection model training method described above.

[0093] According to a seventeenth aspect of the embodiments of this specification, a computing device is provided, comprising:

[0094] Memory and processor;

[0095] The memory is used to store computer-executable instructions, which, when executed by the processor, implement the above-described object detection model training method, the above-described key point detection model training method, the above-described target object detection method, and the steps of the above-described key point detection method.

[0096] According to an eighteenth aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores computer-executable instructions, which, when executed by a processor, implement the above-described object detection model training method, the above-described keypoint detection model training method, the above-described target object detection method, and the steps of the above-described keypoint detection method.

[0097] One embodiment of this specification implements an object detection model training method, including acquiring sample images and labeling detection boxes for initial objects in the sample images; inputting the sample images into a feature extraction model at an initial angle to obtain an initial feature map of the sample images at the initial angle and initial detection boxes corresponding to the initial objects in the initial feature map; rotating the initial feature map according to a preset rotation rule to obtain a rotated feature map and rotated detection boxes corresponding to the rotated objects in the rotated feature map; and training the object detection model based on the initial feature map, the rotated feature map, the initial detection boxes, and the rotated detection boxes to obtain the object detection model. Specifically, the object detection model training method obtains a rotated feature map by rotating the initial feature map according to a preset rotation rule with a very small increase in computation. The object detection model is then trained based on this initial feature map and the rotated feature map, improving the training efficiency of the object detection model. This method enables the trained object detection model to quickly and accurately detect the detection boxes of target objects in images during subsequent applications, improving the user experience. Attached Figure Description

[0098] Figure 1a This is an example diagram illustrating a specific application scenario of an object detection model training method provided in one embodiment of this specification;

[0099] Figure 1b This is an example diagram illustrating a specific application scenario of a target object detection method provided in one embodiment of this specification;

[0100] Figure 2 This is a flowchart of a first object detection model training method provided in one embodiment of this specification;

[0101] Figure 3 This is a schematic diagram of sample images in an object detection model training method provided in one embodiment of this specification;

[0102] Figure 4This is a flowchart of a first target object detection method provided in one embodiment of this specification;

[0103] Figure 5 This is a flowchart illustrating the processing steps of a first keypoint detection model training method provided in one embodiment of this specification.

[0104] Figure 6 This is a flowchart of a first key point detection method provided in one embodiment of this specification;

[0105] Figure 7 This is a schematic diagram of the structure of an object detection model training device provided in one embodiment of this specification;

[0106] Figure 8 This is a schematic diagram of the structure of a key point detection model training device provided in one embodiment of this specification;

[0107] Figure 9 This is a schematic diagram of the structure of a target object detection device provided in one embodiment of this specification;

[0108] Figure 10 This is a schematic diagram of a key point detection device provided in one embodiment of this specification;

[0109] Figure 11 This is a structural block diagram of a computing device provided in one embodiment of this specification. Detailed Implementation

[0110] Many specific details are set forth in the following description to provide a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0111] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a,” “described,” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0112] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0113] First, the terms and concepts used in one or more embodiments of this specification will be explained.

[0114] NMS: Non-Maximum Suppression, as the name suggests, suppresses elements that are not maxima. In object detection, it extracts high-confidence bounding boxes while suppressing low-confidence false positives. Generally, it's used when a parsing model outputs numerous bounding boxes, the exact number determined by the number of anchors; many of these boxes may be duplicates pointing to the same target. NMS removes these duplicates to obtain the true target bounding boxes. For example, in an image with many bounding boxes for objects like people, horses, and cars, NMS can obtain unique detection boxes.

[0115] API: Application Programming Interface.

[0116] In practice, there are two main methods for detecting objects at arbitrary rotation angles within a plane. The first is through cascading, rotating the objects in the image step by step into objects within the detection plane at angles of 0-90 degrees. The second is through data augmentation, training the detection model to detect objects at arbitrary angles. However, the first method requires a multi-level model, rotating each object step by step, and the time consumption is directly proportional to the number of objects; when there are many objects in the image, the time consumption becomes significant. The second method requires a detection model with strong feature representation capabilities, resulting in a larger computational load and higher training costs.

[0117] This specification provides various object detection model training methods. One or more embodiments of this specification simultaneously relate to multiple keypoint detection model training methods, multiple target object detection methods, multiple keypoint detection methods, an object detection model training device, a keypoint detection model training device, a target object detection device, a keypoint detection device, a computing device, and a computer-readable storage medium, which will be described in detail in the following embodiments.

[0118] See Figure 1a , Figure 1a The diagram illustrates a specific application scenario of an object detection model training method provided in one embodiment of this specification.

[0119] Figure 1a The application scenarios include terminal 102 and server 104. Specifically, taking an image as an example, the user obtains an image a containing faces from various angles, and marks detection boxes for face 1, face 2, face 3, face 4, and face 5 in image a. The image a is then sent to server 104 through terminal 102.

[0120] After receiving image a, server 104 inputs image a into feature extraction model M. Feature extraction model M can only process frontal faces within a -90 to 90 degree angle. Therefore, when image a is input into feature extraction model M, face 1 and face 2 in image a are frontal faces.

[0121] The feature extraction model M outputs a frontal face feature map f0 of image a (-90-90 degrees) and detection boxes y0 for frontal faces 1 and 2 within feature map f0. The feature map rotation module rotates feature map f0 90 degrees clockwise to obtain feature map f1 and detection boxes y1 for faces 2 and 3 within feature map f1. At this point, faces 2 and 3 in feature map f1 become frontal faces. Similarly, feature map f0 is rotated clockwise by 180 degrees and 270 degrees to obtain feature maps f2 and f3, as well as detection boxes y2 for face 4 in feature map f2 and y3 for faces 5 and 1 in feature map f3. Here, face 4 in feature map f2 is a frontal face, and faces 5 and 1 in feature map f3 are frontal faces.

[0122] Based on f0, f1, f2, f3 and y0, y1, y2, y3, calculate the regression loss function loss_reg and the classification loss function loss_cls, and train to obtain the object detection model.

[0123] In practical applications, users obtain multiple images containing the target object (such as a human face). These multiple images are used as training samples to train the object detection model in the manner described above.

[0124] In the embodiments described in this specification, feature map extraction can be performed on the image only once, followed by rotation of the feature map at any angle. This allows for the acquisition of feature maps of target objects in each direction within the image. Based on these feature maps, an object detection model can be trained. In subsequent applications, this model only requires a single feature map extraction of the input image to quickly obtain detection boxes for target objects in each direction, thus identifying the target objects. Compared to detection models that first rotate the image and then extract features, this method requires significantly less computation and is much faster.

[0125] See Figure 1b , Figure 1b The diagram illustrates a specific application scenario of a target object detection method provided in one embodiment of this specification.

[0126] Figure 1b The application scenarios include terminal 1022 and server 1044. Specifically, taking an image as an example, the user obtains an image b containing faces from various angles and sends image b to server 1044 through terminal 1022.

[0127] After receiving image b, server 1044 inputs image b into the object detection model, wherein the object detection model uses the above... Figure 1a The training method is as follows. Specifically, after inputting image b into the object detection model, the feature extraction model M in the object detection model extracts features from image b at angles of -90 to 90 degrees. The feature extraction model M outputs a forward face feature map f0 of image b at angles of -90 to 90 degrees, and displays the forward face feature map f0 on the terminal 1022. The forward face feature map f0 includes forward face 1 and forward face 2.

[0128] The feature map rotation module in the object detection model rotates feature map f0 90 degrees clockwise to obtain feature map f1, which is then displayed on terminal 1022. Feature map f1 includes frontal face 2 and frontal face 3. Similarly, feature map f0 is rotated 180 degrees and 270 degrees clockwise to obtain feature maps f2 and f3, which are also displayed on terminal 1022. Feature map f2 includes frontal face 4, and feature map f3 includes frontal face 5 and frontal face 1.

[0129] Finally, the object detection model outputs the face detection bounding boxes for face 1, face 2, face 3, face 4, and face 5 in image b.

[0130] See Figure 2 , Figure 2A flowchart of a first object detection model training method according to an embodiment of this specification is shown, which specifically includes the following steps.

[0131] Step 202: Obtain sample images and label the initial objects in the sample images with detection boxes.

[0132] The sample image can be understood as an image obtained through any means, such as a screenshot, a photograph, an image downloaded from the internet, or a hand-drawn image, etc. The initial object can be understood as any type of object, such as a person, a car, a pet, etc.

[0133] In practical applications, when the initial object is a person, the object detection model trained using this method can output bounding boxes for the person in the image. When the initial object is a car, the model can output bounding boxes for the car in the image. When the initial object is a pet, the model can output bounding boxes for the pet in the image.

[0134] For ease of understanding, the embodiments in this specification all use images containing human faces as sample images, with the initial object being a face, to provide a detailed introduction to the object detection model training method.

[0135] Specifically, acquiring sample images and labeling detection boxes for initial objects in the sample images can be understood as acquiring multiple sample images and labeling detection boxes for each initial object in each sample image.

[0136] See Figure 3 , Figure 3 A schematic diagram of sample images is shown in an object detection model training method provided according to an embodiment of this specification.

[0137] Figure 3 This is a sample image containing faces, specifically face 1, face 2, face 3, face 4, and face 5.

[0138] Taking face 1, face 2, face 3, face 4, and face 5 as initial objects as an example, we obtain sample images and annotate detection boxes for the initial objects in the sample images. This can be understood as obtaining... Figure 3 The sample image is used to label the detection boxes for face 1, face 2, face 3, face 4, and face 5 in the sample image.

[0139] Step 204: Input the sample image into the feature extraction model at the initial angle to obtain the initial feature map of the sample image at the initial angle and the initial detection box corresponding to the initial object in the initial feature map.

[0140] The initial angle can be understood as the positive angle of the sample image, still based on... Figure 3 For example, the initial angle is -90 to 90 degrees, which is the original horizontal angle of the sample image.

[0141] Specifically, the sample image is first input into the feature extraction model at its original horizontal angle at the time of acquisition to obtain the initial feature map of the sample image at the original horizontal angle and the initial detection box corresponding to the initial object in the initial feature map. Here, the feature extraction model can be understood as a layer in the object detection model; and the initial object in the initial feature map can be understood as the initial object in the initial feature map corresponding to the initial angle.

[0142] In specific implementation, obtaining the initial feature map of the sample image at the initial angle and the initial detection box corresponding to the initial object in the initial feature map includes:

[0143] Obtain the initial feature map of the sample image at the initial angle, and the initial detection box of the initial object in the initial feature map at the initial angle.

[0144] In practical applications, the initial feature map of the sample image at the initial angle, the initial object determined according to the initial angle in the initial feature map, and the initial detection box corresponding to the initial object are obtained.

[0145] Using the previous example, the initial objects in the initial feature map can be understood as objects 1 and 2 facing forward at the initial angle. Then, the initial detection boxes corresponding to the initial objects can be understood as the initial detection boxes of object 1 and object 2.

[0146] In the embodiments of this specification, an initial feature map of each sample image and an initial detection box corresponding to the initial object in the initial feature map of each sample image are first obtained. Subsequently, the initial feature map can be rotated to quickly obtain a rotated feature map corresponding to the initial feature map and a rotated detection box corresponding to the initial object in the rotated feature map.

[0147] Step 206: Rotate the initial feature map according to a preset rotation rule to obtain a rotated feature map and a rotation detection box corresponding to the rotated object in the rotated feature map.

[0148] The preset rotation rules can be set according to actual applications, and this manual does not impose any restrictions on them. For example, rotation can be performed according to four preset rotation angles.

[0149] Specifically, the initial feature map is rotated according to a preset rotation rule to obtain a rotated feature map, and rotation detection boxes corresponding to the rotated objects in the rotated feature map. The rotated objects in the rotated feature map can be understood as objects rotating in the positive direction according to the rotation angle.

[0150] Continuing with the previous example, if the initial feature map is rotated by 90 degrees, then... Figure 3 In the rotation feature map obtained after rotation, faces 2 and 3 are oriented positively. Therefore, the rotated objects in the rotation feature map can be understood as faces 2 and 3. At this time, the rotation detection boxes corresponding to the rotated objects in the rotation feature map are the rotation detection boxes of face 2 and face 3, respectively.

[0151] In specific implementation, rotating the initial feature map according to a preset rotation rule to obtain a rotated feature map and a rotation detection box corresponding to the rotated object in the rotated feature map includes:

[0152] The initial feature map is rotated according to the rotation angle to obtain at least one rotated feature map and a rotation detection box corresponding to the initial object in each rotated feature map, wherein the sum of the initial angle and the rotation angle is 360 degrees.

[0153] The entity that rotates the initial feature map according to the rotation angle can be the feature map rotation module in the object detection model. In practical applications, the feature map rotation module can also be regarded as a layer in the object detection model. It mainly rotates the initial feature map according to the preset rotation angle to obtain the rotated feature map of the sample image at any angle.

[0154] Specifically, the initial feature map is rotated by a rotation angle so that each target object in the initial feature map is positive in its rotated feature map. For example, if an image contains multiple faces, and each face is at a different angle, the image needs to be rotated by multiple rotation angles to ensure that each target object in the initial feature map is positive in its rotated feature map.

[0155] Finally, the initial feature map of the image is rotated 360 degrees to ensure that the rotated objects in each rotated feature map are in the positive orientation, thus ensuring that the object detection model trained subsequently can accurately detect the bounding boxes of target objects at any angle in the image.

[0156] In practical applications, the 360-degree space in a plane can be divided into K overlapping parts. Taking K=4 as an example, the plane can be divided into four subspaces: -90-90, 0-180, 90-270, and 180-360 degrees. The feature extraction model of the object detection model only needs to have good feature extraction capabilities in one subspace. For the other three subspaces, the feature map extracted by the feature extraction model in the first subspace can be rotated according to the angles of the other three subspaces to obtain the feature maps of the other three spaces. Then, the object detection model can be trained based on the feature maps of these four spaces. The specific implementation method is as follows:

[0157] The step of rotating the initial feature map according to a rotation angle to obtain at least one rotated feature map and a rotation detection box corresponding to the initial object in each rotated feature map includes:

[0158] The initial feature map is rotated according to a first rotation angle to obtain a first rotated feature map of the initial feature map at the first rotation angle and a first rotation detection box corresponding to the rotated object in the first rotated feature map;

[0159] The initial feature map is rotated according to the second rotation angle to obtain a second rotated feature map of the initial feature map at the second rotation angle and a second rotation detection box corresponding to the rotated object in the second rotated feature map;

[0160] The initial feature map is rotated according to a third rotation angle to obtain a third rotation feature map of the initial feature map at the third rotation angle and a third rotation detection box corresponding to the rotated object in the third rotation feature map.

[0161] The initial angle, first rotation angle, second rotation angle, and third rotation angle add up to 360 degrees. In practical applications, sample images are generally regular squares. Therefore, after extracting the initial feature map from the sample image with the horizontal angle (-90-90) using the feature extraction model, the initial feature map can be rotated by three horizontal angles. This allows for the extraction of the positive rotation feature map of each target object in the sample image.

[0162] Using the initial feature map as Figure 3 Taking the initial feature map of the image as an example, with a first rotation angle of 90 degrees, a second rotation angle of 180 degrees, and a third rotation angle of 270 degrees, this paper provides a detailed explanation of how to rotate the initial feature map according to the rotation angle to obtain at least one rotated feature map and the rotation detection box corresponding to the initial object in each rotated feature map.

[0163] The initial feature map is rotated by 90 degrees to obtain a first rotated feature map of the initial feature map at 90 degrees, as well as the rotated object face 2 and the first rotation detection box corresponding to face 2 in the first rotated feature map;

[0164] The initial feature map is rotated by 180 degrees to obtain a second rotated feature map of the initial feature map at 180 degrees and a second rotation detection box corresponding to the rotated object face 4 in the second rotated feature map;

[0165] The initial feature map is rotated by 270 degrees to obtain a third rotated feature map of the initial feature map at 270 degrees, and a third rotation detection box corresponding to the rotated objects face 5 and face 1 in the third rotated feature map.

[0166] In the embodiments of this specification, by rotating the initial feature map according to the first rotation angle, the second rotation angle, and the third rotation angle, the rotation feature map of the target object in the sample image at any angle in the 360-degree space in the plane can be quickly obtained. With the above method, only a small feature extraction model needs to achieve good detection results in the first subspace, and then the same good detection results can be achieved in the other three subspaces. This can greatly reduce the computational load of the object detection model in subsequent applications, and at the same time, it can achieve good detection results for target objects at any angle in the 360-degree space in the plane.

[0167] In practical applications, the rotation angle can be set according to actual needs; for example, when the image is of other shapes, in order to ensure the completeness of the detection of the target object in the image, its 360-degree space in the plane can be divided into 4, 5, or 6, etc.

[0168] Step 208: Train the object detection model based on the initial feature map, the rotated feature map, the initial detection box, and the rotated detection box to obtain the object detection model.

[0169] In specific implementation, the step of training the object detection model based on the initial feature map, the rotated feature map, the initial detection box, and the rotated detection box to obtain the object detection model includes:

[0170] The initial feature map and the rotated feature map are used as training samples, and the initial detection box corresponding to the initial feature map and the rotated detection box corresponding to the rotated feature map are used as the sample labels corresponding to the training samples.

[0171] The object detection model is trained based on the training samples and the corresponding sample labels to obtain the object detection model.

[0172] In this embodiment, the initial feature map and the rotated feature map obtained by rotating the initial feature map are used as training samples. The initial detection box corresponding to the initial object in the initial feature map and the rotated detection box corresponding to the rotated object in the rotated feature map are used as sample labels. The object detection model can be trained quickly and accurately using the training samples and the corresponding sample labels, and the speed and accuracy of the object detection model in subsequent applications can be guaranteed.

[0173] In another embodiment of this specification, training the object detection model based on the initial feature map, the rotated feature map, the initial detection box, and the rotated detection box to obtain the object detection model includes:

[0174] The initial feature map, the first rotated feature map, the second rotated feature map, and the third rotated feature map are used as training samples;

[0175] The initial detection box corresponding to the initial feature map, the first rotation detection box corresponding to the first rotation feature map, the second rotation detection box corresponding to the second rotation feature map, and the third rotation detection box corresponding to the third rotation feature map are used as the sample labels corresponding to the training samples;

[0176] The object detection model is trained based on the training samples and the corresponding sample labels to obtain the object detection model.

[0177] In this embodiment, the initial feature map and the first, second, and third rotated feature maps obtained by rotating the initial feature map are used as training samples. The initial detection box corresponding to the initial object in the initial feature map, the first rotated detection box corresponding to the rotated object in the first rotated feature map, the second rotated detection box corresponding to the rotated object in the second rotated feature map, and the third rotated detection box corresponding to the rotated object in the third rotated feature map are used as sample labels. Through these training samples and the corresponding sample labels, the object detection model can be trained quickly and accurately, and the speed and accuracy of the object detection model in subsequent applications can be guaranteed.

[0178] Furthermore, training the object detection model based on the training samples and the corresponding sample labels to obtain the object detection model includes:

[0179] Calculate the regression loss function and the classification loss function based on the training samples and the corresponding sample labels of the training samples;

[0180] The object detection model is obtained when the regression loss function and the classification loss function reach the preset training conditions.

[0181] Continuing with the previous example, let's take the training of an object detection model using feature maps F = [f0, f1, f2, f3] and detection boxes Y = [y0, y1, y2, y3] as an example. The regression loss function loss_reg is specifically implemented through the following formulas 1 and 2:

[0182]

[0183]

[0184] Specifically, the regression loss function loss_reg is obtained through formulas 1 and 2, where v = (v x v y v w v h ) represents the label box Y, This represents the prediction box, where x represents...

[0185] Furthermore, the classification loss function loss_cls can be implemented using the following formulas 3 and 4:

[0186] loss cls =-α t (1-p t ) γ log(p t ) Formula 3

[0187]

[0188] Where, α t γ are manually set hyperparameters, t represents the target object of the t-th category, and p t This represents the confidence level of the predicted category t.

[0189] In the embodiments of this specification, the object detection model training method rotates the initial feature map according to a preset rotation rule. By increasing the computational load slightly, a rotated feature map at any angle can be obtained. The object detection model is then trained based on the initial feature map and the rotated feature map, thereby improving the training efficiency of the object detection model. In this way, the trained object detection model can quickly and accurately detect the bounding boxes of target objects in images during subsequent applications, thus improving the user experience.

[0190] See Figure 4 , Figure 4 A flowchart of a first target object detection method provided in one embodiment of this specification is shown, which specifically includes the following steps.

[0191] Step 402: Input the image carrying the target object into the object detection model to obtain the initial detection box of the target object in the image.

[0192] In this context, "image" can be understood as any image obtained through any means, such as screenshots, photographs, images downloaded from the internet, or hand-drawn images. "Target object" can be understood as any type of object, such as a person, a car, a pet, etc.

[0193] Here, the object detection model can be understood as the object detection model obtained through any of the object detection model training methods in the above embodiments.

[0194] In practical applications, an image containing the target object is input into the object detection model to obtain the initial detection bounding box of the target object in the image.

[0195] Following the previous example, Figure 3 After using the image input object detection model, you can obtain... Figure 3 The initial detection bounding boxes for face 1, face 2, face 3, face 4, and face 5 in the image.

[0196] Optionally, the object detection model is trained through the following steps:

[0197] Acquire sample images and label detection boxes for initial objects in the sample images;

[0198] The sample image is input into the feature extraction model at an initial angle to obtain the initial feature map of the sample image at the initial angle and the initial detection box corresponding to the initial object in the initial feature map;

[0199] The initial feature map is rotated according to a preset rotation rule to obtain a rotated feature map and a rotation detection box corresponding to the rotated object in the rotated feature map;

[0200] The object detection model is trained based on the initial feature map, the rotated feature map, the initial detection box, and the rotated detection box to obtain the object detection model.

[0201] Optionally, obtaining the initial feature map of the sample image at the initial angle and the initial detection box corresponding to the initial object in the initial feature map includes:

[0202] Obtain the initial feature map of the sample image at the initial angle, and the initial detection box of the initial object in the initial feature map at the initial angle.

[0203] Optionally, rotating the initial feature map according to a preset rotation rule to obtain a rotated feature map and a rotation detection box corresponding to the rotated object in the rotated feature map includes:

[0204] The initial feature map is rotated according to the rotation angle to obtain at least one rotated feature map and a rotation detection box corresponding to the initial object in each rotated feature map, wherein the sum of the initial angle and the rotation angle is 360 degrees.

[0205] Optionally, rotating the initial feature map by a rotation angle to obtain at least one rotated feature map and a rotation detection box corresponding to the initial object in each rotated feature map includes:

[0206] The initial feature map is rotated by a first rotation angle to obtain a first rotated feature map of the initial feature map at the first rotation angle and a first rotation detection box corresponding to the initial object in the first rotated feature map;

[0207] The initial feature map is rotated according to the second rotation angle to obtain a second rotated feature map of the initial feature map at the second rotation angle and a second rotation detection box corresponding to the initial object in the second rotated feature map;

[0208] The initial feature map is rotated according to a third rotation angle to obtain a third rotation feature map of the initial feature map at the third rotation angle and a third rotation detection box corresponding to the initial object in the third rotation feature map.

[0209] Optionally, training the object detection model based on the initial feature map, the rotated feature map, the initial detection box, and the rotated detection box to obtain the object detection model includes:

[0210] The initial feature map and the rotated feature map are used as training samples, and the initial detection box corresponding to the initial feature map and the rotated detection box corresponding to the rotated feature map are used as the sample labels corresponding to the training samples.

[0211] The object detection model is trained based on the training samples and the corresponding sample labels to obtain the object detection model.

[0212] Optionally, training the object detection model based on the initial feature map, the rotated feature map, the initial detection box, and the rotated detection box to obtain the object detection model includes:

[0213] The initial feature map, the first rotated feature map, the second rotated feature map, and the third rotated feature map are used as training samples;

[0214] The initial detection box corresponding to the initial feature map, the first rotation detection box corresponding to the first rotation feature map, the second rotation detection box corresponding to the second rotation feature map, and the third rotation detection box corresponding to the third rotation feature map are used as the sample labels corresponding to the training samples;

[0215] The object detection model is trained based on the training samples and the corresponding sample labels to obtain the object detection model.

[0216] Optionally, training the object detection model based on the training samples and the corresponding sample labels to obtain the object detection model includes:

[0217] Calculate the regression loss function and the classification loss function based on the training samples and the corresponding sample labels of the training samples;

[0218] The object detection model is obtained when the regression loss function and the classification loss function reach the preset training conditions.

[0219] Step 404: Filter the initial detection box based on a preset filtering method to obtain the target detection box of the target object.

[0220] The preset filtering methods include, but are not limited to, the non-maximum suppression method (NMS), which can be set according to the actual application. This manual does not impose any restrictions on this.

[0221] Specifically, object detection models will detect bounding boxes for target objects in an image at any angle. It's possible that a single target object may have multiple bounding boxes. In this case, the Non-Maximum Search (NMS) method is used to filter out the redundant initial bounding boxes, retaining the final target bounding box for each target object in the image.

[0222] Step 406: Rotate the target object in the target detection box to a preset angle to obtain the rotated target object in the target detection box.

[0223] Specifically, after obtaining the target detection box for each target object in the image, the target detection box is rotated to a preset angle, such as rotating all of them to a horizontal angle, so that the target object is facing forward; then the target object is obtained from each target detection box, namely face 1, face 2, face 3, face 4 and face 5.

[0224] Following the previous example, in specific implementation, Figure 3The image is input into a pre-trained object detection model. The feature extraction model in the object detection model only extracts the initial feature map of the image from -90 to 90 degrees. Then, the feature map rotation module in the object detection model rotates the initial feature map clockwise by 90 degrees, 180 degrees, and 270 degrees respectively, obtaining three rotated feature maps. Finally, the initial face detection boxes of the image at all angles are obtained from the initial feature map and the three rotated feature maps. Finally, NMS is used to filter out overlapping face detection boxes, obtaining the target face detection boxes of the image at all angles.

[0225] In practical implementation, the embodiments of this solution can be extended to the detection of any target object based on images, such as pedestrian detection, car detection, or pet detection, etc. The specific detection process is the same as the face detection process, and this specification will not go into detail about it.

[0226] The target object detection method provided in the embodiments of this specification offers a solution that enables the detection of target objects at any rotation angle in a plane by rotating an initial feature map by one angle. This improves the detection speed and accuracy of target objects at any rotation angle. In practice, it can be applied to the detection of any image target objects such as faces, pedestrians, cars, and pets, and has strong applicability and a good user experience.

[0227] In another embodiment of this specification, the above-described target object detection method is applied to a face detection scenario, and the specific implementation is as follows:

[0228] The image containing the face is input into the object detection model to obtain the initial detection box of the face in the image;

[0229] The initial detection box is filtered based on a preset filtering method to obtain the target detection box of the face;

[0230] The face in the target detection box is rotated to a preset angle to obtain the rotated face in the target detection box.

[0231] Furthermore, facial landmark localization is an indispensable module in many face-related applications and systems. In some applications, due to the varying orientation of the input image and the different angles of the face within the image, accurate landmark localization of faces at any angle within a plane is required, while maintaining high processing speed. In some video applications, such as live streaming and video beautification, highly accurate and stable landmark localization is necessary for precise beautification. If the model is to handle landmark localization at any angle, achieving stable landmark localization becomes very difficult.

[0232] Currently, there are two main methods for localizing keypoints of faces at arbitrary rotation angles within a plane. The first method uses a cascading approach. First, a small number of facial keypoints (lmks1) are initially located, the face angle is calculated, the face is rotated to the correct orientation, and then accurate keypoint localization (lmks2) is performed on the rotated, upright face image. The main problem with this method is that it requires multiple steps, including initial keypoint localization, angle calculation, face image rotation, and precise keypoint localization, which is time-consuming and unsuitable for many time-sensitive scenarios (such as mobile applications). The second method involves directly inputting faces at arbitrary angles and their corresponding annotations during keypoint model training. Data augmentation is used to help the model fit the keypoints of faces at arbitrary angles. This method typically requires a strong model fitting capability, meaning it requires significant computation and is time-consuming. Given a limited computational load, it is difficult for the model to accurately and stably predict keypoints of faces at arbitrary angles.

[0233] Therefore, this specification proposes a keypoint detection model that can predict facial angles while locating facial keypoints, and use the predicted facial angles to rotate the facial feature map to a zero-degree direction in the plane for keypoint localization. This model can quickly and accurately locate facial keypoints at any rotation angle in the plane. Specifically, the training method for this keypoint detection model is as follows.

[0234] See Figure 5 , Figure 5 The flowchart of the first keypoint detection model training method provided in one embodiment of this specification is shown, which specifically includes the following steps.

[0235] Step 502: Obtain sample images and input the sample images into the object detection model to obtain the initial detection bounding boxes of the target objects in the sample images.

[0236] The sample images and descriptions of the target objects can be found in the above embodiments, and this specification does not limit them in any way.

[0237] Furthermore, the object detection model in the embodiments of this specification can be trained using any of the object detection model training methods described in the above embodiments.

[0238] Step 504: Filter the initial detection box based on a preset filtering method to obtain the target detection box of the target object.

[0239] Specifically, the step of filtering the initial detection box based on a preset filtering method to obtain the target detection box of the target object includes:

[0240] The initial detection box is filtered using a nonmaximum suppression method to obtain the target detection box of the target object.

[0241] In the embodiments of this specification, in order to ensure the accuracy of the target object, the initial detection box of the target object can be detected by the NMS method to obtain the accurate target detection box of the target object.

[0242] Step 506: Rotate the target object in the target detection box to a preset angle and mark the key points of the target object.

[0243] The preset angle can be understood as the positive angle of the target object.

[0244] In practice, the steps of obtaining the initial detection box of the target object in the sample image, filtering the initial detection box using the NMS method, and rotating the target object to the positive orientation can all be found in the detailed description of the above embodiments, and will not be repeated here.

[0245] Specifically, after capturing the target object from each angle in the sample image and rotating it to face the positive direction, the key points of each target object are obtained. The key points differ depending on the target object.

[0246] For example, if the target object is a human face, the key points of the target object can be the human's eyebrows, eyes, nose, mouth, and facial contours; if the target object is a pet, the key points of the target object can be the pet's eyes, mouth, nose, nasal prints, fur, etc.; if the target object is a car, the key points of the target object can be the car's outer contours, headlights, door handles, hood, etc.

[0247] Step 508: Train the key point detection model based on the target object and the key points corresponding to the target object.

[0248] Specifically, the target objects obtained from all angles of the sample images are used as training samples, and the key points corresponding to each target object are used as sample labels. The key point detection model is trained based on the training samples and sample labels to obtain the trained key point detection model.

[0249] In the embodiments of this specification, an object detection model is introduced into the keypoint detection model. The keypoint detection model is trained by obtaining the target objects and keypoints of the target objects from all angles of the sample images obtained by the object detection model. This enables the keypoint detection model to quickly, accurately, and stably predict the keypoints of target objects at any angle in the image during subsequent applications.

[0250] Furthermore, training the keypoint detection model using this method avoids acquiring sample images from multiple angles, thus reducing manual labor costs. By obtaining sample images of the target object from any angle without manually rotating them, the keypoint detection model can be trained using sample images from all angles, significantly saving on labor costs.

[0251] See Figure 6 , Figure 6 A flowchart of a first keypoint detection method provided in one embodiment of this specification is shown, which specifically includes the following steps.

[0252] Step 602: Input the image carrying the target object into the object detection model to obtain the initial detection box of the target object in the image.

[0253] Step 604: Filter the initial detection box based on a preset filtering method to obtain the target detection box of the target object.

[0254] Step 606: Rotate the target object in the target detection box to a preset angle, input the rotated target detection box into the key point detection model, and obtain the key points of the target object in the target detection box.

[0255] Optionally, the keypoint detection model is trained through the following steps:

[0256] Acquire a sample image, input the sample image into the object detection model, and obtain the initial detection box of the target object in the sample image;

[0257] The initial detection box is filtered based on a preset filtering method to obtain the target detection box of the target object;

[0258] Rotate the target object in the target detection box to a preset angle and mark key points on the target object;

[0259] The keypoint detection model is obtained by training based on the target object and the key points corresponding to the target object.

[0260] Specifically, the object detection model is trained using the object detection model training method described above, and the keypoint detection model is trained using the keypoint detection model training method described above.

[0261] Still with Figure 3 In the example, the target object is a face.

[0262] Specifically, by inputting an image containing faces into the object detection model, initial detection bounding boxes for face 1, face 2, face 3, face 4, and face 5 in the image can be obtained.

[0263] The initial detection boxes of face 1, face 2, face 3, face 4 and face 5 are filtered using the NMS filtering method to obtain the target detection boxes of face 1, face 2, face 3, face 4 and face 5.

[0264] The target object in each object detection box is rotated to the positive direction, and then the rotated object detection box carrying the target object is input into the key point detection model to obtain the key points of face 1, face 2, face 3, face 4 and face 5 respectively.

[0265] This specification presents a method that simultaneously predicts the target object's angle and rotates the feature map in the object detection model according to the predicted angle before locating the key points. This method completes multiple steps that would otherwise require cascading within a single model, significantly reducing computational load and improving processing speed. Specifically, in facial key point detection scenarios, by predicting the face angle, rotating the model's feature map to a forward orientation, and then locating key points on the forward-facing face, this method reduces the difficulty of locating facial key points at arbitrary rotation angles, enabling fast, accurate, and stable prediction of facial key points.

[0266] In another embodiment of this specification, the above-described key point detection method is applied to a face key point detection scenario, and the specific implementation is as follows:

[0267] The image containing the face is input into the object detection model to obtain the initial detection box of the face in the image;

[0268] The initial detection box is filtered based on a preset filtering method to obtain the target detection box of the face;

[0269] The face in the target detection box is rotated to a preset angle, and the rotated target detection box is input into the key point detection model to obtain the key points of the face in the target detection box.

[0270] This specification provides an embodiment of a second method for training an object detection model, including:

[0271] Based on the user's request, display the image input interface to the user;

[0272] Obtain the sample image input by the user based on the image input interface, and label the initial objects in the sample image with detection boxes;

[0273] The sample image is input into the feature extraction model at an initial angle to obtain the initial feature map of the sample image at the initial angle and the initial detection box corresponding to the initial object in the initial feature map;

[0274] The initial feature map is rotated according to a preset rotation rule to obtain a rotated feature map and a rotation detection box corresponding to the rotated object in the rotated feature map;

[0275] The object detection model is trained based on the initial feature map, the rotated feature map, the initial detection box, and the rotated detection box to obtain the object detection model and return it to the user.

[0276] It should be noted that the technical solution of the second object detection model training method is based on the same concept as the technical solution of the first object detection model training method described above. For details not described in detail in the technical solution of the second object detection model training method, please refer to the description of the technical solution of the first object detection model training method described above.

[0277] This specification provides an embodiment of a third method for training an object detection model, including:

[0278] Receive a call request sent by the user, wherein the call request carries a sample image;

[0279] Label the initial objects in the sample image with detection boxes;

[0280] The sample image is input into the feature extraction model at an initial angle to obtain the initial feature map of the sample image at the initial angle and the initial detection box corresponding to the initial object in the initial feature map;

[0281] The initial feature map is rotated according to a preset rotation rule to obtain a rotated feature map and a rotation detection box corresponding to the rotated object in the rotated feature map;

[0282] The object detection model is trained based on the initial feature map, the rotated feature map, the initial detection box, and the rotated detection box to obtain the object detection model and return it to the user.

[0283] It should be noted that the technical solution of this third object detection model training method is based on the same concept as the technical solution of the first object detection model training method described above. For details not described in detail in the technical solution of this third object detection model training method, please refer to the description of the technical solution of the first object detection model training method described above.

[0284] This specification provides a second method for training a keypoint detection model, including:

[0285] Based on the user's request, display the image input interface to the user;

[0286] Obtain the sample image input by the user based on the image input interface, input the sample image into the object detection model, and obtain the initial detection box of the target object in the sample image;

[0287] The initial detection box is filtered based on a preset filtering method to obtain the target detection box of the target object;

[0288] Rotate the target object in the target detection box to a preset angle and mark key points on the target object;

[0289] The keypoint detection model is obtained by training based on the target object and the key points corresponding to the target object.

[0290] The object detection model is trained using the object detection model training method described above and then returned to the user.

[0291] It should be noted that the technical solution of the second keypoint detection model training method is based on the same concept as the technical solution of the first keypoint detection model training method described above. For details not described in detail in the technical solution of the second keypoint detection model training method, please refer to the description of the technical solution of the first keypoint detection model training method described above.

[0292] This specification provides a third method for training a keypoint detection model, including:

[0293] Receive a call request sent by the user, wherein the call request carries a sample image;

[0294] The sample image is input into the object detection model to obtain the initial detection bounding box of the target object in the sample image;

[0295] The initial detection box is filtered based on a preset filtering method to obtain the target detection box of the target object;

[0296] Rotate the target object in the target detection box to a preset angle and mark key points on the target object;

[0297] The keypoint detection model is obtained by training based on the target object and the key points corresponding to the target object.

[0298] The object detection model is trained using the object detection model training method described above and then returned to the user.

[0299] It should be noted that the technical solution of the third keypoint detection model training method is based on the same concept as the technical solution of the first keypoint detection model training method described above. For details not described in detail in the technical solution of the third keypoint detection model training method, please refer to the description of the technical solution of the first keypoint detection model training method described above.

[0300] This specification provides a second method for detecting target objects, including:

[0301] Based on the user's request, display the image input interface to the user;

[0302] Obtain the image containing the target object that the user inputs based on the image input interface;

[0303] The image is input into an object detection model to obtain the initial detection bounding box of the target object in the image;

[0304] The initial detection box is filtered based on a preset filtering method to obtain the target detection box of the target object;

[0305] The target object in the target detection box is rotated to a preset angle, and the rotated target object in the target detection box is obtained and returned to the user.

[0306] The object detection model is trained using the object detection model training method described above.

[0307] It should be noted that the technical solution of the second target object detection method is based on the same concept as the technical solution of the first target object detection method described above. For details not described in detail in the technical solution of the second target object detection method, please refer to the description of the technical solution of the first target object detection method described above.

[0308] This specification provides a third method for target object detection in one embodiment, including:

[0309] Receive a call request sent by the user, wherein the call request carries an image containing the target object;

[0310] The image is input into an object detection model to obtain the initial detection bounding box of the target object in the image;

[0311] The initial detection box is filtered based on a preset filtering method to obtain the target detection box of the target object;

[0312] The target object in the target detection box is rotated to a preset angle, and the rotated target object in the target detection box is obtained and returned to the user.

[0313] The object detection model is trained using the object detection model training method described above.

[0314] It should be noted that the technical solution of the third target object detection method is based on the same concept as the technical solution of the first target object detection method described above. For details not described in detail in the technical solution of the third target object detection method, please refer to the description of the technical solution of the first target object detection method described above.

[0315] This specification provides a second keypoint detection method in one embodiment, including:

[0316] Based on the user's request, display an image input interface to the user and obtain the image containing the target object that the user inputs based on the image input interface;

[0317] The image is input into an object detection model to obtain the initial detection bounding box of the target object in the image;

[0318] The initial detection box is filtered based on a preset filtering method to obtain the target detection box of the target object;

[0319] The target object in the target detection box is rotated to a preset angle, and the rotated target detection box is input into the key point detection model to obtain the key points of the target object in the target detection box.

[0320] The object detection model is trained using the object detection model training method described above, and the keypoint detection model is trained using the keypoint detection model training method described above.

[0321] It should be noted that the technical solution of the second key point detection method is based on the same concept as the technical solution of the first key point detection method described above. For details not described in detail in the technical solution of the second key point detection method, please refer to the description of the technical solution of the first key point detection method described above.

[0322] This specification provides a third keypoint detection method in one embodiment, including:

[0323] Receive a call request sent by the user, wherein the call request carries an image containing the target object;

[0324] The image is input into an object detection model to obtain the initial detection bounding box of the target object in the image;

[0325] The initial detection box is filtered based on a preset filtering method to obtain the target detection box of the target object;

[0326] The target object in the target detection box is rotated to a preset angle, and the rotated target detection box is input into the key point detection model to obtain the key points of the target object in the target detection box.

[0327] The object detection model is trained using the object detection model training method described above, and the keypoint detection model is trained using the keypoint detection model training method described above.

[0328] It should be noted that the technical solution of the third key point detection method is based on the same concept as the technical solution of the first key point detection method described above. For details not described in detail in the technical solution of the third key point detection method, please refer to the description of the technical solution of the first key point detection method described above.

[0329] Corresponding to the above method embodiments, this specification also provides an embodiment of an object detection model training device. Figure 7 A schematic diagram of an object detection model training device according to one embodiment of this specification is shown. Figure 7 As shown, the device includes:

[0330] The sample acquisition module 702 is configured to acquire sample images and label detection boxes for initial objects in the sample images.

[0331] The feature map acquisition module 704 is configured to input the sample image into the feature extraction model at an initial angle to obtain an initial feature map of the sample image at the initial angle and an initial detection box corresponding to the initial object in the initial feature map;

[0332] The rotation module 706 is configured to rotate the initial feature map according to a preset rotation rule to obtain a rotated feature map and a rotation detection box corresponding to the rotated object in the rotated feature map;

[0333] The model training module 708 is configured to train the object detection model based on the initial feature map, the rotated feature map, the initial detection box, and the rotated detection box to obtain the object detection model.

[0334] In the embodiments of this specification, the object detection model training device rotates the initial feature map according to a preset rotation rule. By increasing the computational load slightly, a rotated feature map at any angle can be obtained. Based on the initial feature map and the rotated feature map, the object detection model is trained, thereby improving the training efficiency of the object detection model. In this way, the trained object detection model can quickly and accurately detect the detection box of the target object in the image in subsequent applications, thus improving the user experience.

[0335] The above is a schematic scheme of an object detection model training device according to this embodiment. It should be noted that the technical solution of this object detection model training device belongs to the same concept as the technical solution of the first object detection model training method described above. For details not described in detail in the technical solution of the object detection model training device, please refer to the description of the technical solution of the first object detection model training method described above.

[0336] Corresponding to the above method embodiments, this specification also provides an embodiment of a key point detection model training device. Figure 8 A schematic diagram of a keypoint detection model training device according to one embodiment of this specification is shown. Figure 8 As shown, the device includes:

[0337] The detection box acquisition module 802 is configured to acquire a sample image, input the sample image into the object detection model, and obtain the initial detection box of the target object in the sample image.

[0338] The detection box filtering module 804 is configured to filter the initial detection box based on a preset filtering method to obtain the target detection box of the target object;

[0339] The object rotation module 806 is configured to rotate the target object in the target detection box to a preset angle and mark key points for the target object;

[0340] The model training module 808 is configured to train based on the target object and the key points corresponding to the target object to obtain the key point detection model.

[0341] The object detection model is trained using the object detection model training method described above.

[0342] Optionally, the detection frame filtering module 804 is further configured to:

[0343] The initial detection box is filtered using a nonmaximum suppression method to obtain the target detection box of the target object.

[0344] In the embodiments of this specification, an object detection model is introduced into the keypoint detection model. The keypoint detection model is trained by obtaining the target objects and keypoints of the target objects from all angles of the sample images obtained by the object detection model. This enables the keypoint detection model to quickly, accurately, and stably predict the keypoints of target objects at any angle in the image during subsequent applications.

[0345] The above is a schematic scheme of a keypoint detection model training device according to this embodiment. It should be noted that the technical solution of this keypoint detection model training device belongs to the same concept as the technical solution of the first keypoint detection model training method described above. For details not described in detail in the technical solution of the keypoint detection model training device, please refer to the description of the technical solution of the first keypoint detection model training method described above.

[0346] Corresponding to the above method embodiments, this specification also provides an embodiment of a target object detection device. Figure 9 A schematic diagram of a target object detection device according to one embodiment of this specification is shown. Figure 9 As shown, the device includes:

[0347] The detection box acquisition module 902 is configured to input an image carrying a target object into an object detection model to obtain an initial detection box of the target object in the image.

[0348] The detection box filtering module 904 is configured to filter the initial detection box based on a preset filtering method to obtain the target detection box of the target object;

[0349] The object acquisition module 906 is configured to rotate the target object in the target detection box to a preset angle to obtain the target object in the target detection box after rotation.

[0350] Optionally, the device further includes:

[0351] The training module is configured as follows:

[0352] Acquire sample images and label detection boxes for initial objects in the sample images;

[0353] The sample image is input into the feature extraction model at an initial angle to obtain the initial feature map of the sample image at the initial angle and the initial detection box corresponding to the initial object in the initial feature map;

[0354] The initial feature map is rotated according to a preset rotation rule to obtain a rotated feature map and a rotation detection box corresponding to the rotated object in the rotated feature map;

[0355] The object detection model is trained based on the initial feature map, the rotated feature map, the initial detection box, and the rotated detection box to obtain the object detection model.

[0356] Optionally, the training module is further configured to:

[0357] Obtain the initial feature map of the sample image at the initial angle, and the initial detection box of the initial object in the initial feature map at the initial angle.

[0358] Optionally, the training module is further configured to:

[0359] The initial feature map is rotated according to the rotation angle to obtain at least one rotated feature map and a rotation detection box corresponding to the initial object in each rotated feature map, wherein the sum of the initial angle and the rotation angle is 360 degrees.

[0360] Optionally, the training module is further configured to:

[0361] The initial feature map is rotated by a first rotation angle to obtain a first rotated feature map of the initial feature map at the first rotation angle and a first rotation detection box corresponding to the initial object in the first rotated feature map;

[0362] The initial feature map is rotated according to the second rotation angle to obtain a second rotated feature map of the initial feature map at the second rotation angle and a second rotation detection box corresponding to the initial object in the second rotated feature map;

[0363] The initial feature map is rotated according to a third rotation angle to obtain a third rotation feature map of the initial feature map at the third rotation angle and a third rotation detection box corresponding to the initial object in the third rotation feature map.

[0364] Optionally, the training module is further configured to:

[0365] The initial feature map and the rotated feature map are used as training samples, and the initial detection box corresponding to the initial feature map and the rotated detection box corresponding to the rotated feature map are used as the sample labels corresponding to the training samples.

[0366] The object detection model is trained based on the training samples and the corresponding sample labels to obtain the object detection model.

[0367] Optionally, the training module is further configured to:

[0368] Calculate the regression loss function and the classification loss function based on the training samples and the corresponding sample labels of the training samples;

[0369] The object detection model is obtained when the regression loss function and the classification loss function reach the preset training conditions.

[0370] The target object detection device provided in the embodiments of this specification provides a solution that can realize the detection of target objects with any rotation angle in the plane by rotating an initial feature map by one angle, thereby improving the detection speed and accuracy of target objects with any rotation angle.

[0371] The above is a schematic scheme of a target object detection device according to this embodiment. It should be noted that the technical solution of this target object detection device belongs to the same concept as the technical solution of the first target object detection method described above. For details not described in detail in the technical solution of the target object detection device, please refer to the description of the technical solution of the first target object detection method described above.

[0372] Corresponding to the above method embodiments, this specification also provides an embodiment of a key point detection device. Figure 10 A schematic diagram of a key point detection device according to one embodiment of this specification is shown. Figure 10 As shown, the device includes:

[0373] The detection box acquisition module 1002 is configured to input an image carrying a target object into an object detection model to obtain an initial detection box of the target object in the image.

[0374] The detection box filtering module 1004 is configured to filter the initial detection box based on a preset filtering method to obtain the target detection box of the target object;

[0375] The key point acquisition module 1006 is configured to rotate the target object in the target detection box to a preset angle, input the rotated target detection box into the key point detection model, and obtain the key points of the target object in the target detection box.

[0376] The object detection model is trained using the object detection model training method described above.

[0377] Optionally, the device further includes:

[0378] The model training module is configured as follows:

[0379] Acquire a sample image, input the sample image into the object detection model, and obtain the initial detection box of the target object in the sample image;

[0380] The initial detection box is filtered based on a preset filtering method to obtain the target detection box of the target object;

[0381] Rotate the target object in the target detection box to a preset angle and mark key points on the target object;

[0382] The keypoint detection model is obtained by training based on the target object and the key points corresponding to the target object.

[0383] In the embodiments of this specification, a device is proposed that predicts the angle of the target object while locating key points, rotates the feature map in the object detection model according to the predicted angle, and then locates the key points. This device completes multiple steps that would otherwise require cascading within a single model, greatly reducing computation and improving running speed.

[0384] The above is a schematic scheme of a key point detection device according to this embodiment. It should be noted that the technical solution of this key point detection device belongs to the same concept as the technical solution of the first key point detection method described above. For details not described in detail in the technical solution of the key point detection device, please refer to the description of the technical solution of the first key point detection method described above.

[0385] Figure 11 A structural block diagram of a computing device 1100 according to one embodiment of this specification is shown. The components of the computing device 1100 include, but are not limited to, a memory 1110 and a processor 1120. The processor 1120 is connected to the memory 1110 via a bus 1130, and a database 1150 is used to store data.

[0386] The computing device 1100 also includes an access device 1140, which enables the computing device 1100 to communicate via one or more networks 1060. Examples of these networks include a Public Switched Telephone Network (PSTN), a Local Area Network (LAN), a Wide Area Network (WAN), a Personal Area Network (PAN), or a combination of communication networks such as the Internet. The access device 1140 may include one or more of any type of wired or wireless network interface (e.g., a Network Interface Card (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) interface, a Wi-MAX interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, a Near Field Communication (NFC) interface, and so on.

[0387] In one embodiment of this specification, the aforementioned components of the computing device 1100 and Figure 11 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 11 The block diagram of the computing device shown is for illustrative purposes only and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0388] The computing device 1100 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or PCs. The computing device 1100 can also be a mobile or stationary server.

[0389] The processor 1120 is configured to execute the following computer-executable instructions, which, when executed by the processor, implement the above-described object detection model training method, the above-described key point detection model training method, the above-described target object detection method, and the steps of the above-described key point detection method.

[0390] The above is an illustrative scheme of a computing device according to this embodiment. It should be noted that the technical solution of this computing device belongs to the same concept as the technical solutions of the object detection model training method, key point detection model training method, target object detection method, or key point detection method described above. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solutions of the object detection model training method, key point detection model training method, target object detection method, or key point detection method described above.

[0391] An embodiment of this specification also provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the above-described object detection model training method, the above-described keypoint detection model training method, the above-described target object detection method, and the steps of the above-described keypoint detection method.

[0392] The above is an illustrative scheme of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium belongs to the same concept as the technical solutions of the object detection model training method, keypoint detection model training method, target object detection method, or keypoint detection method described above. Details not described in detail in the technical solution of the storage medium can be found in the descriptions of the technical solutions of the object detection model training method, keypoint detection model training method, target object detection method, or keypoint detection method described above.

[0393] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0394] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added to or subtracted according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0395] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments in this specification are not limited to the described order of actions, because according to the embodiments in this specification, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments in this specification.

[0396] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0397] The preferred embodiments disclosed above are merely illustrative of this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments described herein. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of the embodiments, thereby enabling those skilled in the art to better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A target object detection method, comprising: An image carrying a target object is input into an object detection model to obtain an initial detection box of the target object in the image. The object detection model is trained by using an initial feature map and a rotation feature map obtained by rotating the initial feature map as training samples, and using the initial detection box corresponding to the initial object in the initial feature map and the rotation detection box corresponding to the rotated object in the rotation feature map as sample labels. The initial feature map is the feature map of the sample image at an initial angle. The initial detection box is filtered based on a preset filtering method to obtain the target detection box of the target object; Rotate the target object in the target detection box to a preset angle to obtain the rotated target object in the target detection box.

2. The target object detection method according to claim 1, wherein the object detection model is trained through the following steps: Acquire sample images and label detection boxes for initial objects in the sample images; The sample image is input into the feature extraction model at an initial angle to obtain the initial feature map of the sample image at the initial angle and the initial detection box corresponding to the initial object in the initial feature map; The initial feature map is rotated according to a preset rotation rule to obtain a rotated feature map and a rotation detection box corresponding to the rotated object in the rotated feature map; The object detection model is trained based on the initial feature map, the rotated feature map, the initial detection box, and the rotated detection box to obtain the object detection model.

3. The target object detection method according to claim 2, wherein obtaining the initial feature map of the sample image at the initial angle and the initial detection box corresponding to the initial object in the initial feature map includes: Obtain the initial feature map of the sample image at the initial angle, and the initial detection box of the initial object in the initial feature map at the initial angle.

4. The target object detection method according to claim 2, wherein rotating the initial feature map according to a preset rotation rule to obtain a rotated feature map and a rotation detection box corresponding to the rotated object in the rotated feature map includes: The initial feature map is rotated according to the rotation angle to obtain at least one rotated feature map and a rotation detection box corresponding to the initial object in each rotated feature map, wherein the sum of the initial angle and the rotation angle is 360 degrees.

5. The target object detection method according to claim 4, wherein rotating the initial feature map by a rotation angle to obtain at least one rotated feature map and a rotation detection box corresponding to the initial object in each rotated feature map comprises: The initial feature map is rotated by a first rotation angle to obtain a first rotated feature map of the initial feature map at the first rotation angle and a first rotation detection box corresponding to the initial object in the first rotated feature map; The initial feature map is rotated according to the second rotation angle to obtain a second rotated feature map of the initial feature map at the second rotation angle and a second rotation detection box corresponding to the initial object in the second rotated feature map; The initial feature map is rotated by a third rotation angle to obtain a third rotated feature map of the initial feature map at the third rotation angle and a third rotation detection box corresponding to the initial object in the third rotated feature map.

6. The target object detection method according to claim 2 or 5, wherein training the object detection model based on the initial feature map, the rotated feature map, the initial detection box, and the rotated detection box to obtain the object detection model includes: The initial feature map and the rotated feature map are used as training samples, and the initial detection box corresponding to the initial feature map and the rotated detection box corresponding to the rotated feature map are used as the sample labels corresponding to the training samples. The object detection model is trained based on the training samples and the corresponding sample labels to obtain the object detection model.

7. The target object detection method according to claim 6, wherein training the object detection model based on the training samples and the sample labels corresponding to the training samples to obtain the object detection model includes: Calculate the regression loss function and the classification loss function based on the training samples and the corresponding sample labels of the training samples; The object detection model is obtained when the regression loss function and the classification loss function reach the preset training conditions.

8. A method for training an object detection model, comprising: Acquire sample images and label detection boxes for initial objects in the sample images; The sample image is input into the feature extraction model at an initial angle to obtain the initial feature map of the sample image at the initial angle and the initial detection box corresponding to the initial object in the initial feature map; The initial feature map is rotated according to a preset rotation rule to obtain a rotated feature map and a rotation detection box corresponding to the rotated object in the rotated feature map; The object detection model is trained based on the initial feature map, the rotated feature map, the initial detection box, and the rotated detection box to obtain the object detection model.

9. A key point detection method, comprising: The image carrying the target object is input into the object detection model to obtain the initial detection box of the target object in the image; The initial detection box is filtered based on a preset filtering method to obtain the target detection box of the target object; The target object in the target detection box is rotated to a preset angle, and the rotated target detection box is input into the key point detection model to obtain the key points of the target object in the target detection box. The object detection model is trained using the object detection model training method described in claim 8.

10. The keypoint detection method according to claim 9, wherein the keypoint detection model is trained through the following steps: Acquire a sample image, input the sample image into the object detection model, and obtain the initial detection box of the target object in the sample image; The initial detection box is filtered based on a preset filtering method to obtain the target detection box of the target object; Rotate the target object in the target detection box to a preset angle and mark key points on the target object; The keypoint detection model is obtained by training based on the target object and the key points corresponding to the target object.

11. A method for training a keypoint detection model, comprising: Acquire a sample image, input the sample image into the object detection model, and obtain the initial detection box of the target object in the sample image; The initial detection box is filtered based on a preset filtering method to obtain the target detection box of the target object; Rotate the target object in the target detection box to a preset angle and mark key points on the target object; The key point detection model is obtained by training based on the target object and the key points corresponding to the target object. The object detection model is trained using the object detection model training method described in claim 8.

12. A method for detecting a target object, comprising: Based on the user's request, display the image input interface to the user; Obtain the image containing the target object that is input by the user based on the image input interface; The image is input into an object detection model to obtain initial detection boxes of target objects in the image. The object detection model is trained by using an initial feature map and a rotation feature map obtained by rotating the initial feature map as training samples, and using the initial detection boxes corresponding to the initial objects in the initial feature map and the rotation detection boxes corresponding to the rotated objects in the rotation feature map as sample labels. The initial feature map is the feature map of the sample image at an initial angle. The initial detection box is filtered based on a preset filtering method to obtain the target detection box of the target object; The target object in the target detection box is rotated to a preset angle, and the rotated target object in the target detection box is obtained and returned to the user.

13. A method for detecting a target object, comprising: Receive a call request sent by a user, wherein the call request carries an image containing the target object; The image is input into an object detection model to obtain initial detection boxes of target objects in the image. The object detection model is trained by using an initial feature map and a rotation feature map obtained by rotating the initial feature map as training samples, and using the initial detection boxes corresponding to the initial objects in the initial feature map and the rotation detection boxes corresponding to the rotated objects in the rotation feature map as sample labels. The initial feature map is the feature map of the sample image at an initial angle. The initial detection box is filtered based on a preset filtering method to obtain the target detection box of the target object; The target object in the target detection box is rotated to a preset angle, and the rotated target object in the target detection box is obtained and returned to the user.

14. A computing device, comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, they implement the target object detection method according to any one of claims 1-7, 12, and 13, the key point detection method according to any one of claims 9-10, the object detection model training method according to claim 8, and the key point detection model training method according to claim 11.

15. A computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the target object detection method according to any one of claims 1-7, 12, and 13, the key point detection method according to any one of claims 9-10, the object detection model training method according to claim 8, and the steps of the key point detection model training method according to claim 11.

16. A computer program product, characterized in that, The method includes computer instructions that, when executed by a processor, implement the target object detection method according to any one of claims 1-7, 12, and 13, the key point detection method according to any one of claims 9-10, the object detection model training method according to claim 8, and the steps of the key point detection model training method according to claim 11.

Images