Detection frame determination method and device, electronic equipment and readable storage medium

Abstract

The application discloses a detection frame determination method and device, electronic equipment and a readable storage medium, and relates to the technical field of computers. The detection frame determination method provided by the application comprises the following steps: determining at least one preset detection frame in the first image frame of a video; performing key point detection on a target object in a current image frame according to a target detection frame, and determining a first detection frame for the target object according to the result of the key point detection, the current image frame being any image frame in the image frames of the video; obtaining a second detection frame matched with the first detection frame from the at least one preset detection frame; and taking the second detection frame as a target detection frame for performing key point detection on a target object in the next image frame of the current image frame; wherein the target detection frame is used for target object detection or key point detection.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of computer technology, and particularly relates to a detection frame determination method and device, electronic equipment and a readable storage medium. BACKGROUND

[0002] Currently, mainstream video human key point detection methods are divided into two stages: the first stage is human detection and tracking, and the second stage is to crop the detected human and perform key point detection through an algorithm. Video key point detection on a mobile terminal is limited by hardware and cannot simultaneously perform human tracking detection algorithms and key point algorithms, so it is usually to derive a human detection frame of the next frame based on the key point detection result of the last frame. However, the width-height ratio and center point position of the detection frame derived in this way fluctuate greatly, which affects the key point detection result and causes the key point detection result to be not accurate enough. SUMMARY

[0003] The embodiments of the present application provide a detection frame determination method and device, electronic equipment and a readable storage medium, which can solve the problem of not accurate enough key point detection result in the related art.

[0004] In a first aspect, the embodiments of the present application provide a detection frame determination method, comprising:

[0005] determining at least one preset detection frame in the first image frame of the video;

[0006] performing key point detection on a target object in a current image frame according to a target detection frame, and determining a first detection frame for the target object according to the result of the key point detection, the current image frame being any image frame in the image frames of the video;

[0007] obtaining a second detection frame matched with the first detection frame from the at least one preset detection frame;

[0008] taking the second detection frame as a target detection frame for performing key point detection on a target object in the next image frame of the current image frame;

[0009] The target detection frame is used for target object detection or key point detection.

[0010] In a second aspect, the embodiments of the present application provide a detection frame determination device, comprising:

[0011] a first determination module configured to determine at least one preset detection frame in the first image frame of the video;

[0012] The detection module is configured to perform key point detection on a target object in a current image frame according to a target detection frame, and determine a first detection frame for the target object according to a result of the key point detection, the current image frame being any one of image frames of the video.

[0013] The acquisition module is configured to acquire a second detection frame matching the first detection frame from the at least one preset detection frame.

[0014] The second determination module is configured to take the second detection frame as a target detection frame for performing key point detection on a target object in a next image frame of the current image frame.

[0015] The target detection frame is used for target object detection or key point detection.

[0016] In a third aspect, an electronic device is provided, and the electronic device includes a processor and a memory. The memory stores programs or instructions executable on the processor. When the programs or instructions are executed by the processor, the steps of the method for determining a detection frame according to the first aspect are implemented.

[0017] In a fourth aspect, a readable storage medium is provided, and the readable storage medium stores programs or instructions. When the programs or instructions are executed by a processor, the steps of the method for determining a detection frame according to the first aspect are implemented.

[0018] In the embodiments of the present application, the target detection frame is determined from the at least one preset detection frame. Even if the target object shakes or fluctuates, the preset detection frame does not fluctuate, and a detection frame matching the first detection frame of the target object can always be found in the at least one preset detection frame. Therefore, the accuracy of key point detection based on the target detection frame can be effectively ensured, and the accuracy of key point detection is improved. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 is a flowchart of a method for determining a detection frame provided by the embodiments of the present application;

[0020] Figure 2 is a schematic diagram of a preset detection frame in the embodiments of the present application;

[0021] Figure 3 is a flowchart of another method for determining a detection frame provided by the embodiments of the present application;

[0022] Figure 4 is a structural diagram of a detection frame determination device provided by the embodiments of the present application;

[0023] Figure 5is a structural diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0024] The technical solutions in the embodiments of the present application will be clearly described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all of them. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art belong to the scope of protection of the present application.

[0025] The terms "first", "second", and the like in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally a category and do not limit the number of objects, for example, the first object can be one or more. In addition, "and / or" in the specification and claims means at least one of the connected objects, and the character " / ", generally represents a "or" relationship between the front and rear associated objects.

[0026] At present, with the development of the times and the advent of the big data era, the application of artificial intelligence has become one of the important trends of current scientific and technological development. Among them, the human key point detection algorithm is a widely used algorithm, which has been applied in various fields, especially human key point detection in video. In human key point detection, the detection frame needs to be determined first to frame the object to be detected for key point detection, and the determination of the detection frame is an important link in key point detection.

[0027] The determination method, device and electronic equipment of the detection frame provided by the embodiments of the present application will be described in detail below in combination with the drawings, through specific embodiments and their application scenarios.

[0028] Please refer to Figure 1 , Figure 1 is a flowchart of a detection frame determination method provided by an embodiment of the present application. The method can be applied to electronic devices such as mobile phones, tablet computers, computers, smart wearable devices, etc. In subsequent embodiments, the electronic device is taken as the execution subject of the method of the embodiments of the present application to explain and describe the technical solutions of the embodiments of the present application.

[0029] As shown in Figure 1 , the detection frame determination method provided by the embodiments of the present application includes the following steps:

[0030] Step 101, determining at least one preset detection frame in the first image frame of the video.

[0031] It can be understood that the first image frame is the first image frame or the first frame of the video.

[0032] In the embodiment of the application, the preset detection frame can be a detection frame with a preset size. For example, the size of the preset detection frame can be matched with the size of the target object in the video image frame, and the target object can be a person, a vehicle, a building, a road sign, etc. in the video image frame.

[0033] Optionally, in the case where the number of the at least one preset detection frame is two or more, the size of any two preset detection frames can be different, so that multiple preset detection frames with different sizes can be determined in the first image frame of the video, and then the target objects with different sizes in the video image frame can be located by using the preset detection frames with different sizes.

[0034] In step 102, key point detection is performed on the target object in the current image frame according to the target detection frame, and a first detection frame for the target object is determined according to the result of the key point detection.

[0035] The current image frame can be any image frame in the video image frames, for example, the first image frame or an image frame after the first image frame, such as the second image frame, the third image frame, etc.

[0036] In the embodiment of the application, the target detection frame is used for target object detection or key point detection. The target object can be a person, a vehicle, a building, etc. in the video image frame, and the technical solution of the application will be explained and described below by taking the person as an example.

[0037] Optionally, in the case where the current image frame is the first image frame of the video, the target detection frame can be a detection frame for the target person in the first image frame, which is obtained by performing human body frame detection on the target person in the first image frame. Then, key point detection is performed on the target person in the first image frame based on the detection frame, and a first detection frame for the target person is determined according to the result of the key point detection. For example, the first detection frame can be a detection frame obtained by scaling the target detection frame according to the size of the target object in the first image frame.

[0038] Alternatively, if the current image frame is not the first image frame of the video, such as the second image frame, the target detection box can be a detection box used for keypoint detection. This target detection box can be obtained based on the preset detection box, which will be explained in detail in subsequent steps and will not be described in detail here. Similarly, keypoint detection is performed on the target object in the current image frame based on the target detection box, and a first detection box for the target object is determined according to the result of the keypoint detection. For example, the first detection box can be a detection box obtained by scaling the target detection box to a certain extent.

[0039] Step 103: Obtain a second detection box that matches the first detection box from the at least one preset detection box.

[0040] Understandably, the second detection frame is also a certain preset detection frame. For example, the second detection frame may be the one among the at least one preset detection frames that is closest in size to the first detection frame, or the one among the at least one preset detection frames that is closest in position to the first detection frame, etc.

[0041] Step 104: Use the second detection box as the target detection box for key point detection of the target object in the next image frame of the current image frame.

[0042] For example, taking the current image frame as the first image frame, based on the detection box obtained after performing human body bounding box detection on the target person in the first image frame, key point detection is performed on the target person in the first image frame based on the detection box, and a first detection box A1 is determined for the target person according to the key point detection result. A second detection box B1 matching the first detection box A1 is obtained from the at least one preset detection box. For example, the one with the closest size to the first detection box A1 is obtained from the at least one preset detection box as the second detection box B1. Then, the second detection box B1 is used as the target detection box for key point detection on the target person in the second image frame. Furthermore, for the second image frame, key point detection is performed on the target person based on the second detection box B1, and a new first detection box A2 is determined for the target person based on the key point detection results. Then, a second detection box B2 matching the first detection box A2 is obtained from the at least one preset detection box. The second detection box B2 is then used as the target detection box for key point detection of the target person in the third image frame... By executing this process sequentially, the target detection box for key point detection of the target person in each image frame can be determined.

[0043] Understandably, the target detection box is the one that matches the first detection box determined from the at least one preset detection box. That is, the target detection box is actually a preset detection box, so there is no need to perform target object detection on each image frame to determine the target detection box.

[0044] In this embodiment, after key point detection of the target object in the current image frame is performed based on the target detection box determined from the preset detection box, a first detection box is determined according to the key point detection result. Then, a second detection box matching the first detection box is obtained from at least one preset detection box, and the second detection box is used as the target detection box for key point detection of the target object in the next image frame. Compared with the prior art, which requires deriving and calculating the detection box of the next image frame based on the key point detection result of the previous image frame, this application can directly determine the target detection box of the next image frame from the preset detection box without deriving and calculating based on the key point detection result. This effectively saves the calculation process of the electronic device, making the determination process of the detection box simpler and faster, and effectively saving the hardware cost of the electronic device. Furthermore, by determining the target detection box from at least one preset detection box, even if the target object jitters or fluctuates, the preset detection box will not fluctuate, and a detection box matching the first detection box of the target object can always be found in at least one preset detection box. This can more effectively ensure the accuracy of key point detection based on the target detection box, improve the accuracy of key point detection, thereby improving the stability of key point detection and reducing the jitter problem of key points.

[0045] Optionally, step 101 may include:

[0046] The first image frame of the video is divided into a grid, and the at least one preset detection box is determined based on the grid.

[0047] The electronic device can divide the first image frame into a grid based on the size of the first image frame, for example, dividing the first image frame into a certain number of grids. Alternatively, it can divide the grid based on the size of the target object in the first image frame, so that the target object can be located in one or a preset number of grids.

[0048] In this embodiment, the electronic device first divides the first image frame of the video into a grid, and then determines at least one preset detection box based on the divided grid. For example, the size of the preset detection box can be close to or the same as the size of the divided grid. This makes the determination of the preset detection box faster.

[0049] Optionally, the step of dividing the first image frame of the video into a grid includes:

[0050] Object bounding box detection is performed on the target object in the first image frame of the video to obtain a third detection bounding box for the target object;

[0051] Based on the dimensions of the third detection box and the first image frame, the first image frame is divided into a grid, wherein the dimensions of the divided grid match those of the third detection box.

[0052] For example, taking the target object as the target person in the first image frame, a human bounding box detection is performed on the target person in the first image frame to obtain a human bounding box (i.e., the third detection box) for the target person. Based on the size of the human bounding box and the size of the first image frame, the first image frame is divided into a grid so that the size of the divided grid matches the size of the human bounding box. For example, the size of the divided grid is close to or the same as the size of the human bounding box.

[0053] For example, assuming the width and height of the first image frame are w and h respectively, and the width and height of the human detection box (i.e., the third detection box) are w1 and h1 respectively, then the resulting grid is (M, N), where M = w / w1 and N = h / h1. For example, M and N are both 6, meaning the number of grids after division is 6 × 6, a total of 36.

[0054] In this embodiment, a third detection box is obtained by performing object bounding box detection on the target object in the first image frame. Then, the first image frame is divided into grids based on the third detection box so that the size of the divided grid matches the third detection box. The preset detection box is determined based on the divided grid, which ensures that the subsequently obtained preset detection box is closer to the real detection box, thereby ensuring that the subsequent target detection box is closer to the real detection box, thus effectively ensuring the accuracy of key point detection in subsequent image frames.

[0055] Optionally, when the current image frame is the first image frame, the step of performing keypoint detection on the target object in the current image frame based on the target detection box includes:

[0056] Key point detection is performed on the target object in the current image frame based on the third detection box.

[0057] In this embodiment, taking a target person in a video image frame as an example, for the first image frame, a human bounding box is detected for the target person in the first image frame to obtain a human bounding box (i.e., the third detection box) for the target person. Keypoint detection is then performed on the target person in the first image frame based on this human bounding box. For example, the target person can be cropped based on the human bounding box, and the cropped image including the target person can be input into a keypoint network model for keypoint detection to obtain the keypoint detection results output by the network model.

[0058] In this embodiment of the application, for the first image frame of the video, key point detection can be performed based on the third detection box obtained after human body bounding box detection of the target person, thereby effectively ensuring the implementation of subsequent processes and ensuring that each image frame in the video image frame can obtain the corresponding key point detection result.

[0059] Optionally, determining the at least one preset detection box based on the divided grid includes:

[0060] Obtain the first grid center of each grid after grid division;

[0061] Based on the first grid center, at least one first preset detection box is determined for each grid, and the at least one preset detection box includes the at least one first preset detection box;

[0062] Wherein, the center of the first grid of the target grid coincides with the center of the first preset detection frame corresponding to the target grid, the target grid is any one of the divided grids, and in the at least one first preset detection frame corresponding to each grid, any two first preset detection frames have different sizes.

[0063] It should be noted that the first grid center is also the center of the divided grid. For the divided grid, at least one first preset detection frame is determined for each grid based on the first grid center of each grid, and the center of each first preset detection frame coincides with the first grid center of the grid, and the size of any two first preset detection frames of the grid is different.

[0064] For example, such as Figure 2As shown, after determining the first grid center of a certain grid, different proportions of the first preset detection boxes can be set based on this first grid center to obtain multiple first preset detection boxes corresponding to that grid. The settings include the aspect ratio and scaling ratio of the first preset detection boxes. The aspect ratio refers to the ratio to the width and height of the grid, and the scaling ratio refers to the scaling ratio specific to the grid. For example, aspect ratios such as [1.0, 1.0], [1.2, 1.2], [1.0, 1.5], and [1.5, 1.0] can be set, and scaling ratios such as 0.8, 0.9, and 1.0 can be used. For the same aspect ratio, a corresponding first preset detection box can be obtained by scaling based on each scaling ratio. Assuming that the number of aspect ratios is 4 and the number of scaling ratios is 3, a total of 12 first preset detection boxes will be obtained, thus effectively enriching the number and size of the first preset detection boxes.

[0065] In this embodiment, each grid after division corresponds to at least one first preset detection box. Thus, for grids at different positions in the first image frame, each grid corresponds to at least one first preset detection box of different sizes, effectively enriching the number and size of preset detection boxes. This makes it easier to obtain a target detection box that matches the first detection box from at least one preset detection box, which in turn helps to improve the accuracy of key point detection based on the target detection box.

[0066] Optionally, after determining at least one first preset detection box corresponding to each grid based on the first grid center, the method further includes:

[0067] Obtain the offset of the target mesh;

[0068] The offset grid and the second grid center of the offset grid are determined based on the offset amount;

[0069] At least one second preset detection box corresponding to the offset grid is determined based on the second grid center, and the at least one preset detection box includes the at least one first preset detection box and the at least one second preset detection box.

[0070] In this embodiment, after obtaining the divided grid and at least one first preset detection box corresponding to each grid, the offset of any one of the grids is determined. For example, the offset can be determined based on user input, and the offset indicates how much to offset from the original grid. Based on the offset, the offset grid after offset from the original grid and the second grid center of the offset grid can be determined. It should be noted that the original grid does not change position; the offset grid can be a copy of the original grid before offsetting.

[0071] Further, at least one second preset detection frame corresponding to the offset grid is determined based on the center of the second grid. The method for determining the at least one second preset detection frame can refer to the method for determining the at least one first preset detection frame described above, and will not be described in detail here. Among the at least one second preset detection frames corresponding to the same offset grid, any two second preset detection frames have different sizes.

[0072] In this embodiment, an offset grid is determined based on the divided grid, and each offset grid corresponds to at least one second preset detection box. This effectively enriches the number and size of the preset detection boxes, and allows different positions in the image frame to be covered by preset detection boxes as much as possible, thus improving the richness of the preset detection boxes. This, in turn, makes it easier to obtain a target detection box that matches the first detection box from at least one preset detection box, effectively improving the matching degree between the preset detection box and the first detection box, thereby improving the accuracy of key point detection. Furthermore, the movement of the target object in different image frames can be more smoothly matched to the preset detection box, thereby ensuring the stability of key point detection based on the target detection box.

[0073] Optionally, if the current image frame is not the first image frame, after using the second detection box as the target detection box for keypoint detection in the next image frame of the current image frame, the method further includes:

[0074] Obtain the ratio of the size of the target detection box to the size of the third detection box;

[0075] If the ratio is greater than a preset threshold, the first image frame or the current image frame is divided into grids according to the target detection box, and at least one preset detection box is re-determined based on the divided grid.

[0076] For example, assuming the current image frame is the second image frame, keypoint detection is performed on the target object in the second image frame based on the target detection bounding box determined in the first image frame. Based on the keypoint detection results, a first detection bounding box for the target object in the second image frame is determined. Then, a second detection bounding box matching the first detection bounding box is identified from at least one preset detection bounding box. This second detection bounding box is used as the target detection bounding box for keypoint detection of the target object in the third image frame. At this point, the ratio of the size of the matched target detection bounding box to the size of the third detection bounding box (i.e., the size obtained by object bounding box detection of the target object in the first image frame) can be determined. If this ratio is greater than a preset threshold, it indicates that the size ratio of the target object in the current image frame has changed relative to the size ratio of the target object in the first image frame. For example, as a person in a video moves from far to near or from near to far, the size ratio of the person changes in different image frames. Therefore, the first image frame or the current image frame is divided into a grid based on the target detection bounding box. The size of the newly divided grid can match, or be similar to, the size of the target detection bounding box. Then, at least one preset detection bounding box is re-determined based on the divided grid. The determination of target detection bounding boxes in subsequent image frames is then performed by matching and determining these at least one re-determined preset detection bounding box. This ensures that the subsequently determined target detection bounding boxes are closer to the size of the target object, thereby more effectively ensuring the accuracy of keypoint detection based on the target detection bounding box.

[0077] Optionally, in this embodiment of the application, step 102 may include:

[0078] The target object in the current image frame is cropped according to the target detection box to obtain a first image including the target object;

[0079] Perform key point detection on the first image, and determine the minimum bounding box based on the key point detection results;

[0080] The smallest outer bounding box is used as the first detection box for the target object.

[0081] In this embodiment, after determining the target detection box of the current image frame, the target object is cropped based on the target detection box. Understandably, the target detection box is the bounding box of the target object, and the image covered by the target detection box can be cropped to obtain a first image including the target object. Further, keypoint detection is performed on the first image; for example, if the target object is a person, human body keypoint detection is performed, and the minimum outer bounding box is determined based on the coordinates of each detected keypoint—that is, the minimum outer bounding box among these keypoints that can enclose the target person, such as a minimum outer rectangle. This minimum outer rectangle is used as the first detection box for the target object.

[0082] Optionally, after obtaining the minimum outer bounding box, the minimum outer bounding box can be scaled up, and the scaled minimum outer bounding box can be used as the first detection box.

[0083] In this embodiment, the target object in the current image frame is cropped using a target detection box, and then key point detection is performed on the cropped first image. This eliminates the need to perform key point detection on the entire current image frame, effectively saving the computing resources of electronic devices and making key point detection more targeted, thus helping to improve the accuracy of key point detection.

[0084] Optionally, in this embodiment of the application, step 103 may include:

[0085] Obtain the intersection-union ratio (IUU) between each preset detection frame and the first detection frame in the at least one preset detection frame;

[0086] The preset detection box with the largest intersection-union ratio among the at least one preset detection boxes is determined as the second detection box.

[0087] In this embodiment, after determining at least one preset detection box and determining a first detection box based on the key point detection results of the target object in the current image frame, the Intersection over Union (IOU) value of each preset detection box in the at least one preset detection box and the first detection box is obtained. The preset detection box with the largest IOU value is used as the second detection box, which is the target detection box used for key point detection of the target object in the next image frame. In this way, the target detection box that best matches the first detection box can be obtained from multiple preset detection boxes, thereby helping to improve the accuracy of key point detection.

[0088] Please refer to Figure 3 , Figure 3 This is a flowchart of another method for determining a detection frame provided in an embodiment of this application, such as... Figure 3As shown, the method includes the following steps: Human object detection is performed on the first image frame (i.e., the first image frame) of the video to obtain an initial detection box; a suitable preset detection box is determined based on the initial detection box; the human object is cropped according to the initial detection box to obtain a cropped human image; keypoint detection is performed on the human image; an inferred detection box is determined based on the keypoint detection results; IOU calculation is performed between the preset detection box and the inferred detection box; a new detection box that best matches the inferred detection box is determined from the preset detection box based on the IOU calculation result; and based on the new detection box… The detection box performs portrait cropping on the next image frame, i.e., the second image frame, to obtain the cropped portrait. Keypoint detection is also performed on this portrait. Based on the keypoint detection results, a deduced detection box is determined. An IOU calculation is performed between the preset detection box and the deduced detection box. Based on the IOU calculation result, a new detection box that best matches the deduced detection box is determined from the preset detection box. This new detection box is used to crop the portrait on the next image frame for keypoint detection... Following this process, keypoint detection and detection box determination for video image frames are completed sequentially.

[0089] In this embodiment, the target detection box for key point detection in the next image frame can be directly determined from the preset detection box without the need for derivation calculation based on the key point detection results. This effectively saves the calculation process of the electronic device and makes the determination of the detection box simpler and faster.

[0090] Please refer to Figure 4 , Figure 4 This is a structural diagram of a detection frame determination device provided in an embodiment of this application, as shown below. Figure 4 As shown, the detection frame determining device 400 includes:

[0091] The first determining module 401 is used to determine at least one preset detection box in the first image frame of the video.

[0092] The detection module 402 is used to perform key point detection on the target object in the current image frame according to the target detection box, and determine a first detection box for the target object according to the result of the key point detection, wherein the current image frame is any image frame in the video image frame;

[0093] The acquisition module 403 is used to acquire a second detection box that matches the first detection box from the at least one preset detection box;

[0094] The second determining module 404 is used to use the second detection box as a target detection box for key point detection of the target object in the next image frame of the current image frame.

[0095] The target detection box is used for target object detection or key point detection.

[0096] Optionally, the first determining module 401 is further configured to:

[0097] The first image frame of the video is divided into a grid, and the at least one preset detection box is determined based on the grid.

[0098] Optionally, the first determining module 401 is further configured to:

[0099] Object bounding box detection is performed on the target object in the first image frame of the video to obtain a third detection bounding box for the target object;

[0100] Based on the dimensions of the third detection box and the first image frame, the first image frame is divided into a grid, wherein the dimensions of the divided grid match those of the third detection box.

[0101] Optionally, the first determining module 401 is further configured to:

[0102] Obtain the first grid center of each grid after grid division;

[0103] Based on the first grid center, at least one first preset detection box is determined for each grid, and the at least one preset detection box includes the at least one first preset detection box;

[0104] Wherein, the center of the first grid of the target grid coincides with the center of the first preset detection frame corresponding to the target grid, the target grid is any one of the divided grids, and in the at least one first preset detection frame corresponding to each grid, any two first preset detection frames have different sizes.

[0105] Optionally, the first determining module 401 is further configured to:

[0106] Obtain the offset of the target mesh;

[0107] The offset grid and the second grid center of the offset grid are determined based on the offset amount;

[0108] At least one second preset detection box corresponding to the offset grid is determined based on the second grid center, and the at least one preset detection box includes the at least one first preset detection box and the at least one second preset detection box.

[0109] Optionally, when the current image frame is the first image frame, the detection module 402 is further configured to:

[0110] Key point detection is performed on the target object in the current image frame based on the third detection box.

[0111] Optionally, if the current image frame is not the first image frame, the acquisition module 403 is further configured to:

[0112] Obtain the ratio of the size of the target detection box to the size of the third detection box;

[0113] The device further includes a partitioning module, which is used to partition the first image frame or the current image frame into a grid according to the target detection box when the ratio is greater than a preset threshold, and redetermine at least one preset detection box based on the partitioned grid.

[0114] Optionally, the detection module 402 is further configured to:

[0115] The target object in the current image frame is cropped according to the target detection box to obtain a first image including the target object;

[0116] Perform key point detection on the first image, and determine the minimum bounding box based on the key point detection results;

[0117] The smallest outer bounding box is used as the first detection box for the target object.

[0118] Optionally, the acquisition module 403 is further configured to:

[0119] Obtain the intersection-union ratio (IUU) between each preset detection frame and the first detection frame in the at least one preset detection frame;

[0120] The preset detection box with the largest intersection-union ratio among the at least one preset detection boxes is determined as the second detection box.

[0121] In this embodiment of the application, the device determines the target detection box from at least one preset detection box. Even if the target object jitters or fluctuates, the preset detection box will not fluctuate. Moreover, a detection box that matches the first detection box of the target object can always be found in at least one preset detection box. This can more effectively ensure the accuracy of key point detection based on the target detection box and improve the accuracy of key point detection.

[0122] The detection frame determining device 400 in this application embodiment can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc., and this application embodiment does not specifically limit the scope.

[0123] The detection frame determining device 400 in this embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this embodiment does not specifically limit its use.

[0124] The detection frame determination device 400 provided in this application embodiment can achieve Figure 1 The various processes implemented in the method embodiments described above can achieve the same technical effects, and will not be repeated here to avoid duplication.

[0125] This application also provides an electronic device. See also... Figure 5 , Figure 5 This is a structural diagram of an electronic device provided in an embodiment of this application, such as... Figure 5 As shown, the electronic device includes: a processor 500, a memory 520, and a program or instructions stored in the memory 520 and executable on the processor 500. The processor 500 is used to read the program or instructions from the memory 520. The electronic device also includes a bus interface and a transceiver 510.

[0126] Transceiver 510 is used to receive and send data under the control of processor 500.

[0127] Among them, Figure 5In this context, the bus architecture may include any number of interconnected buses and bridges, specifically linking various circuits together, represented by one or more processors (processor 300) and memory (memory 520). The bus architecture may also link various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. A bus interface provides an interface. Transceiver 510 may be multiple elements, including transmitters and transceivers, providing a unit for communicating with various other devices over a transmission medium. Processor 500 is responsible for managing the bus architecture and general processing, and memory 520 may store data used by processor 500 during operation.

[0128] The processor 500 is used to read programs or instructions from the memory 520 and execute the following steps:

[0129] Determine at least one preset detection box in the first image frame of the video;

[0130] Based on the target detection box, key point detection is performed on the target object in the current image frame, and a first detection box for the target object is determined based on the result of the key point detection. The current image frame is any image frame in the video.

[0131] Obtain a second detection box that matches the first detection box from the at least one preset detection box;

[0132] The second detection box is used as the target detection box for key point detection of the target object in the next image frame of the current image frame;

[0133] The target detection box is used for target object detection or key point detection.

[0134] Optionally, the processor 500 is also configured to read programs or instructions from the memory 520 and perform the following steps:

[0135] The first image frame of the video is divided into a grid, and the at least one preset detection box is determined based on the grid.

[0136] Optionally, the processor 500 is also configured to read programs or instructions from the memory 520 and perform the following steps:

[0137] Object bounding box detection is performed on the target object in the first image frame of the video to obtain a third detection bounding box for the target object;

[0138] Based on the dimensions of the third detection box and the first image frame, the first image frame is divided into a grid, wherein the dimensions of the divided grid match those of the third detection box.

[0139] Optionally, the processor 500 is also configured to read programs or instructions from the memory 520 and perform the following steps:

[0140] Obtain the first grid center of each grid after grid division;

[0141] Based on the first grid center, at least one first preset detection box is determined for each grid, and the at least one preset detection box includes the at least one first preset detection box;

[0142] Wherein, the center of the first grid of the target grid coincides with the center of the first preset detection frame corresponding to the target grid, the target grid is any one of the divided grids, and in the at least one first preset detection frame corresponding to each grid, any two first preset detection frames have different sizes.

[0143] Optionally, the processor 500 is also configured to read programs or instructions from the memory 520 and perform the following steps:

[0144] Obtain the offset of the target mesh;

[0145] The offset grid and the second grid center of the offset grid are determined based on the offset amount;

[0146] At least one second preset detection box corresponding to the offset grid is determined based on the second grid center, and the at least one preset detection box includes the at least one first preset detection box and the at least one second preset detection box.

[0147] Optionally, if the current image frame is the first image frame, the processor 500 is further configured to read the program or instructions in the memory 520 and perform the following steps:

[0148] Key point detection is performed on the target object in the current image frame based on the third detection box.

[0149] Optionally, if the current image frame is not the first image frame, the processor 500 is further configured to read the program or instructions in the memory 520 and perform the following steps:

[0150] Obtain the ratio of the size of the target detection box to the size of the third detection box;

[0151] If the ratio is greater than a preset threshold, the first image frame or the current image frame is divided into grids according to the target detection box, and at least one preset detection box is re-determined based on the divided grid.

[0152] Optionally, the processor 500 is also configured to read programs or instructions from the memory 520 and perform the following steps:

[0153] The target object in the current image frame is cropped according to the target detection box to obtain a first image including the target object;

[0154] Perform key point detection on the first image, and determine the minimum bounding box based on the key point detection results;

[0155] The smallest outer bounding box is used as the first detection box for the target object.

[0156] Optionally, the processor 500 is also configured to read programs or instructions from the memory 520 and perform the following steps:

[0157] Obtain the intersection-union ratio (IUU) between each preset detection frame and the first detection frame in the at least one preset detection frame;

[0158] The preset detection box with the largest intersection-union ratio among the at least one preset detection boxes is determined as the second detection box.

[0159] It should be noted that the electronic device provided in the embodiments of this application is capable of achieving... Figure 1 The various processes implemented in the method embodiments described above can achieve the same technical effects, and will not be repeated here to avoid duplication.

[0160] This application embodiment also provides a readable storage medium storing a program or instructions that, when executed by a processor, implement the above-described functionality. Figure 1 The various processes in the method embodiments described herein can achieve the same technical effect, and will not be repeated here to avoid repetition.

[0161] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0162] This application embodiment also provides a chip, the chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the above. Figure 1 The various processes in the method embodiments described herein can achieve the same technical effect, and will not be repeated here to avoid repetition.

[0163] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0164] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0165] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0166] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A method for determining a detection frame, characterized in that, include: Determine at least one preset detection box in the first image frame of the video; Based on the target detection box, key point detection is performed on the target object in the current image frame, and a first detection box for the target object is determined based on the result of the key point detection. The current image frame is any image frame in the video. Obtain a second detection box that matches the first detection box from the at least one preset detection box; The second detection box is used as the target detection box for key point detection of the target object in the next image frame of the current image frame; The target detection box is used for target object detection or key point detection. Determining at least one preset detection box in the first image frame of the video includes: The first image frame of the video is divided into grids, and the at least one preset detection box is determined based on the grids. The step of dividing the first image frame of the video into a grid includes: Object bounding box detection is performed on the target object in the first image frame of the video to obtain a third detection bounding box for the target object; Based on the size of the third detection box and the first image frame, the first image frame is divided into a grid, wherein the size of the divided grid matches that of the third detection box; When the current image frame is not the first image frame, after using the second detection box as the target detection box for keypoint detection in the next image frame of the current image frame, the method further includes: Obtain the ratio of the size of the target detection box to the size of the third detection box; If the ratio is greater than a preset threshold, the first image frame or the current image frame is divided into grids according to the target detection box, and at least one preset detection box is re-determined based on the divided grid.

2. The method according to claim 1, characterized in that, Determining the at least one preset detection box based on the divided grid includes: Obtain the first grid center of each grid after grid division; Based on the first grid center, at least one first preset detection box is determined for each grid, and the at least one preset detection box includes the at least one first preset detection box; Wherein, the center of the first grid of the target grid coincides with the center of the first preset detection frame corresponding to the target grid, the target grid is any one of the divided grids, and in the at least one first preset detection frame corresponding to each grid, any two first preset detection frames have different sizes.

3. The method according to claim 2, characterized in that, After determining at least one first preset detection box corresponding to each grid based on the center of the first grid, the method further includes: Obtain the offset of the target mesh; The offset grid and the second grid center of the offset grid are determined based on the offset amount; At least one second preset detection box corresponding to the offset grid is determined based on the second grid center, and the at least one preset detection box includes the at least one first preset detection box and the at least one second preset detection box.

4. The method according to claim 1, characterized in that, When the current image frame is the first image frame, the step of performing keypoint detection on the target object in the current image frame based on the target detection box includes: Key point detection is performed on the target object in the current image frame based on the third detection box.

5. The method according to any one of claims 1-4, characterized in that, The step of performing keypoint detection on the target object in the current image frame based on the target detection bounding box, and determining a first detection bounding box for the target object based on the result of the keypoint detection, includes: The target object in the current image frame is cropped according to the target detection box to obtain a first image including the target object; Perform key point detection on the first image, and determine the minimum bounding box based on the key point detection results; The smallest outer bounding box is used as the first detection box for the target object.

6. The method according to any one of claims 1-4, characterized in that, The step of obtaining a second detection box that matches the first detection box from the at least one preset detection box includes: Obtain the intersection-union ratio (IUU) between each preset detection frame and the first detection frame in the at least one preset detection frame; The preset detection box with the largest intersection-union ratio among the at least one preset detection boxes is determined as the second detection box.

7. A device for determining a detection frame, characterized in that, include: The first determining module is used to determine at least one preset detection box in the first image frame of the video; The detection module is used to perform key point detection on the target object in the current image frame according to the target detection box, and determine the first detection box for the target object according to the result of the key point detection, wherein the current image frame is any image frame in the video. The acquisition module is used to acquire a second detection box that matches the first detection box from the at least one preset detection box; The second determining module is used to use the second detection box as a target detection box for key point detection of the target object in the next image frame of the current image frame; The target detection box is used for target object detection or key point detection. The first determining module is further configured to: The first image frame of the video is divided into grids, and the at least one preset detection box is determined based on the grids. The first determining module is further configured to: Object bounding box detection is performed on the target object in the first image frame of the video to obtain a third detection bounding box for the target object; Based on the size of the third detection box and the first image frame, the first image frame is divided into a grid, wherein the size of the divided grid matches that of the third detection box; If the current image frame is not the first image frame, the acquisition module is further configured to: Obtain the ratio of the size of the target detection box to the size of the third detection box; The device further includes a partitioning module, which is used to partition the first image frame or the current image frame into a grid according to the target detection box when the ratio is greater than a preset threshold, and redetermine at least one preset detection box based on the partitioned grid.

8. An electronic device, characterized in that, It includes a processor and a memory, the memory storing a program or instructions that can run on the processor, the program or instructions being executed by the processor to implement the steps of the method for determining a detection box as described in any one of claims 1-6.

9. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the method for determining a detection frame as described in any one of claims 1-6.

Images