Object detection method and object detection device

Abstract

An object detection method includes: an AI accelerator receiving an input image and performing object detection inference on the input image through an object detection model to generate a downsampled feature map. The downsampled feature map includes an object bounding box coordinate feature map and an object classification feature map; the AI ​​accelerator performing decoding operations on the object bounding box coordinate feature map through the object detection model to obtain multiple object bounding box coordinates corresponding to multiple feature point positions and corresponding to multiple object bounding box coordinates of the input image; the AI ​​accelerator transmitting each object bounding box coordinate and the confidence score corresponding to each object bounding box coordinate to a processing unit; the processing unit comparing each confidence score with a threshold value; and when any confidence score is greater than the threshold value, the processing unit outputs its corresponding object bounding box coordinates.

Claims

1. An object detection method, applicable to an object detection device, the object detection device comprising an AI accelerator and a processing unit, comprising: the AI ​​accelerator receiving an input image and performing object detection inference on the input image through an object detection model to generate a downsampled feature map, the downsampled feature map comprising an object bounding box coordinate feature map and an object classification feature map, the object classification feature map comprising multiple confidence scores corresponding to multiple feature point locations; the AI ​​accelerator performing a decoding operation on the object bounding box coordinate feature map through the object detection model to obtain multiple object bounding box coordinates corresponding to the feature point locations and corresponding to the input image; the AI ​​accelerator transmitting each object bounding box coordinate and the confidence score corresponding to each object bounding box coordinate to the processing unit; the processing unit comparing each confidence score with a threshold value; and when any confidence score is greater than the threshold value, the processing unit outputs its corresponding object bounding box coordinates; wherein... The AI ​​accelerator performs the object detection inference and the decoding operation, and the processing unit only compares each confidence score with the threshold value, and outputs the corresponding object frame coordinates when any confidence score is greater than the threshold value.

2. The object detection method as described in claim 1, wherein the decoding operation comprises: reading a plurality of boundary offset vectors corresponding to the positions of each feature point from the object bounding box coordinate feature map; performing a softmax operation on the boundary offset vectors and performing an inner product with a basis vector to generate a plurality of offsets; generating a pre-scaled object bounding box coordinate corresponding to the position of the feature point based on the offsets and the coordinates of the corresponding feature point positions; and generating the object bounding box coordinate corresponding to the position of the feature point and corresponding to the input image based on the pre-scaled object bounding box coordinate and an image scaling factor.

3. The object detection method as described in claim 2, wherein the depth of the object bounding box coordinate feature map is 64, the number of boundary offset vectors corresponding to each feature point position is 4, and the dimension of each boundary offset vector is 16.

4. The object detection method as described in claim 3, wherein the boundary offset vectors corresponding to the positions of each feature point include a left offset vector corresponding to the left direction, a right offset vector corresponding to the right direction, an up offset vector corresponding to the up direction, and a down offset vector corresponding to the down direction, and the offsets include a left offset corresponding to the left direction, a right offset corresponding to the right direction, an up offset corresponding to the up direction, and a down offset corresponding to the down direction.

5. The object detection method as claimed in claim 4, wherein the coordinates of the feature point location include a horizontal coordinate and a vertical coordinate, and the step of generating the pre-scaled object frame coordinates corresponding to the feature point location based on the offsets and the corresponding coordinates of the feature point location comprises: subtracting the left offset from the horizontal coordinate to generate the left boundary of the pre-scaled object frame coordinates; adding the right offset to the horizontal coordinate to generate the right boundary of the pre-scaled object frame coordinates; subtracting the upper offset from the vertical coordinate to generate the upper boundary of the pre-scaled object frame coordinates; and adding the lower offset to the vertical coordinate to generate the lower boundary of the pre-scaled object frame coordinates.

6. The object detection method as described in claim 5, wherein the step of generating the object frame coordinates corresponding to the feature point position and the input image based on the object frame coordinates before scaling and the image scaling factor includes: multiplying the left boundary, right boundary, top boundary and bottom boundary of the object frame coordinates before scaling by the image scaling factor; wherein the left boundary and right boundary obtained after multiplication are the horizontal boundaries of the object frame coordinates, and the top boundary and bottom boundary obtained after multiplication are the vertical boundaries of the object frame coordinates.

7. The object detection method as claimed in claim 1, wherein the decoding operation comprises: reading a plurality of boundary offset vectors corresponding to the positions of each feature point from the object bounding box coordinate feature map; generating a plurality of offsets based on the boundary offset vectors and at least one predefined anchor box; generating a pre-scaled object bounding box coordinate corresponding to the position of the feature point based on the offsets and the coordinates of the corresponding feature point positions; and generating the object bounding box coordinate corresponding to the position of the feature point and corresponding to the input image based on the pre-scaled object bounding box coordinate and an image scaling factor.

8. The object detection method as described in claim 7, wherein the depth of the object bounding box coordinate feature map is 64, the number of the boundary offset vectors corresponding to each feature point position is 4, and the dimension of each boundary offset vector is 16.

9. The object detection method as claimed in claim 8, wherein the boundary offset vectors corresponding to the positions of each feature point include a left offset vector corresponding to the left direction, a right offset vector corresponding to the right direction, an upper offset vector corresponding to the upper direction, and a lower offset vector corresponding to the lower direction; the predefined anchor frame includes a horizontal center coordinate and a vertical center coordinate corresponding to the position of the feature point; the offsets include a left offset corresponding to the left direction, a right offset corresponding to the right direction, an upper offset corresponding to the upper direction, and a lower offset corresponding to the lower direction; and the step of generating the offsets based on the boundary offset vectors and at least one predefined anchor frame includes: adding the horizontal center coordinate to the left offset vector to generate the left offset; adding the horizontal center coordinate to the right offset vector to generate the right offset; adding the vertical center coordinate to the upper offset vector to generate the upper offset; and adding the vertical center coordinate to the lower offset vector to generate the lower offset.

10. An object detection apparatus, comprising: an AI accelerator, configured to: receive an input image and perform object detection inference on the input image through an object detection model to generate a downsampled feature map, the downsampled feature map including an object bounding box coordinate feature map and an object classification feature map, the object classification feature map including multiple confidence scores corresponding to multiple feature point locations; and perform a decoding operation on the object bounding box coordinate feature map through the object detection model to obtain multiple object bounding box coordinates corresponding to the feature point locations and corresponding to the input image; and a processing unit coupled to the AI ​​accelerator, configured to: receive each of the object bounding box coordinates and the confidence score corresponding to each of the object bounding box coordinates from the AI ​​accelerator; compare each confidence score with a threshold value; and when any of the confidence scores is greater than the threshold value, output the corresponding object bounding box coordinates; wherein... The AI ​​accelerator performs the object detection inference and the decoding operation, and the processing unit only compares each confidence score with the threshold value, and outputs the corresponding object frame coordinates when any confidence score is greater than the threshold value.

Images