Data processing method, device, electronic device and storage medium

Abstract

Embodiments of the present invention provide a method, apparatus, electronic device, and computer storage medium for target detection and its model training. A data processing method includes: obtaining an image to be detected, wherein the image to be detected includes a target object to be detected, and the target object includes a first dimension value; inputting the image to be detected into a target detection model, wherein the target detection model generates feature maps of multiple levels corresponding to the image to be detected; and obtaining feature maps of the level corresponding to the first dimension value of the target object to detect the target object and output a detection result. Through embodiments of the present invention, more accurate detection results can be obtained.

Claims

1. A data processing method, comprising: Get the model training request for the object detection model; Acquire, according to the model training request, a sample image for training the target detection model, wherein the sample image includes a labeling box and a target object, and the labeling box includes a first dimension value; Inputting the sample image into the convolutional layer of the target detection model to obtain feature maps of multiple levels; Performing a bounding box prediction for the target object in a feature map of a level corresponding to the first dimension value, wherein the feature map of the level corresponding to the first dimension value is modified and determined based on the first dimension value and an offset to a candidate feature map, the candidate feature map being determined based on the feature map of the level corresponding to the labeled box, and the offset being an offset between each boundary of the target object and the labeled box; The object detection model is trained based on the bounding box prediction result and the loss function.

2. The method according to claim 1, wherein The performing bounding box prediction on the target object in the feature map of the level corresponding to the first dimension value includes: Calculating the offset between the target object and each boundary of the annotation box; Determining a feature map of a level corresponding to the first dimension value according to the first dimension value and the offset; In the feature map of the level corresponding to the first dimension value, a bounding box prediction is performed on the target object.

3. The method according to claim 2, wherein: The determining, according to the first dimension value and the offset, a feature map of a level corresponding to the first dimension value includes: determining allocation information of the target object according to the first dimension value and the offset; Determine a feature map of a level corresponding to the first dimension value according to the allocation information.

4. The method according to any one of claims 1 to 3, wherein: The first dimension value is the aspect ratio or width-to-length ratio of the annotation box.

5. The method according to any one of claims 1 to 3, wherein: The acquiring, according to the model training request, sample images for training the object detection model comprises: According to the model training request, sample images for training the object detection model are obtained locally from the SaaS platform; or, According to the model training request, the SaaS platform obtains sample images for training the object detection model from a third party; or, According to the model training request, the SaaS platform obtains sample images for training the object detection model from the client.

6. A data processing method comprising: Acquire an image to be detected, wherein the image to be detected includes a target object to be detected, and the target object includes a first dimension value; Inputting the image to be detected into an object detection model, wherein the object detection model generates feature maps of multiple levels corresponding to the image to be detected; Obtain a detection result of a target object detection output by using a feature map of a level corresponding to the first dimension value, wherein the feature map of the level corresponding to the first dimension value corrects and determines a candidate feature map based on the first dimension value and an offset, and the candidate feature map is determined based on the feature map of the level corresponding to the annotation box, and the offset is the offset between the target object and each boundary of the annotation box.

7. The method according to claim 6, wherein: The feature map of the level corresponding to the first dimension value is: a feature map of the level corresponding to the aspect ratio or the width-to-length ratio of the target object.

8. The method according to claim 6, wherein: The feature map of the level corresponding to the first dimension value is: a feature map of the level corresponding to the aspect ratio or height-to-length ratio of the target object.

9. The method according to claim 6, wherein: The feature map of the level corresponding to the first dimension value is: a feature map of the level corresponding to the horizontal distance determined according to the horizontal coordinate information of the target object and the vertical distance determined according to the vertical coordinate information of the target object.

10. The method according to claim 7, wherein: The acquiring of the feature map of the level corresponding to the first dimension value to perform target object detection and output a detection result includes: According to the aspect ratio or width-to-length ratio of the target object, the target object is assigned to a feature map of a corresponding level for bounding box detection; Obtain a detection result of a bounding box detection output by the target detection model.

11. The method according to claim 10, wherein: The step of allocating the target object to a feature map of a corresponding level for bounding box detection according to the aspect ratio or width-to-length ratio of the target object includes: According to the aspect ratio or the maximum ratio of the aspect ratio of the target object, the target object is assigned to the feature map of the corresponding level for bounding box detection.

12. The method according to any one of claims 6 to 11, wherein: The step of inputting the image to be detected into the target detection model comprises: The image to be detected is scaled by resampling based on the regional pixel relationship, and the scaled image to be detected is input into the target detection model.

13. The method according to claim 6, wherein: The image to be detected includes a plurality of target objects to be detected; Inputting the image to be detected into the target detection model includes: performing image segmentation on the image to be detected to obtain multiple image areas where the multiple target objects to be detected are located; generating corresponding multiple sub-images based on the multiple image areas; and inputting the multiple sub-images into the target detection model respectively.

14. The method according to claim 6, wherein The acquiring of the feature map of the level corresponding to the first dimension value to perform target object detection and output a detection result includes: The target object is detected based on the feature map of the level corresponding to the first dimension value and the color of the target object, and a detection result is output.

15. A data processing method, comprising: Acquire a first image, wherein the first image includes a preset target object, and the target object includes a first dimension value; Inputting the first image into an object detection model, wherein the object detection model generates a plurality of layers of feature maps corresponding to the first image; Obtaining a target object detection result outputted by performing target object detection on a feature map of a level corresponding to the first dimension value, wherein the feature map of the level corresponding to the first dimension value corrects and determines a candidate feature map based on the first dimension value and an offset, the candidate feature map being determined based on the feature map of the level corresponding to the annotation box, and the offset being an offset between the target object and each boundary of the annotation box; A second image is synthesized according to the detection result of the target object and preset image information.

16. A data processing method comprising: Acquire a sample image for training a target detection model, wherein the sample image includes a labeling box and a target object, and the labeling box includes a first dimension value; Inputting the sample image into the convolutional layer of the target detection model to obtain feature maps of multiple levels; Performing a bounding box prediction for the target object in a feature map of a level corresponding to the first dimension value, wherein the feature map of the level corresponding to the first dimension value is modified and determined based on the first dimension value and an offset to a candidate feature map, the candidate feature map being determined based on the feature map of the level corresponding to the labeled box, and the offset being an offset between each boundary of the target object and the labeled box; The object detection model is trained based on the bounding box prediction result and the loss function.

17. The method according to claim 16, wherein The performing bounding box prediction on the target object in the feature map of the level corresponding to the first dimension value includes: Calculating the offset between the target object and each boundary of the annotation box; Determining a feature map of a level corresponding to the first dimension value according to the first dimension value and the offset; In the feature map of the level corresponding to the first dimension value, a bounding box prediction is performed on the target object.

18. The method according to claim 17, wherein The determining, according to the first dimension value and the offset, a feature map of a level corresponding to the first dimension value includes: determining allocation information of the target object according to the first dimension value of the annotation box and the offset; Determine a feature map of a level corresponding to the first dimension value according to the allocation information.

19. The method according to claim 18, wherein The determining, according to the first dimension value of the annotation box and the offset, the allocation information of the target object includes: The allocation information of the target object is determined according to the aspect ratio or width-to-length ratio of the annotation box and the offset.

20. The method according to claim 19, wherein The determining the allocation information of the target object according to the aspect ratio or width-to-length ratio of the annotation box and the offset includes: Determining a maximum ratio from the aspect ratio and the width-to-length ratio of the annotation frame; The allocation information of the target object is determined according to the maximum ratio of the marked boxes and the offset.

21. The method according to claim 20, wherein The determining, according to the maximum ratio of the marked frame and the offset, the allocation information of the target object includes: Processing the maximum ratio of the marked box according to a preset activation function to obtain a processing result; The allocation information of the target object is determined according to the ratio of the offset to the processing result.

22. The method according to claim 18, wherein The determining, according to the first dimension value of the annotation box and the offset, the allocation information of the target object includes: The allocation information of the target object is determined according to the aspect ratio or the height-to-length ratio of the annotation box and the offset.

23. The method according to claim 18, wherein The determining, according to the first dimension value of the annotation box and the offset, the allocation information of the target object includes: The allocation information of the target object is determined according to the horizontal coordinate information and the vertical coordinate information of the annotation box and the offset.

24. The method according to claim 17, wherein The calculating of the offsets between the target object and each boundary of the annotation box includes: Calculating the offset between each pixel point of the target object and each boundary of the annotation box; Among the offsets between each pixel point and each boundary of the annotation box, the maximum offset is determined as the offset between the target object and each boundary of the annotation box.

25. The method according to claim 16, wherein The obtaining of sample images for training the target detection model includes: Perform offset enhancement on the original annotation data of the original sample image to obtain an offset-enhanced annotation frame; A sample image for training an object detection model is obtained based on the original sample image and the annotation box.

26. The method according to claim 25, wherein The offset enhancement of the original annotation data of the original sample image to obtain the offset-enhanced annotation frame includes: Determining a ratio of an area formed by original annotated data in an original sample image to an area of ​​a training sub-region of the original sample image; determining an offset enhancement frequency according to the ratio; Performing disturbance enhancement on the original labeled data according to the offset enhancement frequency; According to the result of disturbance enhancement, the offset enhanced annotation box is obtained.

27. The method according to claim 25, wherein Before performing offset enhancement on the original annotation data of the original sample image to obtain the offset-enhanced annotation frame, the method further includes: The original sample image is scaled by resampling based on the regional pixel relationship.

28. The method according to claim 16, wherein Before inputting the sample image into the convolutional layer of the target detection model, the method further includes: Initializing the target detection model; The initialization includes at least one of the following: Initializing the backbone network of the target detection model by loading pre-trained model parameters, wherein the pre-trained model parameters are parameters jointly trained through multiple preset subtasks; Training unlocks a parameter-locked layer in the object detection model, wherein the parameter-locked layer includes at least a batch normalization layer.

29. The method according to claim 18, wherein The method further comprises: Outputting the correspondence between the first dimension value and the corresponding level; or, Output the first dimension value, allocation information corresponding to the first dimension value, and a correspondence between the levels corresponding to the allocation information.

30. The method of claim 16, wherein The sample image includes multiple target objects and corresponding multiple annotation boxes; Before inputting the sample image into the convolution layer of the object detection model to obtain feature maps of multiple levels, the method further includes: grouping the multiple first dimension values ​​corresponding to the multiple annotation boxes, and determining a ratio of the number of first dimension values ​​in each group to the number of all first dimension values; After inputting the sample image into the convolution layer of the target detection model to obtain feature maps of multiple levels, the method further includes: determining whether the ratio is greater than a preset ratio; if so, performing the operation of performing bounding box prediction on the target object in the feature map of the level corresponding to the first dimension value; otherwise, performing the operation of performing bounding box prediction on the target object in the feature map of the level corresponding to the annotation box.

31. A data processing device comprising: A first acquisition module is configured to acquire a sample image for training a target detection model, wherein the sample image includes a labeling box and a target object, and the labeling box includes a first dimension value; A second acquisition module is used to input the sample image into the convolution layer of the target detection model to obtain feature maps of multiple levels; a prediction module, configured to perform bounding box prediction for the target object in a feature map of a level corresponding to the first dimension value, wherein the feature map of the level corresponding to the first dimension value modifies and determines a candidate feature map based on the first dimension value and an offset, the candidate feature map being determined based on the feature map of the level corresponding to the annotation box, and the offset being an offset between each boundary of the target object and the annotation box; A training module is used to train the object detection model based on the bounding box prediction result and the loss function.

32. A data processing device comprising: a third acquisition module, configured to acquire an image to be detected, wherein the image to be detected includes a target object to be detected, and the target object includes a first dimension value; An input module, configured to input the image to be detected into a target detection model, wherein the target detection model generates feature maps of multiple levels corresponding to the image to be detected; The fourth acquisition module is used to obtain the detection result of the target object detection output by the feature map of the level corresponding to the first dimension value, wherein the feature map of the level corresponding to the first dimension value corrects and determines the candidate feature map based on the first dimension value and the offset, and the candidate feature map is determined based on the feature map of the level corresponding to the annotation box, and the offset is the offset of the target object and each boundary of the annotation box.

33. An electronic device comprising: A processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform an operation corresponding to the data processing method according to any one of claims 1 to 5; or, to perform an operation corresponding to the data processing method according to any one of claims 6 to 14; or, to perform an operation corresponding to the data processing method according to claim 15; or, to perform an operation corresponding to the data processing method according to any one of claims 16 to 30.

34. A computer storage medium having a computer program stored thereon, which, when executed by a processor, implements the data processing method according to any one of claims 1 to 5; or, implements the data processing method according to any one of claims 6 to 14; or, executes the data processing method according to claim 15; or, implements the data processing method according to any one of claims 16 to 30.

Images