Model training methods, image processing methods, devices, and readable storage media
By setting the number of anchor boxes at different granularities in the target detection head and using a clustering algorithm to generate position-related anchor boxes, the problem of high computational load and inaccurate positioning caused by an excessive number of anchor boxes in high depth-of-field scenes is solved, achieving more efficient target detection.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ZHEJIANG DAHUA TECH CO LTD
- Filing Date
- 2022-12-30
- Publication Date
- 2026-05-26
AI Technical Summary
Existing object detection methods suffer from problems such as excessive number of anchor boxes leading to high computational load, overlapping coverage, low training efficiency, and inaccurate localization in high depth-of-field scenes.
By setting the number of anchor box groups at different granularities of the target detection head using a clustering algorithm, the anchor box size related to the location is obtained, and positive and negative sample matching training is performed. The assumption of the probability distribution of anchor boxes at the global location is discarded, and the location-related prior boxes are learned.
It improves the localization accuracy of target detection, reduces computational load, enhances training efficiency, and adapts to target detection tasks in different depth-of-field scenarios.
Smart Images
Figure CN116152598B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image processing technology, and in particular to a model training method, an image processing method, an image processing apparatus, and a computer-readable storage medium. Background Technology
[0002] Object detection (OD) is an important research area in machine vision, artificial intelligence, and other scientific research fields. Various sectors, including transportation, security, and environmental protection, use OD as the underlying support for implementing business logic. The task of OD is to locate and identify people or objects in images or videos, thereby enabling functions such as tracking, behavior analysis, quantity statistics, and alarms. Currently, OD is primarily implemented using deep learning methods.
[0003] Image processing methods can generally be categorized into anchor-based methods and anchor-free methods based on the presence or absence of anchor boxes. The former sets prior anchor boxes on the feature map for target localization, while the latter directly locates the target based on the points in the feature map. Typical anchor-free methods include YOLOv1, while anchor-based methods include YOLOv3 and Faster R-CNN. Generally speaking, anchor-based methods outperform anchor-free methods in localization.
[0004] An anchor box is a prior bounding box assumed on an image or feature map. Localization and recognition tasks are modeled as regression and classification tasks respectively in deep learning, and the appropriate use of prior boxes has been shown to improve localization accuracy and stability. Faster R-CNN sets nine anchor boxes of varying sizes and shapes for each point in the feature map, while YOLOv3 sets three sets of anchor boxes of different shapes and sizes for detection heads at different scales. Anchor boxes exist as a prior; they can be simply understood as the shape and size of the target output by the neural network without any data-driven input, a concept consistent with Bayesian priors in probabilistic models. Priors directly affect the accuracy of modeling; therefore, anchor boxes are usually not randomly assigned but carefully designed or obtained through statistical or learning methods. Clustering algorithms, such as k-means clustering, are commonly used to obtain anchor boxes.
[0005] Generally, larger anchor boxes are suitable for detecting large targets, while smaller anchor boxes are suitable for detecting small targets. Anchor boxes with different aspect ratios are also suitable for detecting anchor boxes with similar shapes. To adapt to and cover targets of different sizes and shapes, multiple combinations are usually provided. For example, Faster R-CNN combines 3 sizes and 3 shapes to obtain 9 anchor boxes, so each point in the feature map has 9 anchor boxes corresponding to it. However, the number of anchor boxes is not always better the more there are. Too many anchor boxes will lead to a large computational burden in the non-maximum suppression process during inference, excessive time consumption in the detection algorithm, and a large degree of overlap in the target intervals covered by anchor boxes, which is also a waste or causes instability between positive and negative samples. For example, for a 17x17 feature map, assigning 9 anchor boxes results in 17x17x9 = 2601 anchor boxes. The time consumption generated by distributing n ground truths to 2601 anchor boxes will reduce training efficiency, and too many anchor boxes will also make real-time detection difficult to achieve. Conversely, too few anchor boxes will result in problems such as too small coverage and inaccurate localization. Summary of the Invention
[0006] This application provides a model training method, an image processing method, an image processing apparatus, and a computer-readable storage medium.
[0007] This application provides a model training method, the model training method comprising:
[0008] Obtain several ground truth bounding boxes and the target feature map of the target detection head;
[0009] The target group anchor frames are set according to the granularity of the target detection head, wherein the number of anchor frame groups corresponding to target detection heads with different granularities is different;
[0010] Obtain the projection points of several real bounding boxes onto the target feature map;
[0011] Clustering is performed based on the actual annotation boxes of the points where the projection points fall in each group of anchor frames to obtain the anchor frame size of each group of anchor frames;
[0012] The model to be trained is trained according to the actual annotation box and its corresponding anchor box size.
[0013] The step of clustering based on the actual annotation boxes of the projected points falling on the corresponding points of each anchor frame to obtain the anchor frame size of each group of anchor frames includes:
[0014] Cluster all the real labeled boxes whose projected points fall on the corresponding points of the current group's anchor boxes;
[0015] Obtain the actual bounding box size corresponding to a preset number of cluster centers as the anchor box size for the current group.
[0016] Among them, the number of target group anchor frames corresponding to the coarse-grained target detection head is one group;
[0017] The number of target group anchor boxes corresponding to the medium-granularity target detection head is equal to the number of columns in the corresponding feature map;
[0018] The number of anchor boxes for the target group corresponding to the fine-grained target detection head is equal to the number of points in the corresponding feature map.
[0019] The step of training the model to be trained according to the dimensions of the ground truth bounding box and its corresponding anchor box includes:
[0020] Obtain the training group anchor boxes associated with the projection points of the real labeled boxes;
[0021] Obtain the intersection-union ratio (IoU) of the ground truth bounding boxes and all anchor boxes in the training group anchor boxes;
[0022] The anchor box with the largest intersection-union ratio is used to perform positive sample regression training on the model to be trained.
[0023] The model to be trained is trained using negative sample regression using the remaining anchor boxes in the training group anchor boxes.
[0024] The model training method further includes, after performing positive sample regression training on the model to be trained using the anchor box with the largest intersection-union ratio, the method further includes:
[0025] The anchor box with the largest intersection-union ratio and its corresponding ground truth bounding box are used to perform category and / or position difference training on the model to be trained.
[0026] Before obtaining the training group anchor boxes associated with the projection points of the ground truth bounding boxes, the model training method further includes:
[0027] Traverse all points on the target feature map of the target detection head and perform the following: obtain the maximum intersection-union ratio (IUU) between the anchor box of each point and all ground truth bounding boxes;
[0028] When the maximum crossover ratio is less than or equal to the negative sample threshold, the anchor box for that point is not trained.
[0029] This application also provides another model training method, which includes:
[0030] Obtain several ground truth bounding boxes and the target feature map of the target detection head;
[0031] Obtain the training group anchor boxes associated with the projection points of the ground truth bounding boxes on the target feature map;
[0032] Obtain the intersection-union ratio (IoU) of the ground truth bounding boxes and all anchor boxes in the training group anchor boxes;
[0033] The anchor box with the largest intersection-union ratio is used to perform positive sample regression training on the model to be trained.
[0034] The model to be trained is trained using negative sample regression using the remaining anchor boxes in the training group anchor boxes.
[0035] The model training method further includes the following steps before obtaining the training group anchor boxes associated with the projection points of the target feature map of the ground truth bounding boxes:
[0036] The target group anchor frames are set according to the granularity of the target detection head, wherein the number of anchor frame groups corresponding to target detection heads with different granularities is different;
[0037] Obtain the projection points of several real bounding boxes onto the target feature map;
[0038] Clustering is performed based on the actual annotation boxes of the points where the projection points fall in each group of anchor frames to obtain the anchor frame size of each group of anchor frames.
[0039] This application also provides an image processing method, the image processing method comprising:
[0040] Obtain the image to be processed;
[0041] The target information of the image to be processed is obtained by using a pre-trained target detection model to perform target detection on the image to be processed.
[0042] The target detection model is trained using the model training method described above.
[0043] This application also provides an image processing apparatus, which includes a processor and a memory, wherein the memory stores program data, and the processor executes the program data to implement the model training method and / or image processing method as described above.
[0044] This application also provides a computer-readable storage medium for storing program data, which, when executed by a processor, is used to implement the above-described model training method and / or image processing method.
[0045] The beneficial effects of this application are as follows: the image processing device acquires several ground truth bounding boxes and a target feature map from the target detection head; it sets target group anchor boxes according to the granularity of the target detection head, wherein the number of anchor box groups corresponds to different granularity target detection heads; it acquires the projection points of several ground truth bounding boxes onto the target feature map; it clusters the ground truth bounding boxes whose projection points fall on the corresponding points of each group of anchor boxes, and acquires the anchor box size of each group of anchor boxes; it trains the training model according to the ground truth bounding boxes and their corresponding anchor box sizes. Through the above method, the image processing device uses a clustering algorithm to cluster anchor boxes at a finer positional granularity, thereby improving the localization accuracy of target detection. Attached Figure Description
[0046] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein:
[0047] Figure 1 This is a flowchart illustrating an embodiment of the model training method provided in this application;
[0048] Figure 2 This is a schematic diagram of the overall process of the target detection prior anchor frame generation method provided in this application;
[0049] Figure 3 This is a schematic diagram of the anchor frame particle size division provided in this application;
[0050] Figure 4 This is a schematic diagram of another embodiment of the model training provided in this application;
[0051] Figure 5 This is a flowchart illustrating the method for assigning positive and negative samples to anchor boxes during the forward propagation process of OD training provided in this application.
[0052] Figure 6 This is a flowchart illustrating yet another embodiment of the model training method provided in this application;
[0053] Figure 7 This is a schematic flowchart of an embodiment of the image processing method provided in this application;
[0054] Figure 8 This application provides the positioning effects of different granularities in high depth-of-field scenes;
[0055] Figure 9 This is a schematic diagram of the structure of an embodiment of the image processing apparatus provided in this application;
[0056] Figure 10 This is a schematic diagram of an embodiment of the computer-readable storage medium provided in this application. Detailed Implementation
[0057] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.
[0058] This application proposes an optimized object detection algorithm for high depth-of-field scenes. This method abandons the original assumption that anchor boxes have the same probability distribution across the image, instead assuming that the probability distribution of anchor boxes differs across the image. This allows for the learning of a series of location-related prior boxes to improve localization accuracy in high depth-of-field scenes. First, the size of the downsampled feature map is determined, and the image is divided into corresponding grids based on the feature map. Then, based on the projection of the ground truth box (GT) center point onto the feature map, the anchor boxes are clustered using k-means clustering according to the horizontal height of the grid or the global position of the projection point, resulting in a set of location-related anchor boxes. Next, during the training of the OD model, the GT boxes are indexed to their corresponding anchor boxes based on their projected positions on the feature map, and positive and negative sample matching or assignment is performed accordingly. Finally, conventional object detection loss calculation and backpropagation are performed.
[0059] Please refer to details. Figure 1 and Figure 2 , Figure 1 This is a flowchart illustrating an embodiment of the model training method provided in this application. Figure 2 This is a schematic diagram of the overall process of the target detection prior anchor frame generation method provided in this application.
[0060] The model training method of this application is applied to an image processing apparatus, which can be a server or a system consisting of a server and a terminal device working together. Accordingly, the various parts of the image processing apparatus, such as units, subunits, modules, and submodules, can be all located in the server, or they can be located separately in the server and the terminal device.
[0061] Furthermore, the aforementioned server can be either hardware or software. When the server is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. When the server is software, it can be implemented as multiple software programs or software modules, such as software or software modules used to provide distributed servers, or as a single software program or software module; no specific limitation is made here. In some possible implementations, the image processing method of this application embodiment can be implemented by a processor calling computer-readable instructions stored in memory.
[0062] Specifically, such as Figure 1 As shown, the model training method in this embodiment of the application specifically includes the following steps:
[0063] Step S11: Obtain several ground truth bounding boxes and the target feature map of the target detection head.
[0064] In this embodiment, the image processing device acquires a training dataset, which includes several manually annotated ground truth bounding boxes. Additionally, the image processing device acquires at least one detection head of the model to be trained and calculates the feature map output by the detection head according to the model depth where the detection head is located. Figure 2 As shown, the feature maps of the M head layers of the model to be trained in this embodiment have sizes of {(W1, H1), (W2, H2)...(W...H1)}, respectively. M H M )}, with granularities of {C1, C2, ... C m}
[0065] Step S12: Set the target group anchor frame according to the granularity of the target detection head, wherein the number of anchor frame groups corresponding to different granularity target detection heads is different.
[0066] In this embodiment, the image processing device sets the target group anchor frames according to the granularity of the target detection head; the finer the granularity, the more anchor frames are set. For example... Figure 2 As shown, for a coarse-grained target detection head, the number of anchor frame groups L is set. m =1; For medium-grained target detection heads, the number of anchor frame groups L is set to 1. m =H m For fine-grained target detection heads, the number of anchor frame groups set is L. m =W m ×H m .
[0067] Specifically, please refer to the anchor frames of different granularities. Figure 3 , Figure 3 This is a schematic diagram of the anchor frame grain division provided in this application. For the coarse-grained head layer, its size is W. m ×Hm All points share a common set of anchor frames, resulting in a total of k different sizes and shapes of anchor frames; for the medium-grained head layer, its size is W. m ×H m Each row of points shares a common set of anchor frames, totaling H. m ×k different sizes and shapes of anchor frames; for the fine-grained head layer, the size is W. m ×H m Each point has a set of anchor frames, totaling W. m ×H m ×k different sizes and shapes of anchor frames. Regardless of the grain size, the number of anchor frames on the Head layer is always W. m ×H m ×k. The finer the granularity, the more types of anchor frames of different sizes and shapes there are.
[0068] Step S13: Obtain the projection points of several real bounding boxes onto the target feature map.
[0069] In this embodiment of the application, the image processing device obtains the projection point of the center point of each real bounding box onto the feature map of each detection head.
[0070] Step S14: Cluster the anchors based on the actual annotation boxes where the projected points fall on the corresponding points of each anchor group, and obtain the anchor size of each anchor group.
[0071] In this embodiment of the application, the image processing device obtains the real annotation boxes where the projection points fall on the corresponding points of each group of anchor boxes, and clusters these real annotation boxes. The size of the real annotation box of the cluster center of the clustering result is the anchor box size of each group of anchor boxes.
[0072] Specifically, since the coarse-grained detection head has only one anchor box group, all ground truth bounding boxes are clustered, and the box sizes of multiple cluster centers in the clustering results are used as the anchor box sizes for all points in the feature map of the coarse-grained detection head. Since the medium-grained detection head has H anchor box groups... m Clustering is performed using the true bounding boxes of projected points falling within the same row of feature map points. The box sizes of multiple cluster centers in the clustering results are used as the anchor box sizes of a set of anchor boxes for the medium-grained detection head. Since the number of anchor box sets for the fine-grained detection head is W... m ×H m Clustering is performed using the true bounding boxes where the projected points fall on the same feature map point. The box sizes of multiple cluster centers in the clustering results are used as the anchor box sizes of the first set of anchor boxes for fine-grained detection.
[0073] It should be noted that the embodiments of this application may use clustering algorithms such as k-means clustering, hierarchical clustering, and spectral clustering to generate prior boxes.
[0074] Step S15: Train the model to be trained according to the dimensions of the real bounding box and its corresponding anchor box.
[0075] In this embodiment of the application, after the image processing device performs clustering algorithm processing in step S14, it obtains L. m Anchor boxes are grouped, with each group containing k classes, and are bound to their respective detection heads. The image processing device can combine detection heads of different granularities to the target detection model, thereby achieving target detection of different granularities based on anchor boxes of different granularities, and can adapt to target detection tasks in various depth-of-field scenes.
[0076] In this embodiment, the image processing device acquires several ground truth bounding boxes and a target feature map from a target detection head; it sets target group anchor boxes according to the granularity of the target detection head, wherein the number of anchor box groups corresponds to different granularity target detection heads; it acquires the projection points of several ground truth bounding boxes onto the target feature map; it clusters the ground truth bounding boxes whose projection points fall on the corresponding points of each anchor box group, and acquires the anchor box size of each anchor box group; it trains the training model according to the ground truth bounding boxes and their corresponding anchor box sizes. Through the above method, the image processing device uses a clustering algorithm to cluster anchor boxes at a finer positional granularity, thereby improving the localization accuracy of target detection.
[0077] Furthermore, during the training phase, the ground truth boxes need to be assigned to the corresponding anchor boxes for loss calculation. When calculating the intersection-union ratio (IOU) between the anchor boxes and the ground truth boxes, the size and shape of the anchor boxes need to be based on the index of the location, and the size of the anchor boxes is generated by clustering through the above embodiment.
[0078] For example, in the medium granularity proposed in this application, the anchor frame size and shape remain the same along the rows because "nearer objects appear larger and farther objects smaller" is a common real-world scenario, and the target shape and size show a strong correlation with the horizontal position. This application assumes that the granularity of each detection head may be different; therefore, using the same granularity for each detection head is also within the scope of protection claimed in this application.
[0079] Please continue reading. Figure 4 and Figure 5 , Figure 4 This is a schematic diagram of another embodiment of the model training provided in this application. Figure 5 This is a flowchart illustrating the method for assigning positive and negative samples to anchor frames during the forward propagation process of OD training provided in this application.
[0080] Specifically, such as Figure 4 As shown, the model training method in this embodiment of the application specifically includes the following steps:
[0081] Step S21: Obtain the training group anchor boxes associated with the projection points of the ground truth bounding boxes.
[0082] In this embodiment of the application, a single training image contains N ground truth (GT) boxes, and the feature sizes of the M detector heads are {(W1, H1, C), (W2, H2, C)...(W...}. M H M C)}, there are C anchor frames at each point, and the size of the anchor frames is determined by C. Figure 1 The model shown is trained and generated, with a negative sample threshold set to v.
[0083] Each detection head needs to be trained independently. Therefore, in the following embodiments, a single granular detection head is used as an example to illustrate the process of assigning positive and negative samples.
[0084] The image processing device initializes the matching matrix Match(N, WxHxk) with each element as 0, initializes the IOU(N, WxHxk) with each element as 0.0, initializes the mask vector maskm(W, H, C) with each element as -1, and initializes the matching GT index Bind = -1.
[0085] The image processing device iterates through all points on the feature map of the target detection head, calculates the maximum intersection-union ratio (IoU) between the anchor box and the ground truth bounding box at each point, and performs IoU calculations. max When v ≤ v, set maskm[i, j, c] = 0.0; in IOU max When >v, keep maskm[i,j,c] = -1.
[0086] After the traversal is complete, the image processing device calculates the projection point of each ground truth bounding box onto the feature map of the target detection head, and the C-value of that projection point. m An anchor frame.
[0087] Step S22: Obtain the intersection-union ratio (IUU) of all anchor boxes in the ground truth bounding boxes and the anchor boxes in the training group.
[0088] Step S23: Use the anchor box with the largest intersection-union ratio to perform positive sample regression training on the model to be trained.
[0089] Step S24: Use the remaining anchor boxes in the training group anchor boxes to perform negative sample regression training on the model to be trained.
[0090] In this embodiment of the application, the image processing device calculates the true bounding box and C m The IOU of each anchor frame, and the IOU max The corresponding anchor box is set to maskm(W, H, C) = 1.0 and Bind(W, H, C) = n, which is the index of the actual annotation box.
[0091] The image processing device regresses anchor boxes with a value of -1 in maskm(W, H, C) to a confidence level of 0.0, even though the confidence level of the training output for anchor boxes with a value of -1 in maskm(W, H, C) is biased towards 0. The image processing device does not perform loss calculation on anchor boxes with a value of 0 in maskm(W, H, C), as these anchor boxes have a low correlation with the ground truth bounding boxes; reducing the training of these anchor boxes helps improve training efficiency. The image processing device regresses anchor boxes with a value of 1 in maskm(W, H, C) to a confidence level of 1.0, even though the confidence level of the training output for anchor boxes with a value of 1 in maskm(W, H, C) is biased towards 1.
[0092] Furthermore, the image processing device can also perform a difference loss on the output category of each anchor box and the true labeled box recorded by the box regression position and the associated Bind(W, H, C), so that the output category and output position of the anchor box are biased towards the associated true labeled box.
[0093] In the embodiments of this application, the image processing device abandons the assumption that the target size and shape have the same probability distribution at global locations. This application believes that the prior bounding boxes should have different sizes and shapes in different scenarios (typically high depth-of-field scenarios). It proposes to use the k-means clustering algorithm to cluster anchor boxes at a finer positional granularity, thereby improving the localization accuracy of target detection. It also proposes a GT allocation algorithm that matches the positional granularity, which is equally applicable to one-stage and two-stage anchor box-based target detection algorithms.
[0094] Please continue reading. Figure 6 , Figure 6 This is a flowchart illustrating another embodiment of the model training method provided in this application.
[0095] Specifically, such as Figure 6 As shown, the model training method in this embodiment of the application specifically includes the following steps:
[0096] Step S31: Obtain several ground truth bounding boxes and the target feature map of the target detection head.
[0097] Step S32: Obtain the training group anchor boxes associated with the projection points of the ground truth bounding boxes onto the target feature map.
[0098] In this embodiment, the image processing device sets target group anchor boxes according to the granularity of the target detection head, wherein the number of anchor box groups corresponding to different granularity target detection heads is different; it obtains the projection points of several real annotation boxes on the target feature map; and it performs clustering based on the real annotation boxes whose projection points fall on the corresponding points of each anchor box group to obtain the anchor box size of each anchor box group. This process is the same as steps S11 to S15 in the above embodiment, and will not be described again here.
[0099] Step S33: Obtain the intersection-union ratio (IUU) of all anchor boxes in the ground truth bounding boxes and the anchor boxes in the training group.
[0100] Step S34: Use the anchor box with the largest intersection-union ratio to perform positive sample regression training on the model to be trained.
[0101] Step S35: Use the remaining anchor boxes in the training group anchor boxes to perform negative sample regression training on the model to be trained.
[0102] In this embodiment, steps S31 to S35 are the same as steps S21 to S24 in the above embodiment, and will not be repeated here.
[0103] Based on the model training method described in the above embodiments, this application utilizes the trained image processing method to achieve target detection performance. For details, please refer to... Figure 7 , Figure 7 This is a flowchart illustrating an embodiment of the image processing method provided in this application.
[0104] Specifically, such as Figure 7 As shown, the image processing method of this application embodiment specifically includes the following steps:
[0105] Step S41: Obtain the image to be processed.
[0106] Step S42: Use a pre-trained object detection model to perform object detection on the image to be processed, and obtain the object information of the image to be processed.
[0107] In this embodiment, a roadside parking detection scenario is used to compare with existing detection methods, demonstrating the effectiveness of the image processing method in this application.
[0108] Please see Figure 8 , Figure 8 This application demonstrates the positioning effects of different granularities in high depth-of-field scenes. Figure 8 In scenarios where distant vehicles are far away and obstructed, their vehicle bounding boxes are significantly larger. Using a medium-grained method for detection improves the accuracy of the vehicle bounding boxes. In fact, the distribution of medium-grained anchor boxes exhibits a clear "nearer larger, farther smaller" pattern. Using the average area of the anchor boxes obtained from coarse-grained clustering as a reference, the average area patterns of medium-grained and fine-grained distant vehicles on a 35x35 feature map are roughly as shown in Table 1:
[0109] Table 1 Comparison of relative dimensions (area) of anchor frames with different particle sizes proposed in the proposal.
[0110]
[0111] Improved positioning accuracy leads to enhanced OD detection performance. Following the Map calculation method of COCO data, using a high-depth-of-field roadside parking scenario as the learning and testing scenario, the mAP (middle accuracy) for pedestrians, motor vehicles, and non-motor vehicles is shown in Table 2 after 10 averages at coarse, medium, and fine granular levels.
[0112] Table 2 lists the mAP values of anchor frames with different particle sizes.
[0113]
[0114] In summary, the method proposed in this application can learn location-related prior boxes and then assign positive and negative samples based on these prior boxes. This application presents two different prior anchor box generation methods, and experiments verify their effectiveness in high-depth-of-field scenarios such as traffic. Compared to existing solutions, these methods achieve better localization results without increasing the number of anchor boxes or computational load.
[0115] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.
[0116] To implement the model training method and / or image processing method described in the above embodiments, this application also proposes an image processing apparatus, which can be found in detail below. Figure 9 , Figure 9 This is a schematic diagram of an embodiment of the image processing apparatus provided in this application.
[0117] The image processing apparatus 500 of this application embodiment includes a memory 51 and a processor 52, wherein the memory 51 and the processor 52 are coupled together.
[0118] The memory 51 is used to store program data, and the processor 52 is used to execute the program data to implement the model training method and / or image processing method described in the above embodiments.
[0119] In this embodiment, processor 52 can also be referred to as a CPU (Central Processing Unit). Processor 52 may be an integrated circuit chip with signal processing capabilities. Processor 52 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The general-purpose processor can be a microprocessor, or processor 52 can be any conventional processor.
[0120] To implement the model training method and / or image processing method of the above embodiments, this application also provides a computer-readable storage medium, such as... Figure 10 As shown, the computer-readable storage medium 600 is used to store program data 61, which, when executed by a processor, is used to implement the model training method and / or image processing method as described in the above embodiments.
[0121] This application also provides a computer program product, wherein the computer program product includes a computer program operable to cause a computer to perform the model training method and / or image processing method as described in the embodiments of this application. The computer program product may be a software installation package.
[0122] The model training method and / or image processing method described in the above embodiments of this application, when implemented as a software functional unit and sold or used as an independent product, can be stored in a device, such as a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.
[0123] The above description is merely an embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.
Claims
1. A model training method, characterized in that, The model training method includes: Obtain several ground truth bounding boxes and the target feature map of the target detection head; The target group anchor frames are set according to the granularity of the target detection head, wherein the number of anchor frame groups corresponding to target detection heads with different granularities is different; Obtain the projection points of several real bounding boxes onto the target feature map; Clustering is performed based on the actual annotation boxes of the points where the projection points fall in each group of anchor frames to obtain the anchor frame size of each group of anchor frames; The training model is performed according to the actual annotation box and its corresponding anchor box size; The number of target group anchor frames corresponding to the coarse-grained target detection head is one group; The number of target group anchor boxes corresponding to the medium-granularity target detection head is equal to the number of columns in the corresponding feature map; The number of anchor boxes for the target group corresponding to the fine-grained target detection head is equal to the number of points in the corresponding feature map.
2. The model training method according to claim 1, characterized in that, The process of clustering based on the actual annotation boxes of the projected points falling on the corresponding points of each anchor frame to obtain the anchor frame size of each anchor frame includes: Cluster all the real labeled boxes whose projected points fall on the corresponding points of the current group's anchor boxes; Obtain the actual bounding box size corresponding to a preset number of cluster centers as the anchor box size for the current group.
3. The model training method according to claim 1, characterized in that, The step of training the model to be trained according to the actual bounding box and its corresponding anchor box size includes: Obtain the training group anchor boxes associated with the projection points of the real labeled boxes; Obtain the intersection-union ratio (IoU) of the ground truth bounding boxes and all anchor boxes in the training group anchor boxes; The anchor box with the largest intersection-union ratio is used to perform positive sample regression training on the model to be trained. The model to be trained is trained using negative sample regression using the remaining anchor boxes in the training group anchor boxes.
4. The model training method according to claim 3, characterized in that, After using the anchor box with the largest intersection-union ratio to perform positive sample regression training on the model to be trained, the model training method further includes: The anchor box with the largest intersection-union ratio and its corresponding ground truth bounding box are used to perform category and / or position difference training on the model to be trained.
5. The model training method according to claim 3, characterized in that, Before obtaining the training group anchor boxes associated with the projection points of the ground truth bounding boxes, the model training method further includes: Traverse all points on the target feature map of the target detection head and perform the following: obtain the maximum intersection-union ratio (IUU) between the anchor box of each point and all ground truth bounding boxes; When the maximum crossover ratio is less than or equal to the negative sample threshold, the anchor box for that point is not trained.
6. An image processing method characterized by, The image processing method includes: Obtain the image to be processed; The target information of the image to be processed is obtained by using a pre-trained target detection model to perform target detection on the image to be processed. The target detection model is trained using the model training method described in any one of claims 1 to 5.
7. An image processing apparatus characterized by comprising: The image processing apparatus includes a processor and a memory, the memory storing program data, and the processor executing the program data to implement the model training method as described in any one of claims 1 to 5, and / or the image processing method as described in claim 6.
8. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store program data, which, when executed by a processor, is used to implement the model training method according to any one of claims 1 to 5, and / or the image processing method according to claim 6.