Target detection method and device, computer device and readable storage medium

Abstract

The target detection method, device, computer device and readable storage medium provided by the application, the method comprises: acquiring a to-be-detected image; using a trained target detection model to detect whether a target exists in the to-be-detected image; if a target exists, using a detection head module of the target detection model to determine a coordinate offset prediction value, a scale offset prediction value and a maximum probability scale classification corresponding to the target from a feature map of the to-be-detected image; determining the coordinates of a prediction box corresponding to the target according to the coordinate offset prediction value, and determining the scale of the prediction box according to the scale offset prediction value and the maximum probability scale classification. The application converts the problem of directly predicting the scale into a multi-scale classification problem of the scale, compared with the method of directly predicting the scale of the target, the application can be not affected by the actual scale of the target, and the accuracy and efficiency of the detection result are improved.

Description

Technical Field

[0001] This invention relates to the field of target detection technology, and more specifically, to a target detection method, apparatus, computer device, and readable storage medium. Background Technology

[0002] Object detection is a fundamental technology in computer vision, directly or indirectly applied across various industries, such as autonomous driving, facial recognition, and security surveillance. The performance of object detection directly impacts the effectiveness of subsequent tasks; therefore, research in object detection technology is trending towards higher accuracy and efficiency.

[0003] With the development of computer vision and artificial intelligence technologies, convolutional neural networks have become the mainstream framework for object detection. The current mainstream detection method uses direct prediction to output the position and scale of the target. This direct prediction method is easily affected by the distribution of target scale in the training set during model training, causing the predicted value range to fluctuate greatly when predicting targets of different scales, which affects network convergence and model performance. Summary of the Invention

[0004] One objective of this invention is to provide a target detection method, apparatus, computer device, and readable storage medium to improve the accuracy and efficiency of target detection. Embodiments of this invention can be implemented as follows:

[0005] In a first aspect, the present invention provides a target detection method, the method comprising: acquiring an image to be detected; using a trained target detection model to detect whether a target exists in the image to be detected; if a target exists, using the detection head module of the target detection model to determine the predicted coordinate offset, predicted scale offset, and maximum probability scale classification of the target from the feature map of the image to be detected; determining the coordinates of the prediction box corresponding to the target based on the predicted coordinate offset, and determining the scale of the prediction box based on the predicted scale offset and the maximum probability scale classification.

[0006] Secondly, the present invention provides a target detection apparatus, the method comprising: an acquisition module for acquiring an image to be detected; a detection module for detecting whether a target exists in the image to be detected using a trained target detection model; a prediction module for, if present, using a detection head module of the target detection model to determine, from the feature map of the image to be detected, a predicted coordinate offset value, a predicted scale offset value, and a maximum probability scale classification corresponding to the target; the prediction module is further configured to determine, based on the predicted coordinate offset value, the coordinates of a prediction box corresponding to the target, and to determine the scale of the prediction box based on the predicted scale offset value and the maximum probability scale classification.

[0007] Thirdly, the present invention provides a computer device including a processor and a memory, the memory storing a computer program executable by the processor, the processor being able to execute the computer program to implement the method described in the first aspect.

[0008] Fourthly, the present invention provides a readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect.

[0009] The target detection method, apparatus, computer device, and readable storage medium provided by this invention first detect the presence of a target in an image to be detected using a target detection model. If a target is found, the detection head module of the target detection model determines the predicted coordinate offset, predicted scale offset, and maximum probability scale classification of the target from the feature map of the image to be detected. Finally, based on the predicted coordinate offset, the coordinates of the prediction box corresponding to the target are determined, and the scale of the prediction box is determined based on the predicted scale offset and the maximum probability scale classification. This process allows the target detection model to predict the maximum probability scale classification of the target, and then calculates the target's scale based on the maximum probability scale classification. The entire process transforms the problem of directly predicting the scale into a multi-scale classification problem. Compared to methods that directly predict the target scale, this invention is not affected by the actual scale of the target, thus improving the accuracy and efficiency of the detection results. Attached Figure Description

[0010] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a schematic diagram of an application scenario provided by an embodiment of this application;

[0012] Figure 2 This is a schematic diagram of the backbone network structure of the target detection model provided in an embodiment of the present invention;

[0013] Figure 3 This is a schematic diagram of the structure of the detection head module of the target detection model provided in an embodiment of the present invention;

[0014] Figure 4 A schematic flowchart illustrating the training method of the target detection model provided in an embodiment of the present invention;

[0015] Figure 5 The detection effect diagram of the target detection model provided in the embodiment of the present invention is shown.

[0016] Figure 6 A schematic flowchart of the target detection method provided in an embodiment of the present invention;

[0017] Figure 7 A schematic flowchart of step S52 provided in an embodiment of the present invention;

[0018] Figure 8 A schematic flowchart of subsidy S53 provided in an embodiment of the present invention;

[0019] Figure 9 This is a functional block diagram of the target detection device provided in an embodiment of the present invention;

[0020] Figure 10 A structural block diagram of a computer device provided in an embodiment of the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0022] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0023] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0024] In the description of this invention, it should be noted that if terms such as "upper," "lower," "inner," or "outer" are used to indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship in which the product of this invention is usually placed, they are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this invention.

[0025] Furthermore, the terms "first" and "second" are used only to distinguish descriptions and should not be interpreted as indicating or implying relative importance.

[0026] It should be noted that, where there is no conflict, the features in the embodiments of the present invention can be combined with each other.

[0027] Please see Figure 1 , Figure 1 This is a schematic diagram of an application scenario provided by an embodiment of this application. The scenario includes: user 10, terminal device 11, and server 12. After establishing a communication connection between terminal device 11 and server 12 through a network, terminal device 11 can send an image to server 12, which will then perform target detection to obtain the position, width, and height of the target's predicted bounding box.

[0028] Among them, the terminal device 11 may be, but is not limited to, a scanner, a digital camera, various personal computers with image information acquisition functions, laptops, smartphones, tablets, smart wearable devices and other computer equipment.

[0029] Server 12 can be implemented using a standalone server, a server cluster composed of multiple servers, or a distributed system. It can also be a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. Terminal device 11 and server 12 can be connected directly or indirectly via wired or wireless communication; this application does not impose any restrictions on this connection.

[0030] Terminal device 11 and server 12 can communicate with each other through a communication network. The communication network can be a wireless fidelity (Wi-Fi) hotspot network, Bluetooth (BT) network, or near field communication (NFC) network, or it can be a third-generation wireless telephone technology (3G) network, a fourth-generation mobile communication technology (4G) network, a fifth-generation mobile communication technology (5G) network, a future public land mobile network (PLMN), or the Internet.

[0031] See also Figure 1Object detection algorithms can generally be divided into two main categories: anchor-based detection algorithms and anchor-free detection algorithms.

[0032] Anchor-based detection algorithms require a certain number of anchor boxes to guide target box prediction. This is typically achieved through clustering to generate anchor boxes, often in conjunction with multi-scale feature layer outputs. When the target box's scale is closest to that of a particular anchor box, that feature layer is responsible for target prediction. This method converts target scale into predictions based on anchor box offsets, resulting in a relatively stable range of predicted values ​​for the same feature layer. However, the selection and generation of anchor boxes require manual configuration, and the anchor box ratios and scales are dependent on the training data. This leads to poor model generalization performance, making it unable to handle diverse real-world scenarios. Furthermore, this multi-scale detection approach increases model complexity, hindering runtime reduction.

[0033] Anchor-free detection algorithms abandon the concept of anchor boxes and no longer make predictions based on anchor boxes. They are specifically divided into two modes: the first is key point prediction, which predicts the target range by predicting the corner points and center points of the target box, and finally generates the prediction box by key point matching; the second is direct prediction, which directly predicts the height and width of the target on the feature map, or predicts the distance from the center point to the boundary of the target box to obtain the prediction result.

[0034] However, while the first method is unaffected by the target scale, it involves complex keypoint matching algorithms, which is detrimental to inference timeliness. The second method can directly obtain the prediction results, but in reality, whether directly predicting the height and width of the target or predicting the distance from the center point to the boundary, it is actually related to the target scale and the dataset, which will also affect model performance.

[0035] Based on the above summary and analysis of the limitations of existing technologies, this invention proposes an improved target detection method on the basis of anchor-free detection algorithms. This method transforms the scale prediction problem into a multi-scale classification problem, thereby reducing the model's sensitivity to target scale. The target scale is decomposed into two parts, scale offset and scale vector, through encoding. These two parts are predicted separately and are not affected by the target scale or the number of targets in the training set. Only the scaling parameters and encoding length need to be specified according to actual usage requirements, without the need for complex post-processing.

[0036] Before introducing the target detection method provided in the embodiments of the present invention, the target detection model and its training method involved in the embodiments of the present invention will be described in detail.

[0037] Please see Figure 2 , Figure 2 This is a schematic diagram of the backbone network of the target detection model provided in this embodiment of the invention. The backbone network plays the role of feature extraction and multi-scale feature fusion in this embodiment of the invention. It can use, but is not limited to, an FPN structure. The backbone network can downsample the input image to be detected by multiple ratios in sequence, and then use transposed convolution to upsample to obtain the feature map of the image to be detected.

[0038] For example, see continue. Figure 2 , Figure 2 The backbone network uses an FPN structure. The input image to be detected is 960x512x3 in size. It is downsampled by 1 / 8, 1 / 16 and 1 / 32 in sequence. Finally, it is upsampled to 1 / 8 as the feature output by transposed convolution. That is, the size of the final output feature map is 1 / 8 of the size of the network input image, that is, the size of the output feature map is 120x64.

[0039] In this embodiment of the invention, the output channels of the feature map output by the backbone network can be determined according to a pre-set scale encoding length and number of categories. Combined with the feature map output by the backbone network, the target detection model of this embodiment also includes a detection head module, the structure of which is as follows: Figure 3 As shown, Figure 3 This is a schematic diagram of the detection head module of the target detection model provided in an embodiment of the present invention.

[0040] like Figure 3 As shown, in this embodiment of the invention, the detection head module can form multiple detection branches, which, from left to right, are: coordinate offset detection branch (offset... x and offset y ), Scale offset detection branch (offset) w and offset h ), object (obj) detection branch, class detection branch (class) and scale-coded vector detection branch (g) cw and g ch Each detection branch corresponds to a corresponding output channel of the feature map output by the backbone network. For example, the coordinate offset detection branch corresponds to the coordinate offset channel of the feature map, and the scale encoding vector detection branch corresponds to multiple scale encoding vector channels in the feature map to generate the target's coordinate offset, scale offset, object label, category label, and scale classification label.

[0041] Based on the above-described object detection model, this embodiment of the invention provides a training method for the object detection model. Please refer to [link to relevant documentation]. Figure 4 , Figure 4 A schematic flowchart illustrating the training method for the target detection model provided in this embodiment of the invention includes:

[0042] Step S41: Obtain image samples for training; the image samples have annotation information;

[0043] In this embodiment of the invention, the image annotation information includes: the location information of the target box, its width and height (the scale in this embodiment of the invention), and the target type. Considering the differences in the number of samples, scale, and color during the training process, after obtaining the image samples, a series of image enhancement operations are performed on the image samples, including image scaling, random flipping, and small-amplitude color jitter. The enhanced image samples are used as network input. At the same time, the image annotations are also transformed accordingly with the scale changes in the image enhancement.

[0044] Step S42: Obtain the feature map of the image sample using the backbone network of the object detection model, and extract the feature prediction value corresponding to each feature point from the feature map; wherein, the feature prediction value includes: coordinate offset prediction value, scale offset prediction value, object prediction value, category prediction value, and scale classification prediction value.

[0045] Step S43: Using the feature detection head module of the object detection model, determine the actual feature values ​​corresponding to each feature point based on the feature map; wherein, the actual feature values ​​include: coordinate offset, scale offset, object label, category label and scale classification label;

[0046] Step S44: Calculate the loss value of the pre-constructed loss function based on the predicted and actual feature values ​​of each feature point, and update the network parameters of the object detection model based on the loss value until the loss function converges.

[0047] In conjunction with the aforementioned detection head module, during model training, each target box is mapped to coordinates on the feature layer based on its center point, and the actual coordinate offset is calculated using these coordinates. Next, the height and width of the target box are decomposed into scale offset and scale encoding vector based on scaling parameters. Finally, the mean squared error loss function is used to calculate the losses for coordinate offset and scale offset, and the cross-entropy loss function is used to calculate the losses for object, category, and scale encoding vectors. Therefore, in this embodiment of the invention, the loss function of the target detection model includes a mean squared error loss term and a cross-entropy loss term; wherein the mean squared error loss term is obtained by summing the coordinate offset loss and scale offset loss, respectively; and the cross-entropy loss term is obtained by summing the object loss, category loss, and scale encoding loss.

[0048] For example, assuming the loss function is denoted as L, then L = L x +L y +L w +L h +L obj +L class +L code Among them, L xand L y It is the coordinate offset loss, L y and L w It is the scale offset loss, L obj It is object loss, L class It is a class loss, L code It is a scale-based classification loss.

[0049] Among them, the coordinate offset loss and scale offset loss can be solved by formula (1), and the object loss, class loss and scale classification loss can be solved by formula (2):

[0050] loss = ∑(py) 2 …(1)

[0051] loss=∑-[y log p+(1-y)log(1-p)]…(2)

[0052] Where p is the actual feature value and y is the predicted feature value.

[0053] During training, the above p is the value obtained by activating the feature points on each output channel of the feature map. Specifically, the coordinate offset, scale offset, and obj channels are activated using logistic activation, while the class and scale encoding vectors are activated using softmax activation to obtain the activated value p. For example, formula (3) can be used on the coordinate offset, scale offset, and obj channels, formula (4) can be used on the class channel, and formula (5) can be used on the scale encoding vector channel to obtain the value p for each feature point x. i，j The activation value p in different channels.

[0054]

[0055]

[0056]

[0057] Where N is the number of categories, L is the scale coding length, and c represents the channel position.

[0058] During training, for each feature point x i，j The predicted feature value y, when a target exists at the feature point, can be determined as follows: that is, step S43 above may include the following steps:

[0059] a1: Determine the coordinate offset of the feature point based on the position of the feature point, the size of the feature map, and the coordinates of the target box;

[0060] a2: Based on the size of the image sample, the size of the feature map, the preset scale scaling factor, and the preset scale encoding length, determine the scale encoding and scale offset of the feature points, and based on the scale encoding, set the label of the scale category that matches the scale encoding as the first preset value, and set the label of other scale categories as the second preset value.

[0061] a3: Determine the object label of the feature point as the first preset value, and based on the category corresponding to the target box, determine the category label that matches the category as the first preset value and determine the other category labels as the second preset value.

[0062] For ease of understanding, assume the image sample has a size of H×W, and there exists a bounding box, whose center point coordinates, height, and width are represented as b. x b y b w b h Its value is normalized to 0 to 1 according to the input image size. The size of the feature map obtained by outputting the image sample to the backbone network is h×w. Then the coordinates of the target center point mapped to the feature map are... Representing rounding down, the predicted coordinate offset is: offset x =w×b x -i, offset y =h×b y -j.

[0063] For predicting the label obj of an object, due to feature point x i，j This corresponds to the bounding box, proving that a target exists at this point, so the feature point x i，j If a feature point does not have a bounding box, it is considered a positive sample. If a feature point does not have a bounding box, it is considered a negative sample. The object prediction label for a positive sample is obj in the obj channel. i，i The value is 1 (first preset value), and the object prediction label obj for negative samples in the obj channel is 1. i，j It is 0 (second preset value).

[0064] Predicting labels for categories c，i，j If there are N categories, then the category channels are channels 0 to N-1. This is represented by the target category having a label of 1 for its corresponding channel, and the other categories having labels of 0 for their respective channels. For example, suppose there are two categories: animal and human. Animals correspond to category channel 0, and humans correspond to category channel 1. If feature point x... i，j The corresponding target is a person, then the feature point x i，j The predicted category label output on category channel 0 is "class". 0，i，j The predicted category label output on category channel 1 is class.1，i，j The value is 1.

[0065] Predicting label offset for scale offset w and offset h Let the scaling factor be S, and the scaling length of the height and width be L. Then, the base coefficients B of the height and width encoding can be obtained first according to formula (6). w and B h They are respectively:

[0066]

[0067] Then, the scale coding value code is calculated according to formulas (7) and (8). w and code h and scale offset w and offset h :

[0068]

[0069]

[0070] The prediction target of the scale vector channel is determined by the scale value. When the channel order is equal to the scale value, the prediction label of the channel is 1, and the prediction label of the other channels is 0. The mapping relationship is shown in formula (9). The height and width encoding are processed in the same way.

[0071]

[0072] Among them, g c，w To predict labels for wide-scale classification, g c，h Predict labels for high-scale classification.

[0073] In other words, for the scale classification label g c，h and g c，w The scale-encoded vector channels are determined to be from 0 to L-1 based on the scale-encoded length L. When feature point x i，j If the corresponding scale encoding value matches the index of one of the scale encoding vector channels, then the scale classification label output by that scale encoding vector channel is 1, and the scale classification label output by the other scale encoding vector channels is 0.

[0074] In summary, for each feature point in the feature map, the final predicted value output in this embodiment of the invention is: offset. x offset y offset w offset h obj, class c0 …classcN-1 g c0w ...g cL-1w g c0h ...g cL-1h Substitute the actual and predicted feature values ​​output from each channel of each feature point into formulas (3) and (4), and according to L = L x +L y +L w +L h +L obj +L class-(0) +…+L class-(N-1) +L code-(0) +…+L code-(L-1) , where L class-(0) To L class-(N-1) That is, the loss values ​​corresponding to the N category channels, L code-(0) +…+L code-(L-1 That is, the loss value corresponding to the L scale-coded vector channels. The total loss value can be calculated using the above formula.

[0075] Once the loss function is in a convergent state, the target detection model of this invention can be obtained, and target detection can be performed based on the target detection model.

[0076] After obtaining the target detection model through the above implementation method, image samples can be obtained for testing to test the target detection model until the test results meet the detection requirements.

[0077] To verify the effectiveness of the target detection model trained according to the present invention, this embodiment of the invention takes a flame detection task as an example. A flame detection training set and a test set are constructed by collecting publicly available data and monitoring data. The training set contains 9740 images, and the test set contains 1546 images. The ratio of positive to negative samples in both the training set and the test set is 1:2.

[0078] During training, a scale scaling factor of s = 8 was selected, the scale encoding length for height and width was L = 6, and the test threshold was T = 0.5. Using the variance of the target value predicted by the feature layer as a reference, the scale-based classification detection method provided in this invention was compared with the method that directly predicts height and width. The detection results are as follows: Figure 5 As shown, Figure 5 This is a detection effect diagram of the target detection model provided in the embodiments of the present invention. Figure 5 As can be seen, for targets of different scales, the target detection model provided in this embodiment of the invention predicts target bounding boxes with a scale that is almost consistent with the actual scale of the target, and the accuracy is high. Specific detection indicators are also shown in Table 1.

[0079] Table 1

[0080]

[0081] Among them, the larger the variance value, the wider the distribution range of the predicted target value, and the greater the influence of the training set data distribution, which affects the generalization performance of the model. In addition, accuracy (representing the proportion of correctly detected targets) and recall (representing the proportion of correctly detected targets) are also defined. As can be seen from Table 1, the test results on the flame test set demonstrate that the method for predicting height and width in scale classification provided by the embodiments of the present invention helps to improve model performance.

[0082] Based on the trained target detection model described above, this invention provides a target detection method. Please refer to [link to relevant documentation]. Figure 6 , Figure 6 This is a schematic flowchart of a target detection method provided in an embodiment of the present invention. The method includes:

[0083] Step S51: Acquire the image to be detected;

[0084] Step S52: Use the trained target detection model to detect whether there is a target in the image to be detected;

[0085] Step S53: If it exists, use the detection head module of the target detection model to determine the predicted coordinate offset, predicted scale offset, and maximum probability scale classification of the target from the feature map of the image to be detected.

[0086] Step S51: Determine the coordinates of the prediction box corresponding to the target based on the predicted coordinate offset value, and determine the scale of the prediction box based on the predicted scale offset value and the maximum probability scale classification.

[0087] The target detection method provided in this embodiment of the invention first detects whether a target exists in the image to be detected using a target detection model. If a target exists, the detection head module of the target detection model determines the predicted coordinate offset, predicted scale offset, and maximum probability scale classification of the target from the feature map of the image to be detected. Finally, based on the predicted coordinate offset, the coordinates of the prediction box corresponding to the target are determined, and the scale of the prediction box is determined based on the predicted scale offset and the maximum probability scale classification. This process allows the target detection model to predict the maximum probability scale classification of the target, and then calculates the target's scale based on the maximum probability scale classification. The entire process transforms the problem of directly predicting the scale into a multi-scale classification problem, which reduces the impact of the target scale on the prediction results and improves the accuracy and efficiency of the detection results.

[0088] The following will provide a detailed explanation of each of the above steps in conjunction with the relevant accompanying drawings.

[0089] In step S51: the image to be detected is acquired.

[0090] In this embodiment of the invention, the image to be detected can be an image captured in real time, an image stored locally, or an image uploaded by the user in real time. The image to be detected may contain targets, and the number of targets is not limited.

[0091] In step S52: the trained target detection model is used to detect whether there is a target in the image to be detected.

[0092] In this embodiment of the invention, if a target exists in the image to be detected, subsequent steps S53 and S54 can be executed; if no target exists, a prompt message indicating that no target exists can be returned directly or no processing can be performed.

[0093] Regarding step S52 above, the embodiments of the present invention provide the following... Figure 7 For the implementation methods shown, please refer to [link / reference]. Figure 7 , Figure 7 A schematic flowchart of step S52 provided in an embodiment of the present invention:

[0094] Step S52-1: Using the backbone network of the target detection model, feature extraction and fusion are performed on the image to be detected to obtain a feature map;

[0095] Step S52-2: Using the detection head module, read the object prediction value of each feature point from the object channel of the feature map;

[0096] Step S52-3: If there is a target feature point whose predicted object value is greater than a preset threshold, then it is determined that there is a target at the image location corresponding to the target feature point.

[0097] From the above Figure 3 As can be seen, in this embodiment of the invention, the output channels of the feature map include a coordinate offset channel, a scale offset channel, an obj channel, a category channel, and a scale encoding vector channel. As is known from the above training content, during the training process, the obj of feature points with targets is 1, and the obj of feature points without targets is 0. Therefore, when detecting whether there is a target in the target image to be detected, the judgment can be made based on the predicted value of the obj channel in the feature map.

[0098] Therefore, in this embodiment of the invention, the object prediction value p of each feature point is first read. obj Then, if there is a target feature point whose predicted value is greater than the preset threshold, it is determined that there is a target at the image location corresponding to the target feature point. If there is no target, a prompt message indicating that the target does not exist is output or no action is taken.

[0099] To improve the confidence level of the target detection results, this embodiment of the invention also proposes an implementation method, namely, after the above steps S52-3, the following steps may be included:

[0100] b1: Using the detection head module, read the predicted class value corresponding to the target feature point from the class channel of the feature map;

[0101] b2: If the product of the category prediction value and the object prediction value is greater than the preset confidence level, then proceed to step S53.

[0102] In this embodiment of the invention, assuming there are N category channels, the detection head module can read the N category prediction values ​​P corresponding to the target feature point. class Then predict the value p for each category. class The predicted value p of the target feature point obj If the product is greater than the preset confidence level, step S53 can be executed; otherwise, the detection result of the feature point can be left unoutput.

[0103] In step S53: If it exists, the detection head module of the target detection model is used to determine the predicted coordinate offset, predicted scale offset, and maximum probability scale classification of the target from the feature map of the image to be detected.

[0104] Regarding step S53 above, the embodiments of the present invention provide the following: Figure 8 The implementation method shown, Figure 8 A schematic flowchart of subsidy S53 provided for embodiments of the present invention:

[0105] Step S53-1: Using the detection head module, read the predicted coordinate offset and scale offset values ​​of the feature points corresponding to the target from the coordinate offset channel and scale offset channel of the feature map, respectively.

[0106] In this embodiment of the invention, there is only one coordinate offset channel and one scale offset channel. The predicted coordinate offset value p of the feature point corresponding to the target can be directly read from the coordinate offset channel. x p y The predicted scale offset value p of the feature point corresponding to the target is read directly from the scale offset channel. w p h .

[0107] Step S53-2: Traverse the multiple scale encoding vector channels of the feature map, read the maximum scale classification prediction value corresponding to the feature point, and determine the channel number of the scale encoding vector channel corresponding to the maximum scale classification prediction value as the maximum probability scale classification.

[0108] In this embodiment of the invention, there are multiple scale encoding vector channels, and the maximum probability scale classification c is determined. w c hDuring the process, the scale classification prediction value of each scale encoding vector channel can be traversed, and then the maximum scale classification prediction value can be found. The channel number of the scale encoding vector channel corresponding to the maximum scale classification prediction value is taken as the maximum probability scale classification.

[0109] In step S54: Based on the predicted coordinate offset value, the coordinates of the prediction box corresponding to the target are determined, and the scale of the prediction box is determined based on the predicted scale offset value and the maximum probability scale classification.

[0110] In this embodiment of the invention, the predicted coordinate offset value p is obtained. x p y Then, the coordinates of the predicted bounding box can be calculated based on the size of the feature map, the position of the feature point corresponding to the target, and the predicted coordinate offset.

[0111] For example, assuming the feature map has a size of h×w and the feature points are located at positions i and j, the coordinates of the prediction box can be obtained according to formula (10):

[0112]

[0113] Among them, b x and b y These are the x-coordinate and y-coordinate of the center of the prediction box, respectively.

[0114] The scale of the prediction box can be obtained according to formula (11):

[0115]

[0116] Where W and H are the width and height of the image to be detected, respectively, S is the preset scaling parameter, and p w p h For the predicted value of scale offset, c w c h For maximum probability scale classification, the channel number and b of the corresponding scale encoding vector channels are... w b h This is for predicting the width and height of the bounding box.

[0117] In an optional implementation, after obtaining the coordinates and scale of the predicted bounding boxes, the feature layer can be traversed to obtain all predicted bounding boxes. After non-maximum suppression (NMS) processing, the final prediction result can be obtained, thus improving the accuracy of the detection result.

[0118] Based on the same inventive concept, this application also provides a target detection device, the structure of which is as follows: Figure 9 As shown, Figure 9The present invention provides a functional block diagram of a target detection device 600, which may include: an acquisition module 601, a detection module 602, and a prediction module 603.

[0119] The acquisition module 601 is used to acquire the image to be detected;

[0120] Detection module 602 is used to detect whether there is a target in the image to be detected using the trained target detection model;

[0121] The prediction module 603 is used to determine the predicted coordinate offset, predicted scale offset, and maximum probability scale classification of the target from the feature map of the image to be detected, if the target exists, using the detection head module of the target detection model.

[0122] The prediction module 603 is also used to determine the coordinates of the prediction box corresponding to the target based on the predicted coordinate offset value, and to determine the scale of the prediction box based on the predicted scale offset value and the maximum probability scale classification.

[0123] It is understandable that the acquisition module 601, the detection module 602, and the prediction module 603 can execute collaboratively. Figure 9 Each step in the process is used to achieve the corresponding technical effect.

[0124] In an optional implementation, the prediction module 603 is used to use the detection head module to read the predicted coordinate offset and scale offset values ​​of the feature points corresponding to the target from the coordinate offset channel and scale offset channel of the feature map, respectively; to traverse multiple scale encoding vector channels of the feature map, to read the maximum scale classification prediction value corresponding to the feature point, and to determine the channel number of the scale encoding vector channel corresponding to the maximum scale classification prediction value as the maximum probability scale classification.

[0125] In an optional implementation, the detection module 602 is used to extract and fuse features of the image to be detected using the backbone network of the target detection model to obtain a feature map; using the detection head module, the object prediction label of each feature point is read from the object channel of the feature map; when there is a target feature point whose object prediction value is greater than a preset threshold, it is determined that there is a target at the image location corresponding to the target feature point.

[0126] In an optional implementation, the detection module 602 is further configured to use the detection head module to read the category prediction value corresponding to the target feature point from the category channel of the feature map; if the product of the category prediction value and the object prediction value is greater than a preset confidence level, then the detection head module of the target detection model is used to determine the coordinate offset prediction value, scale offset prediction value and maximum probability scale classification corresponding to the target from the feature map of the image to be detected.

[0127] In an optional implementation, the prediction module 603 is used to calculate the coordinates of the prediction box based on the size of the feature map, the position of the feature point corresponding to the target, and the predicted value of the coordinate offset; and to calculate the scale of the prediction box based on the size of the image to be detected, the preset scale scaling parameter, the predicted value of the scale offset, and the maximum probability scale classification.

[0128] In an optional implementation, the object detection device 600 may further include a training module for acquiring image samples for training; the image samples have annotation information; a feature map of the image samples is obtained using the backbone network of the object detection model, and feature prediction values ​​corresponding to each feature point are extracted from the feature map; wherein, the feature prediction values ​​include: coordinate offset prediction value, scale offset prediction value, object prediction label, category prediction label, and scale classification prediction label; the feature detection head module of the object detection model determines the actual feature value corresponding to each feature point based on the feature map; wherein, the actual feature value includes: coordinate offset, scale offset, object label, category label, and scale classification label; based on the feature prediction value and the actual feature value of each feature point, the loss value of the pre-constructed loss function is calculated, and the network parameters of the object detection model are updated based on the loss value until the loss function converges.

[0129] In an optional implementation, the loss function includes a mean squared error loss term and a cross-entropy loss term; wherein the mean squared error loss term is obtained by summing the coordinate offset loss and the scale offset loss, respectively; and the cross-entropy loss term is obtained by summing the object loss, the category loss, and the scale classification loss.

[0130] In an optional implementation, the training module is configured to: determine the coordinate offset of the feature points based on the location of the feature points, the size of the feature map, and the coordinates of the target box; determine the scale code and scale offset of the feature points based on the size of the image sample, the size of the feature map, a preset scale scaling factor, and a preset scale encoding length; and, based on the scale code, determine the label of the scale category that matches the scale code as a first preset value, and determine the labels of other scale categories as a second preset value; determine the object label of the feature points as the first preset value, and, based on the category corresponding to the target box, determine the category label that matches the category as the first preset value and determine the labels of other categories as the second preset value.

[0131] It should be noted that the module division in the above embodiments of this application is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods. Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, exist as separate physical entities, or have two or more units integrated into one unit. The integrated units described above can be implemented in hardware or as software functional units.

[0132] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned readable storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0133] Based on the above embodiments, this application also provides a schematic diagram of a computer device used to implement the target detection method in the above embodiments. (See also...) Figure 10 As shown, Figure 10 The structural block diagram of the computer device provided in the embodiment of the present invention shows that the computer device 700 includes: a memory 701, a processor 702, and a communication interface 703. The memory 701, the processor 702, and the communication interface 703 are electrically connected to each other directly or indirectly to realize data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines.

[0134] Optionally, the bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, Figure 10 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0135] In this embodiment, the processor 702 can be a general-purpose processor, digital signal processor, application-specific integrated circuit, field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in this embodiment. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in this embodiment can be directly implemented by the hardware processor, or implemented by a combination of hardware and software modules within the processor. The software modules can be located in the memory 701. The processor 702 reads the program instructions from the memory 701 and, in conjunction with its hardware, completes the steps of the aforementioned methods.

[0136] In this embodiment, the memory 701 can be non-volatile memory, such as a hard disk drive (HDD) or a solid-state drive (SSD), or it can be volatile memory, such as RAM. The memory can also be any other medium capable of carrying or storing desired program code in the form of instructions or data structures, and accessible by a computer, but is not limited thereto. The memory in this embodiment can also be a circuit or any other device capable of implementing storage functions, used to store instructions and / or data.

[0137] The memory 701 can be used to store software programs and modules, such as the instructions / modules of the target detection device 600 provided in this embodiment of the invention. These can be stored in the memory 701 in the form of software or firmware, or embedded in the operating system (OS) of the computer device 700. The processor 702 executes various functional applications and data processing by executing the software programs and modules stored in the memory 701. The communication interface 703 can be used for signaling or data communication with other node devices.

[0138] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the above-described apparatus and unit can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0139] Understandable. Figure 10 The structure shown is for illustrative purposes only; the computer device 700 may also include more than [other components]. Figure 10 The more or fewer components shown, or having the same Figure 10 The different configurations shown. Figure 10 The components shown can be implemented using hardware, software, or a combination thereof.

[0140] Based on the above embodiments, this application also provides a readable storage medium storing a computer program, which, when executed by a computer, causes the computer to perform the target detection method provided in the above embodiments.

[0141] Based on the above embodiments, this application also provides a computer program that, when run on a computer, causes the computer to execute the target detection method provided in the above embodiments.

[0142] Based on the above embodiments, this application also provides a chip for reading a computer program stored in a memory and executing the target detection method provided in the above embodiments.

[0143] This application also provides a computer program product, including instructions that, when run on a computer, cause the computer to execute the target detection method provided in the above embodiments.

[0144] This application describes embodiments of methods, apparatus (systems), and computer program products according to embodiments of this application with reference to flowchart illustrations and / or block diagrams. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by instructions. These instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0145] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0146] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0147] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A target detection method, characterized in that, The method includes: Acquire the image to be detected; The trained target detection model is used to detect whether a target exists in the image to be detected; If present, the detection head module of the target detection model is used to determine the predicted coordinate offset, predicted scale offset, and maximum probability scale classification of the target from the feature map of the image to be detected. The detection head module has a scale-encoded vector detection branch, which corresponds to multiple scale-encoded vector channels in the feature map. Determining the maximum probability scale classification includes: traversing the multiple scale-encoded vector channels of the feature map, reading the maximum probability scale classification prediction value corresponding to the feature point, and determining the channel number of the scale-encoded vector channel corresponding to the maximum probability scale classification as the maximum probability scale classification. Based on the predicted coordinate offset value, the coordinates of the prediction box corresponding to the target are determined. The scale of the prediction box is calculated based on the size of the image to be detected, the preset scale scaling parameter, the predicted scale offset value, and the maximum probability scale classification. The scale of the prediction box is expressed by the following formula: Where W and H are the dimensions of the image to be detected, i.e., width and height, respectively; and S is the preset scale scaling parameter. , The predicted value of the scale offset; , For the maximum probability scale classification, the channel number of the corresponding scale encoding vector channel; , B represents the scale of the prediction box, i.e., its width and height. w and B h These are the basis coefficients encoding the width and height of the image to be detected, respectively. L represents the dimensional coding length of the width and height.

2. The target detection method according to claim 1, characterized in that, If present, the detection head module of the target detection model is used to determine the predicted coordinate offset, predicted scale offset, and maximum probability scale classification of the target from the feature map of the image to be detected. This also includes: Using the detection head module, the predicted coordinate offset and the predicted scale offset of the feature points corresponding to the target are read from the coordinate offset channel and the scale offset channel of the feature map, respectively.

3. The method according to claim 1, characterized in that, Using a trained target detection model, the detection of whether a target exists in the image to be detected includes: The target detection model's backbone network is used to extract and fuse features from the image to be detected, resulting in the feature map. Using the detection head module, the object prediction label for each feature point is read from the object channel of the feature map; If there is a target feature point whose predicted object value is greater than a preset threshold, then it is determined that the target exists at the image location corresponding to the target feature point.

4. The method according to claim 3, characterized in that, After determining that the target exists at the image location corresponding to the target feature point when there is a target feature point with a predicted object value greater than a preset threshold, the method further includes: Using the detection head module, the predicted category value corresponding to the target feature point is read from the category channel of the feature map; If the product of the category prediction value and the object prediction value is greater than the preset confidence level, then the detection head module of the target detection model is used to determine the predicted coordinate offset value, the predicted scale offset value, and the maximum probability scale classification of the target from the feature map of the image to be detected.

5. The method according to claim 1, characterized in that, Based on the predicted coordinate offset value, the coordinates of the predicted bounding box corresponding to the target are determined, including: The coordinates of the prediction box are calculated based on the size of the feature map, the position of the feature point corresponding to the target, and the predicted coordinate offset value.

6. The method according to any one of claims 1-5, characterized in that, The target detection model was trained in the following manner: Obtain image samples for training; the image samples have annotation information. The feature map of the image sample is obtained using the backbone network of the target detection model, and the feature prediction value corresponding to each feature point is extracted from the feature map; wherein, the feature prediction value includes: coordinate offset prediction value, scale offset prediction value, object prediction label, category prediction label, and scale classification prediction label; Using the feature detection head module of the target detection model, the actual feature values ​​corresponding to each feature point are determined based on the feature map; wherein, the actual feature values ​​include: coordinate offset, scale offset, object label, category label, and scale classification label; Based on the predicted feature value and the actual feature value of each feature point, the loss value of the pre-constructed loss function is calculated, and the network parameters of the target detection model are updated based on the loss value until the loss function converges.

7. The method according to claim 6, characterized in that, The loss function includes a mean squared error loss term and a cross-entropy loss term; wherein the mean squared error loss term is obtained by summing the coordinate offset loss and the scale offset loss, respectively; and the cross-entropy loss term is obtained by summing the object loss, the category loss, and the scale classification loss.

8. The method according to claim 6, characterized in that, When a target bounding box exists for a feature point, the feature detection head module of the target detection model determines the actual feature value corresponding to each feature point based on the feature map, including: The coordinate offset of the feature point is determined based on the position of the feature point, the size of the feature map, and the coordinates of the target box. Based on the size of the image sample, the size of the feature map, and the preset scale scaling factor and preset scale encoding length, the scale encoding and scale offset of the feature points are determined. Based on the scale encoding, the label of the scale category that matches the scale encoding is determined as the first preset value, and the labels of other scale categories are determined as the second preset value. The object label of the feature point is determined as the first preset value, and based on the category corresponding to the target, the category label that matches the category is determined as the first preset value and other category labels are determined as the second preset value.

9. A target detection device, characterized in that, include: The acquisition module is used to acquire the image to be detected; The detection module is used to detect whether a target exists in the image to be detected using a trained target detection model; The prediction module, if present, uses the detection head module of the target detection model to determine the predicted coordinate offset, predicted scale offset, and maximum probability scale classification of the target from the feature map of the image to be detected. The detection head module has a scale-encoded vector detection branch, which corresponds to multiple scale-encoded vector channels in the feature map. Determining the maximum probability scale classification includes: traversing the multiple scale-encoded vector channels of the feature map, reading the maximum probability scale classification prediction value corresponding to the feature point, and determining the channel number of the scale-encoded vector channel corresponding to the maximum probability scale classification as the maximum probability scale classification. The prediction module is further configured to determine the coordinates of the prediction box corresponding to the target based on the predicted coordinate offset value, and calculate the scale of the prediction box based on the size of the image to be detected, the preset scale scaling parameter, the predicted scale offset value, and the maximum probability scale classification; the scale of the prediction box is expressed by the following formula: Where W and H are the dimensions of the image to be detected, i.e., width and height, respectively; and S is the preset scale scaling parameter. , This is the predicted value of the scale offset; , For the maximum probability scale classification, the channel number of the corresponding scale encoding vector channel; , B represents the scale of the prediction box, i.e., its width and height. w and B h These are the basis coefficients encoding the width and height of the image to be detected, respectively. L represents the dimensional coding length of the width and height.

10. A computer device, characterized in that, It includes a processor and a memory, the memory storing a computer program executable by the processor, the processor being able to execute the computer program to implement the method of any one of claims 1 to 8.

11. A readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 8.

Images