Target detection method and device, computer readable storage medium and terminal equipment

Abstract

The invention belongs to the technical field of target detection, and particularly relates to a target detection method and device, a computer readable storage medium and terminal equipment. The method comprises the steps of obtaining a to-be-detected image; performing target detection on the to-be-detected image based on the target detection model to obtain a target detection result of the to-be-detected image; the target detection model is a neural network model obtained through training and used for target detection, the training process of the target detection model comprises the step of obtaining a matched prediction frame corresponding to each real frame from each candidate prediction frame through matching, and each candidate prediction frame is determined by a candidate anchor frame. Through the above model training, an initial prediction frame (candidate prediction frame) can be determined by using prior information provided by an anchor frame, and a prediction frame (matched prediction frame) closer to a real frame can be further screened from the initial prediction frame, so that a target detection model with relatively high precision can be obtained through training. And the target detection precision and accuracy in the dense small target scene can be improved.

Description

Technical Field

[0001] The present application belongs to the field of target detection technology, and in particular, relates to a target detection method, device, computer-readable storage medium and terminal device. Background Art

[0002] Object detection is an important task in the field of computer vision, which aims to identify the target object in the image and determine the position and size of the target object. In the existing scheme, the relevant neural network model can be used to perform the object detection task; specifically, in the process of object detection, the model used for detection needs to identify the category of the object in the image and needs to accurately locate the specific position of the object in the image. The specific position of the object in the image can usually be marked and displayed by a bounding box. However, the existing target detection scheme is difficult to obtain high-precision detection results when detecting dense small targets. Therefore, there is an urgent need for a target detection method that can accurately detect dense small targets. Summary of the invention

[0003] In view of this, the embodiments of the present application provide a target detection method, apparatus, computer-readable storage medium and terminal device to solve the problem that existing target detection methods are difficult to obtain high-precision detection results when detecting dense small targets.

[0004] A first aspect of an embodiment of the present application provides a target detection method, which may include:

[0005] Acquire the image to be detected;

[0006] Performing target detection on the image to be detected based on the target detection model to obtain a target detection result of the image to be detected;

[0007] Among them, the target detection model is a trained neural network model for target detection, and the training process of the target detection model includes matching each candidate prediction frame to obtain a matching prediction frame corresponding to each real frame, and each candidate prediction frame is determined by a candidate anchor frame.

[0008] A second aspect of an embodiment of the present application provides a target detection device, which may include:

[0009] An acquisition module, used for acquiring an image to be detected;

[0010] A detection module is used to perform target detection on the image to be detected based on a target detection model to obtain a target detection result of the image to be detected; wherein the target detection model is a trained neural network model for target detection, and the training process of the target detection model includes matching each candidate prediction frame to obtain a matching prediction frame corresponding to each real frame, and each candidate prediction frame is determined by a candidate anchor frame.

[0011] A third aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any of the above-mentioned target detection methods are implemented.

[0012] The fourth aspect of an embodiment of the present application provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any one of the above-mentioned target detection methods when executing the computer program.

[0013] A fifth aspect of an embodiment of the present application provides a computer program product. When the computer program product is run on a terminal device, the terminal device executes the steps of any one of the above-mentioned target detection methods.

[0014] Compared with the prior art, the embodiments of the present application have the following beneficial effects: the embodiments of the present application obtain an image to be detected; perform target detection on the image to be detected based on a target detection model to obtain a target detection result of the image to be detected; wherein the target detection model is a trained neural network model for target detection, and the training process of the target detection model includes matching each candidate prediction frame to obtain a matching prediction frame corresponding to each real frame, and each candidate prediction frame is determined by a candidate anchor frame. Through the model training of the embodiments of the present application, the initial prediction frame (candidate prediction frame) can be first determined using the prior information provided by the (candidate) anchor frame, and then a prediction frame (matching prediction frame) that is closer to the real frame can be further screened out from the initial prediction frame, so that a target detection model with higher accuracy can be trained, which helps to improve the precision and accuracy of target detection in dense small target scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0016] Figure 1A schematic diagram of an application process of the initial detection model;

[0017] Figure 2 Schematic diagram of the prediction module of the initial detection model;

[0018] Figure 3 is a schematic diagram of the prediction layer of the prediction module;

[0019] Figure 4 A schematic diagram of a label allocation process according to an embodiment of the present application;

[0020] Figure 5 This is a flow chart of an embodiment of a target detection method in an embodiment of the present application;

[0021] Figure 6 This is a structural diagram of an embodiment of a target detection device in an embodiment of the present application;

[0022] Figure 7 This is a schematic block diagram of a terminal device in an embodiment of the present application. DETAILED DESCRIPTION

[0023] In order to make the purpose, features, and advantages of the invention of this application more obvious and easy to understand, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the embodiments described below are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0024] It should be understood that when used in this specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0025] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0026] It should be further understood that the term “and / or” used in the specification and appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0027] As used in this specification and the appended claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined" or "if [described condition or event] is detected" may be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0028] In addition, in the description of the present application, the terms "first", "second", "third", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0029] Object detection is an important task in the field of computer vision, which aims to identify target objects in images and determine their location and size.

[0030] Object detection is widely used in various scenarios. For example, in video surveillance scenarios, object detection technology can be used to identify and track objects such as people and vehicles for security monitoring, crowd counting, or abnormal behavior detection. For another example, in autonomous driving scenarios, object detection can be used to identify and locate objects such as vehicles, pedestrians, traffic signs, and signal lights on the road to ensure driving safety, which is crucial for the decision-making and path planning of autonomous driving systems. For another example, in manufacturing scenarios, object detection can be used to detect defective products on the production line, monitor machine status, or guide robots to perform precise grasping and placement operations to achieve the purpose of automated production.

[0031] In existing solutions, relevant neural network models can be used to perform target detection tasks; specifically, in the process of target detection, the model used for detection needs to identify the category of the object in the image and accurately locate the specific position of the object in the image. The specific position of the object in the image can usually be marked and displayed by a bounding box. However, existing target detection solutions are difficult to obtain high-precision detection results when detecting dense small targets. Therefore, there is an urgent need for a target detection method that can accurately detect dense small targets.

[0032] In view of this, the embodiments of the present application provide a target detection method, apparatus, computer-readable storage medium and terminal device to solve the problem that existing target detection methods are difficult to obtain high-precision detection results when detecting dense small targets.

[0033] It should be noted that the executor of the method of the present application is a terminal device, specifically, it can be a common computing device such as a desktop computer, notebook, PDA, workstation, server, mobile phone, video surveillance, etc., or it can be other computing devices.

[0034] In an embodiment of the present application, a target detection model can be used to perform target detection on an image that requires target detection (referred to as an image to be detected) to obtain a target detection result of the image to be detected.

[0035] The target detection model in the embodiment of the present application is a neural network model for target detection. Before using the target detection model for actual target detection, it is necessary to perform model training on the initial detection model. After the model training is completed, the target detection model in the embodiment of the present application can be obtained.

[0036] Since the detection model may predict a large number of overlapping bounding boxes during the actual target detection process, it is usually necessary to filter the predicted bounding boxes (this process is called label assignment), and the qualified bounding boxes can be used as the result of target detection.

[0037] During the model training process of the embodiment of the present application, a label assignment strategy with multiple screenings can be used to perform multiple screenings on the predicted bounding box from loose to strict, and finally a bounding box with higher precision is obtained. Based on this, a high-precision target detection model can be effectively trained, thereby improving the detection accuracy in dense small target scenarios.

[0038] The following will first introduce in detail the training process of the target detection model in the embodiment of the present application.

[0039] Specifically, a training sample set can be first obtained, which is a data set used for model training; the training sample set may include a preset number of sample images and corresponding sample labels, and the sample labels of the sample images may specifically include each real box of the sample image and the category corresponding to each real box; wherein the real box of the sample image is a bounding box marked in the sample image and containing the target, and the category corresponding to the real box is the category to which the target contained in the real box belongs.

[0040] The number of sample images in the above training sample set can be specific and situational according to actual needs, and the embodiments of the present application do not limit this.

[0041] In a specific implementation, in order to improve the applicability of the trained target detection model in various scenarios, the training sample set can be enriched; specifically, images in various target detection application scenarios can be obtained, for example, images that require target detection in various scenarios such as video surveillance, autonomous driving, and production manufacturing can be obtained as sample images, and the sample images can be marked to obtain sample labels corresponding to each sample image; based on each sample image and the corresponding sample label, a training sample set can be determined.

[0042] Afterwards, the initial detection model can be trained based on the training sample set. Here, the initial detection model can specifically be a fully convolutional neural network model.

[0043] When the initial detection model is trained, each sample image in the training sample set can be used as an input of the initial detection model to obtain a prediction result corresponding to each sample image output by the initial detection model.

[0044] In a specific implementation, if the original scale of each sample image in the training sample set does not meet the scale requirement of the initial detection model for the input image, each sample image can be preprocessed by image scaling and pixel filling to obtain each sample image that meets the scale requirement; then, the model training is performed based on each preprocessed sample image.

[0045] After obtaining the prediction results corresponding to each sample image output by the initial detection model, the model parameters of the initial detection model can be adjusted based on the sample labels and prediction results of each sample image to obtain the target detection model of the embodiment of the present application.

[0046] The above training process will be described in detail below with reference to the accompanying drawings.

[0047] Figure 1 The application process of an initial detection model in an embodiment of the present application is shown; specifically, the initial detection model may include a network backbone module, a feature fusion module and a prediction module.

[0048] The network backbone module can receive a sample image, extract and output the deep semantic features of the sample image at different scales; specifically, Figure 1 Feature 1 is the deep semantic feature of the sample image at scale 1, feature 2 is the deep semantic feature of the sample image at scale 2, and feature 3 is the deep semantic feature of the sample image at scale 3.

[0049] The number of the above scales and the specific value of each scale can be set according to actual needs, and the embodiments of the present application are not limited to this; in the above example, scale 1 can be 1 / 8 of the image input scale, scale 2 can be 1 / 16 of the image input scale, and scale 3 can be 1 / 32 of the image input scale.

[0050] In addition, the network backbone module in the embodiment of the present application can be any common network backbone module with a fully convolutional structure, including but not limited to network backbone modules with a fully convolutional structure such as the ResNet series, MobileNet series, ShuffleNet series or YOLO series.

[0051] The feature fusion module can be used to receive the deep semantic features of the sample image at different scales output by the network backbone module, and can fuse the deep semantic features of the sample image at different scales to strengthen the information exchange between the deep semantic features at different scales, and then can output the features of the sample image at different scales after deep fusion. Specifically, Figure 1 Feature 4 is the deep semantic feature at scale 1 after deep fusion, feature 5 is the deep semantic feature at scale 2 after deep fusion, and feature 6 is the deep semantic feature at scale 3 after deep fusion.

[0052] The feature fusion module in the embodiment of the present application can be a common feature fusion network, including but not limited to any common feature fusion network such as Feature Pyramid Network (FPN) and Bidirectional Feature Pyramid Network (BiFPN), and the embodiment of the present application does not make specific limitations on this.

[0053] The prediction module may specifically include prediction layers of different scales, each of which may predict the category of the target contained in the sample image and the specific location of the target in the sample image based on the features output by the feature fusion module at the corresponding scale, and output the prediction result. Figure 1 The prediction layer 1 is the prediction layer corresponding to scale 1, the prediction layer 2 is the prediction layer corresponding to scale 2, and the prediction layer 3 is the prediction layer corresponding to scale 3; the prediction result 1 is the prediction result output by the prediction layer 1, the prediction result 2 is the prediction result output by the prediction layer 2, and the prediction result 3 is the prediction result output by the prediction layer 3.

[0054] In an embodiment of the present application, each prediction layer may specifically include a classification prediction sublayer and a positioning prediction sublayer. The classification prediction sublayer can be used to predict the category confidence vector corresponding to each initial anchor box in the sample image (referred to as the classification prediction result), and the positioning prediction sublayer can be used to predict the position of the initial prediction box corresponding to each initial anchor box in the sample image (referred to as the positioning prediction result).

[0055] The prediction module in the above example can be Figure 2 As shown, the prediction layer 1 may specifically include a classification prediction sublayer 1 and a positioning prediction sublayer 1. The classification prediction sublayer 1 may predict the category confidence vector (i.e., Figure 2 The classification prediction result 1 in the positioning prediction sublayer 1 can predict the prediction frame decoding parameters corresponding to each initial anchor frame of the sample image at scale 1 based on feature 4 (i.e. Figure 2The prediction layer 2 may specifically include a classification prediction sublayer 2 and a positioning prediction sublayer 2. The classification prediction sublayer 2 may predict the category confidence vector (i.e., the category confidence vector corresponding to each initial anchor frame of the sample image at scale 2) based on feature 5. Figure 2 The classification prediction result 2 in the positioning prediction sublayer 2 can predict the prediction frame decoding parameters corresponding to each initial anchor frame of the sample image at scale 2 based on feature 5 (i.e. Figure 2 The location prediction result 2 in the prediction box is obtained. Based on the prediction box decoding parameters, the position of the initial prediction box corresponding to each initial anchor box can be determined. Similarly, the prediction layer 3 can specifically include a classification prediction sublayer 3 and a positioning prediction sublayer 3. The classification prediction sublayer 3 can predict the category confidence vector (i.e., the category confidence vector corresponding to each initial anchor box of the sample image at scale 1) based on feature 6. Figure 2 The classification prediction result 3 in the positioning prediction sublayer 3 can predict the prediction frame decoding parameters corresponding to each initial anchor frame of the sample image at scale 3 based on feature 6 (i.e. Figure 2 The positioning prediction result 3) in the prediction box decoding parameter can determine the position of the initial prediction box corresponding to each initial anchor box.

[0056] A prediction layer of the prediction module in the above example can be specifically as follows Figure 3 As shown; each classification prediction sublayer and positioning prediction sublayer can include two 3×3 convolution submodules ( Figure 3 3×3conv in), a 1×1 convolution submodule ( Figure 3 1×1conv) and activation submodules in ; among them, the 3×3 convolution submodule can be followed by a normalization layer ( Figure 3 bn in) and an activation layer ( Figure 3 The activation layer can be implemented by an activation function. The present application does not limit the type of the activation function. For example, it can be a common activation function such as ReLU and SiLU. In addition, the activation submodule can be implemented by a Sigmoid activation function. The Sigmoid activation function can be used to constrain the prediction result to a numerical range of 0-1.

[0057] Here, the size of the output classification prediction result can be (B, C*N, H', W'); where B is the number of batch sample images, C represents the number of predicted categories, N represents the number of initial anchor boxes contained in a 1×1 network unit (for example, 3), and H' and W' represent the height and width of the deep semantic features after deep fusion of the corresponding scale, respectively; in this example, scale 1 can be 1 / 8 of the image input scale, scale 2 can be 1 / 16 of the image input scale, and scale 3 can be 1 / 32 of the image input scale. ; then at scale 1, H' and W' are 1 / 8 of the sample image height and 1 / 8 of the width respectively; at scale 2, H' and W' are 1 / 16 of the sample image height and 1 / 16 of the width respectively; at scale 3, H' and W' are 1 / 32 of the sample image height and 1 / 32 of the width respectively; the size of the output positioning prediction result can be (B, 4*N, H', W'), where 4 refers to the 4 prediction box decoding parameters included. Based on the prediction box decoding parameters, the initial anchor box can be decoded to obtain the corresponding initial prediction box (position).

[0058] In the embodiment of the present application, the network backbone module in the initial detection model can be used to extract the initial features corresponding to each sample image in the training sample set; then, the feature fusion module in the initial detection model can be used to perform feature fusion on the initial features corresponding to each sample image in the training sample set; then, based on the fusion features corresponding to each sample image in the training sample set, the prediction module of the initial detection model can be used to perform result prediction on each sample image to obtain the prediction results corresponding to each sample image in the training sample set. Here, the prediction results corresponding to the sample images can include the category confidence vectors corresponding to each initial anchor box of the sample image and the prediction box decoding parameters.

[0059] Taking any sample image in the training sample set (called the target sample image) as an example, the target sample image can be input into the initial detection model to obtain the category confidence vector and prediction box decoding parameters corresponding to each initial anchor box in the output target sample image. Based on the prediction box decoding parameters, the position of the initial prediction box corresponding to each initial anchor box can be determined.

[0060] The above-mentioned initial anchor frame is a pre-set anchor frame; specifically, multiple (for example, 3) initial anchor frames can be set on a network unit, and different initial anchor frames on a network unit can have different sizes and aspect ratios; in the target detection process, each initial anchor frame can provide a reference for the possible position of the target in the image and the size of the target.

[0061] In addition, each initial anchor frame may correspond to a category confidence vector, and the category confidence vector of the initial anchor frame may include the confidences of each category of the initial anchor frame. The category confidence of a category of the initial anchor frame is the confidence of the target of the category contained in the initial anchor frame.

[0062] For example, the initial detection model is used to detect targets of category 1, category 2, and category 3, and the confidence of the target of category 1 contained in the initial anchor frame is confidence 1, the confidence of the target of category 2 is confidence 2, and the confidence of the target of category 3 is confidence 3. Then confidence 1, confidence 2, and confidence 3 can be combined and expressed in the form of a vector, which is the category confidence vector corresponding to the initial anchor frame.

[0063] Based on this, the (initial) anchor box can be introduced as prior information to provide a reference for the possible location and size of the target, so that target detection can be performed more specifically, which helps to improve the efficiency and accuracy of target detection.

[0064] Based on the initial anchor frame and the prediction frame decoding parameters, the initial prediction frame corresponding to the initial anchor frame can be located; the initial detection frame may include the detected target, and the position of the initial prediction frame is the position of the detected target in the image; in the embodiment of the present application, the initial prediction frame can be adjusted by the corresponding initial anchor frame; specifically, the initial anchor frame can be decoded using the prediction frame decoding parameters to obtain the corresponding initial prediction frame; wherein the prediction frame decoding parameters may include 4 parameters, namely, Δx, Δy, Δw and Δh, which can be specifically expressed as (Δx, Δy, Δw, Δh); Δx is the horizontal offset of the center point of the initial prediction frame relative to the center point of the initial anchor frame, Δy is the vertical offset of the center point of the initial prediction frame relative to the center point of the initial anchor frame, Δw is the scaling ratio of the width of the initial prediction frame relative to the width of the initial anchor frame, and Δh is the scaling ratio of the height of the initial prediction frame relative to the height of the initial anchor frame.

[0065] In an embodiment of the present application, after obtaining the category confidence vector and the initial prediction frame corresponding to each initial anchor frame in the target sample image, the label assignment strategy (also referred to as a post-processing process) in the embodiment of the present application can be used to screen each initial prediction frame and finally determine the prediction result corresponding to the target sample image.

[0066] Figure 4 The label assignment process of the embodiment of the present application is shown, which may specifically include the process of selecting candidate anchor frames from the initial anchor frames, the process of selecting candidate prediction frames from the initial prediction frames corresponding to the candidate anchor frames, and the process of matching the candidate prediction frames with the real frames. Figure 4 Each process in is described in detail.

[0067] Specifically, each candidate anchor frame in the target sample image can be determined based on each initial anchor frame and each true frame in the target sample image. Here, the initial anchor frame that is closer to the true frame can be preliminarily screened out in the network unit where the true frame is located and the adjacent network units of the network unit, that is, the initial anchor frame that is closer to the marked target can be screened out, and the screened initial anchor frame is used as the candidate anchor frame, and then further screening can be performed based on each candidate anchor frame.

[0068] Taking any real box (referred to as the target real box) as an example, the network unit where the target real box is located can be referred to as the first network unit, the adjacent network unit to the first network unit can be referred to as the second network unit, and any one of the first network unit and the second network unit can also be referred to as the target network unit; for any initial anchor box in the target network unit (referred to as the target anchor box), the intersection over Union (IoU) between the target initial anchor box and the target real box can be specifically calculated. If the IoU is greater than the preset first IoU threshold, it can be considered that the target initial anchor box is closer to the target real box, and the target initial anchor box can be determined as a candidate anchor box. Among them, the first IoU threshold can be set according to actual needs, and the embodiment of the present application does not limit this. For example, the first IoU threshold can be set to 0.5 or 0.7. Through the above preliminary screening, each candidate anchor box in the target sample image can be determined; accordingly, the initial anchor box that does not contain the target can be filtered out, thereby reducing subsequent unnecessary matching.

[0069] Since the initial anchor frames corresponding to the candidate anchor frames may be relatively close, that is, the differences between the initial anchor frames are small and there is a certain degree of redundancy. Therefore, in the embodiment of the present application, the category-independent non-maximum threshold (NMS) method can be used to further screen the initial anchor frames corresponding to the candidate anchor frames to obtain the candidate prediction frames with large differences.

[0070] Specifically, the maximum confidence in the category confidence vector of the initial anchor box can be determined as the confidence of the predicted category of the initial anchor box, and the confidence of the predicted category is also the confidence of the predicted category of the initial prediction box corresponding to the initial anchor box; the initial prediction boxes of each candidate anchor box are sorted in descending order of the confidence of the prediction category to obtain a first queue; here, the initial prediction box that is ranked highest in the first queue and has not been screened can be recorded as the first prediction box, and each initial prediction box in the first queue that is ranked after the first prediction box can be recorded as the second prediction box. For example, there is an initial prediction box 1 corresponding to candidate anchor box 1, an initial prediction box 2 corresponding to candidate anchor box 2, an initial prediction box 3 corresponding to candidate anchor box 3, and an initial prediction box 4 corresponding to candidate anchor box 4; each initial prediction box can be sorted in order of confidence from large to small, and the result of the sorting is initial prediction box 2, initial prediction box 4, initial prediction box 1, and initial prediction box 3. According to the order of initial prediction box 2, initial prediction box 4, initial prediction box 1, and initial prediction box 3, a first queue can be obtained; at this time, the initial prediction box that has not been screened and is ranked highest in the first queue is initial prediction box 2, then initial prediction box 2 can be used as the first prediction box, and initial prediction box 4, initial prediction box 1, and initial prediction box 3 ranked after initial prediction box 2 can be used as second prediction boxes.

[0071] Afterwards, the intersection-and-union ratio between the first prediction box and each second prediction box can be calculated to obtain each intersection-and-union ratio; if the intersection-and-union ratio between the first prediction box and a second prediction box is less than the preset second intersection-and-union ratio threshold, it can be considered that the first prediction box and the second prediction box are less similar (i.e., the difference is large), and the second prediction box can be retained; if the intersection-and-union ratio between the first prediction box and a second prediction box is greater than or equal to the second intersection-and-union ratio threshold, it can be considered that the first prediction box and the second prediction box are more similar (i.e., the difference is small), and the second prediction box can be removed from the first queue; after each second prediction box completes the above judgment process, the first prediction box can be determined as an initial prediction box that has been screened; afterward, the first prediction box and each second prediction box can be re-determined, and the above process can be repeated until each initial prediction box in the first queue becomes an initial prediction box that has been screened; afterward, each initial prediction box in the first queue can be determined as a candidate prediction box. Among them, the second intersection-and-union ratio threshold can be set according to actual needs, and the embodiment of the present application does not limit this. For example, the second intersection-and-union ratio threshold can be set to 0.5 or 0.7.

[0072] For example, the first queue includes initial prediction box 1, initial prediction box 2, initial prediction box 3 and initial prediction box 4; at this time, the initial prediction box that is ranked first and has not been screened in the first queue is initial prediction box 1, then initial prediction box 1 can be used as the first prediction box, and initial prediction box 2, initial prediction box 3 and initial prediction box 4 can be used as respective second prediction boxes; then, the intersection-and-union ratio 1 of initial prediction box 1 and initial prediction box 2, the intersection-and-union ratio 2 of initial prediction box 1 and initial prediction box 3, and the intersection-and-union ratio 3 of initial prediction box 1 and initial prediction box 4 can be calculated; wherein, the intersection-and-union ratio 1 is greater than the second intersection-and-union ratio threshold, and the intersection-and-union ratio 1 and the intersection-and-union ratio 3 are less than the second intersection-and-union ratio threshold; therefore, the initial prediction box 2 can be removed from the first queue, and the initial prediction box 3 and the initial prediction box 4 can be retained; then, the initial prediction box 1 can be determined as the screened initial prediction box, And re-determine that the initial prediction box that has not been screened and is ranked the highest in the first queue is the initial prediction box 3, then the initial prediction box 3 can be determined as the first prediction box, and the initial prediction box 4 that is ranked after the initial prediction box 3 can be determined as the second prediction box, and then the intersection-and-union ratio 4 between the initial prediction box 3 and the initial prediction box 4 can be calculated, and the intersection-and-union ratio 4 is less than the second intersection-and-union ratio threshold, therefore, the initial prediction box 3 and the initial prediction box 4 can be retained, and the initial prediction box 3 can be determined as the screened initial prediction box; since there is only the initial prediction box 4 that has not been screened in the first queue, and there are no other prediction boxes ranked after the initial prediction box 4, therefore, the initial prediction box 4 can also be determined as the screened initial prediction box; so far, each initial prediction box in the first queue has been screened, and each initial prediction box in the first queue can be determined as a candidate prediction box.

[0073] Based on this, candidate prediction boxes with certain differences can be determined, which helps to reduce redundancy and thus alleviates the data size of the initial prediction boxes required to match in the subsequent matching process; at the same time, it also reduces the matching ambiguity caused by the high overlap between the initial prediction boxes, providing a data basis for the subsequent stable matching process.

[0074] Afterwards, the matching cost between each real box and each candidate prediction box can be calculated respectively to obtain a matching cost matrix; based on the matching cost matrix, the Hungarian matching algorithm can be used to match the real box with the candidate prediction box, and a matching prediction box corresponding to each real box can be determined when the global matching cost is minimized.

[0075] As an example, the matching cost v of the i-th real box in each real box and the j-th candidate prediction box in each candidate prediction box is ij The calculation formula can be shown as follows:

[0076]

[0077] in, is the confidence of the predicted category corresponding to the jth candidate prediction box; is the intersection-and-union ratio of the ith real box and the jth candidate prediction box, e1 and e2 are respectively the preset first index and second index, and the specific values ​​of e1 and e2 can be set according to actual needs, which is not limited in the embodiment of the present application.

[0078] According to the above formula, the matching cost between each real box and each candidate prediction box can be calculated to obtain a matching cost matrix; based on the matching cost matrix, the Hungarian matching algorithm can be used to match each real box to obtain a matching prediction box corresponding to each real box.

[0079] Through the above matching process, only a unique predicted box can be matched for each real box, thereby reducing the need to remove overlapping bounding boxes in the subsequent process, greatly simplifying the workload of subsequent processing.

[0080] Afterwards, the prediction result of the target sample image may be determined based on each matching prediction box of the target sample image and the corresponding category confidence vector.

[0081] After obtaining the prediction results of each sample image, the model parameters of the initial detection model can be adjusted based on the sample labels and prediction results corresponding to each sample image to obtain the target detection model. Specifically, the training loss can be calculated based on the sample labels and prediction results corresponding to each sample image, and the model parameters of the initial detection model can be adjusted based on the training loss.

[0082] Specifically, the classification loss and positioning loss of the initial detection model can be calculated to determine the training loss; here, the classification loss can be used to characterize the degree of difference between the predicted category and the true category (i.e., the category corresponding to the true box). The smaller the classification loss, the more accurate the classification of the target; the positioning loss can be used to characterize the degree of difference between the position of the matching prediction box and the position of the true box. The smaller the positioning loss, the more accurate the positioning of the target.

[0083] The following takes the target true box in the target sample image and the target matching prediction box corresponding to the target true box as an example to introduce the calculation process of the above training loss.

[0084] Specifically, the classification loss of the initial detection model can be calculated based on the category confidence vector of the target matching prediction box and the category of the target real box; the classification loss Loss cls The calculation formula can be expressed as:

[0085]

[0086] in, is the category confidence vector of the target matching prediction box, is the category of the target real box, Specifically, it can be a one-hot form; here, the calculation formula of the classification loss can specifically be a cross entropy loss function or an improved function of the cross entropy loss function, such as a focal loss (FocalLoss) function.

[0087] In addition, the positioning loss of the initial detection model can be calculated based on the position of the target matching prediction box and the position of the target real box; positioning loss Loss bbox The calculation formula can be expressed as:

[0088]

[0089] in, Match the location of the prediction box for the target, is the position of the target real box; and can be expressed as [x min ,y min ,x max ,y max ] in coordinate form, x min is the x value of the upper left corner of the bounding box, y max is the y value of the upper left corner of the bounding box, x max is the x value of the lower right corner of the bounding box, y min is the y value of the lower right corner of the bounding box. Here, the calculation formula of the positioning loss can be specifically an intersection over union loss (IoULoss) function or an improved function of the intersection over union loss, such as a generalized intersection over union loss (GIoULoss) function or a corrected intersection over union loss (CIoULoss) function.

[0090] Based on the classification loss and positioning loss, the training loss can be determined. Specifically, the classification loss and positioning loss can be weighted averaged using the preset classification loss weight and positioning loss weight to obtain the training loss. total The calculation formula can be expressed as:

[0091] Loss total =ɑLoss cls +βLoss bbox ,

[0092] Among them, ɑ is the classification loss weight, β is the positioning loss weight, and the values ​​of ɑ and β can be set according to actual needs, which is not limited in the embodiment of the present application.

[0093] Based on the classification loss and positioning loss (i.e., training loss), the model parameters of the initial detection model can be adjusted from the two dimensions of classification and positioning, so that more accurate classification results and positioning results can be predicted; for example, the model parameters of the initial detection model can be adjusted based on the gradient descent method to obtain the adjusted model parameters. Afterwards, the above training process can be repeated using the adjusted model parameters until the preset stop training condition is met; after stopping the training, the target detection model of the embodiment of the present application can be obtained. Here, the stop training condition can be that the number of training iterations is greater than or equal to the preset training iteration number threshold; or, it can be that the detection accuracy of the detection model is greater than or equal to the preset accuracy threshold.

[0094] After obtaining the target detection model in the embodiment of the present application, it can be applied to actual target detection. Figure 5 , an embodiment of a target detection method in the embodiment of the present application may include:

[0095] Step S501: Acquire an image to be detected.

[0096] In an embodiment of the present application, the image that requires target detection can be called an image to be detected. The image to be detected can specifically be an image uploaded by a user, or it can be an image captured by an image acquisition module (such as a camera) of a terminal device. The embodiment of the present application does not limit the method of acquiring the image to be detected.

[0097] Step S502: Perform target detection on the image to be detected based on the target detection model to obtain a target detection result of the image to be detected.

[0098] After obtaining the image to be detected, the image to be detected can be input into the target detection model, and the target detection model can perform target detection on the image to be detected and output the target detection result of the image to be detected. The target detection result of the image to be detected may include each target prediction box and the prediction category corresponding to each target prediction box.

[0099] Specifically, the prediction module of the target detection model can output the category confidence vector and prediction frame decoding parameters corresponding to each initial anchor frame of the image to be detected. Afterwards, for each initial anchor frame, the initial prediction frame corresponding to the initial anchor frame can be determined based on the prediction frame decoding parameters; specifically, the coordinates of the center point of the prediction frame (x b ,y b ), width w b With height h b The decoding method is as follows:

[0100] x b =(2Δx-0.5+x a )*S,

[0101] y b =(2Δy-0.5+y a )*S,

[0102] w b =4*Δw 2 *w a ,

[0103] h b =4*Δh 2 *h a ,

[0104] Among them, x a ,y a are the x and y values ​​of the center point of the corresponding initial anchor box; w a 、h a are the width and height of the initial anchor box respectively; S represents the downsampling rate, for example, if the scale of the prediction layer is 1 / 8 of the image input scale, S is 8; if the scale of the prediction layer is 1 / 16 of the image input scale, S is 16; if the scale of the prediction layer is 1 / 32 of the image input scale, S is 32.

[0105] In addition, for each initial anchor box, the predicted category corresponding to the initial anchor box can be determined from the category confidence vector corresponding to the initial anchor box; here, the maximum confidence in the category confidence vector corresponding to the initial anchor box can be determined, and the category corresponding to the maximum confidence is the predicted category corresponding to the initial anchor box, that is, the category of the initial prediction box corresponding to the initial anchor box.

[0106] In a specific implementation, the category confidence vector corresponding to the initial anchor box can be expressed as cls_pred = [c1, c2, ..., c k ], where c1 is the confidence of category 1 of the initial anchor box, c2 is the confidence of category 2 of the initial anchor box, ..., c k is the confidence of category k of the initial anchor box, then the predicted category corresponding to the initial anchor box is c1, c2, …, c k The category corresponding to the maximum value.

[0107] According to the above method, each initial prediction frame of the image to be detected can be decoded and the prediction category corresponding to each initial prediction frame can be determined. After that, each initial prediction frame of the image to be detected can be subsequently screened.

[0108] Specifically, the category confidences of the prediction categories of the initial prediction boxes can be sorted in descending order, and the initial prediction boxes corresponding to the top M category confidences can be determined; wherein, the value of M can be set according to actual needs, and the embodiment of the present application does not limit this, for example, M can be 3000; then, the M initial prediction boxes can be screened using the NMS method. Here, the M initial prediction boxes can be determined as the second queue, and the top unscreened initial prediction box in the second queue can be determined as the third prediction box, and each initial prediction box sorted after the third prediction box in the second queue can be determined as the fourth prediction box; then, the intersection-and-union ratio between the third prediction box and each fourth prediction box can be calculated, and if the intersection-and-union ratio of a fourth prediction box to the third prediction box is greater than or equal to the preset third intersection-and-union ratio threshold, the fourth prediction box can be removed from the second queue; if the intersection-and-union ratio of a fourth prediction box to the third prediction box is less than the third intersection-and-union ratio threshold, the fourth prediction box can be retained; after each fourth prediction box has completed the above judgment process, the fourth prediction box can be removed from the second queue. The three prediction boxes are determined as the initial prediction boxes that have been screened; thereafter, the third prediction box can be re-determined, and the above process can be repeated until all the initial prediction boxes in the second queue have become initial prediction boxes that have been screened; thereafter, each initial prediction box in the second queue can be determined as a target prediction box; here, the third intersection-over-union threshold can be set according to actual needs, and the embodiment of the present application is not limited to this; since a higher-precision target detection model can be obtained after the above-mentioned model training, a more relaxed NMS strategy can be adopted, that is, the third intersection-over-union threshold can be set to a larger value, for example, the third intersection-over-union threshold can be set to 0.7; accordingly, each target prediction box can be determined.

[0109] Since the target detection model will extract features of the image to be detected at different scales, it is also necessary to scale each target prediction box to the original scale of the image to be detected to obtain the final output target prediction box.

[0110] As an example, the coordinates of the center point of the target prediction box are (x b ,y b ), with a width of w b , height h b , the calculation formula for scaling the target prediction box to the original scale can be shown as follows:

[0111] x min ′=(x b -w b / 2-pad_w)*S,

[0112] y min ′=(y b -h b / 2-pad_h)*S,

[0113] x max ′=(x b +w b / 2-pad_w)*S,

[0114] y max ′=(y b +h b / 2-pad_h)*S,

[0115] Among them, x min ′ is the x value of the upper left corner of the target prediction box, y max ′ is the y value of the upper left corner of the target prediction box, x max ′ is the x value of the lower right corner of the target prediction box, y min ′ is the y value of the lower right corner of the target prediction box, pad_w is the width filled on the left side of the distance image in the preprocessing stage, and pad_h is the height filled on the top of the distance image in the preprocessing stage.

[0116] Afterwards, each target prediction box of the image to be detected and the prediction category corresponding to each target prediction box can be used as the target detection result of the image to be detected.

[0117] In summary, the embodiment of the present application obtains an image to be detected; performs target detection on the image to be detected based on a target detection model to obtain a target detection result of the image to be detected; wherein the target detection model is a trained neural network model for target detection, and the training process of the target detection model includes matching each candidate prediction frame to obtain a matching prediction frame corresponding to each real frame, and each candidate prediction frame is determined by a candidate anchor frame. Through the model training of the embodiment of the present application, the initial prediction frame (candidate prediction frame) can be first determined using the prior information provided by the (candidate) anchor frame, and then the prediction frame (matching prediction frame) that is closer to the real frame can be further screened out from the initial prediction frame, so that a target detection model with higher accuracy can be trained, which helps to improve the precision and accuracy of target detection in dense small target scenes.

[0118] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0119] Corresponding to a target detection method described in the above embodiment, Figure 6 A structural diagram of an embodiment of a target detection device provided in an embodiment of the present application is shown.

[0120] In an embodiment of the present application, a target detection device may include:

[0121] An acquisition module 601 is used to acquire an image to be detected;

[0122] The detection module 602 is used to perform target detection on the image to be detected based on the target detection model to obtain the target detection result of the image to be detected; wherein the target detection model is a trained neural network model for target detection, and the training process of the target detection model includes matching each candidate prediction frame to obtain a matching prediction frame corresponding to each real frame, and each candidate prediction frame is determined by a candidate anchor frame.

[0123] In a specific implementation of the embodiment of the present application, the target detection device further includes:

[0124] A sample set acquisition module, used to acquire a training sample set; wherein the training sample set includes a preset number of sample images and corresponding sample labels, and the sample labels include the corresponding real frames of the sample images;

[0125] A prediction module, used to use each of the sample images in the training sample set as an input of an initial detection model to obtain a prediction result corresponding to each of the sample images; wherein the prediction result includes each of the matching prediction boxes of the corresponding sample image, and one of the matching prediction boxes of the sample image corresponds to one of the real boxes of the sample image;

[0126] A parameter adjustment module is used to adjust the model parameters of the initial detection model based on the sample labels corresponding to each of the sample images and the prediction results to obtain the target detection model.

[0127] In a specific implementation of the embodiment of the present application, the prediction module includes:

[0128] An extraction submodule, used for extracting initial features corresponding to each of the sample images in the training sample set;

[0129] A fusion submodule, used for performing feature fusion on the initial features respectively corresponding to the sample images in the training sample set to obtain fused features respectively corresponding to the sample images in the training sample set;

[0130] The prediction submodule is used to perform result prediction based on the fusion features respectively corresponding to each of the sample images in the training sample set, so as to obtain the prediction results corresponding to each of the sample images in the training sample set.

[0131] In a specific implementation of the embodiment of the present application, the prediction module further includes:

[0132] A decoding submodule, configured to determine an initial prediction frame corresponding to each of the initial anchor frames based on the prediction frame decoding parameters of each of the initial anchor frames in the target sample image; wherein the target sample image is any one of the sample images in the training sample set;

[0133] A determination submodule, configured to determine each of the candidate anchor frames in the target sample image based on each of the initial anchor frames and each of the real frames in the target sample image;

[0134] A non-maximum suppression processing submodule, used for performing non-maximum suppression processing on the initial prediction frame of each candidate anchor frame to obtain each candidate prediction frame;

[0135] The matching submodule is used to match each of the candidate prediction frames of the target sample image to obtain a matching prediction frame corresponding to each of the real frames.

[0136] In a specific implementation of the embodiment of the present application, the determining submodule includes:

[0137] an IoU calculation unit, configured to calculate an IoU between a target initial anchor box and a target true box; wherein the target initial anchor box is any one of the initial anchor boxes in the target network unit, the target true box is any one of the true boxes of the target sample image, the target network unit is a first network unit or a second network unit, the first network unit is a network unit where the target true box is located, the second network unit is an adjacent network unit of the first network unit, and the target true box is any one of the true boxes of the target sample image;

[0138] A determination unit is configured to determine the target initial anchor frame as the candidate anchor frame if an intersection-and-union ratio (IoU) between the target initial anchor frame and the target real frame is greater than a first IoU threshold.

[0139] In a specific implementation of the embodiment of the present application, the matching submodule includes:

[0140] A cost calculation unit, used to calculate the matching cost between each of the real boxes and each of the candidate prediction boxes to obtain a matching cost matrix;

[0141] The matching unit is used to determine, based on the matching cost matrix, a matching prediction frame corresponding to each of the real frames when the global matching cost is minimized.

[0142] In a specific implementation of the embodiment of the present application, the parameter adjustment module includes:

[0143] A first loss calculation submodule, used to calculate the classification loss of the initial detection model based on the category confidence vector of the target matching prediction box in the target sample image and the category of the target real box;

[0144] A second loss calculation submodule, used to calculate the positioning loss of the initial detection model based on the position of the target matching prediction box and the position of the target real box;

[0145] The parameter adjustment submodule is used to adjust the model parameters of the initial detection model based on each of the classification losses and each of the positioning losses to obtain the target detection model.

[0146] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices, modules and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0147] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0148] Figure 7 A schematic block diagram of a terminal device provided in an embodiment of the present application is shown. For ease of explanation, only the parts related to the embodiment of the present application are shown.

[0149] like Figure 7 As shown, the terminal device 7 of this embodiment includes: a processor 70, a memory 71, and a computer program 72 stored in the memory 71 and executable on the processor 70. When the processor 70 executes the computer program 72, the steps in the above-mentioned target detection method embodiments are implemented, for example Figure 5 Alternatively, when the processor 70 executes the computer program 72, the functions of each module / unit in the above-mentioned device embodiments are realized, for example Figure 6 The functions of modules 601 to 602 are shown.

[0150] Exemplarily, the computer program 72 may be divided into one or more modules / units, which are stored in the memory 71 and executed by the processor 70 to complete the present application. The one or more modules / units may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program 72 in the terminal device 7.

[0151] The terminal device 7 may be a computing device such as a desktop computer, a notebook, a PDA, a smart phone, a smart TV, etc. Those skilled in the art will appreciate that Figure 7 It is only an example of the terminal device 7 and does not constitute a limitation on the terminal device 7. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the terminal device 7 may also include input and output devices, network access devices, buses, etc.

[0152] The processor 70 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor 70 may be the nerve center and command center of the terminal device 7. The processor 70 may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0153] The memory 71 may be an internal storage unit of the terminal device 7, such as a hard disk or memory of the terminal device 7. The memory 71 may also be an external storage device of the terminal device 7, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal device 7. Further, the memory 71 may also include both an internal storage unit and an external storage device of the terminal device 7. The memory 71 is used to store the computer program and other programs and data required by the terminal device 7. The memory 71 may also be used to temporarily store data that has been output or is to be output.

[0154] The terminal device 7 may also include a communication module, which may provide communication solutions including wireless local area networks (WLAN) (such as Wi-Fi networks), Bluetooth, Zigbee, mobile communication networks, global navigation satellite systems (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to network devices. The communication module may be one or more devices integrating at least one communication processing module. The communication module may include an antenna, which may have only one array element or an antenna array including multiple array elements. The communication module may receive electromagnetic waves through the antenna, frequency modulate and filter the electromagnetic wave signals, and send the processed signals to the processor. The communication module may also receive the signal to be sent from the processor, frequency modulate and amplify it, and convert it into electromagnetic waves for radiation through the antenna.

[0155] The terminal device 7 may further include a power management module, which may receive input from an external power source, a battery and / or a charger to supply power to the processor, the memory, the communication module, and the like.

[0156] The terminal device 7 may also include a display module, which may be used to display information input by a user or information provided to a user. The display module may include a display panel, and optionally, the display panel may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), etc. Further, the touch panel may cover the display panel, and when the touch panel detects a touch operation on or near it, it is transmitted to the processor to determine the type of the touch event, and then the processor provides a corresponding visual output on the display panel according to the type of the touch event.

[0157] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0158] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0159] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0160] In the embodiments provided in the present application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0161] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0162] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0163] An embodiment of the present application provides a computer program product. When the computer program product runs on the terminal device, the terminal device can implement the steps in the above-mentioned various method embodiments.

[0164] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable storage medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable storage media do not include electric carrier signals and telecommunication signals.

[0165] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A target detection method, characterized in that: include: Acquire the image to be detected; Performing target detection on the image to be detected based on the target detection model to obtain a target detection result of the image to be detected; Among them, the target detection model is a trained neural network model for target detection, and the training process of the target detection model includes matching each candidate prediction frame to obtain a matching prediction frame corresponding to each real frame, and each candidate prediction frame is determined by a candidate anchor frame.

2. The target detection method according to claim 1, characterized in that: The training process of the target detection model includes: Acquire a training sample set; wherein the training sample set includes a preset number of sample images and corresponding sample labels, and the sample labels include the corresponding real frames of the sample images; Using each of the sample images in the training sample set as an input of the initial detection model to obtain prediction results corresponding to each of the sample images; wherein the prediction results include category confidence vectors corresponding to each initial anchor frame of the corresponding sample image and prediction frame decoding parameters; Based on the sample labels corresponding to each of the sample images and the prediction results, the model parameters of the initial detection model are adjusted to obtain the target detection model.

3. The target detection method according to claim 2, characterized in that: The step of taking each of the sample images in the training sample set as an input of an initial detection model to obtain a prediction result corresponding to each of the sample images includes: Extracting initial features corresponding to each of the sample images in the training sample set; Performing feature fusion on the initial features respectively corresponding to the sample images in the training sample set to obtain fused features respectively corresponding to the sample images in the training sample set; Result prediction is performed based on the fusion features respectively corresponding to each of the sample images in the training sample set to obtain the prediction result corresponding to each of the sample images in the training sample set.

4. The target detection method according to claim 3, characterized in that: After performing result prediction based on the fusion features respectively corresponding to each of the sample images in the training sample set to obtain the prediction result corresponding to each of the sample images in the training sample set, the method further includes: Determine the initial prediction frame corresponding to each of the initial anchor frames based on the prediction frame decoding parameters of each of the initial anchor frames in the target sample image; wherein the target sample image is any one of the sample images in the training sample set; Determine each of the candidate anchor frames in the target sample image based on each of the initial anchor frames and each of the real frames in the target sample image; Performing non-maximum suppression processing on the initial prediction box of each candidate anchor box to obtain each candidate prediction box; A matching prediction frame corresponding to each of the real frames is obtained by matching each of the candidate prediction frames in the target sample image.

5. The target detection method according to claim 4, characterized in that: The determining each of the candidate anchor frames in the target sample image based on each of the initial anchor frames and each of the real frames in the target sample image comprises: Calculate the intersection and union ratio of the target initial anchor box and the target real box; wherein the target initial anchor box is any one of the initial anchor boxes in the target network unit, the target real box is any one of the real boxes of the target sample image, the target network unit is the first network unit or the second network unit, the first network unit is the network unit where the target real box is located, and the second network unit is the adjacent network unit of the first network unit; If the intersection-and-union ratio of the target initial anchor frame and the target real frame is greater than a first intersection-and-union ratio threshold, the target initial anchor frame is determined as the candidate anchor frame.

6. The target detection method according to claim 5, characterized in that: The step of matching each candidate prediction frame in the target sample image to obtain a matching prediction frame corresponding to each real frame, respectively, comprises: Calculate the matching cost between each of the real boxes and each of the candidate prediction boxes to obtain a matching cost matrix; Based on the matching cost matrix, a matching prediction frame corresponding to each of the real frames is determined when the global matching cost is minimized.

7. The target detection method according to claim 6, characterized in that: The adjusting the model parameters of the initial detection model based on the sample labels corresponding to each of the sample images and the prediction results to obtain the target detection model includes: Calculating the classification loss of the initial detection model based on the category confidence vector of the target matching prediction box in the target sample image and the category of the target real box; Calculating the positioning loss of the initial detection model based on the position of the target matching prediction box and the position of the target real box; Based on each of the classification losses and each of the positioning losses, the model parameters of the initial detection model are adjusted to obtain the target detection model.

8. A target detection device, characterized in that: include: An acquisition module, used for acquiring an image to be detected; A detection module is used to perform target detection on the image to be detected based on a target detection model to obtain a target detection result of the image to be detected; wherein the target detection model is a trained neural network model for target detection, and the training process of the target detection model includes matching each candidate prediction frame to obtain a matching prediction frame corresponding to each real frame, and each candidate prediction frame is determined by a candidate anchor frame.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the target detection method according to any one of claims 1 to 7 are implemented.

10. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the target detection method according to any one of claims 1 to 7 are implemented.

Images