Target detection method, apparatus, device, and medium

Abstract

The application provides a target detection method, device, equipment and medium. In the method, at least one process feature matrix and a result feature matrix are obtained by inputting a to-be-detected image into a feature extraction network model; then the result feature matrix is input into a to-be-detected target extraction network model to obtain at least one to-be-detected target data matrix; then for each to-be-detected target data matrix, a corresponding to-be-classified matrix is generated according to the position data of the process target frame in the to-be-detected image, the at least one process feature matrix, the result feature matrix and the to-be-detected target data matrix; finally, for each to-be-classified matrix, the to-be-classified matrix is input into a to-be-detected target type detection network model to obtain a detection result data matrix. Through the feature extraction network model, the to-be-detected target extraction network model and the target type detection network model, the target detection of the image is realized, and the detection accuracy is effectively improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and in particular to a target detection method, apparatus, device, and medium. Background Technology

[0002] Object detection is a method for determining the location and type of a target in an image.

[0003] In existing technologies, deep learning models are used. The image to be detected is input into the deep learning model, which then determines the location and type of the target within the image. However, since deep learning models are typically designed for images in well-lit scenes, and are used for both training and application, these images have higher resolution. Therefore, the detection results are usually poor for images with lower resolution.

[0004] In summary, existing target detection methods are designed for images with poor resolution, resulting in low detection accuracy. Summary of the Invention

[0005] This application provides a target detection method, apparatus, device, and medium to address the problem that existing target detection methods suffer from low detection accuracy when detecting images with poor resolution.

[0006] In a first aspect, embodiments of this application provide a target detection method, including:

[0007] Acquire the image to be detected;

[0008] The image to be detected is input into a feature extraction network model to obtain at least one process feature matrix and one result feature matrix; wherein, the feature extraction network model is pre-trained and is used as a computational model to determine the process feature matrix and the result feature matrix based on the image;

[0009] The resulting feature matrix is ​​input into the target extraction network model to obtain at least one target data matrix. Each target data matrix includes the position data of the process target box and the probability of the target being detected corresponding to the process target box. The target existence probability is used to represent the probability that the target being detected exists in the process target box corresponding to the target existence probability. The target extraction network model is pre-trained and is used to determine the calculation model of the target data matrix based on the resulting feature matrix.

[0010] For each target data matrix to be detected, a corresponding classification matrix is ​​generated based on the image to be detected, the at least one process feature matrix, the result feature matrix, and the position data of the process target bounding boxes in the target data matrix to be detected.

[0011] For each matrix to be classified, the matrix to be classified is input into the target type detection network model to obtain the detection result data matrix. The detection result data matrix includes the target type identifier and the position data of the target bounding box. The target type detection network model is a pre-trained computational model used to determine the detection result data matrix based on the matrix to be classified.

[0012] In one specific implementation, the target type detection network model includes a spatial location feature mixing layer, a channel information feature mixing layer, and a classification and regression layer. The step of inputting the classification matrix into the target type detection network model to obtain a detection result data matrix includes:

[0013] The matrix to be classified is input into the spatial location feature mixing layer to obtain the spatial location feature mixing matrix;

[0014] The spatial location feature mixing matrix is ​​input into the channel information feature mixing layer to obtain the location channel feature mixing matrix;

[0015] The location channel feature mixing matrix is ​​input into the classification and regression layer to obtain the detection result data matrix.

[0016] In one specific implementation, the resulting feature matrix is ​​input into the target extraction network model to obtain at least one target data matrix, including:

[0017] Based on the size of the resulting feature matrix, multiple initial target boxes are generated;

[0018] For each initial target box, a binary classification process is performed based on the data corresponding to the initial target box in the result feature matrix to obtain the probability of the existence of the target to be detected;

[0019] For each initial target box, bounding box regression is performed based on the data corresponding to the initial target box in the result feature matrix to obtain the position data of the corrected target box;

[0020] Based on the preset threshold for the probability of the target to be detected and the probability of the target to be detected corresponding to each initial target box, all initial target boxes are filtered to obtain filtered target boxes;

[0021] Based on the position data of the corrected target box corresponding to each filtered target box, nonmaximum suppression processing is performed to obtain the at least one target data matrix to be detected.

[0022] In one specific embodiment, before acquiring the image to be detected, the method further includes:

[0023] Obtain a preset number of low-resolution training images;

[0024] The first initial neural network model is trained using the preset number of low-resolution training images until the first training iteration equals the first preset iteration, thus obtaining the feature extraction network model.

[0025] In one specific implementation, the step of training the first initial neural network model using the preset number of low-resolution training images until the first training iteration equals the first preset iteration, to obtain the feature extraction network model, includes:

[0026] Select one training image from the preset number of low-resolution training images;

[0027] The training image is input into the first initial neural network model to obtain at least one training process feature matrix and one training result feature matrix;

[0028] Update the first training iteration;

[0029] If the updated first training iteration is equal to the first preset iteration, the feature extraction network model is obtained;

[0030] If the updated first training iterations are less than the first preset iterations, then the first initial neural network model is updated to obtain the trained neural network model; and a training image is selected again from the preset number of low-resolution training images and input into the trained neural network model to update the first training iterations, until the updated first training iterations are equal to the first preset iterations, thus obtaining the feature extraction network model.

[0031] In one specific embodiment, the method further includes:

[0032] The preset number of low-resolution training images are input into the feature extraction network model to obtain at least one training process feature matrix corresponding to each training image, and a training result feature matrix corresponding to each training image.

[0033] The second initial neural network model is trained using the feature matrix of the training result corresponding to each training image until the second training iteration equals the second preset iteration, thus obtaining the target extraction network model.

[0034] In one specific implementation, the step of training the second initial neural network model using the feature matrix of the training result corresponding to each training image until the second training iteration equals the second preset iteration, to obtain the target extraction network model, includes:

[0035] Select one training result matrix from the feature matrix of the training result corresponding to each training image;

[0036] The training result matrix is ​​input into the second initial neural network model to obtain at least one training data matrix of the target to be detected.

[0037] Update the second training iteration;

[0038] If the updated second training iterations are equal to the second preset iterations, the target extraction network model is obtained;

[0039] If the updated second training iterations are less than the second preset iterations, then the second initial neural network model is updated to obtain the trained neural network model; and a training result matrix is ​​selected from the training result feature matrix corresponding to each training image and input into the trained neural network model to update the second training iterations until the updated second training iterations are equal to the second preset iterations, thus obtaining the target extraction network model.

[0040] In one specific embodiment, the method further includes:

[0041] The training result feature matrix corresponding to each training image is input into the feature extraction network model to obtain at least one training process feature matrix corresponding to each training image, and a training result feature matrix corresponding to each training image.

[0042] The feature matrix of the training result corresponding to each training image is input into the target extraction network model to obtain at least one target training data matrix corresponding to each training image;

[0043] For each training data matrix of a target to be detected corresponding to a training image, a corresponding training matrix to be classified is generated based on the training image, at least one training process feature matrix corresponding to the training image, and the training result feature matrix corresponding to the training image.

[0044] The third initial neural network model is trained using the training matrix corresponding to each training image until the third training iteration equals the third preset iteration, thus obtaining the target type detection network model.

[0045] In one specific implementation, a third initial neural network model is trained using the training matrix corresponding to each training image until the third training iteration equals the third preset iteration, thereby obtaining the target type detection network model, including:

[0046] Select one training matrix corresponding to a training image from all the training matrices corresponding to each training image;

[0047] The training matrix to be classified is input into the third initial neural network model to obtain the training detection result data matrix;

[0048] Update the third training iteration;

[0049] If the updated third training iteration is equal to the third preset iteration, the target type detection network model is obtained;

[0050] If the updated third training iterations are less than the third preset iterations, then the third initial neural network model is updated to obtain the trained neural network model; and a training matrix corresponding to a training image is selected from all the training matrices corresponding to each training image and input into the trained neural network model to update the third training iterations until the updated third training iterations are equal to the third preset iterations, thus obtaining the target type detection network model.

[0051] Secondly, embodiments of this application provide a target detection device, comprising:

[0052] The acquisition module is used to acquire the image to be detected;

[0053] Processing module, used for:

[0054] The image to be detected is input into a feature extraction network model to obtain at least one process feature matrix and one result feature matrix; wherein, the feature extraction network model is pre-trained and is used as a computational model to determine the process feature matrix and the result feature matrix based on the image;

[0055] The resulting feature matrix is ​​input into the target extraction network model to obtain at least one target data matrix. Each target data matrix includes the position data of the process target box and the probability of the target being detected corresponding to the process target box. The target existence probability is used to represent the probability that the target being detected exists in the process target box corresponding to the target existence probability. The target extraction network model is pre-trained and is used to determine the calculation model of the target data matrix based on the resulting feature matrix.

[0056] For each target data matrix to be detected, a corresponding classification matrix is ​​generated based on the image to be detected, the at least one process feature matrix, the result feature matrix, and the position data of the process target bounding boxes in the target data matrix to be detected.

[0057] For each matrix to be classified, the matrix to be classified is input into the target type detection network model to obtain the detection result data matrix. The detection result data matrix includes the target type identifier and the position data of the target bounding box. The target type detection network model is a pre-trained computational model used to determine the detection result data matrix based on the matrix to be classified.

[0058] Thirdly, embodiments of this application provide an electronic device, including:

[0059] Processor, memory, communication interface;

[0060] The memory is used to store the executable instructions of the processor;

[0061] The processor is configured to execute the target detection method according to any one of the first aspects by executing the executable instructions.

[0062] Fourthly, embodiments of this application provide a readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the target detection method described in any of the first aspects.

[0063] Fifthly, embodiments of this application provide a computer program product, including a computer program, which, when executed by a processor, is used to implement the target detection method described in any of the first aspects.

[0064] The target detection method, apparatus, device, and medium provided in this application involve inputting the image to be detected into a feature extraction network model to obtain at least one process feature matrix and one result feature matrix; then, inputting the result feature matrix into a target extraction network model to obtain at least one target data matrix; furthermore, for each target data matrix, generating a corresponding classification matrix based on the image to be detected, at least one process feature matrix, the result feature matrix, and the position data of the process target bounding boxes in the target data matrix; finally, inputting each classification matrix into a target type detection network model to obtain a detection result data matrix. This solution achieves target detection in images through a feature extraction network model, a target extraction network model, and a target type detection network model, effectively improving detection accuracy. Attached Figure Description

[0065] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0066] Figure 1 A flowchart illustrating an embodiment of the target detection method provided in this application;

[0067] Figure 2 A flowchart illustrating Embodiment 2 of the target detection method provided in this application;

[0068] Figure 3 A flowchart illustrating Embodiment 3 of the target detection method provided in this application;

[0069] Figure 4 This is a schematic diagram of the structure of an embodiment of the target detection device provided in this application;

[0070] Figure 5 This is a schematic diagram of the structure of an electronic device provided in this application. Detailed Implementation

[0071] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments made by those skilled in the art under the guidance of these embodiments are within the scope of protection of this application.

[0072] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0073] Object detection is one of the most fundamental and commonly used applications in computer vision, forming the basis for many computer vision tasks and finding applications in numerous fields. Object detection has consistently received significant attention, evolving from traditional feature extraction methods based on wavelet transform and scale-invariant features to the widely used deep learning methods. Current technologies input the image to be detected into a deep learning model to obtain the location and type of the target object within the image. However, since deep learning models are typically designed for images in well-lit scenes, and both model training and usage utilize images from such high-resolution scenes, the detection results are usually poor for images with lower resolution.

[0074] In some scenarios, the images captured by cameras are often less than ideal. For example, in security scenarios, inspections may need to be carried out in blurry environments such as night or rain. In other scenarios, underwater autonomous vehicles are used to inspect the conditions of seabed aquaculture or the natural environment. In these scenarios, the images collected are low-resolution images.

[0075] Therefore, existing target detection methods suffer from low detection accuracy when detecting images with poor resolution.

[0076] To address the problems existing in the prior art, the inventors, during their research on object detection methods, discovered that to improve detection accuracy, object detection can be divided into multiple stages. First, the bounding box of the target to be detected can be determined, followed by identification and determination of the target's type and more precise location. Furthermore, during model training, lower-resolution images are used for training. Based on the above inventive concept, the object detection scheme in this application was designed.

[0077] The object detection method in this application can be executed by a computer, or by a server, terminal device, camera, video recording device, or other device capable of running a neural network model. This application does not limit it. The following explanation uses a computer as an example.

[0078] The following provides examples illustrating the application scenarios of the target detection method provided in this application.

[0079] For example, in this scenario, the user needs to perform target detection on a low-resolution image, which could be taken in an underwater scene, in rainy or foggy weather, or at night for security purposes.

[0080] The user inputs a low-resolution image to be detected into the computer, which then acquires the image and inputs it into the image input feature extraction network model for feature extraction processing, resulting in at least one process feature matrix and one result feature matrix.

[0081] The resulting feature matrix is ​​then input into the target extraction network model to obtain the target bounding boxes containing the target. This yields at least one target data matrix, where each target data matrix includes the location data of the process target bounding box and the probability of the target being detected corresponding to the process target bounding box.

[0082] For each target data matrix to be detected, a corresponding classification matrix is ​​generated based on the image to be detected, at least one process feature matrix, the result feature matrix, and the position data of the process target bounding boxes in the target data matrix to be detected. The features of the process target bounding boxes in the image to be detected can be extracted from the process feature matrix and the result feature matrix and fused to obtain the classification matrix.

[0083] Finally, for each matrix to be classified, the matrix is ​​input into the target type detection network model for classification and refined target bounding box processing to obtain the detection result data matrix. The detection result data matrix includes the target type identifier and the position data of the target bounding box.

[0084] The computer can also display the target bounding box of the result in the image to be detected for the user to view.

[0085] It should be noted that the above scenario is only an example of an application scenario provided by the embodiments of this application. The embodiments of this application do not limit the actual form of the various devices included in the scenario. In the specific application of the solution, it can be set according to actual needs.

[0086] The technical solution of this application will now be described in detail through specific embodiments. It should be noted that the following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0087] Figure 1 This is a flowchart illustrating an embodiment of the target detection method provided in this application. This embodiment describes how, after a computer acquires an image to be detected, it performs model detection through a feature extraction network model, a target extraction network model, and a target type detection network model to obtain the detection result. The method in this embodiment can be implemented through software, hardware, or a combination of both. Figure 1 As shown, the target detection method specifically includes the following steps:

[0088] S101: Acquire the image to be detected.

[0089] In this step, the user uses a computer to perform target detection. The user inputs the image to be detected into the computer, and the computer can then obtain the image.

[0090] It should be noted that the executing entity can also be a server, a camera device, etc., and the method of acquiring the image to be detected can also be automatic acquisition. For example, when the executing entity is a camera device, capturing an image is equivalent to acquiring the image. This application does not limit the executing entity or the method of acquiring the image to be detected; these can be determined according to the actual situation.

[0091] S102: Input the image to be detected into the feature extraction network model to obtain at least one process feature matrix and one result feature matrix.

[0092] In this step, after the computer acquires the image to be detected, it needs to extract features from the image in order to identify the location of the target. Therefore, the image to be detected needs to be input into the feature extraction network model to obtain at least one process feature matrix and one result feature matrix. The feature extraction network model is pre-trained and is a computational model used to determine the process feature matrix and the result feature matrix based on the image.

[0093] The feature extraction network model performs standardization processing on the image to be detected, that is, scaling the image to be detected to a preset size, and then generates a three-dimensional matrix based on the data of the image to be detected. Feature extraction processing is then performed on this three-dimensional matrix to obtain at least one process feature matrix and one result feature matrix. The size of this three-dimensional matrix can be 224*224*3, or it can be 112*112*3, 448*448*3, etc. This application embodiment does not limit the size of the three-dimensional matrix and can determine it according to the actual situation.

[0094] It should be noted that the feature extraction network model can be divided into multiple stages when extracting features. Each stage includes convolutional layers, activation function layers and pooling layers. After multiple stages of processing, feature pyramids are used to fuse features, resulting in at least one process feature matrix and one result feature matrix.

[0095] For example, the three-dimensional matrix generated by standardizing the image to be detected is 224*224*3. Five stages of feature extraction are performed. In each stage, the number of padding layers in the convolutional layer is 1, and a convolution with a stride of 1 is performed. The number of channels in the convolutional kernel is 256. The pooling layer can halve the number of the two dimensions other than the number of channels. Therefore, there are four process feature matrices with sizes of 112*112*256, 56*56*256, 28*28*256, and 14*14*256, respectively. The size of a result feature matrix is ​​7*7*256.

[0096] For example, the three-dimensional matrix generated by standardizing the image to be detected is C1, where u represents upsampling, lc represents lateral connection, C2 = u(C1), C3 = u(C2), C4 = u(C3), C5 = u(C4), P5 = lc(C5), P4 = lc(C4) + P5, P3 = lc(C3) + P4, P2 = lc(C2) + P3, P1 = lc(C1) + P2, where P2, P3, P3, and P5 represent process feature matrices, and P1 represents result feature matrix.

[0097] S103: Input the resulting feature matrix into the target extraction network model to obtain at least one target data matrix.

[0098] In this step, after the computer obtains at least one process feature matrix and one result feature matrix, it can determine the location of the target to be detected based on the result feature matrix. The result feature matrix is ​​then input into the target extraction network model to obtain at least one target data matrix. Each target data matrix includes the location data of the process target bounding box and the probability of the target's presence corresponding to that bounding box. The target presence probability represents the probability that the target to be detected exists within the process target bounding box corresponding to the target presence probability. The target extraction network model is pre-trained and is used as a computational model to determine the target data matrix based on the result feature matrix.

[0099] For example, the target data matrix to be detected is a 1*4 matrix, and the values ​​in the matrix represent the coordinates of the top left corner of the target bounding box, the length of the target bounding box, the width of the target bounding box, and the probability of the target being detected.

[0100] It should be noted that multiple target extraction network models can be used to process the resulting feature matrix serially, which can improve the detection speed.

[0101] S104: For each target data matrix to be detected, generate a corresponding classification matrix based on the image to be detected, at least one process feature matrix, the result feature matrix, and the position data of the process target bounding boxes in the target data matrix to be detected.

[0102] In this step, after the computer obtains at least one target data matrix to be detected, for each target data matrix to be detected, the region corresponding to the image to be detected is determined based on the position data of the target box in the target data matrix to be detected, and then the feature data corresponding to the process feature matrix and the result feature matrix are determined, and then feature mixing is performed to obtain the classification matrix.

[0103] S105: For each matrix to be classified, input the matrix to be classified into the target type detection network model to obtain the detection result data matrix.

[0104] In that step, after the computer obtains at least one matrix to be classified, in order to determine the type of the corresponding target to be detected and its more accurate location, for each matrix to be classified, it needs to be input into the target type detection network model to obtain the detection result data matrix. The detection result data matrix includes the target type identifier and the location data of the target bounding box. The target type detection network model is a pre-trained computational model used to determine the detection result data matrix based on the matrix to be classified.

[0105] The target type detection network model mixes spatial location features and channel information features into the classification matrix, and then performs classification and calculates the target box position to obtain the detection result data matrix.

[0106] For example, the detection result data matrix is ​​a 1*5 matrix, where the values ​​in the matrix represent, in order, the x-coordinate of the center point of the target box, the y-coordinate of the center point of the target box, the x-coordinate of the top left corner of the target box, the y-coordinate of the top left corner of the target box, and the identifier of the target type to be detected.

[0107] It should be noted that the computer can also mark the target boxes in the image to be detected based on the target type identifier and the position data of the target boxes in the detection result matrix, and display the corresponding type for the user to view.

[0108] The target detection method provided in this embodiment involves inputting the acquired image to be detected into a feature extraction network model to obtain at least one process feature matrix and one result feature matrix. The result feature matrix is ​​then input into a target extraction network model to obtain at least one target data matrix. For each target data matrix, a corresponding classification matrix is ​​generated by combining the image to be detected, at least one process feature matrix, the result feature matrix, and the position data of the bounding boxes in the target data matrix. Finally, the classification matrix is ​​input into a target type detection network model to obtain a detection result data matrix. Compared to existing technologies that suffer from low detection accuracy when detecting images with poor resolution, this solution effectively improves the detection accuracy of target type and location by extracting and fusing features in the extraction network model, performing feature fusion during the generation of the classification matrix, and mixing spatial location features and channel information features in the target type detection network model, while accurately calculating the bounding box position.

[0109] Figure 2This is a flowchart illustrating a second embodiment of the target detection method provided in this application. Based on the above embodiments, this application describes how the target type detection network model performs spatial location feature mixing and channel information mixing on the classification matrix, classifies the target, determines the precise location of the bounding box, and obtains the detection result data matrix. For example... Figure 2 As shown, the target detection method specifically includes the following steps:

[0110] S201: Input the matrix to be classified into the spatial location feature mixing layer to obtain the spatial location feature mixing matrix.

[0111] In this step, after the computer obtains the matrix to be classified, it needs to input it into the target type detection network model to classify and determine the precise location of the target box. Since the target type detection network model includes a spatial location feature mixing layer, a channel information feature mixing layer, and a classification regression layer, and in order to make the classification and the obtained target box more accurate, the matrix to be classified needs to be input into the spatial location feature mixing layer to obtain the spatial location feature mixing matrix.

[0112] The spatial location feature mixing layer first reshapes the matrix to be classified, converting it into a two-dimensional matrix. For example, if the size of the matrix to be classified is 7*7*256, the two dimensions other than the dimension corresponding to the number of channels are expanded to form a matrix of size 49*256. Then, group normalization is performed, that is, the channels are divided into multiple groups, and each group is normalized. Finally, spatial location feature mixing is performed on the matrix after group normalization to obtain the spatial location feature mixing matrix.

[0113] It should be noted that, expressed as a formula, it can be written as U T =X T +W2σ(W1Norm(X T ), where U represents the spatial location feature mixing matrix, X T Norm(X) represents the reshaped matrix to be classified. T ) indicates that group normalization is performed on the reshaped matrix to be classified; X T +W2σ(W1Norm(X T )) indicates that after group normalization is performed on the reshaped matrix to be classified, spatial location features are mixed; σ function is a nonlinear function, and W2 and W1 are constants.

[0114] S202: Input the spatial location feature mixing matrix into the channel information feature mixing layer to obtain the location channel feature mixing matrix.

[0115] In this step, after the computer obtains the spatial location feature mixing matrix, in order to make the classification and the resulting target boxes more accurate, it needs to be input into the channel information feature mixing layer to obtain the location channel feature mixing matrix.

[0116] The channel information feature mixing layer first transposes the spatial location feature mixing matrix, then performs a group normalization process, and then performs channel information feature mixing on the group normalized matrix to obtain the location channel feature mixing matrix.

[0117] It should be noted that, expressed by the formula, it can be written as Y=U+W4σ(W3Norm(U)), where Y represents the location channel feature mixing matrix, U represents the spatial location feature mixing matrix, Norm(U) represents the group normalization processing of the spatial location feature mixing matrix; U+W4σ(W3Norm(U)) represents the group normalization processing of the spatial location feature mixing matrix, followed by channel information feature mixing; σ is a nonlinear function, and W3 and W4 are constants.

[0118] S203: Input the location channel feature mixing matrix into the classification and regression layer to obtain the detection result data matrix.

[0119] In this step, the computer obtains a location channel feature mixing matrix. Since the spatial location features and channel information features in the matrix to be classified have been fully mixed, the classification and determination of the target box position will be more accurate. Therefore, the location channel feature mixing matrix is ​​input into the classification and regression layer to classify and determine the position of the target box, resulting in a detection result data matrix. The classification and regression layer consists of two parts: one part classifies to obtain the type of the target to be detected, and the other part regresses to obtain the position of the target box. The two parts are independent and parallel. The classification part uses a fully connected layer, and the regression part uses a convolutional neural network. The location channel feature mixing matrix is ​​input into the fully connected layer for classification to obtain the target type identifier; the location channel feature mixing matrix is ​​then processed by the convolutional neural network to obtain the position data of the target box.

[0120] The target detection method provided in this embodiment effectively improves the accuracy of the detection results by mixing spatial location features and channel information features of the matrix to be classified before classifying and determining the position of the target box.

[0121] Figure 3 This is a flowchart illustrating Embodiment 3 of the target detection method provided in this application. Based on the above embodiments, this embodiment describes how the target extraction network model performs binary classification, bounding box regression, filtering, and non-maximum suppression on the resulting feature matrix to obtain the target data matrix. Figure 3 As shown, the target detection method specifically includes the following steps:

[0122] S301: Generate multiple initial target boxes based on the size of the resulting feature matrix.

[0123] In this step, after the computer obtains the result feature matrix, in order to determine the target matrix to be detected, it first generates multiple initial target boxes based on the size of the result feature matrix.

[0124] Determine the size of the two dimensions other than the dimension corresponding to the number of channels, and then determine the number of nodes based on the product of the values ​​of these two dimensions; for each node, generate a preset number of initial target boxes of different sizes.

[0125] For example, the size of the resulting feature matrix is ​​7*7*256, the dimension size corresponding to the number of channels is 256, so the number of nodes is 49. For each node, 9 initial target boxes can be generated, so the total number of initial target boxes is 49*9=441.

[0126] S302: For each initial target box, perform binary classification based on the data corresponding to the initial target box in the result feature matrix to obtain the probability of the existence of the target to be detected.

[0127] In this step, after the computer obtains the initial target bounding boxes, it performs binary classification processing on each initial target bounding box based on the data corresponding to the initial target bounding box in the result feature matrix, that is, distinguishing between the foreground and the background, and obtaining the probability of the existence of the target to be detected.

[0128] S303: For each initial target box, perform bounding box regression processing based on the data corresponding to the initial target box in the result feature matrix to obtain the position data of the corrected target box.

[0129] In this step, after the computer obtains the initial target box, if there is a target to be detected in the initial target box, the size of the initial target box may not match the size of the target to be detected because the initial target box was generated according to a set size. Therefore, it is necessary to perform bounding box regression processing based on the data corresponding to the initial target box in the result feature matrix to obtain the position data of the corrected target box.

[0130] It should be noted that the execution order of steps S302 and S303 can be: step S302 is executed first and then step S303 is executed; step S303 is executed first and then step S302 is executed; or step S302 and step S303 are executed simultaneously. This application embodiment does not limit the execution order of steps S302 and S303, and it can be determined according to the actual situation.

[0131] S304: Based on the preset target existence probability threshold and the target existence probability corresponding to each initial target box, filter all initial target boxes to obtain filtered target boxes.

[0132] In this step, after the computer obtains the probability of the target's existence and the position data of the corrected target bounding boxes, there is a correspondence between the initial target bounding boxes, the probability of the target's existence, and the corrected target bounding boxes. Since some initial target bounding boxes do not contain the target, meaning their corresponding corrected target bounding boxes also do not contain the target, these boxes need to be filtered out. This requires filtering all initial target bounding boxes based on a preset threshold for the probability of the target's existence and the probability of the target's existence corresponding to each initial target bounding box, resulting in filtered target bounding boxes. Initial target bounding boxes whose probability of the target's existence is greater than or equal to the preset threshold for the probability of the target's existence are used as filtered target bounding boxes.

[0133] It should be noted that the preset probability threshold for the existence of the target to be detected can be 0.5, 0.6, or 0.7, etc. This application embodiment does not specifically limit the preset probability threshold for the existence of the target to be detected, and it can be set according to the actual situation.

[0134] S305: Based on the position data of the corrected target box corresponding to each filtered target box, perform non-maximum suppression processing to obtain at least one target data matrix to be detected.

[0135] In this step, after the computer obtains the filter target boxes, since some of the targets to be detected in the filter target boxes are duplicates, that is, the targets to be detected in the corresponding correction target boxes are duplicates, deduplication processing is required. Therefore, based on the position data of the correction target box corresponding to each filter target box, nonmaximum suppression processing is performed to obtain at least one target data matrix.

[0136] First, based on the probability of the target's existence, select the modified target box with the highest probability of the target's existence from all the modified target boxes corresponding to the filtered target boxes. Use its corresponding position data and the probability of the target's existence as the position data of the process target box in the target data matrix, and the probability of the target's existence corresponding to the process target box. This modified target box is the process target box. Next, calculate the intersection-union ratio (IUR) between the remaining modified target boxes and this process target box; delete the modified target boxes whose IUR is greater than a preset IUR threshold. Then, from all the modified target boxes corresponding to the filtered target boxes, excluding the process target boxes, select the modified target box with the highest probability of the target's existence and determine it as the process target box. Similar to the above steps, calculate the IUR, delete modified target boxes whose IUR is greater than a preset IUR threshold, and repeat the above steps until all modified target boxes are process target boxes.

[0137] It should be noted that the preset intersection-union ratio threshold can be 0.6, 0.7, or 0.8, etc. This application embodiment does not limit the preset intersection-union ratio threshold, and it can be set according to the actual situation.

[0138] The target detection method provided in this embodiment obtains the target data matrix by performing binary classification, bounding box regression, filtering out target boxes that do not contain the target to be detected, and deduplication through nonmaximum suppression on the result data matrix, which can effectively improve the accuracy of the detection results.

[0139] The training process of the feature extraction network model, the target extraction network model, and the target type detection network model provided in this application is described below.

[0140] This application first trains a first initial neural network model to obtain a feature extraction network model; then trains a second initial neural network model based on the feature extraction network model to obtain a target extraction network model; finally, trains a third initial neural network model based on the feature extraction network model and the target extraction network model to obtain a target type detection network model.

[0141] First, obtain a preset number of low-resolution training images.

[0142] For example, the underwater target detection dataset UTDAC2020 or the VOC Devkit dataset can be selected. It should be noted that the preset number can be 1000, 2000, or 3000, etc. This application embodiment does not limit the preset number and can be determined according to the actual situation.

[0143] Then, a preset number of low-resolution training images are used to train the first initial neural network model until the first training iteration equals the first preset iteration, thus obtaining the feature extraction network model.

[0144] Specifically, select one training image from a preset number of low-resolution training images;

[0145] Input the training image into the first initial neural network model to obtain at least one training process feature matrix and one training result feature matrix;

[0146] The first initial neural network model performs standardization processing on the training images, that is, it scales the training images to a preset size, then generates a three-dimensional matrix based on the data of the training images, and then performs feature extraction processing on the three-dimensional matrix to obtain at least one process feature matrix and one result feature matrix.

[0147] It should be noted that the first initial neural network model can be divided into multiple stages when extracting features. Each stage includes convolutional layers, activation function layers and pooling layers. After multiple stages of processing, feature pyramids are used for feature fusion to obtain at least one process feature matrix and one result feature matrix.

[0148] Then update the first training iteration, which means incrementing the first training iteration by one;

[0149] If the updated first training iteration equals the first preset iteration, a feature extraction network model is obtained.

[0150] If the updated first training iterations are less than the first preset iterations, then the first initial neural network model is updated to obtain the trained neural network model; and a training image is selected again from the preset number of low-resolution training images and input into the trained neural network model to update the first training iterations, until the updated first training iterations are equal to the first preset iterations, thus obtaining the feature extraction network model.

[0151] It should be noted that the first preset number of times can be 500, 1000, or 2000, etc. This application embodiment does not limit the first preset number of times, and it can be set according to the actual situation.

[0152] Once the feature extraction network model is trained, the second initial neural network model can be trained to obtain the target extraction network model.

[0153] A predetermined number of low-resolution training images are input into the feature extraction network model to obtain at least one training process feature matrix corresponding to each training image, and a training result feature matrix corresponding to each training image.

[0154] The second initial neural network model is trained using the feature matrix of the training result corresponding to each training image until the second training iteration equals the second preset iteration, thus obtaining the target extraction network model.

[0155] Specifically, a training result matrix is ​​selected from the feature matrix of the training result corresponding to each training image;

[0156] Input the training result matrix into the second initial neural network model to obtain at least one training data matrix of the target to be detected;

[0157] The second initial neural network model generates multiple initial target boxes based on the size of the feature matrix of the training results. For each initial target box, binary classification is performed based on the data corresponding to the initial target box in the training result matrix to obtain the probability of the existence of the target to be detected. For each initial target box, bounding box regression is performed based on the data corresponding to the initial target box in the training result matrix to obtain the position data of the corrected target box. All initial target boxes are filtered according to a preset threshold for the probability of the target to be detected and the probability of the target to be detected corresponding to each initial target box to obtain filtered target boxes. Non-maximum suppression is performed based on the position data of the corrected target boxes corresponding to each filtered target box to obtain at least one training data matrix of the target to be detected.

[0158] Then update the second training iteration, which means incrementing the second training iteration by one;

[0159] If the updated second training iterations are equal to the second preset iterations, the target extraction network model is obtained.

[0160] If the updated second training iterations are less than the second preset iterations, then the second initial neural network model is updated to obtain the trained neural network model; and a training result matrix is ​​selected from the training result feature matrix corresponding to each training image and input into the trained neural network model to update the second training iterations, until the updated second training iterations are equal to the second preset iterations, thus obtaining the target extraction network model.

[0161] It should be noted that the second preset number of times can be 500, 1000, or 2000, etc. This application embodiment does not limit the second preset number of times, and it can be set according to the actual situation.

[0162] It should be noted that during the training of the target extraction network model, for each initial target box, after obtaining the corresponding target existence probability and the corrected target box location data, the loss function can be calculated. The loss function can be expressed by the formula... We obtain, where L cls (obj i ,y i )=-y i log(obj i )-(1-y i log(1-obj) i ), N represents the total number of initial bounding boxes; i belongs to [1, N], representing the corresponding initial bounding box; obj i y represents the probability of the detected target corresponding to the initial bounding box i; i N represents the label corresponding to the initial target box i, with a value of 1 or 0;pos This represents the number of initial bounding boxes whose intersection-union ratio (IUU) with the ground truth bounding boxes is greater than a preset threshold; t i This represents the predicted offset corresponding to the initial target box i; This represents the actual offset corresponding to the initial target box i.

[0163] Once the target extraction network model is trained, the third initial neural network model can be trained to obtain the target type detection network model.

[0164] Input the training result feature matrix corresponding to each training image into the feature extraction network model to obtain at least one training process feature matrix corresponding to each training image, and the training result feature matrix corresponding to each training image.

[0165] Input the feature matrix of the training result corresponding to each training image into the target extraction network model to obtain at least one target training data matrix corresponding to each training image;

[0166] For each training data matrix of each target to be detected corresponding to each training image, a corresponding training matrix to be classified is generated based on the training image, at least one training process feature matrix corresponding to the training image, and the training result feature matrix corresponding to the training image.

[0167] For each training data matrix of a target to be detected, the region corresponding to the training image is determined based on the position data of the target box in the training data matrix. Then, the feature data corresponding to the feature matrix of the training process and the feature matrix of the training result are determined, and feature mixing is performed to obtain the matrix to be classified.

[0168] The third initial neural network model is trained using the training matrix corresponding to each training image until the third training iteration equals the third preset iteration, thus obtaining the target type detection network model.

[0169] Specifically, select one training matrix corresponding to each training image from all the training matrices to be classified.

[0170] The training matrix to be classified is input into the third initial neural network model to obtain the training detection result data matrix;

[0171] The training matrix to be classified is input into the spatial location feature mixing layer in the third initial neural network model to obtain the spatial location feature mixing matrix; the spatial location feature mixing matrix is ​​input into the channel information feature mixing layer in the third initial neural network model to obtain the location channel feature mixing matrix. The location channel feature mixing matrix is ​​then input into the classification and regression layer in the third initial neural network model to obtain the training detection result data matrix.

[0172] Update the third training iteration, which means incrementing the third training iteration by one;

[0173] If the updated third training iteration equals the third preset iteration, the target type detection network model is obtained.

[0174] If the updated third training iterations are less than the third preset iterations, then the third initial neural network model is updated to obtain the trained neural network model; and a training matrix corresponding to a training image is selected from all the training matrices corresponding to each training image and input into the trained neural network model to update the third training iterations, until the updated third training iterations are equal to the third preset iterations, thus obtaining the target type detection network model.

[0175] It should be noted that the third preset number of times can be 500, 1000, or 2000, etc. This application embodiment does not limit the third preset number of times, and it can be set according to the actual situation.

[0176] The target detection method provided in this embodiment trains a first initial neural network model using low-resolution training images to obtain a feature extraction network model. Then, it trains a second initial neural network model using the feature extraction network model to obtain a target extraction network model. Finally, it trains a third initial neural network model using the feature extraction network model and the target extraction network model to obtain a target type detection network model. This sequential training method improves the performance of the feature extraction network model, the target extraction network model, and the target type detection network model, resulting in more accurate target detection results when using these three network models to detect targets in the images.

[0177] The test results provided in this application are described below.

[0178] This application selects ResNet-50 pre-trained on ImageNet as the feature extraction network model. The performance of the object detection method in this application is tested on the UTDAC2020 dataset. SGD was selected as the optimizer, with a learning rate of 0.005, trained for 12 epochs, and the learning rate decayed to 0.1 in epochs 8 and 11. The momentum coefficient was 0.9, and the training batch size was 4. Comparison with other existing object detection methods is shown in Table 1.

[0179] Table 1

[0180]

[0181]

[0182] In Table 1, AP represents the average accuracy, AP50 represents the accuracy with an intersection-union ratio (IU) greater than 50%, AP75 represents the accuracy with an IU greater than 75%, APS represents the accuracy for small targets, APM represents the accuracy for medium targets, and AML represents the accuracy for large targets. It can be seen that the target detection method in this application has a higher detection accuracy. Furthermore, using a multi-scale training paradigm and a soft maximum suppression method during model training can further improve the detection accuracy of the target detection method.

[0183] To verify the overall performance of this application, the VOC 0712 dataset was added for verification, and the results are shown in Table 2:

[0184] Table 2

[0185]

[0186] In Table 2, mAP represents the average accuracy across all classes, showing that the object detection method in this application has a higher detection accuracy. Furthermore, for the VOC 0712 dataset, the detection speed can reach 14.8 FPS, compared to Faster R-CNN's 13.8 FPS, demonstrating that this application achieves improvements in both speed and detection accuracy.

[0187] The following are embodiments of the apparatus described in this application, which can be used to execute the embodiments of the method described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method described in this application.

[0188] Figure 4 This is a schematic diagram of the structure of an embodiment of the target detection device provided in this application. Figure 4 As shown, the target detection device 40 includes:

[0189] Acquisition module 41 is used to acquire the image to be detected;

[0190] Processing module 42 is used for:

[0191] The image to be detected is input into a feature extraction network model to obtain at least one process feature matrix and one result feature matrix; wherein, the feature extraction network model is pre-trained and is used as a computational model to determine the process feature matrix and the result feature matrix based on the image;

[0192] The resulting feature matrix is ​​input into the target extraction network model to obtain at least one target data matrix. Each target data matrix includes the position data of the process target box and the probability of the target being detected corresponding to the process target box. The target existence probability is used to represent the probability that the target being detected exists in the process target box corresponding to the target existence probability. The target extraction network model is pre-trained and is used to determine the calculation model of the target data matrix based on the resulting feature matrix.

[0193] For each target data matrix to be detected, a corresponding classification matrix is ​​generated based on the image to be detected, the at least one process feature matrix, the result feature matrix, and the position data of the process target bounding boxes in the target data matrix to be detected.

[0194] For each matrix to be classified, the matrix to be classified is input into the target type detection network model to obtain the detection result data matrix. The detection result data matrix includes the target type identifier and the position data of the target bounding box. The target type detection network model is a pre-trained computational model used to determine the detection result data matrix based on the matrix to be classified.

[0195] Furthermore, the target type detection network model includes a spatial location feature mixing layer, a channel information feature mixing layer, and a classification and regression layer. The processing module 42 is specifically used for:

[0196] The matrix to be classified is input into the spatial location feature mixing layer to obtain the spatial location feature mixing matrix;

[0197] The spatial location feature mixing matrix is ​​input into the channel information feature mixing layer to obtain the location channel feature mixing matrix;

[0198] The location channel feature mixing matrix is ​​input into the classification and regression layer to obtain the detection result data matrix.

[0199] Furthermore, the processing module 42 is specifically used for:

[0200] Based on the size of the resulting feature matrix, multiple initial target boxes are generated;

[0201] For each initial target box, a binary classification process is performed based on the data corresponding to the initial target box in the result feature matrix to obtain the probability of the existence of the target to be detected;

[0202] For each initial target box, bounding box regression is performed based on the data corresponding to the initial target box in the result feature matrix to obtain the position data of the corrected target box;

[0203] Based on the preset threshold for the probability of the target to be detected and the probability of the target to be detected corresponding to each initial target box, all initial target boxes are filtered to obtain filtered target boxes;

[0204] Based on the position data of the corrected target box corresponding to each filtered target box, nonmaximum suppression processing is performed to obtain the at least one target data matrix to be detected.

[0205] Furthermore, the acquisition module 41 is also used to acquire a preset number of low-resolution training images;

[0206] Furthermore, the processing module 42 is also used to train the first initial neural network model using the preset number of low-resolution training images until the first training iterations equal the first preset iterations, thereby obtaining the feature extraction network model.

[0207] Furthermore, the processing module 42 is also used for:

[0208] Select one training image from the preset number of low-resolution training images;

[0209] The training image is input into the first initial neural network model to obtain at least one training process feature matrix and one training result feature matrix;

[0210] Update the first training iteration;

[0211] If the updated first training iteration is equal to the first preset iteration, the feature extraction network model is obtained;

[0212] If the updated first training iterations are less than the first preset iterations, then the first initial neural network model is updated to obtain the trained neural network model; and a training image is selected again from the preset number of low-resolution training images and input into the trained neural network model to update the first training iterations, until the updated first training iterations are equal to the first preset iterations, thus obtaining the feature extraction network model.

[0213] Furthermore, the processing module 42 is also used for:

[0214] The preset number of low-resolution training images are input into the feature extraction network model to obtain at least one training process feature matrix corresponding to each training image, and a training result feature matrix corresponding to each training image.

[0215] The second initial neural network model is trained using the feature matrix of the training result corresponding to each training image until the second training iteration equals the second preset iteration, thus obtaining the target extraction network model.

[0216] Furthermore, the processing module 42 is also used for:

[0217] Select one training result matrix from the feature matrix of the training result corresponding to each training image;

[0218] The training result matrix is ​​input into the second initial neural network model to obtain at least one training data matrix of the target to be detected.

[0219] Update the second training iteration;

[0220] If the updated second training iterations are equal to the second preset iterations, the target extraction network model is obtained;

[0221] If the updated second training iterations are less than the second preset iterations, then the second initial neural network model is updated to obtain the trained neural network model; and a training result matrix is ​​selected from the training result feature matrix corresponding to each training image and input into the trained neural network model to update the second training iterations until the updated second training iterations are equal to the second preset iterations, thus obtaining the target extraction network model.

[0222] Furthermore, the processing module 42 is also used for:

[0223] The training result feature matrix corresponding to each training image is input into the feature extraction network model to obtain at least one training process feature matrix corresponding to each training image, and a training result feature matrix corresponding to each training image.

[0224] The feature matrix of the training result corresponding to each training image is input into the target extraction network model to obtain at least one target training data matrix corresponding to each training image;

[0225] For each training data matrix of a target to be detected corresponding to a training image, a corresponding training matrix to be classified is generated based on the training image, at least one training process feature matrix corresponding to the training image, and the training result feature matrix corresponding to the training image.

[0226] The third initial neural network model is trained using the training matrix corresponding to each training image until the third training iteration equals the third preset iteration, thus obtaining the target type detection network model.

[0227] Furthermore, the processing module 42 is also used for:

[0228] Select one training matrix corresponding to a training image from all the training matrices corresponding to each training image;

[0229] The training matrix to be classified is input into the third initial neural network model to obtain the training detection result data matrix;

[0230] Update the third training iteration;

[0231] If the updated third training iteration is equal to the third preset iteration, the target type detection network model is obtained;

[0232] If the updated third training iterations are less than the third preset iterations, then the third initial neural network model is updated to obtain the trained neural network model; and a training matrix corresponding to a training image is selected from all the training matrices corresponding to each training image and input into the trained neural network model to update the third training iterations until the updated third training iterations are equal to the third preset iterations, thus obtaining the target type detection network model.

[0233] The target detection device provided in this embodiment is used to execute the technical solution in any of the foregoing method embodiments. Its implementation principle and technical effect are similar, and will not be described again here.

[0234] Figure 5 This is a schematic diagram of the structure of an electronic device provided in this application. Figure 5 As shown, the electronic device 50 includes:

[0235] Processor 51, memory 52, and communication interface 53;

[0236] The memory 52 is used to store the executable instructions of the processor 51;

[0237] The processor 51 is configured to execute the technical solutions in any of the foregoing method embodiments by executing the executable instructions.

[0238] Optionally, the memory 52 can be either standalone or integrated with the processor 51.

[0239] Optionally, when the memory 52 is a device independent of the processor 51, the electronic device 50 may further include:

[0240] Bus 54, memory 52 and communication interface 53 are connected to processor 51 through bus 54 and complete communication with each other. Communication interface 53 is used to communicate with other devices.

[0241] Optionally, the communication interface 53 can be implemented using a transceiver. The communication interface is used to enable communication between the database access device and other devices (e.g., clients, read-write databases, and read-only databases). The memory may include random access memory (RAM) and may also include non-volatile memory, such as at least one disk drive.

[0242] Bus 54 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, only one thick line is used in the diagram, but this does not indicate that there is only one bus or one type of bus.

[0243] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0244] The electronic device is used to execute the technical solutions in any of the foregoing method embodiments. Its implementation principle and technical effect are similar, and will not be described again here.

[0245] This application also provides a readable storage medium storing a computer program thereon, which, when executed by a processor, implements the technical solutions provided in any of the foregoing method embodiments.

[0246] This application also provides a computer program product, including a computer program, which, when executed by a processor, is used to implement the technical solutions provided in any of the foregoing method embodiments.

[0247] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0248] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A target detection method, characterized in that, include: Acquire the image to be detected; The image to be detected is input into a feature extraction network model to obtain at least one process feature matrix and one result feature matrix; wherein, the feature extraction network model is pre-trained and is used as a computational model to determine the process feature matrix and the result feature matrix based on the image; wherein, the image to be detected is scaled to a preset size, and a three-dimensional matrix is ​​generated based on the data of the image to be detected. The three-dimensional matrix is ​​subjected to multi-stage feature extraction processing. After multi-stage processing, feature fusion is performed using a feature pyramid to obtain at least one process feature matrix and one result feature matrix; The resulting feature matrix is ​​input into the target extraction network model to obtain at least one target data matrix. Each target data matrix includes the location data of the process target box and the probability of the target being detected corresponding to the process target box. The target existence probability is used to represent the probability that the target being detected exists in the process target box corresponding to the target existence probability. The target extraction network model is pre-trained and is used to determine the calculation model of the target data matrix based on the resulting feature matrix. The process target box is obtained based on the following steps: the initial target boxes generated based on the resulting feature matrix undergo binary classification processing, bounding box regression processing, and filtering of all initial target boxes according to a preset target existence probability threshold and the target existence probability corresponding to each initial target box to obtain filtered target boxes; based on the target existence probability, the modified target box with the highest target existence probability is selected as the process target box from all the modified target boxes corresponding to the filtered target boxes. For each target data matrix to be detected, a corresponding classification matrix is ​​generated based on the image to be detected, the at least one process feature matrix, the result feature matrix, and the position data of the process target bounding boxes in the target data matrix to be detected. For each matrix to be classified, the matrix to be classified is input into the target type detection network model to obtain the detection result data matrix. The detection result data matrix includes the target type identifier and the position data of the target bounding box. The target type detection network model is a pre-trained computational model used to determine the detection result data matrix based on the matrix to be classified.

2. The method according to claim 1, characterized in that, The target type detection network model includes a spatial location feature mixing layer, a channel information feature mixing layer, and a classification and regression layer. The step of inputting the target classification matrix into the target type detection network model yields a detection result data matrix, including: The matrix to be classified is input into the spatial location feature mixing layer to obtain the spatial location feature mixing matrix; The spatial location feature mixing matrix is ​​input into the channel information feature mixing layer to obtain the location channel feature mixing matrix; The location channel feature mixing matrix is ​​input into the classification and regression layer to obtain the detection result data matrix.

3. The method according to claim 1, characterized in that, The resulting feature matrix is ​​input into the target extraction network model to obtain at least one target data matrix, including: Based on the size of the resulting feature matrix, multiple initial target boxes are generated; For each initial target box, a binary classification process is performed based on the data corresponding to the initial target box in the result feature matrix to obtain the probability of the existence of the target to be detected; For each initial target box, bounding box regression is performed based on the data corresponding to the initial target box in the result feature matrix to obtain the position data of the corrected target box; Based on the preset threshold for the probability of the target to be detected and the probability of the target to be detected corresponding to each initial target box, all initial target boxes are filtered to obtain filtered target boxes; Based on the position data of the corrected target box corresponding to each filtered target box, nonmaximum suppression processing is performed to obtain the at least one target data matrix to be detected.

4. The method according to claim 1, characterized in that, Before acquiring the image to be detected, the method further includes: Obtain a preset number of low-resolution training images; The first initial neural network model is trained using the preset number of low-resolution training images until the first training iteration equals the first preset iteration, thus obtaining the feature extraction network model.

5. The method according to claim 4, characterized in that, The step of training the first initial neural network model using the preset number of low-resolution training images until the first training iteration equals the first preset iteration, to obtain the feature extraction network model, includes: Select one training image from the preset number of low-resolution training images; The training image is input into the first initial neural network model to obtain at least one training process feature matrix and one training result feature matrix; Update the first training iteration; If the updated first training iteration is equal to the first preset iteration, the feature extraction network model is obtained; If the updated first training iterations are less than the first preset iterations, then the first initial neural network model is updated to obtain the trained neural network model; and a training image is selected again from the preset number of low-resolution training images and input into the trained neural network model to update the first training iterations, until the updated first training iterations are equal to the first preset iterations, thus obtaining the feature extraction network model.

6. The method according to claim 4, characterized in that, The method further includes: The preset number of low-resolution training images are input into the feature extraction network model to obtain at least one training process feature matrix corresponding to each training image, and a training result feature matrix corresponding to each training image. The second initial neural network model is trained using the feature matrix of the training result corresponding to each training image until the second training iteration equals the second preset iteration, thus obtaining the target extraction network model.

7. The method according to claim 6, characterized in that, The step of training the second initial neural network model using the feature matrix of the training result corresponding to each training image until the second training iteration equals the second preset iteration, to obtain the target extraction network model, includes: Select one training result matrix from the feature matrix of the training result corresponding to each training image; The training result matrix is ​​input into the second initial neural network model to obtain at least one training data matrix of the target to be detected. Update the second training iteration; If the updated second training iterations are equal to the second preset iterations, the target extraction network model is obtained; If the updated second training iterations are less than the second preset iterations, then the second initial neural network model is updated to obtain the trained neural network model; and a training result matrix is ​​selected from the training result feature matrix corresponding to each training image and input into the trained neural network model to update the second training iterations until the updated second training iterations are equal to the second preset iterations, thus obtaining the target extraction network model.

8. The method according to claim 6, characterized in that, The method further includes: The training result feature matrix corresponding to each training image is input into the feature extraction network model to obtain at least one training process feature matrix corresponding to each training image, and a training result feature matrix corresponding to each training image. The feature matrix of the training result corresponding to each training image is input into the target extraction network model to obtain at least one target training data matrix corresponding to each training image; For each training data matrix of a target to be detected corresponding to a training image, a corresponding training matrix to be classified is generated based on the training image, at least one training process feature matrix corresponding to the training image, and the training result feature matrix corresponding to the training image. The third initial neural network model is trained using the training matrix corresponding to each training image until the third training iteration equals the third preset iteration, thus obtaining the target type detection network model.

9. The method according to claim 8, characterized in that, The third initial neural network model is trained using the training matrix corresponding to each training image until the third training iteration equals the third preset iteration, resulting in the target type detection network model, including: Select one training matrix corresponding to a training image from all the training matrices corresponding to each training image; The training matrix to be classified is input into the third initial neural network model to obtain the training detection result data matrix; Update the third training iteration; If the updated third training iteration is equal to the third preset iteration, the target type detection network model is obtained; If the updated third training iterations are less than the third preset iterations, then the third initial neural network model is updated to obtain the trained neural network model; and a training matrix corresponding to a training image is selected from all the training matrices corresponding to each training image and input into the trained neural network model to update the third training iterations until the updated third training iterations are equal to the third preset iterations, thus obtaining the target type detection network model.

10. A target detection device, characterized in that, include: The acquisition module is used to acquire the image to be detected; Processing module, used for: The image to be detected is input into a feature extraction network model to obtain at least one process feature matrix and one result feature matrix; wherein, the feature extraction network model is pre-trained and is used as a computational model to determine the process feature matrix and the result feature matrix based on the image; wherein, the image to be detected is scaled to a preset size, and a three-dimensional matrix is ​​generated based on the data of the image to be detected. The three-dimensional matrix is ​​subjected to multi-stage feature extraction processing. After multi-stage processing, feature fusion is performed using a feature pyramid to obtain at least one process feature matrix and one result feature matrix; The resulting feature matrix is ​​input into the target extraction network model to obtain at least one target data matrix. Each target data matrix includes the location data of the process target box and the probability of the target being detected corresponding to the process target box. The target existence probability is used to represent the probability that the target being detected exists in the process target box corresponding to the target existence probability. The target extraction network model is pre-trained and is used to determine the calculation model of the target data matrix based on the resulting feature matrix. The process target box is obtained based on the following steps: the initial target boxes generated based on the resulting feature matrix undergo binary classification processing, bounding box regression processing, and filtering of all initial target boxes according to a preset target existence probability threshold and the target existence probability corresponding to each initial target box to obtain filtered target boxes; based on the target existence probability, the modified target box with the highest target existence probability is selected as the process target box from all the modified target boxes corresponding to the filtered target boxes. For each target data matrix to be detected, a corresponding classification matrix is ​​generated based on the image to be detected, the at least one process feature matrix, the result feature matrix, and the position data of the process target bounding boxes in the target data matrix to be detected. For each matrix to be classified, the matrix to be classified is input into the target type detection network model to obtain the detection result data matrix. The detection result data matrix includes the target type identifier and the position data of the target bounding box. The target type detection network model is a pre-trained computational model used to determine the detection result data matrix based on the matrix to be classified.

11. An electronic device, characterized in that, include: Processor, memory, communication interface; The memory is used to store the executable instructions of the processor; The processor is configured to execute the target detection method according to any one of claims 1 to 9 by executing the executable instructions.

12. A readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the target detection method according to any one of claims 1 to 9.

13. A computer program product, characterized in that, It includes a computer program, which, when executed by a processor, is used to implement the target detection method according to any one of claims 1 to 9.

Images