Image target detection method, device, equipment and storage medium

Abstract

The application discloses an image target detection method and device, equipment and a storage medium, relates to the technical field of machine vision, and is used for improving the accuracy of search result presentation and improving content search efficiency. The image target detection process of the method is based on One-stage target detection technology, and in the whole process, the cross-feature scale-based feature enhancement processing makes the feature information exist in coarse granularity and fine granularity, so the feature information is more abundant. The weighted feature enhancement processing process enhances the feature information through the attention mechanism, and performs weighted fusion on the enhanced feature information and the original feature information, thereby improving the expression ability of the feature information. Therefore, after secondary feature enhancement, the final obtained feature information contains more abundant content and has higher expression ability, so that the accuracy of the target detection result obtained based on the enhanced feature information is higher, thereby meeting the accuracy requirements in actual scenes.

Description

Technical Field

[0001] This application relates to the field of machine vision technology, providing an image target detection method, apparatus, device, and storage medium. Background Technology

[0002] Computer vision (CV) object detection technology typically employs a multi-scale feature fusion optimization model based on convolutional neural networks (CNNs). As the demands for object detection and the scenarios for detection become more diverse, the complexity of object detection also increases.

[0003] Currently, object detection technology can be divided into two-stage and one-stage object detection technologies. Two-stage object detection first obtains candidate bounding boxes as samples using relevant algorithms, and then classifies these samples using a CNN. Since this process involves two steps, it is called two-stage object detection. In contrast, one-stage object detection directly regresses the object's class probability and location coordinates to complete the prediction process. Because two-stage object detection requires two processing steps, while one-stage object detection only requires one regression, it is faster. However, despite its higher efficiency, one-stage object detection's accuracy is less than satisfactory and cannot meet the accuracy requirements of real-world scenarios. Summary of the Invention

[0004] This application provides an image target detection method, apparatus, device, and storage medium to improve the accuracy of target detection results.

[0005] On the one hand, an image target detection method is provided, the method comprising:

[0006] Basic features are extracted from the target image to be detected to obtain basic image features corresponding to multiple feature scales.

[0007] For the multiple basic image features obtained, perform the following operations respectively:

[0008] For a given basic image feature, based on the basic image feature, the depth image feature obtained by depth feature extraction is subjected to cross-feature scale feature enhancement processing to obtain the corresponding first enhanced image feature.

[0009] Based on the first enhanced image features, the attention image features obtained by the attention mechanism are subjected to weighted feature enhancement processing to obtain the corresponding second enhanced image features.

[0010] Target prediction is performed based on multiple second enhanced image features to obtain target detection results for the target image. The target detection results represent the number and location of the detected targets in the target image.

[0011] On the one hand, an image target detection device is provided, the device comprising:

[0012] The basic feature extraction unit is used to extract basic features from the target image to be detected, and obtain basic image features corresponding to multiple feature scales.

[0013] The feature enhancement unit is used to perform the following operations on the multiple basic image features obtained through the following sub-units:

[0014] The first enhancement subunit is used to perform cross-feature scale feature enhancement processing on the depth image features obtained by depth feature extraction based on a base image feature, so as to obtain the corresponding first enhanced image feature.

[0015] The second enhancement subunit is used to perform weighted feature enhancement processing on its own attention image features obtained through attention mechanism processing based on the first enhanced image features, so as to obtain the corresponding second enhanced image features.

[0016] The prediction unit is used to predict targets based on multiple second enhanced image features to obtain target detection results of the target image. The target detection results represent the number and location of the detected targets in the target image.

[0017] On one hand, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of any of the above methods.

[0018] On the one hand, a computer storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps of any of the above methods.

[0019] On one hand, a computer program product is provided, comprising a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium, and executes the computer program, causing the computer device to perform the steps of any of the methods described above.

[0020] In this embodiment, after basic feature extraction of the target image, feature enhancement processing is performed across feature scales on the depth image features obtained through depth feature extraction, based on each basic image feature itself. Then, for the obtained first enhanced image features, weighted feature enhancement processing is performed again on the attention image features obtained through attention mechanism processing, based on the first enhanced image features themselves. Finally, target prediction is achieved based on the obtained second enhanced image features. The image target detection process in this embodiment is based on One-stage target detection technology. Throughout the process, the cross-feature scale feature enhancement processing results in feature information with both coarse and fine granularity, thus enriching the feature information. The weighted feature enhancement process enhances feature information through an attention mechanism and weightedly fuses the enhanced feature information with the original feature information, improving the expressive power of the feature information. Therefore, after secondary feature enhancement, the final feature information contains richer content and has higher expressive power, resulting in higher accuracy of the target detection results based on the enhanced feature information, meeting the accuracy requirements of real-world scenarios. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0022] Figure 1 This is a schematic diagram illustrating an application scenario provided in the embodiments of this application;

[0023] Figure 2 A schematic flowchart illustrating the model training method provided in this application embodiment;

[0024] Figure 3 A schematic flowchart of the image target detection method provided in the embodiments of this application;

[0025] Figure 4 This is a schematic diagram of the structure of an image target detection model provided in an embodiment of this application;

[0026] Figure 5 This is a schematic diagram of a possible model structure provided for an embodiment of this application;

[0027] Figure 6 This is a possible structural diagram of the feature enhancement module provided in an embodiment of this application;

[0028] Figure 7A flowchart of the feature extraction process of the deep feature extraction network provided in the embodiments of this application;

[0029] Figure 8 A schematic diagram illustrating the addition of side image features and depth image features provided in an embodiment of this application;

[0030] Figure 9 A schematic diagram illustrating the process of obtaining the second enhanced image feature provided in an embodiment of this application;

[0031] Figure 10 This is a schematic diagram of the structure of an image target detection device provided in an embodiment of this application;

[0032] Figure 11 This is a schematic diagram of the composition structure of a computer device provided in an embodiment of this application;

[0033] Figure 12 This is a schematic diagram of the composition structure of another computer device provided in an embodiment of this application. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application. Unless otherwise specified, the embodiments and features in the embodiments of this application can be arbitrarily combined with each other. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than that shown here.

[0035] To facilitate understanding of the technical solutions provided in the embodiments of this application, some key terms used in the embodiments of this application will be explained below:

[0036] Convolutional Layers: Convolution is the core operation for image feature extraction and quantization. Before further processing and learning of an image, feature extraction and quantization are necessary. A convolutional layer is a model structure that performs convolution operations on an image or includes convolution operations. Its main function is to treat the input image as a two-dimensional linear matrix with multiple channels of information, and then use convolution kernels corresponding to the two-dimensional matrix to perform convolution operations on the matrix of each channel to obtain the feature information of the input image and generate the corresponding feature image.

[0037] Receptive field: In convolutional neural networks (CNNs), the receptive field refers to the size of the input image's pixel region corresponding to the data in each cell of the output feature matrix of a given layer. It can also be called the size of the convolution kernel. In other words, the receptive field describes the mapping relationship between the output unit of a convolutional neural network layer and the pixel region of the input image. For example, a 1x1 convolution kernel represents a receptive field corresponding to the size of one pixel in the image, a 3x3 convolution kernel represents a receptive field corresponding to the size of a 3x3 region of nine pixels, and so on.

[0038] Attention mechanisms are a method of using high-level information to measure intermediate features in a network, causing the network to focus on information that aids in judgment within an image while ignoring irrelevant information. The essence of attention mechanisms comes from human visual attention. When perceiving things, people generally don't look at a scene from beginning to end every time; instead, they observe and pay attention to specific parts based on their needs. Furthermore, when people discover that a scene frequently contains what they want to observe, they learn to focus their attention on that part when similar scenes appear again. Therefore, attention mechanisms are essentially a means of filtering high-value information from a large amount of data. Different pieces of information have different importance to the result, and this importance can be reflected by assigning different weights. In other words, attention mechanisms can be understood as a rule for allocating weights when synthesizing multiple sources. It is often used to solve the problem of obtaining a reasonable final vector representation when the model's input sequence is long. The approach is to retain the model's intermediate results, learn from them with a new model, and associate them with the output, thereby achieving the purpose of information filtering.

[0039] Currently, there are generally three types of attention mechanisms in the field of computer vision: channel attention mechanism, spatial attention mechanism, and hybrid spatial and channel attention mechanism. These three mechanisms target different dimensions of the image, including the length × width dimension, the channel dimension, and the length × width × channel dimension, and learn feature information from the image feature matrix in a targeted manner.

[0040] Backpropagation (BP): The BP algorithm is a learning algorithm suitable for multi-layer neural networks, based on gradient descent. The BP learning process consists of forward propagation and backward propagation. During forward propagation, input information is processed layer by layer through the input layer and hidden layers before being passed to the output layer. If the desired output value is not obtained at the output layer, the sum of the squares of the errors between the output and the desired value is taken as the objective function. Backpropagation then begins, calculating the partial derivatives of the objective function with respect to the weights of each neuron layer. This constitutes the gradient of the objective function with respect to the weight vector, serving as the basis for modifying the weights. The network's learning is completed during this weight modification process.

[0041] The technical concept of the embodiments of this application will be briefly described below.

[0042] Currently, while the detection efficiency of the first-stage target detection technology is relatively high, the final detection accuracy is unsatisfactory and cannot meet the accuracy requirements in real-world scenarios.

[0043] Therefore, considering that in real-world scenarios, the images containing the detected targets often exhibit diverse features such as morphology, texture, edge transitions, size, contour representation, background complexity, and lighting, it is essential to fully and efficiently utilize these feature information in target detection models. Strengthening the feature information extracted by the model is highly effective for improving the model's detection and recognition results. Thus, to improve the accuracy of today's diverse target detection scenarios, we can consider starting from the perspective of strengthening feature information.

[0044] In view of this, embodiments of this application provide an image target detection method. After extracting basic features from a target image, this method performs cross-scale feature enhancement processing on the depth image features obtained through depth feature extraction for each basic image feature. Then, for the obtained first enhanced image features, it further performs weighted feature enhancement processing on the attention image features obtained through attention mechanism processing, based on the first enhanced image features themselves. Finally, target prediction is achieved based on the obtained second enhanced image features. The image target detection process in this embodiment is based on One-stage target detection technology. Throughout the process, the cross-scale feature enhancement processing results in feature information with both coarse and fine granularity, thus enriching the feature information. The weighted feature enhancement process enhances the feature information through an attention mechanism and weightedly fuses the enhanced feature information with the original feature information, improving the expressive power of the feature information. Therefore, after secondary feature enhancement, the final feature information contains richer content and has higher expressive power, resulting in higher accuracy of the target detection results based on the enhanced feature information, meeting the accuracy requirements in real-world scenarios.

[0045] In this application, considering that some feature information becomes coarse-grained specialized semantic information after feature extraction through deep neural networks, and the features may be weakened, this embodiment adds a bypass neural network as a Shortcut branch during feature enhancement processing across feature scales. This bypasses the deep neural network to pass fine-grained feature information. Finally, the fine-grained feature information and the coarse-grained feature information are superimposed, so that the first enhanced image features have both coarse and fine granularity, thus the feature information is richer and the expressive ability is better.

[0046] Furthermore, to enhance the extracted feature information, channel attention and spatial attention mechanisms are incorporated into the weighted feature enhancement process. The enhanced feature information is then fused with the original feature information using a weighted fusion. This weight is the optimal weight that is tuned during training. This allows the second enhanced image feature, resulting from the fusion of the original feature information and the attention-enhanced feature information, to be more effective in the target prediction process, thereby improving the accuracy of the final target prediction result.

[0047] The following is a brief introduction to the application scenarios to which the technical solutions of the embodiments of this application are applicable. It should be noted that the application scenarios described below are only for illustrating the embodiments of this application and are not intended to limit the scope. In specific implementation, the technical solutions provided by the embodiments of this application can be flexibly applied according to actual needs.

[0048] The solution provided in this application can be applied to any scenario involving image target detection, such as detecting unfamiliar customers in banks, detecting abnormal behavior in a specified scene, detecting counterfeit banknotes, detecting defective banknotes, detecting different types of banknotes, and detecting strangers in a specified area. Furthermore, the technical solution of this application is relatively easy to implement and can be applied to lightweight computer devices, such as lightweight camera devices, making it easier to achieve target detection in various scenarios.

[0049] like Figure 1 The diagram shown is an application scenario provided by an embodiment of this application. In this scenario, an image source device 101 and an image target detection device 102 may be included.

[0050] Image source device 101 is a device that provides the target image to be detected. It can be a mobile phone, tablet computer (PAD), laptop computer, desktop computer, smart TV, smart vehicle device, smart wearable device, or other terminal device capable of uploading target images, or it can be an image acquisition device such as a camera, camcorder, or camera.

[0051] The image target detection device 102 is used to perform target detection on the target image provided by the image source device 101 based on the image target detection method provided in the embodiments of this application, and obtain the target detection result. Therefore, the image target detection device 102 is a computer device with certain computing power. This computer device can be a terminal device or a server that meets the computing power requirements. When it is a server, it can be, for example, an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, i.e., Content Delivery Network (CDN), and big data and artificial intelligence (AI) platforms, but it is not limited to these.

[0052] The image target detection device 102 may include one or more processors 1021, a memory 1022, and an interactive I / O interface 1023. Furthermore, the image target detection device 102 may be configured with a database 1024, which can be used to store model parameters of the trained image target detection model. The memory 1022 of the image target detection device 102 may store program instructions for the image target detection method provided in this application embodiment. When these program instructions are executed by the processor 1021, they can be used to implement the training process of the image target detection model or the image target detection process provided in this application embodiment.

[0053] It should be noted that the above application scenarios may also include only one of the image source device 101 and the image target detection device 102, for example, only the image target detection device 102, or the image source device 101 and the image target detection device 102 may be considered as the same device.

[0054] In this embodiment, the image source device 101 and the image target detection device 102 can be directly or indirectly connected via one or more networks 103. The network 103 can be a wired network or a wireless network; for example, the wireless network can be a mobile cellular network or a Wireless-Fidelity (WIFI) network, and of course, it can be other possible networks. This embodiment does not limit the types of networks used. It should be noted that... Figure 1 The examples shown are merely illustrative; in reality, the number of terminal devices and servers is unlimited and is not specifically limited in the embodiments of this application.

[0055] The technical solution of this application embodiment belongs to an end-to-end one-stage target detection method. This method enhances the accuracy of the one-stage target detection method by enhancing features. That is, it further ensures accuracy without significantly increasing computational complexity. It is particularly suitable for some lightweight camera devices. These camera devices have limited computing power, but it is sufficient to ensure the computing power required by the method of this application embodiment, so as to realize the target detection process in some scenarios through these camera devices.

[0056] For example, the solutions of the embodiments of this application will be described below using several specific scenarios.

[0057] In one possible implementation, the solution of this application embodiment can be applied to the detection and identification of abnormal behavior in a bank lobby setting. Specifically, a target detection model for abnormal behavior detection can be pre-trained. Then, in practical applications, images of the target to be detected are captured by camera equipment in the bank lobby setting. The trained target detection model is then used to identify whether abnormal behavior exists in the target image, and when abnormal behavior is found, an alarm is triggered promptly, and the object corresponding to the abnormal behavior is located. For example, abnormal behavior may include theft, harming others, etc., thereby assisting banking institutions in their daily security work.

[0058] In one possible implementation, the solution of this application embodiment can be applied to the detection and identification of abnormal banknotes in banking institutions. Specifically, a target detection model for detecting abnormal banknotes can be pre-trained. The target is abnormal banknotes appearing in the image, such as counterfeit or defective banknotes. In practical applications, the bank's camera equipment can capture images of the target to be detected, for example, by placing them in the corresponding area for tellers to verify banknotes to be deposited into the bank. The trained target detection model can then identify whether abnormal banknotes exist in the target image, and when abnormal banknotes are found, it can promptly issue an alarm and locate which banknote is abnormal. Of course, besides being used for abnormal banknote identification in banking institutions, it can also be used in other banknote-related scenarios, such as in store checkout scenarios.

[0059] In one possible implementation, the solution of this application embodiment can be applied to the detection and identification of different types of banknotes in banking institutions. Specifically, a target detection model for banknote detection can be pre-trained. The target to be detected is the banknotes appearing in the image and the type of banknote they belong to. In actual application, the bank's camera equipment captures the target image to be detected. Then, the trained target detection model can be used to identify whether there are banknotes in the target image and, if there are banknotes, to mark their type.

[0060] In one possible implementation, the solution of this application embodiment can be applied to stranger detection and identification in important areas of banking institutions. Specifically, a target detection model for human body detection and human body category (e.g., whether it is a stranger) can be pre-trained. The target to be detected is the human body appearing in the image and its category. In actual application, the target image to be detected is acquired by camera equipment deployed in important areas for monitoring. Then, the trained target detection model can be used to identify whether there is a human body in the target image and, if so, label its category.

[0061] It should be noted that when the embodiments of this application are applied to specific products or technologies, the user data involved needs to obtain relevant licenses or consents, and the collection, use and processing of the relevant data need to comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0062] The following describes the method provided by exemplary embodiments of this application, in conjunction with the application scenarios and system architecture described above and with reference to the accompanying drawings. It should be noted that the above application scenarios are merely shown to facilitate understanding of the spirit and principles of this application, and the embodiments of this application are not limited in any way. Furthermore, it should be noted that the following method can be executed by the aforementioned terminal device or server, or by both the terminal device and server; specifically, it is shown here as being executed by either the terminal device or the server.

[0063] Since the image target detection method provided in this application is based on the trained image target detection model, the model training process will be introduced first before introducing the actual image target detection process.

[0064] See Figure 2 The diagram shown is a flowchart illustrating the model training method provided in this embodiment. This method can be executed by a computer device, which may be... Figure 1 The image target detection device shown, or a computer device with equivalent computing power, is used. The specific implementation process of this method is as follows:

[0065] Step 201: Obtain multiple image samples, each image sample is associated with the labeled detection results.

[0066] Each image sample can be an image, and it is labeled to obtain its corresponding labeled detection result. The labeled detection result can indicate the number and location of the detected targets contained in the image. In different implementation scenarios, the detection targets can be set according to needs. For example, when it is necessary to perform human body detection, that is, to detect whether the image contains a human body, the detection target is the human body. When it is necessary to detect whether the image contains a specific item, such as a water cup, the detection target can be set to the specified item.

[0067] Step 202: Based on multiple image samples, perform multiple iterations to train the assigned image target detection model until the iteration termination condition is met, and obtain the trained image target detection model.

[0068] It should be noted that the training process for each iteration of the image object detection model is similar. Therefore, the following mainly uses one training process as an example. Each iteration of training may include the following steps:

[0069] Step 2021: Based on the difference between the target prediction results obtained by the image target detection model used in this study and the labeled detection results of the corresponding image samples, obtain the model loss value for this training.

[0070] The image object detection model used in this iteration refers to the model actually used in this iteration of training. In the first iteration of training, the image object detection model used in this iteration is the initial model. In subsequent iterations, the image object detection model used in this iteration is the model updated in the previous iteration. For example, the image object detection model used in the second iteration of training is the image object detection model after the parameter update in the first iteration of training.

[0071] Furthermore, through forward inference of the image target detection model, the sample detection result corresponding to the current input image sample is obtained. The sample detection result is obtained by model prediction. It can be compared with the labeled detection result to measure the accuracy of the current model. Therefore, the model loss value for this training can be obtained based on the difference between the sample detection result obtained by target prediction and the labeled detection result of the corresponding image sample.

[0072] Specifically, the model loss value can be calculated using any possible loss function, such as the cross-entropy loss function, the mean squared error (MSE) loss function, the squared absolute error loss function, the maximum likelihood loss (LHL) function, etc. Of course, other possible loss functions can also be used, and this application embodiment does not limit this.

[0073] Step 2022: Determine whether the model has reached the training termination condition. If yes, the model training ends.

[0074] In this embodiment of the application, the training termination condition may include at least one of the following conditions:

[0075] (1) The number of iterations reaches the set threshold.

[0076] (2) The model loss value is less than the set loss threshold.

[0077] Of course, other possible conditions may also apply, and this application does not limit the specific conditions.

[0078] Step 2023: If the judgment result of step 2022 is negative, then adjust the parameters of the image target detection model used in this step.

[0079] The image target detection model consists of various image processing layers, each containing its own corresponding neuron parameters. When the model fails to converge, it indicates that the accuracy of the current model is not yet up to standard, and the neuron parameters need to be updated further.

[0080] Specifically, the image object detection model includes a preset weight matrix. This preset weight matrix is ​​used to obtain the first enhanced image features of the target image. Then, based on this preset weight matrix, the first enhanced image features and the attention image features are weighted and fused to obtain the second enhanced image features. When updating the parameters of the preset weight matrix, an adaptive weighting (Re-weighting) method can be used. That is, if the loss value of the model trained in the current iteration is lower than the loss value of the model trained in the previous iteration, it indicates that the weights used in the current iteration are better than those used in the previous iteration. In this case, the preset weight matrix adjusted in the previous iteration is retained, which is the weight used in the current iteration, and the preset weight matrix is ​​updated with the weights used in the current iteration. Conversely, if the loss value of the model trained in the current iteration is higher than the loss value of the model trained in the previous iteration, it indicates that the weights used in the current iteration are worse than those used in the previous iteration. In this case, the weights used in the previous iteration are retained, and the preset weight matrix does not need to be updated.

[0081] Adaptive weighting is widely used in data mining, machine learning, and artificial intelligence. Its basic principle is to automatically adjust weights based on the characteristics of the data, thereby improving the accuracy and efficiency of data processing. Specifically, adaptive weighting adjusts weights based on factors such as data distribution, data importance, and data relevance, resulting in more accurate and reliable data processing results. In this embodiment, the role of re-weighting is to continuously update neuron weight information during multiple rounds of model training. Backpropagation is used to determine whether the weights in each iteration are optimal. If the weights in a given round are worse than the previous round, no update is needed; conversely, if the weights are better, they are updated. This adaptive weighting, by adaptively selecting the weights between the original and enhanced feature information, allows for more efficient use of the feature matrix information, resulting in better feature enhancement and improved accuracy of subsequent target detection results. Besides comparing model loss values, other metrics can be used, such as the confidence level of the predicted target. When the confidence level is closer to the true value, the weights are considered better; conversely, a lower confidence level indicates worse weights.

[0082] Of course, in addition to tuning the preset weight matrix, other model parameters will also be tuned. For example, the model weight parameters can be updated using optimization methods such as gradient descent and batch gradient descent, and then training can continue with the updated model weight parameters.

[0083] Upon completion of model training and the acquisition of the trained image object detection model, this embodiment of the application can test the trained image object detection model using test samples to verify its effectiveness. If the evaluation metrics meet the preset evaluation results, the image object detection model can be used in actual image object detection processes. If the model fails the evaluation, training and testing need to continue until the image object detection model achieves the preset evaluation results.

[0084] See Figure 3 The diagram shown is a flowchart illustrating an image target detection method provided in an embodiment of this application. This method can be executed by a computer device, which may be... Figure 1 The image target detection device shown, or a computer device with equivalent computing power, is used in this method, which is based on the image target detection model trained above. After the target image is acquired or input, the following steps can be performed through the image target detection model:

[0085] Step 301: Extract basic features from the target image to be detected to obtain basic image features corresponding to each of the multiple feature scales.

[0086] In practical scenarios, target detection can be performed on each target image acquired or input to determine whether it contains the target object. For example, in a surveillance scenario, target detection can be performed on image frames within the video stream captured by the image acquisition device to determine whether a specified event or a specified target exists within the monitored scene.

[0087] See Figure 4 The diagram shown is a structural schematic of an image target detection model according to an embodiment of this application. The image target detection model comprises three main parts: a feature extraction module, a feature fusion module, and a feature prediction module. The basic feature extraction process can be implemented by the feature extraction module.

[0088] Specifically, the feature extraction module is responsible for extracting features from the input target image. This can be achieved using an extraction model based on a convolutional neural network. The extracted features can be diverse in dimensions such as image shape and texture, edge transition, size, contour representation, background complexity, and light and dark.

[0089] In practical applications, the output of the feature extraction module can typically be base image features at various feature scales. The feature scale usually refers to the size of the two-dimensional image, which can be represented by the size of the pixel matrix, for example, 20. 20 or 40 40, etc. However, in practical applications, certain feature scales of basic image features can be selected as input for the next module; see [reference needed]. Figure 4 As shown, 20 can be selected. 20 1024, 40 40 512 and 80 80 256 can be used as the input for the next module, but the specific selection can be made according to the needs of the actual scenario or experimental data. This application embodiment does not limit this.

[0090] In one possible implementation, see Figure 5 The diagram shown is a possible model structure provided in an embodiment of this application. However, it should be noted that... Figure 5 This is merely an illustration of one possible implementation and is not intended to limit the scope of the embodiments described in this application.

[0091] See Figure 5 As shown, the feature extraction module consists of data processing layers such as the focus layer, convolutional layer, and Fast Spatial Pyramid Pooling Mudule (SPPF) layer.

[0092] The focus layer is used to split the pixels in width and height and reassemble them into the channel dimension, thereby halving the feature scale without losing feature information. It consists of a feature reorganization layer and a standard convolutional module. See [link to relevant documentation]. Figure 5 As shown, the input target image is 640. 640 3. The size of the feature obtained after processing by the focus layer is 320. 320 64. Its function is to improve the model's ability to detect small targets. This is because in target detection tasks, small targets are small in size and have inconspicuous features, making them difficult to detect. The focus layer adds a convolutional layer before the input layer to downsample the input feature map, and then another convolutional layer upsamples the feature map, thereby making the model pay more attention to the features of small targets and improving the detection effect.

[0093] A convolutional layer is typically a standard convolutional module, named Conv. It consists of a two-dimensional convolutional layer (Conv2D), a batch normalization layer (BatchNorm2D), and an activation layer, all concatenated together. The main function of the convolutional layer is feature extraction. See [link to documentation]. Figure 5 As shown, it contains four convolutional layers, which progressively increase the depth of feature extraction, and output 160 respectively. 160 128, 80 80 256, 40 40 512 and 20 20 Image features of size 1024.

[0094] The SPPF layer consists of three pooling layers and two standard convolutional modules. Its function is to achieve the fusion of local and global features at the feature map level. In practical scenarios, in addition to the SPPF layer, other similar functional layers can also be used, such as the spatial pyramid pooling module (SPP) layer.

[0095] See Figure 5As shown, the feature extraction process involves passing the input red-green-blue (RGB) three-channel image through a series of convolutional layers. Each convolutional operation doubles the number of image channels, and the matrix of each channel represents the image's feature information. In this embodiment, considering constraints such as computational speed and the number of model parameters, the number of image channels is extracted to 1024, at which point the image feature dimensions are 20 pixels wide and 20 pixels high. 20, that is, 20 The matrix tensor is 20, where each value represents the overall information of a partial region in the original input image. Of course, in real-world scenarios, the feature extraction depth can be increased or decreased; this embodiment does not impose such limitations. In the final matrix tensor, since each value represents the overall information, the deeper the number of channels, the greater the amount of information represented by the feature data, which is called coarse-grained information or deep specialized semantic information. Conversely, the shallower the number of channels, such as an initial RGB image with dimensions of 640... 640, a single pixel is a matrix value, which can only represent shallow information at the pixel level. The amount of information is very low and cannot represent the range features of an image, such as shape, outline, texture, color, etc. Such information is called fine-grained information.

[0096] After feature extraction, the 1024-channel image contains the most information, representing coarse-grained information; the 512-channel image contains the next most information, but includes some fine-grained information; and the 256-channel image contains even richer fine-grained information. Typically, image information from these three scales can be fused for prediction. Of course, this embodiment does not limit the specific selection; the number of channels can be reduced, or features from other channels can be combined for prediction.

[0097] Step 302: Perform feature enhancement operations on the multiple basic image features obtained to obtain their respective enhanced image features.

[0098] In this embodiment, feature enhancement can be performed during feature fusion. Therefore, the feature enhancement process can be executed through a feature fusion module; that is, a feature enhancement module can be added to the feature fusion module to implement the feature enhancement process. Placing the feature enhancement module in the three-scale fusion part can enhance the amount of information fused between each scale, enabling the feature enhancement module to optimize the effective information of the feature image after feature fusion at different scales. Therefore, placing the module after the neck network feature fusion forms a feature fusion optimization method based on the feature enhancement algorithm module. This improves the expressive power of features at a single feature scale.

[0099] See Figure 6The diagram shown illustrates a possible structure of the feature enhancement module provided in this embodiment. The feature enhancement module may include a side-branch neural network, a deep feature extraction network, an attention mechanism module, and a preset weight matrix. Since the enhancement process for each basic image feature is similar, this section uses a single basic image feature as an example, which may include the following steps:

[0100] Step 3021: For a basic image feature, perform cross-feature scale feature enhancement processing on the depth image feature obtained by depth feature extraction based on the basic image feature to obtain the corresponding first enhanced image feature.

[0101] In this embodiment, the feature enhancement module includes a backbone network and branch networks. The backbone network is mainly used for further deep feature extraction. The image features obtained by further extraction are deeper, but the feature granularity is coarser, which may miss detailed information. Therefore, the branch network is used to make up for this deficiency. Through a certain fast branch network, fine-grained information is passed over and combined with the coarse-grained information obtained by the backbone network before continuing the subsequent process.

[0102] The feature enhancement module is divided into two stages. The first stage of enhancement produces the first enhanced image feature mentioned above, and the second stage of enhancement is based on the first enhanced image feature to produce the subsequent second enhanced image feature.

[0103] In one possible implementation, see Figure 6 As shown, in the first stage of reinforcement, the branch network, also known as the side branch neural network or ShortCut shortcut branch, is used to transmit fine-grained information. One way is to directly transmit the basic image features, and another way is to process them through certain data processing methods without changing their feature scale. That is, the basic image features can be bypassed and short-circuited through the side branch neural network to obtain the corresponding side branch image features.

[0104] Specifically, the side-branch neural network may include a first convolutional unit based on channel dimension transformation. This first convolutional unit can then perform channel dimension transformation on the base image features to obtain side-branch image features. The side-branch image features have the same feature scale as the base image features, and the number of channel dimensions in the side-branch image features is greater than that in the base image features; that is, the feature scale remains unchanged, but the number of channels increases.

[0105] For example, see Figure 6 As shown, a 1×1 convolutional structure can be used for data processing, so that only the number of channels of the feature is changed, without changing the feature scale.

[0106] The backbone network can be a deep feature extraction network designed based on Conv convolutional modules. This deep feature extraction network can then be used to extract deep features from the basic image features to obtain the corresponding deep image features. The deep image features and the side image features have the same number of feature channels but different feature scales.

[0107] In one possible implementation, the deep feature extraction network includes a second convolutional unit based on feature scale transformation, a batch normalization unit, and an activation unit. That is, the network adds a batch normalization layer and an activation layer after convolution to form a typical standard convolutional (Conv-BatchNorm-ReLU, CBR) module, thereby enabling the neural network to converge faster and strengthening gradient information, avoiding the gradient vanishing problem.

[0108] For details, see Figure 7 The diagram shown is a flowchart of the feature extraction process of the deep feature extraction network provided in this application embodiment. Based on the above network structure, when extracting depth image features through the deep feature extraction network, the feature scale transformation processing of the basic image features is first performed through the second convolutional unit to obtain convolutional image features. The feature scale of the obtained convolutional image features is lower than that of the basic image features. Then, the convolutional image features are used as input to the batch normalization unit. The batch normalization processing of the convolutional image features is performed through the batch normalization unit to obtain batch normalized image features. The batch normalized image features are also used as input to the activation unit to perform feature activation processing on the batch normalized image features, and finally, the depth image features are obtained.

[0109] For example, such as Figure 7 As shown, the feature depth of the deep image features obtained through the deep feature extraction network is further increased, and the feature channels are doubled. For example, the dimension of the basic image features is "80". 80 When the depth image feature dimension is 256", the resulting depth image feature dimension is 40. 40 512", the dimension of the basic image features is "40". 40 When the depth image feature dimension is 512", the resulting dimension is 20. 20 1024".

[0110] Finally, after obtaining the side image features and depth image features, cross-feature scale feature enhancement processing can be performed on the depth image features based on the side image features to obtain the first enhanced image feature. The meaning of cross-feature scale feature enhancement processing is that the side image features transmitted from the side neural network contain fine-grained feature information, while the depth image features after passing through the depth feature extraction network are coarse-grained feature information. The feature scales of the two are different, and the first enhanced image feature obtained by combining them has both coarse and fine granularity, thus the features are richer.

[0111] In one possible implementation, the feature enhancement processing of depth image features based on side image features across feature scales can be achieved by measuring and adding the side image features and depth image features to obtain the first enhanced image feature.

[0112] In one possible implementation, since the feature scales of the side image features and the depth image features are different, it is inconvenient to directly superimpose them. Therefore, in this embodiment, the depth image features can be interpolated to obtain interpolated image features with the same feature scale as the side image features. Then, the side image features and the interpolated image features are superimposed based on feature channels to obtain the first enhanced image features.

[0113] For example, see Figure 8 The diagram illustrates the addition of side image features and depth image features. Each value in the feature represents a pixel or image region. Interpolation of the depth image features ensures that the interpolated image features have the same feature scale as the side image features. Figure 8 As shown, zeros are interpolated at missing values. Of course, other interpolation methods can be used in practical applications, and this embodiment does not limit this. Furthermore, the side image features and the interpolated image features are superimposed on the channels, increasing the number of feature channels. For example, adding 256 channels of side image features and 256 channels of interpolated image features yields a 512-channel first enhanced image feature. However, it should be noted that... Figure 8 This is only for illustrating the data processing procedure and does not limit the actual characteristics.

[0114] It should be noted that the above example only uses a 1×1 convolutional structure for the side image network and a CBR module for the deep feature extraction network. However, in real-world scenarios, other structures can also be used, such as using multiple 1×1 convolutional structures as the side image network or using other deep feature extraction networks. This application does not impose any restrictions on these.

[0115] For example, see Figure 6As shown, the design uses the CBR module as the backbone and is a unidirectional two-dimensional structure based on the Bypass approach. The main feature of this design is the addition of a ShortCut branch for weight shortening, providing discriminative information for backpropagation parameter updates. Simultaneously, to control the dimensionality of the shortcut during feature transmission, a 1×1 convolutional structure is added to the shortcut for dimensionality reduction. This will enhance parameter optimization during multiple backpropagation iterations in the model's training process. In simpler terms, consider c1 feature matrices with dimensions w×h. The features are already optimal in this iteration. After processing by the CBR module, some feature information becomes coarse-grained specialized semantic information, meaning the features may be weakened. Therefore, in this embodiment, a ShortCut branch is added for weight shortening, bypassing the CBR module to transmit the optimal feature information from this iteration. However, the CBR module causes changes in the number of channels, hence the addition of a 1×1 convolutional structure for dimensionality reduction. Finally, before subsequent processing, the fine-grained feature matrix passed from the 1×1 convolutional structure and the coarse-grained feature matrix tensor processed by the CBR module are added together. In this way, the feature information has both coarse and fine granularity, so the features are richer, thereby realizing the fusion of cross-structure feature information and extracting richer feature information.

[0116] Step 3022: Based on the first enhanced image features, perform weighted feature enhancement processing on the attention image features obtained by the attention mechanism to obtain the corresponding second enhanced image features.

[0117] In this embodiment, the design of the second-stage feature enhancement process is mainly based on the attention mechanism to enhance feature information. The first-stage enhancement has already mixed coarse-grained and fine-grained information. To further enhance the extracted feature information, a processing mechanism based on channel attention and spatial attention is added in the subsequent processing. The enhanced feature information is then fused with the original feature information based on a preset weight matrix. The preset weight matrix is ​​obtained through a re-weight adaptive weight adjustment method. Therefore, the re-weight adaptive weight fusion allows for continuous updating of neuron weight information during multiple rounds of model training. Backpropagation is used to determine whether the weights in the current iteration are optimal. If the weights in the current iteration are not as good as those in the previous iteration, no weight update is needed; otherwise, if the weights in the current iteration are better than those in the previous iteration, they are updated. This achieves adaptive weight updating, adaptively selecting the weights between the original feature information and the enhanced feature information, and can more fully utilize the feature matrix information.

[0118] For details, see Figure 9The diagram illustrates the process of obtaining the second enhanced image feature. It describes how the first enhanced image feature is fused using a channel attention mechanism to obtain a first attention sub-feature, and how the first enhanced image feature is fused using a spatial attention mechanism to obtain a second attention sub-feature. Finally, based on the first and second attention image features, an attention image feature is obtained. The fusion of the first and second attention image features can be achieved through methods such as concatenation, pooling, or positional addition.

[0119] Finally, the obtained attention image features and the first enhanced image features are weighted and fused based on their respective preset weight matrices to obtain the second enhanced image features. It should be noted that, although... Figure 9 Both are represented by a preset weight matrix, but the values ​​in the preset weight matrices of the attention image features and the first enhanced image features can be different.

[0120] Step 303: Based on the obtained multiple second enhanced image features, perform target prediction to obtain the target detection result of the target image. The target detection result represents the number and position of the detected target in the target image.

[0121] In this embodiment of the application, after obtaining multiple second enhanced image features, the subsequent target prediction process can be carried out based on the second enhanced image features. Since the second enhanced image features contain richer feature information, they can provide more feature information in the target prediction process, which can effectively improve the accuracy of the target prediction process.

[0122] In practical applications, the target detection results can be represented by prediction boxes. The target detection results can include prediction boxes and their detection scores, which are used to characterize whether the detected target exists in the target image and the confidence level of each detected target.

[0123] In one possible implementation, after introducing a feature enhancement module into the feature fusion module, cross-scale fusion will also be performed on image features at different feature scales.

[0124] Specifically, at least one image feature pair can be constructed based on the obtained multiple second enhanced image features. Each image feature pair contains at least one second enhanced image feature. Then, for each image feature pair, cross-feature scale feature fusion processing is performed on the two image features included to obtain fused image features. Finally, target prediction is performed based on the obtained multiple fused image features to obtain the sub-detection results corresponding to each of the multiple fused image features. Based on the obtained sub-detection results, the target detection result is obtained.

[0125] In practical applications, image features from two adjacent feature scales can be used to construct an image feature pair. For example, a second enhanced image feature from two adjacent feature scales can be used to construct an image feature pair, or a second enhanced image feature from one feature scale and a base image feature from the other feature scale can be used to construct an image feature pair.

[0126] For example, see Figure 5 As shown, a possible implementation of the feature fusion process is illustrated. After feature extraction by the feature extraction module, 256-channel, 512-channel, and 1024-channel basic image features are selected to continue the subsequent prediction process. The feature fusion method adopts the inverted pyramid feature fusion method.

[0127] Specifically, for the 1024-channel basic image features, after processing through a convolutional layer, the number of channels is changed to 512. After upsampling, it is concatenated with the 512-channel basic image features to obtain an image feature with 1024 channels. This feature is then processed through a C3x3 layer, a convolutional layer, and an upsampling layer, and then concatenated with the 256-channel basic image features. The resulting 256-channel image feature is then input into a 256-channel feature prediction module to obtain the corresponding sub-detection results.

[0128] On the other hand, the obtained 256-channel image features are input into the feature enhancement module of this application embodiment for feature enhancement, resulting in a second enhanced image feature of 256 channels. After concatenating the second enhanced image feature with the 256-channel image feature output by the convolutional layer, the fused image feature output by the C3x3 layer has 512 channels, which are input into the 512-channel feature prediction module to obtain the corresponding sub-detection result.

[0129] Furthermore, the fused image features output by the C3x3 layer are input into the feature enhancement module of this application embodiment for feature enhancement, resulting in a second enhanced image feature with 512 channels. After concatenating the second enhanced image feature with the 512-channel image feature output by the convolutional layer, the number of channels of the fused image feature output by the C3x3 layer is 1024. This fused image feature is then input into the 1024-channel feature prediction module to obtain the corresponding sub-detection result.

[0130] The sub-detection results obtained above are detection results obtained from different feature scales. Different feature scales result in different receptive fields, and different receptive fields can perceive different targets. Therefore, combining the sub-detection results obtained above and performing target detection from different receptive fields can help improve the accuracy of the final target detection results and reduce the possibility of missed detections.

[0131] In summary, in this embodiment, the fine-grained feature matrix passed from the ShortCut branch and the coarse-grained feature matrix tensor processed by the standard convolution module are added before the standard convolution module and the attention mechanism. This results in a richer feature set, containing both coarse and fine-grained information. Simultaneously, the 1×1 convolution structure ensures that the number of fine-grained and coarse-grained feature matrices is consistent, and the subsequent attention mechanism further enhances the richness of both coarse and fine-grained feature information, improving the expressive power of the feature information and ultimately increasing the accuracy of the target detection results.

[0132] After multiple rounds of data verification, the mean precision (mAP), precision, and recall were used as indicators for comparison. The experimental results are shown in Table 1 below. To compare the performance of the proposed solution, comparative experiments were conducted with YOLOv5s (where 's' represents a lightweight small model, suitable for most object detection and model embedding), YOLOv5m (where 'm' represents a medium-scale parameter model, requiring a certain amount of computing power), and ESB-YOLOv5s. YOLOv5s-SCConv-bN is the solution proposed in this application. As can be seen, compared with other lightweight small models, the proposed solution performs no worse than other solutions in all indicators, and shows significant improvements in accuracy and recognition precision, outperforming other solutions overall. Furthermore, even compared with medium-scale parameter models, although it cannot achieve the accuracy effect brought by the large number of parameters of medium-scale parameter models, it is still no worse than medium-scale parameter models in terms of recognition precision and recognition recall. Moreover, compared with medium-scale parameter models, the model proposed in this application is more lightweight and has lower computing power requirements. Therefore, the proposed solution has a wider range of applications and higher practicality.

[0133]

[0134] Table 1

[0135] Please see Figure 10 Based on the same inventive concept, embodiments of this application also provide an image target detection device 100, which includes:

[0136] The basic feature extraction unit 1001 is used to extract basic features from the target image to be detected, and obtain basic image features corresponding to multiple feature scales.

[0137] The feature enhancement unit 1002 is used to perform the following operations on the obtained basic image features through the following sub-units:

[0138] The first enhancement subunit 10021 is used to perform cross-feature scale feature enhancement processing on the depth image features obtained by the depth feature extraction based on a basic image feature, so as to obtain the corresponding first enhanced image feature.

[0139] The second enhancement subunit 10022 is used to perform weighted feature enhancement processing on the attention image features obtained by itself through the attention mechanism based on the first enhanced image features, so as to obtain the corresponding second enhanced image features.

[0140] The prediction unit 1003 is used to predict targets based on multiple second enhanced image features to obtain target detection results of the target image. The target detection results represent the number and location of the detected targets in the target image.

[0141] In one possible implementation, the first reinforcing subunit 10021 is specifically used for:

[0142] By using a side-branch neural network, the basic image features are bypassed and short-circuited to obtain the corresponding side-branch image features.

[0143] A deep feature extraction network is used to extract deep features from the basic image features to obtain the corresponding deep image features. The deep image features and the side image features have the same number of feature channels but different feature scales.

[0144] Based on the side image features, feature enhancement processing is performed on the depth image features across feature scales to obtain the first enhanced image features.

[0145] In one possible implementation, the side-branch neural network includes a first convolutional unit based on channel dimension transformation; then the first reinforcement subunit 10021 is specifically used for:

[0146] By performing channel dimension transformation on the basic image features through the first convolutional unit, side image features are obtained.

[0147] Among them, the feature scale of the side image features is the same as that of the basic image features, and the number of channel dimensions of the side image features is greater than that of the basic image features.

[0148] In one possible implementation, the deep feature extraction network includes a second convolutional unit based on feature scale transformation, a batch normalization unit, and an activation unit; then the first enhancement subunit 10021 is specifically used for:

[0149] The second convolutional unit performs feature scale transformation on the base image features to obtain convolutional image features; wherein the feature scale of the convolutional image features is lower than that of the base image features.

[0150] Batch normalization is performed on the convolutional image features by batch normalization unit to obtain batch normalized image features;

[0151] By using activation units, feature activation processing is performed on batch normalized image features to obtain depth image features.

[0152] In one possible implementation, the first reinforcing subunit 10021 is specifically used for:

[0153] Interpolate the depth image features to obtain interpolated image features with the same feature scale as the side image features;

[0154] The side image features and the interpolated image features are superimposed based on feature channels to obtain the first enhanced image features.

[0155] In one possible implementation, the second reinforcing subunit 10022 is specifically used for:

[0156] The first enhanced image features are subjected to feature fusion processing based on a channel attention mechanism to obtain the first attention sub-features;

[0157] The first enhanced image features are subjected to feature fusion processing based on spatial attention mechanism to obtain the second attention sub-features;

[0158] Attention image features are obtained based on the first attention image features and the second attention image features;

[0159] Based on the preset weight matrices of the first enhanced image features and the attention image features respectively, the first enhanced image features and the attention image features are weighted and fused to obtain the second enhanced image features.

[0160] In one possible implementation, the image target detection device performs the above-described image target detection process using a trained image target detection model, which includes a preset weight matrix. The device then further includes a training unit 1004, used for:

[0161] The image object detection model is trained iteratively multiple times until the iteration termination condition is met, resulting in a trained image object detection model. Each iteration includes the following steps:

[0162] The difference between the target detection results obtained by predicting targets in image samples using the image target detection model used in this study and the labeled detection results of the corresponding image samples is used to obtain the model loss value trained in this study.

[0163] If the model loss value in this training iteration is lower than the model loss value in the previous iteration, update the preset weight matrix with the weight values ​​used in this iteration.

[0164] If the model loss value in this training iteration is higher than the model loss value in the previous iteration, the weight values ​​used in the previous iteration will be retained.

[0165] In one possible implementation, the prediction unit 1003 is specifically used for:

[0166] Based on the obtained multiple second-enhanced image features, at least one image feature pair is constructed, and each image feature pair contains at least one second-enhanced image feature;

[0167] For each image feature pair, cross-feature scale feature fusion processing is performed on the image features included in it to obtain fused image features;

[0168] Target prediction is performed based on the obtained multiple fused image features, and sub-detection results corresponding to each of the multiple fused image features are obtained. Based on the obtained sub-detection results, the target detection result is obtained.

[0169] The aforementioned device allows for the addition of the fine-grained feature matrix passed from the ShortCut branch and the coarse-grained feature matrix tensor processed by the standard convolutional module, before the standard convolutional module and attention mechanism. This results in a richer feature set, encompassing both coarse and fine-grained aspects. Furthermore, the 1×1 convolutional structure ensures that the number of fine-grained and coarse-grained feature matrices is consistent, and the subsequent attention mechanism further enhances the richness of both coarse and fine-grained feature information, improving the expressive power of feature information and ultimately increasing the accuracy of target detection results.

[0170] This device can be used to execute the methods shown in the various embodiments of this application. Therefore, the functions that each functional module of this device can achieve can be referred to the description of the foregoing embodiments, and will not be repeated here.

[0171] Please see Figure 11 Based on the same technical concept, embodiments of this application also provide a computer device. In one embodiment, the computer device may be, for example, a server. Figure 11 As shown, it includes a memory 1101, a communication module 1103, and one or more processors 1102.

[0172] The memory 1101 is used to store computer programs executed by the processor 1102. The memory 1101 may mainly include a program storage area and a data storage area. The program storage area may store the operating system and programs required to run the functions of the embodiments of this application; the data storage area may store various functional information and operation instruction sets.

[0173] Memory 1101 may be volatile memory, such as random-access memory (RAM); memory 1101 may also be non-volatile memory, such as read-only memory, flash memory, hard disk drive (HDD), or solid-state drive (SSD); or memory 1101 may be any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto. Memory 1101 may be a combination of the above-described memories.

[0174] The processor 1102 may include one or more central processing units (CPUs) or digital processing units, etc. The processor 1102 is used to implement the above-described image target detection method when it calls the computer program stored in the memory 1101.

[0175] The communication module 1103 is used to communicate with terminal devices and other servers.

[0176] This application embodiment does not limit the specific connection medium between the memory 1101, communication module 1103, and processor 1102. This application embodiment... Figure 11 The memory 1101 and the processor 1102 are connected via a bus 1104, and the bus 1104 is in Figure 11 The diagram uses thick lines to describe the connections between other components; these are for illustrative purposes only and should not be considered limiting. Bus 1104 can be divided into address bus, data bus, control bus, etc. For ease of description, Figure 11 It is described using only a thick line, but does not indicate that there is only one bus or one type of bus.

[0177] The memory 1101 stores a computer storage medium, which stores computer-executable instructions. The computer-executable instructions are used to implement the image target detection method of the embodiments of this application, and the processor 1102 is used to execute the image target detection methods of the above embodiments.

[0178] In another embodiment, the computer device may also be a terminal device. In this embodiment, the structure of the computer device may be as follows: Figure 12 As shown, it includes components such as: communication component 1210, memory 1220, display unit 1230, camera 1240, sensor 1250, audio circuit 1260, Bluetooth module 1270, processor 1280, etc.

[0179] The communication component 1210 is used to communicate with the server. In some embodiments, it may include a Circuit-Wireless Fidelity (WiFi) module. WiFi is a short-range wireless transmission technology, and computer devices can use WiFi modules to help users send and receive information.

[0180] The memory 1220 can be used to store software programs and data. The processor 1280 executes various functions of the terminal device and data processing by running the software programs or data stored in the memory 1220. The memory 1220 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. The memory 1220 stores an operating system that enables the terminal device to run. In this application, the memory 1220 can store the operating system and various application programs, and may also store code that executes the image target detection method of the embodiments of this application.

[0181] The display unit 1230 can also be used to display information input by the user or information provided to the user, as well as various menus of the terminal device, in a graphical user interface (GUI). Specifically, the display unit 1230 may include a display screen 1232 disposed on the front of the terminal device. The display screen 1232 may be configured as a liquid crystal display, a light-emitting diode, or the like. The display unit 1230 can be used to display target detection results, etc., in the embodiments of this application.

[0182] The display unit 1230 can also be used to receive input digital or character information and generate signal inputs related to user settings and function control of the terminal device. Specifically, the display unit 1230 may include a touch screen 1231 disposed on the front of the terminal device, which can collect touch operations of the user on or near it, such as clicking a button, dragging a scroll box, etc.

[0183] The touchscreen 1231 can be placed on top of the display screen 1232, or the touchscreen 1231 and the display screen 1232 can be integrated to realize the input and output functions of the terminal device. After integration, it can be referred to as a touch display screen. In this application, the display unit 1230 can display the application program and the corresponding operation steps.

[0184] Camera 1240 can be used to capture still images, which users can then post comments on via an application. There can be one or multiple cameras 1240. An object is projected onto a photosensitive element through a lens, generating an optical image. This photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then transmitted to the processor 1280 for conversion into a digital image signal.

[0185] The terminal device may also include at least one sensor 1250, such as an accelerometer 1251, a proximity sensor 1252, a fingerprint sensor 1253, and a temperature sensor 1254. The terminal device may also be equipped with other sensors such as a gyroscope, barometer, hygrometer, thermometer, infrared sensor, light sensor, and motion sensor.

[0186] Audio circuitry 1260, speaker 1261, and microphone 1262 provide an audio interface between the user and the terminal device. Audio circuitry 1260 converts received audio data into electrical signals, which are then transmitted to speaker 1261, where they are converted into sound signals for output. The terminal device can also be equipped with volume buttons for adjusting the volume of the sound signal. Conversely, microphone 1262 converts collected sound signals into electrical signals, which are then received by audio circuitry 1260, converted back into audio data, and output to communication component 1210 for transmission to, for example, another terminal device, or to memory 1220 for further processing.

[0187] The Bluetooth module 1270 is used to interact with other Bluetooth devices that also have a Bluetooth module via the Bluetooth protocol. For example, a terminal device can establish a Bluetooth connection with a wearable computer device (such as a smartwatch) that also has a Bluetooth module through the Bluetooth module 1270, thereby exchanging data.

[0188] The processor 1280 is the control center of the terminal device, connecting various parts of the terminal through various interfaces and lines. It executes various functions and processes data by running or executing software programs stored in the memory 1220 and calling data stored in the memory 1220. In some embodiments, the processor 1280 may include one or more processing units; the processor 1280 may also integrate an application processor and a baseband processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the baseband processor mainly handles wireless communication. It is understood that the baseband processor may not be integrated into the processor 1280. In this application, the processor 1280 can run the operating system, applications, user interface display and touch response, and the image target detection method of this embodiment. Furthermore, the processor 1280 is coupled to the display unit 1230.

[0189] Based on the same inventive concept, embodiments of this application also provide a computer storage medium storing a computer program that, when run on a computer device, causes the computer device to perform the steps of the image target detection method according to the various exemplary embodiments of this application described above.

[0190] In some possible implementations, various aspects of the image target detection method provided in this application can also be implemented in the form of a computer program product, which includes a computer program that, when run on a computer device, causes the computer device to perform the steps in the image target detection method according to the various exemplary embodiments of this application described above. For example, the computer device can perform the steps of the various embodiments.

[0191] The computer program product may take the form of any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0192] The computer program product of the embodiments of this application may employ a portable compact disc read-only memory (CD-ROM) and include a computer program, and may run on a computer device. However, the computer program product of this application is not limited thereto. In this application, the readable storage medium may be any tangible medium that contains or stores a program, and the computer program included therein may be used by or in conjunction with a command execution system, apparatus, or device.

[0193] A readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying a readable computer program. This propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting a program for use by or in conjunction with a command execution system, apparatus, or device.

[0194] Computer programs contained on readable media may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0195] Computer programs for performing the operations of this application can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages.

[0196] It should be noted that although several units or sub-units of the device have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of this application, the features and functions of two or more units described above can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided and embodied by multiple units.

[0197] Furthermore, although the operations of the method of this application are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all the operations shown must be performed to achieve the desired result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0198] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer programs.

[0199] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0200] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. An image target detection method, characterized in that, The method includes: Basic features are extracted from the target image to be detected to obtain basic image features corresponding to multiple feature scales. When performing basic feature extraction on the target image to be detected, the image features output by the i-th feature extraction layer are input into the (i+1)-th feature extraction layer. One feature extraction layer is used to extract basic image features for one feature scale. For the multiple basic image features obtained, perform the following operations respectively: For a basic image feature, a side-branch neural network is used to bypass and short-circuit the basic image feature to obtain the corresponding side-branch image feature. A deep feature extraction network is used to perform deep feature extraction processing on the basic image features to obtain corresponding deep image features; wherein the number of feature channels of the deep image features and the side image features are the same, but the feature scales are different. Based on the side image features, feature enhancement processing is performed on the depth image features across feature scales to obtain a first enhanced image feature, including: interpolating the depth image features to obtain interpolated image features with the same feature scale as the side image features; and performing feature channel-based superposition processing on the side image features and the interpolated image features to obtain the first enhanced image feature. Based on the first enhanced image features, the attention image features obtained by the attention mechanism are subjected to weighted feature enhancement processing to obtain the corresponding second enhanced image features. Target prediction is performed based on multiple second enhanced image features to obtain target detection results for the target image. The target detection results represent the number and location of the detected targets in the target image.

2. The method as described in claim 1, characterized in that, The side-branch neural network includes a first convolutional unit based on channel dimension transformation; The process of bypassing and short-circuiting the basic image features through a side-branch neural network to obtain the corresponding side-branch image features includes: The basic image features are transformed by channel dimension transformation through the first convolution unit to obtain the side image features; Wherein, the side image features have the same feature scale as the basic image features, and the number of channel dimensions of the side image features is greater than the number of channel dimensions of the basic image features.

3. The method as described in claim 1, characterized in that, The deep feature extraction network includes a second convolutional unit based on feature scale transformation, a batch normalization unit, and an activation unit; Then, a deep feature extraction network is used to perform deep feature extraction processing on the basic image features to obtain the corresponding deep image features, including: The second convolutional unit performs feature scale transformation on the base image features to obtain convolutional image features; wherein the feature scale of the convolutional image features is lower than that of the base image features. The batch normalization unit performs batch normalization processing on the convolutional image features to obtain batch normalized image features. The activation unit performs feature activation processing on the batch normalized image features to obtain the depth image features.

4. The method according to any one of claims 1 to 3, characterized in that, Based on the first enhanced image features, the attention image features obtained through the attention mechanism are subjected to weighted feature enhancement processing to obtain corresponding second enhanced image features, including: The first enhanced image features are subjected to feature fusion processing based on a channel attention mechanism to obtain the first attention sub-features; The first enhanced image features are subjected to feature fusion processing based on a spatial attention mechanism to obtain the second attention sub-features; The attention image features are obtained based on the first attention image features and the second attention image features; Based on the preset weight matrices of the first enhanced image feature and the attention image feature respectively, the first enhanced image feature and the attention image feature are subjected to weighted fusion processing to obtain the second enhanced image feature.

5. The method as described in claim 4, characterized in that, The image target detection method is executed through a trained image target detection model, which includes the preset weight matrix. The method further includes: The image object detection model is trained iteratively multiple times until the iteration termination condition is met, thereby obtaining the trained image object detection model; wherein each iteration training includes the following steps: The difference between the target detection results obtained by predicting targets in image samples using the image target detection model used in this study and the labeled detection results of the corresponding image samples is used to obtain the model loss value trained in this study. If the model loss value in this training iteration is lower than the model loss value in the previous iteration, the preset weight matrix is ​​updated with the weight values ​​used in this iteration. If the model loss value in this training iteration is higher than the model loss value in the previous iteration, the weight values ​​used in the previous iteration are retained.

6. The method according to any one of claims 1 to 3, characterized in that, The step of predicting the target based on multiple obtained second enhanced image features to obtain the target detection result of the target image includes: Based on the obtained multiple second-enhanced image features, at least one image feature pair is constructed, and each image feature pair contains at least one second-enhanced image feature; For each image feature pair, cross-feature scale feature fusion processing is performed on the image features included in it to obtain fused image features; Target prediction is performed based on the obtained multiple fused image features, and sub-detection results corresponding to each of the multiple fused image features are obtained. Based on the obtained sub-detection results, the target detection result is obtained.

7. An image target detection device, characterized in that, The device includes: The basic feature extraction unit is used to extract basic features from the target image to be detected, and obtain basic image features corresponding to multiple feature scales. When extracting basic features from the target image to be detected, the image features output by the i-th feature extraction layer are input into the (i+1)-th feature extraction layer. One feature extraction layer is used to extract basic image features for one feature scale. The feature enhancement unit is used to perform the following operations on the multiple basic image features obtained through the following sub-units: The first enhancement subunit is configured to, for a given base image feature, perform bypass shorting processing on the base image feature using a side-branch neural network to obtain a corresponding side-branch image feature; and perform depth feature extraction processing on the base image feature using a deep feature extraction network to obtain a corresponding depth image feature; wherein the depth image feature and the side-branch image feature have the same number of feature channels but different feature scales; and perform cross-feature scale feature enhancement processing on the depth image feature based on the side-branch image feature to obtain a first enhanced image feature, including: performing interpolation processing on the depth image feature to obtain an interpolated image feature with the same feature scale as the side-branch image feature; and performing feature channel-based superposition processing on the side-branch image feature and the interpolated image feature to obtain the first enhanced image feature; The second enhancement subunit is used to perform weighted feature enhancement processing on its own attention image features obtained through attention mechanism processing based on the first enhanced image features, so as to obtain the corresponding second enhanced image features. The prediction unit is used to predict targets based on multiple second enhanced image features to obtain target detection results of the target image. The target detection results represent the number and location of the detected targets in the target image.

8. The apparatus as claimed in claim 7, characterized in that, The second strengthening subunit is specifically used for: The first enhanced image features are subjected to feature fusion processing based on a channel attention mechanism to obtain the first attention sub-features; The first enhanced image features are subjected to feature fusion processing based on a spatial attention mechanism to obtain the second attention sub-features; The attention image features are obtained based on the first attention image features and the second attention image features; Based on the preset weight matrices of the first enhanced image feature and the attention image feature respectively, the first enhanced image feature and the attention image feature are subjected to weighted fusion processing to obtain the second enhanced image feature.

9. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program performs the steps of the method according to any one of claims 1 to 6.

11. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program performs the steps of the method according to any one of claims 1 to 6.

Images