Target detection methods, devices, electronic equipment, storage media and products

Abstract

This disclosure relates to a target detection method, apparatus, electronic device, storage medium, and product. The target detection method includes: acquiring an image to be detected; and acquiring a first target detection result of the image to be detected based on a target detection model, wherein the target detection model includes a first network, a second network, and a third network. The first network is used to extract features from the image to be detected, the second network is used to fuse the features extracted by the first network, and the third network is used to generate an original detection result of the image to be detected based on the fused features obtained by the second network. The first target detection result is obtained by post-processing the original detection result. The second network includes n feature fusion structures, and different feature fusion structures include different numbers of feature extraction modules, where n is a positive integer greater than 2. This disclosure improves the accuracy of target detection.

Description

Technical Field

[0001] This disclosure relates to the field of computer vision, and more particularly to a target detection method, apparatus, electronic device, storage medium, and product. Background Technology

[0002] Deep learning has been widely used in the field of computer vision, and convolutional neural networks are one of the important research directions in this field.

[0003] In related technologies, convolutional neural networks (CNNs) have been widely used for object detection. For example, they can be used to identify application pop-ups. However, CNN models in these technologies are mainly used to identify targets with obvious features. For targets with indistinct features, they suffer from low detection accuracy. For instance, when the pop-up scenario is complex and the features are not obvious, these technologies cannot effectively detect application pop-ups. Summary of the Invention

[0004] To overcome the problems existing in related technologies, this disclosure provides a target detection method, apparatus, electronic device, storage medium, and product.

[0005] According to a first aspect of the present disclosure, a target detection method is provided, comprising: acquiring an image to be detected; acquiring a first target detection result of the image to be detected based on a target detection model, the target detection model comprising a first network, a second network, and a third network, wherein the first network is used to extract features from the image to be detected, the second network is used to fuse the features extracted by the first network, and the third network is used to generate an original detection result of the image to be detected based on the fused features obtained by the second network, and the first target detection result is obtained by post-processing the original detection result; wherein the second network comprises n feature fusion structures, and different feature fusion structures include different numbers of feature extraction modules, and n is a positive integer greater than 2.

[0006] In one embodiment, the second network fuses the features extracted by the first network in the following manner: It acquires the features extracted by the first network; based on the i-th feature fusion structure, it extracts features from the (i-1)-th fused feature to obtain the i-th extracted feature, and fuses the i-th extracted feature to obtain the i-th fused feature; it inputs the i-th fused feature into the (i+1)-th feature fusion structure for feature extraction and feature fusion to obtain the (i+1)-th fused feature, where i is a positive integer greater than or equal to 1 and less than n, and the 0th fused feature is the feature extracted by the first network; it repeats the above feature extraction and feature fusion steps until the n-th fused feature is obtained, and uses the n-th fused feature as the fused feature obtained by the second network.

[0007] In one implementation, the number of feature extraction processes in the i-th feature fusion structure is less than the number of feature extraction processes in the (i+1)-th feature fusion structure.

[0008] In one embodiment, the step of fusing the i-th extracted feature to obtain the i-th fused feature includes: performing convolution processing on the i-th extracted feature to obtain the i-th first feature, and performing convolution processing on the features input to the feature fusion structure to obtain the i-th second feature, wherein the i-th first feature and the i-th second feature have the same number of channels; combining the local features and global dependencies of the i-th first feature based on a dual attention network to obtain the i-th third feature; concatenating the i-th second feature and the i-th third feature to obtain the i-th concatenated feature, and processing the i-th concatenated feature to obtain the i-th fused feature.

[0009] In one embodiment, processing the i-th spliced ​​feature to obtain the i-th fused feature includes: performing cross-iterative batch normalization on the i-th spliced ​​feature to obtain the i-th normalized feature; and obtaining the i-th fused feature based on the i-th normalized feature.

[0010] In one embodiment, the first network extracts features of the image to be detected in the following manner: scaling the image to be detected to a preset size to obtain an image of a preset size; sequentially performing feature extraction, feature scaling, and feature combination on the image of the preset size to obtain the features of the image to be detected.

[0011] In one implementation, the first target detection result is obtained by adjusting and selecting the original detection result.

[0012] In one embodiment, the method further includes: acquiring a second target detection result of the image to be detected, the second target detection result being obtained by processing the image to be detected based on a mask region convolutional neural network; performing weighted fusion on the first target detection result and the second target detection result to determine the probability of the presence of a target in the image to be detected; determining that a target exists in the image to be detected in response to the probability satisfying a probability condition; and determining that no target exists in the image to be detected in response to the probability not satisfying a probability condition.

[0013] In one embodiment, the step of sequentially performing feature extraction, feature scaling, and feature combination on the image of the preset size to obtain the features of the image to be detected includes: expanding the channels of the image of the preset size based on a first point convolutional layer to obtain features including a first preset number of channels; extracting features from each channel of the features of the first preset number of channels based on a depth convolutional layer to obtain depth convolutional features; combining the depth convolutional features based on a second point convolutional layer to obtain features of a second preset number of channels, and using the features of the second preset number of channels as the features of the image to be detected.

[0014] In one embodiment, after feature extraction, the method further includes: performing m iterations of attention calculation based on the features obtained from the feature extraction to obtain a first target index of the feature in each iteration of attention calculation, wherein the first target index is used to identify a preset number of positional features with the highest attention scores in the feature, and m is a positive integer greater than 1; obtaining the first target index obtained in the j-th iteration of attention calculation, wherein j is a positive integer greater than or equal to 1 and less than or equal to m; and, in response to j being 1, using the average of the first target index obtained in the j-th iteration of attention calculation and the first target index obtained in the (j+1)-th iteration of attention calculation as the j-th iteration of attention... The calculated second target index; in response to j being greater than 1 and less than m, the average of the first target index calculated based on the (j-1)th iteration attention, the first target index calculated based on the j-th iteration attention, and the first target index calculated based on the (j+1)th iteration attention is used as the second target index calculated based on the j-th iteration attention; in response to j being m, the average of the first target index calculated based on the j-th iteration attention and the first target index calculated based on the (j-1)th iteration attention is used as the second target index calculated based on the j-th iteration attention; based on the second target index, local context enhancement is performed on the features obtained from feature extraction.

[0015] According to a second aspect of the present disclosure, a target detection apparatus is provided, comprising: an acquisition unit for acquiring an image to be detected; and a processing unit for acquiring a first target detection result of the image to be detected based on a target detection model, wherein the target detection model includes a first network, a second network, and a third network, the first network for extracting features from the image to be detected, the second network for fusing the features extracted by the first network, and the third network for generating an original detection result of the image to be detected based on the fused features obtained by the second network, wherein the first target detection result is obtained by post-processing the original detection result; wherein the second network includes n feature fusion structures, and different feature fusion structures include different numbers of feature extraction modules, and n is a positive integer greater than 2.

[0016] In one embodiment, the second network fuses the features extracted by the first network in the following manner: It acquires the features extracted by the first network; based on the i-th feature fusion structure, it extracts features from the (i-1)-th fused feature to obtain the i-th extracted feature, and fuses the i-th extracted feature to obtain the i-th fused feature; it inputs the i-th fused feature into the (i+1)-th feature fusion structure for feature extraction and feature fusion to obtain the (i+1)-th fused feature, where i is a positive integer greater than or equal to 1 and less than n, and the 0th fused feature is the feature extracted by the first network; it repeats the above feature extraction and feature fusion steps until the n-th fused feature is obtained, and uses the n-th fused feature as the fused feature obtained by the second network.

[0017] In one implementation, the number of feature extraction processes in the i-th feature fusion structure is less than the number of feature extraction processes in the (i+1)-th feature fusion structure.

[0018] In one embodiment, the step of fusing the i-th extracted feature to obtain the i-th fused feature includes: performing feature extraction and feature fusion, which includes: performing convolution processing on the i-th extracted feature to obtain the i-th first feature, and performing convolution processing on the feature input to the feature fusion structure to obtain the i-th second feature, wherein the i-th first feature and the i-th second feature have the same number of channels; combining the local features and global dependencies of the i-th first feature based on a dual attention network to obtain the i-th third feature; concatenating the i-th second feature and the i-th third feature to obtain the i-th concatenated feature, and processing the i-th concatenated feature to obtain the i-th fused feature.

[0019] In one embodiment, processing the i-th spliced ​​feature to obtain the i-th fused feature includes: performing cross-iterative batch normalization on the i-th spliced ​​feature to obtain the i-th normalized feature; and obtaining the i-th fused feature based on the i-th normalized feature.

[0020] In one embodiment, the first network extracts features of the image to be detected in the following manner: scaling the image to be detected to a preset size to obtain an image of a preset size; sequentially performing feature extraction, feature scaling, and feature combination on the image of the preset size to obtain the features of the image to be detected.

[0021] In one implementation, the first target detection result is obtained by adjusting and selecting the original detection result.

[0022] In one embodiment, the processing unit is further configured to: acquire a second target detection result of the image to be detected, the second target detection result being obtained by processing the image to be detected based on a mask region convolutional neural network; perform weighted fusion of the first target detection result and the second target detection result to determine the probability of the existence of a target in the image to be detected; determine that a target exists in the image to be detected in response to the probability satisfying a probability condition; and determine that no target exists in the image to be detected in response to the probability not satisfying a probability condition.

[0023] In one embodiment, the step of sequentially performing feature extraction, feature scaling, and feature combination on the image of the preset size to obtain the features of the image to be detected includes: expanding the channels of the image of the preset size based on a first point convolutional layer to obtain features including a first preset number of channels; extracting features from each channel of the features of the first preset number of channels based on a depth convolutional layer to obtain depth convolutional features; combining the depth convolutional features based on a second point convolutional layer to obtain features of a second preset number of channels, and using the features of the second preset number of channels as the features of the image to be detected.

[0024] In one embodiment, after feature extraction, the processing unit is further configured to: perform m iterations of attention calculation based on the features obtained by feature extraction, to obtain a first target index of the feature in each iteration of attention calculation, wherein the first target index is used to identify a preset number of positional features with the highest attention scores in the feature, and m is a positive integer greater than 1; obtain the first target index obtained in the j-th iteration of attention calculation, wherein j is a positive integer greater than or equal to 1 and less than or equal to m; and, in response to j being 1, use the average of the first target index obtained in the j-th iteration of attention calculation and the first target index obtained in the (j+1)-th iteration of attention calculation as the j-th iteration of attention calculation. The second target index is calculated by force; in response to j being greater than 1 and less than m, the average of the first target index calculated by the (j-1)th iteration attention calculation, the first target index calculated by the j-th iteration attention calculation, and the first target index calculated by the (j+1)th iteration attention calculation is used as the second target index calculated by the j-th iteration attention calculation; in response to j being m, the average of the first target index calculated by the j-th iteration attention calculation and the first target index calculated by the (j-1)th iteration attention calculation is used as the second target index calculated by the j-th iteration attention calculation; based on the second target index, local context enhancement is performed on the features extracted by the feature extraction.

[0025] According to a third aspect of the present disclosure, a target detection apparatus is provided, comprising:

[0026] A processor; a memory for storing processor-executable instructions; wherein the processor is configured to execute the target detection method described in the first aspect or any embodiment of the first aspect.

[0027] According to a fourth aspect of the present disclosure, a storage medium is provided, the storage medium storing instructions that, when executed by a processor of a terminal, can perform the method described in the first aspect or any embodiment of the first aspect.

[0028] According to a fifth aspect of the present disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the method described in the first aspect or any embodiment of the first aspect.

[0029] The technical solution provided by the embodiments of this disclosure can include the following beneficial effects: A target detection model is used to detect an image to be detected, obtaining a first target detection result. The target detection model includes a first network, a second network, and a third network. The second network includes more than two feature fusion structures, and each feature fusion structure includes one or more feature extraction modules for feature extraction. Furthermore, different feature fusion structures include different numbers of feature extraction modules. By fusing the features obtained by the first network according to depth levels through multiple feature fusion structures including different feature extraction modules, the model can detect targets in the image to be detected at different depth levels, improving the accuracy of the first target detection result obtained based on the second network and increasing the success rate of target detection.

[0030] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0031] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0032] Figure 1 This is a schematic diagram illustrating an automated testing scenario according to an exemplary embodiment.

[0033] Figure 2 This is a schematic diagram illustrating an advertising pop-up window according to an exemplary embodiment.

[0034] Figure 3 This is a flowchart of a target detection method according to an exemplary embodiment. Figure 1 .

[0035] Figure 4 This is a flowchart of a target detection method according to an exemplary embodiment. Figure 2 .

[0036] Figure 5 This is a flowchart illustrating a method for obtaining the i-th fusion feature according to an exemplary embodiment.

[0037] Figure 6 This is a flowchart illustrating a normalization process according to an exemplary embodiment.

[0038] Figure 7 This is a schematic diagram illustrating a third feature fusion structure according to an exemplary embodiment.

[0039] Figure 8 This is a schematic diagram illustrating a fourth feature fusion structure according to an exemplary embodiment.

[0040] Figure 9 This is a flowchart illustrating a feature extraction method for an image to be detected according to an exemplary embodiment. Figure 1 .

[0041] Figure 10 This is a flowchart illustrating a feature extraction method for an image to be detected according to an exemplary embodiment. Figure 2 .

[0042] Figure 11 This is a flowchart of a target detection method according to an exemplary embodiment. Figure 3 .

[0043] Figure 12 This is a schematic diagram illustrating target detection of an application pop-up window according to an exemplary embodiment.

[0044] Figure 13 This is a flowchart of a target detection method according to an exemplary embodiment. Figure 4 .

[0045] Figure 14 This is a block diagram illustrating a target detection device according to an exemplary embodiment.

[0046] Figure 15 This is a block diagram illustrating an electronic device for target detection according to an exemplary embodiment.

[0047] Figure 16 This is a block diagram illustrating an apparatus for target detection according to an exemplary embodiment. Detailed Implementation

[0048] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure.

[0049] In the accompanying drawings, the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The described embodiments are some, but not all, of the embodiments of this disclosure. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this disclosure, and should not be construed as limiting this disclosure. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without inventive effort are within the scope of protection of this disclosure. The embodiments of this disclosure will be described in detail below with reference to the accompanying drawings.

[0050] The target detection method provided in this disclosure is mainly applied to image recognition scenarios. For example, in scenarios where pop-up information in the display area of ​​an electronic device is identified.

[0051] In one exemplary embodiment, the target detection method of the present disclosure can be applied to scenarios involving automated testing of applications in electronic devices, such as... Figure 1 As shown, Figure 1 This is a schematic diagram illustrating an automated testing scenario according to an exemplary embodiment. Figure 1 In this setup, the test host is connected to the electronic device (also known as the device under test). The test host can control the device under test to run test cases from the test case database; these test cases can be applications. When the device under test is running test cases, there is a possibility that pop-up advertisements may appear during the execution of the test cases. For example... Figure 1 As shown, there is a 65% probability that the pop-up window will not appear during the execution of test cases on the device under test (DUT), and this will not affect the test host's ability to test the DUT. However, there is a 35% probability that a pop-up window will appear during the execution of test cases on the DUT, causing the test to terminate. In this case, manual maintenance is required to ensure that the test cases on the DUT run normally. For scenarios where the test host controls the DUT to run a large number of test cases, manually performing target detection on the pop-up window consumes a significant amount of labor and time costs.

[0052] In related technologies, image capture of electronic devices is mainly based on Robotic Process Automation (RPA) screenshot components, and target detection is performed on the images using detection algorithms. However, the detection algorithms in these technologies primarily target and recognize characters in images, but their performance in detecting objects other than characters is poor. Furthermore, the target detection models in these technologies suffer from limited feature extraction and detection capabilities, resulting in low accuracy in detecting targets with indistinct features and lacking fixed feature information. Examples of targets with indistinct features and lacking fixed feature information include, for instance, advertising pop-ups, etc. Figure 2 As shown, Figure 2 This is a schematic diagram illustrating an advertising pop-up window according to an exemplary embodiment. The size, position, and content of advertising pop-ups can vary, and related technologies cannot accurately identify advertising pop-ups.

[0053] In view of this, embodiments of this disclosure provide a target detection method, which uses a target detection model including a first network for feature extraction, a second network for feature fusion, and a third network for generating detection results based on the features to detect targets in an image to be detected, thereby obtaining a first target detection result. The second network includes more than two feature fusion structures, and the number of feature extraction modules included in different feature fusion structures varies.

[0054] Figure 3 This is a flowchart of a target detection method according to an exemplary embodiment. Figure 1 ,like Figure 3 As shown, it includes the following steps.

[0055] In step S11, the image to be detected is acquired.

[0056] In this embodiment of the disclosure, the image to be detected can be a single image, a group of images including multiple images, or a video including multiple images. For example, in the scenario of detecting pop-up ads in an application, the image to be detected can be a screenshot of the electronic device running the application, or a screen recording video of the electronic device running the application.

[0057] In step S12, the first target detection result of the image to be detected is obtained based on the target detection model.

[0058] In this embodiment, the target detection model includes a first network, a second network, and a third network. The first network extracts features from the image to be detected. The first network can be a pre-trained convolutional neural network, and the features obtained by the first network can be a feature matrix or a feature map. The second network fuses the features extracted by the first network. The features extracted by the first network can have different scales. The second network fuses the features extracted by the first network according to depth levels, enabling the model to detect targets in the image to be detected at different depth levels. After integrating or fusing the features extracted by the first network, the second network obtains fused features and passes these fused features to the third network. The third network generates the original detection result of the image to be detected based on the fused features obtained by the second network. However, the original detection result may contain content that interferes with the target detection result. Post-processing can be performed on the original detection result to obtain the first target detection result, making the first target detection result more accurate and reducing the possibility of missed detections.

[0059] It should be understood that if the image to be detected is a video, an object detection model can be used to detect each frame of the video, and the first object detection result of the video can be determined based on the object detection result of each frame of the video.

[0060] According to embodiments of this disclosure, in application pop-up detection scenarios, a target detection model can be used to detect screenshots of application operation, extract features from the screenshots, and perform fusion processing on the extracted screenshot features based on a feature fusion structure in a second network that includes multiple feature extraction modules of varying numbers, to obtain features at different depth levels. Based on these features, it can be determined whether a pop-up exists in the screenshot, thereby enabling the detection of different pop-ups included in different applications and improving the success rate of application pop-up detection.

[0061] In this embodiment, the second network includes n feature fusion structures, with different feature fusion structures containing different numbers of feature extraction modules, where n is a positive integer greater than 2. The feature extraction module is used to extract features. The feature fusion structure mainly segments the features, processes some of the segmented features through a residual network (ResNet-like blocks), and directly passes the remaining features. The results of the residual processing and the directly passed parts are merged, avoiding residual processing on all features, reducing the computational load of the model, and achieving feature extraction and fusion.

[0062] It should be understood that the feature fusion structure can also include a combination of network structures such as Feature Pyramid Networks (FPN) and Path Aggregation Networks (PAN) to achieve effective fusion of multi-scale features.

[0063] In one exemplary embodiment, the feature fusion structure may be, for example, a Cross Stage Partial Network (CSPNet) structure, and the feature extraction module may be, for example, a convolutional layer (Conv) + batch normalization layer (BN) + Leaky ReLU activation function (Leaky ReLU) module, which may also be called a CBL module.

[0064] In this embodiment of the disclosure, by setting multiple feature fusion structures including different numbers of feature extraction modules, the extraction and detection of features based on different depth levels in the image to be detected are realized, thereby improving the accuracy of target detection.

[0065] In this embodiment of the disclosure, the second network can use a feature fusion structure to extract and fuse features from the features obtained by the first network, and based on the obtained fused features, use other feature fusion structures to extract and fuse features from the obtained fused features, until all feature fusion structures have completed feature extraction and feature fusion, resulting in features including multiple levels.

[0066] Figure 4This is a flowchart of a target detection method according to an exemplary embodiment. Figure 2 ,like Figure 4 As shown, it includes the following steps.

[0067] In step S21, the features extracted from the first network are obtained.

[0068] In step S22, based on the i-th feature fusion structure, feature extraction is performed on the (i-1)-th fusion feature to obtain the i-th extracted feature, and feature fusion is performed on the i-th extracted feature to obtain the i-th fusion feature. The i-th fusion feature is then input into the (i+1)-th feature fusion structure for feature extraction and feature fusion to obtain the (i+1)-th fusion feature.

[0069] In this embodiment, i is a positive integer greater than or equal to 1 and less than n. The 0th fusion feature is the feature extracted by the first network. The feature fusion structure performs feature extraction and feature fusion based on the features extracted by the first network or the fusion features obtained by the previous feature fusion structure. Features belonging to the same depth level are extracted and fused to obtain the fusion feature of that depth level. Features at different depth levels can also be called features at different levels.

[0070] In step S23, the above feature extraction and feature fusion steps are repeated until the nth fused feature is obtained, and the nth fused feature is used as the fused feature obtained by the second network.

[0071] In this embodiment of the disclosure, by using n fusion feature structures, the fusion features of the features extracted by the first network at n depth levels are obtained, thereby realizing the extraction of multi-scale fusion features corresponding to the features extracted by the first network.

[0072] In this embodiment, the number of feature extraction modules included in the i-th feature fusion structure can be set to be less than the number of feature extraction modules included in the (i+1)-th feature fusion structure. The number of feature extraction modules is directly proportional to or positively correlated with the number of feature extraction operations. This can also be understood as the number of feature extraction operations in the i-th feature fusion structure being less than the number of feature extraction operations in the (i+1)-th feature fusion structure. By progressively extracting features from the first network from low-depth to high-depth levels, the object detection model can adaptively handle features of varying complexity, mitigating the gradient vanishing problem.

[0073] In this embodiment of the disclosure, the feature extraction module can capture the global feature dependencies in the features after feature extraction by performing a nonlinear transformation on the i-th extracted feature and by using a dual attention mechanism, and enhance the extracted features, which helps to segment the target content when performing target detection.

[0074] Figure 5 This is a flowchart illustrating a method for obtaining the i-th fusion feature according to an exemplary embodiment, such as... Figure 5 As shown, it includes the following steps.

[0075] In step S31, the i-th extracted feature is convolved to obtain the i-th first feature, and the feature input to the feature fusion structure is convolved to obtain the i-th second feature.

[0076] In this embodiment, the feature input to the feature fusion structure can be the (i-1)th fused feature or a preprocessed feature of the (i-1)th fused feature. The i-th first feature and the i-th second feature have the same number of channels. By using the same convolutional layer, convolution processing is performed on the features before and after feature extraction, so that the number of channels and feature dimensions (height and width) of the i-th first feature and the i-th second feature obtained after convolution processing are consistent, enabling feature fusion operations.

[0077] In step S32, based on the dual attention network, the local features and global dependencies of the i-th first feature are combined to obtain the i-th third feature.

[0078] In this embodiment, the Dual Attention Network (DANet) primarily determines the contextual information between any two positions (also referred to as elements) of a feature based on the sum of position attention and channel attention introduced into the feature. It then determines the global dependency of the i-th first feature and combines local features and global dependencies within the feature to enhance the expression of the i-th first feature. Local features refer to small regions within the feature, while global dependencies represent the relationships between different regions within the feature.

[0079] In step S33, the i-th second feature and the i-th third feature are concatenated to obtain the i-th concatenated feature, and the i-th concatenated feature is processed to obtain the i-th fused feature.

[0080] In this embodiment of the disclosure, a concat layer can be used to concatenate the i-th second feature and the i-th third feature based on the channel dimension to obtain the i-th concatenated feature. The feature dimensions of the i-th concatenated feature, the i-th second feature, and the i-th third feature are the same, and the number of channels of the i-th concatenated feature is the sum of the number of channels of the i-th second feature and the i-th third feature.

[0081] In this embodiment of the disclosure, before feature fusion, the local features and global dependencies of the extracted features to be spliced ​​can be combined using a dual attention mechanism to enhance the i-th first feature and improve the representation of the i-th first feature.

[0082] It should be understood that in the embodiments of this disclosure, each feature fusion structure performs the same feature fusion operation on the extracted features. The process of performing feature fusion on the i-th extracted feature is the same as the process of inputting the i-th fused feature into the i+1-th feature fusion structure for feature extraction and feature fusion, and will not be described again here.

[0083] In this embodiment, after concatenating the i-th second feature and the i-th third feature, batch normalization (BN) can be applied to the resulting i-th concatenated feature to improve the stability of the object detection model. Furthermore, cross-iteration batch normalization (CBN) can be used to further improve the performance degradation when normalizing small batches of data.

[0084] Figure 6 This is a flowchart illustrating a normalization processing method according to an exemplary embodiment, such as... Figure 6 As shown, it includes the following steps.

[0085] In step S41, the i-th spliced ​​feature is subjected to cross-iterative batch normalization to obtain the i-th normalized feature.

[0086] In this embodiment, the cross-iterative batch normalization mainly uses global statistics (which may include the mean and variance) determined by the object detection model during the training phase to normalize the concatenated features. The global statistics are determined by the mean and variance obtained from multiple iterations of normalization during the training process of the object detection model. That is, the global statistics can characterize the statistical information from multiple iterations.

[0087] In step S42, the i-th fused feature is obtained based on the i-th normalized feature.

[0088] In this embodiment, after obtaining the i-th normalized feature, an activation function and feature extraction operation can be used to process the i-th normalized feature to obtain the i-th fused feature after feature fusion. The activation function can be, for example, the Leaky ReLU activation function, which maps negative elements in the input features to reduce their size, thus addressing the neuron death problem present in activation functions such as ReLU and improving the gradient vanishing problem. Furthermore, a feature fusion structure can be used to extract and fuse features obtained from the activation function, thereby improving the model's performance in processing data with complex structures.

[0089] In one exemplary embodiment, the number of feature fusion structures in the second network can be set to four, including a first feature fusion structure (CSP1_X), a second feature fusion structure (CSP2_X), a third feature fusion structure (CSP3_X), and a fourth feature fusion structure (CSP4_X). Each feature fusion structure includes a feature extraction module, a convolutional layer, a dual attention network, a concatenation layer, a cross-iterative batch normalization module, and a Leaky ReLU activation function. Specifically, CSP1_X includes two feature extraction modules, CSP2_X includes three feature extraction modules, CSP3_X includes four feature extraction modules, and CSP4_X includes five feature extraction modules. Schematic diagrams of the third and fourth feature fusion structures are shown below. Figure 7 and Figure 8 As shown. Figure 7 This is a schematic diagram illustrating a third feature fusion structure according to an exemplary embodiment. Figure 7 In this process, the features output by the second feature fusion structure (also known as the second fusion feature) are obtained. Three feature extraction modules are used to extract features from the output of the second feature fusion structure three times, resulting in the third extracted feature. Convolutional layers and a dual attention mechanism are used to process the features extracted three times. A convolutional layer is used to perform convolution processing on the features output by the second feature fusion structure. A concatenation layer is used to concatenate the features output by the dual attention mechanism (also known as the third second feature) and the features output by the convolutionally processed second feature fusion structure (also known as the third first feature), resulting in the concatenated feature (also known as the third concatenated feature). A cross-iterative batch normalization module is used to perform cross-iterative batch normalization on the concatenated feature, resulting in the normalized feature (also known as the third normalized feature). The Leaky ReLU activation function and a feature extraction module are then used to process the normalized feature, resulting in the features output by the third feature fusion structure (also known as the third fusion feature).

[0090] Figure 8 This is a schematic diagram illustrating a fourth feature fusion structure according to an exemplary embodiment. Figure 8 In this process, the features output by the third feature fusion structure are obtained. Four feature extraction modules are used to extract features from the output of the third feature fusion structure four times, resulting in the fourth extracted feature (also known as the fourth extracted feature). Convolutional layers and a dual attention mechanism are used to process the features extracted four times. Convolutional layers are used to process the features output by the third feature fusion structure. A concatenation layer is used to concatenate the features output by the dual attention mechanism (also known as the fourth second feature) and the features output by the convolutionally processed third feature fusion structure (also known as the fourth first feature), resulting in the concatenated feature (also known as the fourth concatenated feature). A cross-iterative batch normalization module is used to normalize the concatenated feature, resulting in the normalized feature (also known as the fourth normalized feature). The Leaky ReLU activation function and a feature extraction module are then used to process the normalized feature, resulting in the features output by the fourth feature fusion structure (also known as the fourth fused feature).

[0091] Figure 9 This is a flowchart illustrating a feature extraction method for an image to be detected according to an exemplary embodiment. Figure 1 ,like Figure 9 As shown, it includes the following steps.

[0092] In step S51, the image to be detected is scaled according to a preset size to obtain an image of the preset size.

[0093] In this embodiment of the disclosure, the image to be detected can be preprocessed to obtain an image that meets the preset size requirements, so that the size of the scaled image to be detected can be adapted to the network structure for feature extraction.

[0094] In step S52, feature extraction, feature scaling, and feature combination are performed sequentially on an image of a preset size to obtain the features of the image to be detected.

[0095] In this embodiment, feature extraction is primarily achieved by progressively extracting features from a scaled image of a preset size through multiple convolutional and pooling layers. The convolutional layers can employ depthwise separable convolutions to improve model performance and reduce the number of parameters. Feature scaling is based on the pooling features obtained during feature extraction, determining the average pooling (avgpool) and converting the feature vectors from the convolutional layers into fixed-length vectors. For example, a 1x1 convolutional layer can be used to map features to a preset fixed dimension, ensuring consistent network performance across images of different resolutions. Feature combination is used to fuse feature information obtained from different dimensions, resulting in richer feature information.

[0096] In this embodiment, fully connected layers or convolutional layers can also be used to classify or regress the features after feature combination. For example, a softmax activation function can be used to output the class probability distribution or to output the location of the bounding box representing the target. Furthermore, feedback information from the target detection results can be obtained as a label for loss calculation. A loss function is used to calculate the loss value and determine the difference between the predicted value of the target detection model and the label. For example, a cross-entropy loss function or a mean squared error loss function can be used. The network parameters are updated using the loss value and the backpropagation algorithm to minimize the loss function and improve the detection accuracy of the target detection model.

[0097] In this embodiment of the disclosure, by performing operations such as feature extraction, feature scaling, and feature combination on the image to be detected after converting it to a preset size, the features of the image to be detected are adjusted in terms of depth, width, and resolution, thereby improving the accuracy of feature extraction and enhancing the efficiency and performance of feature extraction.

[0098] In this embodiment of the disclosure, features of the image to be detected can be obtained by extracting, scaling and combining features of an image of a preset size through point convolutional layers and depth convolutional layers.

[0099] Figure 10 This is a flowchart illustrating a feature extraction method for an image to be detected according to an exemplary embodiment. Figure 2 ,like Figure 10 As shown, it includes the following steps.

[0100] In step S61, based on the first point convolutional layer, the image of a preset size is channel-expanded to obtain features including a first preset number of channels.

[0101] In this embodiment, the pointwise convolutional layer is a convolutional layer with a kernel size of 1*1. The first pointwise convolutional layer is used to increase the number of channels in the feature map.

[0102] In step S62, based on the deep convolutional layer, feature extraction is performed on the features of each channel in the first preset number of channels to obtain deep convolutional features.

[0103] In this embodiment of the disclosure, during the deep convolution process using a deep convolutional layer, each channel in the features of the first preset number of channels has a corresponding convolutional kernel. The convolution operation is performed using the corresponding convolutional kernel for the features of each channel, so as to realize the independent transport of the features of each channel and avoid combining information from different channels during the convolution operation.

[0104] In step S63, based on the second point convolutional layer, the number of channels of the depth convolution feature is reduced to obtain the feature with a second preset number of channels, and the feature with the second preset number of channels is used as the feature of the image to be detected.

[0105] In this embodiment of the disclosure, the deep convolutional feature may include multiple feature maps. A second point-convolutional layer can be used to perform point-by-point convolution on the deep convolutional feature, combining multiple feature maps from the deep convolutional feature, and adjusting the number of channels of the combined feature. This achieves the fusion of the deep convolutional feature and the adjustment of the number of channels, resulting in a second preset number of channels that meets the output channel requirements, which serves as the feature of the image to be detected.

[0106] In this embodiment, features of the image to be detected are acquired by using depthwise convolutional layers and pointwise convolutional layers, thereby reducing computational costs and preserving the representational power of the features during feature extraction. Furthermore, depthwise separable convolution is performed using depthwise convolutional layers to reduce computational load, while width / depth / resolution scaling balances model complexity and performance.

[0107] In this embodiment, the original target detection result can be the original target image generated by the target detection model. The original target image includes multiple detection boxes for identifying targets and a confidence score corresponding to each detection box. The confidence score is used to characterize the degree of certainty of the target detection model regarding the presence of a target within the detection box. If the original target detection result includes a large number of overlapping detection results, for example, if the target detection image output by the third network includes a large number of overlapping detection boxes, there is a problem of unclear target detection results. In this embodiment, the original detection result can be adjusted by using a soft non-maximum suppression (Soft-NMS) algorithm model. Based on the degree of overlap between the detection box and the detection box with the highest confidence score, the confidence score of the detection box is reduced. Based on the reduced confidence scores of each detection box, the original detection result is selected, and the detection box with the highest confidence score is selected as the first target detection result, thereby reducing false detection results and interference items in the target detection result.

[0108] Furthermore, during the training of the Soft-NMS algorithm model, the loss can be calculated using the Alpha-Intersection over Union (Alpha-IoU) loss function based on the degree of overlap between the detection box and the detection box with the highest confidence score.

[0109] In an exemplary embodiment, the formula for the Alpha-IoU loss function used to adjust the original detection result in this disclosure embodiment can be as follows:

[0110]

[0111] In the formula, L α-CIoU The Alpha-IoU loss function, derived from the CIoU (Complete IoU) loss function, represents the intersection-union ratio (IoU) between the predicted detection box b and the detection box b representing the ground truth (gt). gt The degree of overlap. βv represents the predicted detection box b and the detection box b representing the ground truth (gt). gt The absolute value of the difference between the aspect ratios. α is a preset positive integer. It should be understood that α is not overly sensitive to different models or datasets, and can be set to a fixed value, such as α = 3. By using the Alpha-IoU loss function for loss calculation during the training process of the Soft-NMS algorithm model, the recognition performance of the Soft-NMS algorithm model can be effectively improved without introducing additional parameters or increasing training and inference time.

[0112] In this embodiment of the disclosure, the presence of a target in the image to be detected can be comprehensively determined based on the first target detection result and the target detection results of other target detection models.

[0113] Figure 11 This is a flowchart of a target detection method according to an exemplary embodiment. Figure 3 ,like Figure 11 As shown, it includes the following steps.

[0114] In step S71, the second target detection result of the image to be detected is obtained.

[0115] In this embodiment, the second target detection result is obtained by processing the image to be detected using a Mask Region-based Convolutional Neural Network (MASK-RCNN). The second target detection result includes a confidence score, which characterizes the probability of the presence of a target in the image. It should be understood that the second target detection result can also be determined based on other conventional target detection algorithms or models, and this embodiment does not limit the method for determining the second target detection result.

[0116] In step S72, the first target detection result and the second target detection result are weighted and fused to determine the probability of the presence of the target in the image to be detected.

[0117] In this embodiment, a first weight can be set for the first target detection result, and a second weight can be set for the second target detection result. If the first target detection result includes multiple detection boxes and corresponding confidence scores, the value with the largest confidence score among the multiple detection boxes is selected for weighted fusion to obtain the probability of the target's existence determined based on the combined first and second target detection results.

[0118] In step S73a, in response to the probability satisfying the probability condition, it is determined that a target exists in the image to be detected.

[0119] In this embodiment of the disclosure, the probability condition can be a threshold representing the probability of the target's existence. If the probability of existence is determined to be greater than or equal to the threshold, it indicates that the target exists in the image to be detected. If the probability of existence is determined to be less than the threshold, it indicates that the target does not exist in the image to be detected.

[0120] In step S73b, in response to the probability not satisfying the probability condition, it is determined that there is no target in the image to be detected.

[0121] In one exemplary embodiment, the following example illustrates the target detection method provided in this disclosure by taking the scenario of detecting application pop-up targets during the operation of an electronic device as an example. A second target detection result of the image to be detected can be obtained through a MASK-RCNN model. Figure 12 This is a schematic diagram illustrating target detection in an application pop-up window according to an exemplary embodiment. Figure 12 In this method, the first object detection result is the probability of a pop-up window appearing in the image output by the object detection model, with a weight of a = 0.7. The second object detection result is the probability of a pop-up window appearing in the image output by the MASK-RCNN model, with a weight of b = 0.8. The probability condition is that the probability value is greater than a preset empirical threshold. Images of electronic devices running applications are acquired, and the first and second object detection results for these images are obtained using both the object detection model and the MASK-RCNN model. Based on the formula: probability of a pop-up window appearing in an image = probability of a pop-up window appearing in the image output by the object detection model * a + probability of a pop-up window appearing in the image output by the MASK-RCNN model * b, the probability of a pop-up window appearing in the image is obtained. The relationship between the probability of a pop-up window appearing in the image and the preset empirical threshold is determined. If the probability of a pop-up window appearing in the image is greater than or equal to the preset empirical threshold, then a pop-up window is determined to exist, and an intelligent recognition click operation is performed to eliminate the pop-up window appearing when the electronic device is running the application. If the probability of a pop-up window appearing in the image is less than the preset empirical threshold, then a pop-up window is determined not to exist, and the electronic device continues to run. The object detection method provided by this embodiment improves the execution efficiency of object detection and reduces human maintenance costs.

[0122] In this embodiment of the disclosure, by weighted fusion of the first target detection result and the second target detection result, multiple detection results are integrated to make a comprehensive judgment on the existence of the target in the image to be detected, thereby improving the accuracy of target detection in the image to be detected.

[0123] In this embodiment of the disclosure, an attention mechanism can also be set to enhance the extracted features after each feature extraction operation.

[0124] Figure 13 This is a flowchart of a target detection method according to an exemplary embodiment. Figure 4 ,like Figure 13 As shown, it includes the following steps.

[0125] In step S81, based on the features obtained from feature extraction, m iterations of attention calculation are performed to obtain the first target index of the feature in each iteration of attention calculation.

[0126] In this embodiment of the disclosure, m is a positive integer greater than 1. The first target index is used to identify a preset number of positional features with the highest attention scores among the features obtained from feature extraction.

[0127] In step S82, the first target index obtained by the attention calculation in the j-th iteration is obtained.

[0128] In this embodiment, j is a positive integer greater than or equal to 1 and less than or equal to m. The first target index for the j-th iteration of the feature extraction can be denoted as I1. j Among them, I j It can be determined based on the following formula: I1 j =topkIndex(A j ), where A j The tensor data representing the j-th attention computation, A j The query (Q) vector, key (K) vector, and value (K) vector can be determined based on the features extracted from the features. For example, Where T is the transpose operator. TopIndex() is a method to retrieve the k largest values ​​of the tensor within the parentheses and their corresponding indices.

[0129] In step S83a, in response to j being 1, the average of the first target index calculated based on the attention in the j-th iteration and the first target index calculated based on the attention in the (j+1)-th iteration is used as the second target index calculated based on the attention in the j-th iteration.

[0130] In this embodiment of the disclosure,

[0131] In step S83b, in response to j being greater than 1 and less than m, the average of the first target index obtained by the attention calculation in the (j-1)th iteration, the first target index obtained by the attention calculation in the jth iteration, and the first target index obtained by the attention calculation in the (j+1)th iteration is used as the second target index obtained by the attention calculation in the jth iteration.

[0132] In step S83c, in response to j being m, the average of the first target index calculated based on the attention in the j-th iteration and the first target index calculated based on the attention in the (j-1)-th iteration is used as the second target index calculated based on the attention in the j-th iteration.

[0133] In this embodiment of the disclosure, the second target index for the j-th iteration of the features extracted from the features can be denoted as I2. j Among them, I j It can be determined based on the following formula: I = (I j +I j+1 +I j-1 ) / 3, where I 0 and I m+1 It is 0.

[0134] In step S84, local context enhancement is performed on the features obtained from feature extraction based on the second target index.

[0135] In this embodiment of the disclosure, by setting an attention mechanism and determining a second target index that can integrate target indices from multiple iterations based on the first target index of the current iteration and the first target index of the adjacent iterations during attention calculation, the amount of feature information acquired is increased, and the enhancement effect of the attention mechanism is improved.

[0136] In this embodiment, a target detection model is used to detect the image to be detected, obtaining a first target detection result. The target detection model includes a first network, a second network, and a third network. The second network has more than two feature fusion structures, and each feature fusion structure includes one or more feature extraction modules for feature extraction. Furthermore, different feature fusion structures include different numbers of feature extraction modules. By using multiple feature fusion structures with different feature extraction modules, the features obtained by the first network are fused according to depth levels, enabling the model to detect targets in the image to be detected at different depth levels. This improves the accuracy of the first target detection result obtained based on the second network and increases the success rate of target detection.

[0137] Based on the same concept, embodiments of this disclosure also provide a target detection device.

[0138] It is understood that the target detection device provided in this disclosure includes hardware structures and / or software modules corresponding to each function in order to achieve the above-mentioned functions. In conjunction with the units and algorithm steps of the various examples disclosed in this disclosure, this disclosure can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the technical solutions of this disclosure.

[0139] Figure 14 This is a block diagram illustrating a target detection device 100 according to an exemplary embodiment. (Refer to...) Figure 14 The device includes an acquisition unit 101 and a processing unit 102.

[0140] The acquisition unit 101 is used to acquire the image to be detected.

[0141] The processing unit 102 is used to obtain a first target detection result of the image to be detected based on the target detection model. The target detection model includes a first network, a second network, and a third network. The first network is used to extract features of the image to be detected. The second network is used to fuse the features extracted by the first network. The third network is used to generate the original detection result of the image to be detected based on the fused features obtained by the second network. The first target detection result is obtained by post-processing the original detection result. The second network includes n feature fusion structures. Different feature fusion structures include different numbers of feature extraction modules, and n is a positive integer greater than 2.

[0142] In one embodiment, the second network fuses the features extracted by the first network in the following manner: Features extracted by the first network are obtained; based on the i-th feature fusion structure, feature extraction is performed on the (i-1)-th fused feature to obtain the i-th extracted feature, and feature fusion is performed on the i-th extracted feature to obtain the i-th fused feature; the i-th fused feature is input into the (i+1)-th feature fusion structure for feature extraction and feature fusion to obtain the (i+1)-th fused feature, where i is a positive integer greater than or equal to 1 and less than n, and the 0th fused feature is the feature extracted by the first network; the above feature extraction and feature fusion steps are repeated until the n-th fused feature is obtained, and the n-th fused feature is used as the fused feature obtained by the second network.

[0143] In one implementation, the number of feature extraction processes in the i-th feature fusion structure is less than the number of feature extraction processes in the (i+1)-th feature fusion structure.

[0144] In one implementation, feature fusion is performed on the i-th extracted feature to obtain the i-th fused feature, including: performing feature extraction and feature fusion, including: performing convolution processing on the i-th extracted feature to obtain the i-th first feature, and performing convolution processing on the features input to the feature fusion structure to obtain the i-th second feature, wherein the i-th first feature and the i-th second feature have the same number of channels; combining the local features and global dependencies of the i-th first feature based on a dual attention network to obtain the i-th third feature; concatenating the i-th second feature and the i-th third feature to obtain the i-th concatenated feature, and processing the i-th concatenated feature to obtain the i-th fused feature.

[0145] In one implementation, the i-th spliced ​​feature is processed to obtain the i-th fused feature, including: performing cross-iterative batch normalization on the i-th spliced ​​feature to obtain the i-th normalized feature; and obtaining the i-th fused feature based on the i-th normalized feature.

[0146] In one embodiment, the first network extracts features of the image to be detected in the following manner: scaling the image to be detected to a preset size to obtain an image of the preset size; sequentially performing feature extraction, feature scaling, and feature combination on the image of the preset size to obtain the features of the image to be detected.

[0147] In one implementation, the first target detection result is obtained by adjusting and selecting the original detection result.

[0148] In one embodiment, the processing unit 102 is further configured to: acquire a second target detection result of the image to be detected, the second target detection result being obtained by processing the image to be detected based on a mask region convolutional neural network; perform weighted fusion of the first target detection result and the second target detection result to determine the probability of the existence of a target in the image to be detected; determine that a target exists in the image to be detected in response to the probability satisfying the probability condition; and determine that no target exists in the image to be detected in response to the probability not satisfying the probability condition.

[0149] In one embodiment, feature extraction, feature scaling, and feature combination are performed sequentially on an image of a preset size to obtain features of the image to be detected. This includes: expanding the channels of the image of the preset size based on a first point convolutional layer to obtain features including a first preset number of channels; extracting features from each channel of the features of the first preset number of channels based on a depth convolutional layer to obtain depth convolution features; combining the depth convolution features based on a second point convolutional layer to obtain features of a second preset number of channels, and using the features of the second preset number of channels as features of the image to be detected.

[0150] In one embodiment, after feature extraction, the processing unit 102 is further configured to: perform m iterations of attention calculation based on the features obtained from feature extraction, to obtain a first target index of the feature in each iteration of attention calculation, wherein the first target index is used to identify a preset number of positional features with the highest attention scores, and m is a positive integer greater than 1; obtain the first target index obtained in the j-th iteration of attention calculation, wherein j is a positive integer greater than or equal to 1 and less than or equal to m; and, in response to j being 1, use the average of the first target index obtained in the j-th iteration of attention calculation and the first target index obtained in the (j+1)-th iteration of attention calculation as the attention index for the j-th iteration. The calculated second target index; in response to j being greater than 1 and less than m, the average of the first target index calculated in the (j-1)th iteration of attention, the first target index calculated in the jth iteration of attention, and the first target index calculated in the (j+1)th iteration of attention is used as the second target index calculated in the jth iteration of attention; in response to j being m, the average of the first target index calculated in the jth iteration of attention and the first target index calculated in the (j-1)th iteration of attention is used as the second target index calculated in the jth iteration of attention; based on the second target index, local context enhancement is performed on the features extracted from the feature extraction.

[0151] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0152] Figure 15 This is a block diagram illustrating an electronic device 200 for target detection according to an exemplary embodiment. For example, the electronic device 200 may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, etc.

[0153] Reference Figure 15 The electronic device 200 may include one or more of the following components: processing component 202, memory 204, power component 206, multimedia component 208, audio component 210, input / output (I / O) interface 212, sensor component 214, and communication component 216.

[0154] Processing component 202 typically controls the overall operation of electronic device 200, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 202 may include one or more processors 220 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 202 may include one or more modules to facilitate interaction between processing component 202 and other components. For example, processing component 202 may include a multimedia module to facilitate interaction between multimedia component 208 and processing component 202.

[0155] Memory 204 is configured to store various types of data to support the operation of electronic device 200. Examples of such data include instructions for any application or method operating on electronic device 200, contact data, phonebook data, messages, pictures, videos, etc. Memory 204 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0156] Power component 206 provides power to various components of electronic device 200. Power component 206 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 200.

[0157] Multimedia component 208 includes a screen that provides an output interface between the electronic device 200 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 208 includes a front-facing camera and / or a rear-facing camera. When the electronic device 200 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0158] Audio component 210 is configured to output and / or input audio signals. For example, audio component 210 includes a microphone (MIC) configured to receive external audio signals when electronic device 200 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 204 or transmitted via communication component 216. In some embodiments, audio component 210 also includes a speaker for outputting audio signals.

[0159] I / O interface 212 provides an interface between processing component 202 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0160] Sensor assembly 214 includes one or more sensors for providing state assessments of various aspects of electronic device 200. For example, sensor assembly 214 can detect the on / off state of electronic device 200, the relative positioning of components such as the display and keypad of electronic device 200, changes in position of electronic device 200 or a component of electronic device 200, the presence or absence of user contact with electronic device 200, orientation or acceleration / deceleration of electronic device 200, and temperature changes of electronic device 200. Sensor assembly 214 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 214 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 214 may also include an accelerometer, gyroscope, magnetometer, pressure sensor, or temperature sensor.

[0161] Communication component 216 is configured to facilitate wired or wireless communication between electronic device 200 and other devices. Electronic device 200 can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 216 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 216 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0162] In an exemplary embodiment, the electronic device 200 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0163] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 204 including instructions, which can be executed by a processor 220 of an electronic device 200 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0164] Figure 16 This is a block diagram illustrating an apparatus 300 for target detection according to an exemplary embodiment. For example, apparatus 300 may be provided as a server. (Refer to...) Figure 16 The device 300 includes a processing component 322, which further includes one or more processors, and memory resources represented by memory 332 for storing instructions, such as application programs, that can be executed by the processing component 322. The application programs stored in memory 332 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 322 is configured to execute instructions to perform the aforementioned target detection method.

[0165] Device 300 may also include a power supply component 326 configured to perform power management of device 300, a wired or wireless network interface 350 configured to connect device 300 to a network, and an input / output (I / O) interface 358. Device 300 may operate on an operating system stored in memory 332, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, or similar.

[0166] This disclosure also provides a computer program product, including a computer program that, when executed by a processor, implements the target detection method involved in any of the above embodiments.

[0167] In one exemplary embodiment, the processor may be deployed in an electronic device, for example.

[0168] It is understood that in this disclosure, "multiple" refers to two or more, and other quantifiers are similar. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. The singular forms "a," "the," and "the" are also intended to include the plural forms unless the context clearly indicates otherwise.

[0169] It is further understood that the terms "first," "second," etc., are used to describe various types of information, but this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another, and do not indicate a specific order or degree of importance. In fact, the expressions "first," "second," etc., are completely interchangeable. For example, without departing from the scope of this disclosure, first information can also be referred to as second information, and similarly, second information can also be referred to as first information.

[0170] It can be further understood that, unless otherwise specified, "connection" includes both direct connections where no other components exist between the two parties and indirect connections where other components exist between them.

[0171] It is further understood that although operations are described in a specific order in the accompanying drawings in the embodiments of this disclosure, this should not be construed as requiring these operations to be performed in the specific order or serial order shown, or requiring all of the shown operations to be performed to obtain the desired result. In certain environments, multitasking and parallel processing may be advantageous.

[0172] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein.

[0173] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A target detection method characterized by, include: Acquire the image to be detected; Based on the target detection model, a first target detection result of the image to be detected is obtained. The target detection model includes a first network, a second network, and a third network. The first network is used to extract features of the image to be detected. The second network is used to fuse the features extracted by the first network. The third network is used to generate an original detection result of the image to be detected based on the fused features obtained by the second network. The first target detection result is obtained by post-processing the original detection result. The second network includes n feature fusion structures, and the number of feature extraction modules included in different feature fusion structures is different, where n is a positive integer greater than 2.

2. The method according to claim 1, characterized in that, The second network fuses the features extracted by the first network in the following manner: Obtain the features extracted from the first network; Based on the i-th feature fusion structure, feature extraction is performed on the (i-1)-th fusion feature to obtain the i-th extracted feature, and feature fusion is performed on the i-th extracted feature to obtain the i-th fusion feature. The i-th fusion feature is then input into the (i+1)-th feature fusion structure for feature extraction and feature fusion to obtain the (i+1)-th fusion feature, where i is a positive integer greater than or equal to 1 and less than n, and the 0th fusion feature is the feature extracted by the first network. Repeat the above steps of feature extraction and feature fusion until the nth fused feature is obtained, and use the nth fused feature as the fused feature obtained by the second network.

3. The method according to claim 2, characterized in that, The number of feature extraction operations performed in the i-th feature fusion structure is less than the number of feature extraction operations performed in the (i+1)-th feature fusion structure.

4. The method according to claim 2 or 3, characterized in that, The step of fusing the extracted i-th feature to obtain the fused i-th feature includes: The i-th extracted feature is convolved to obtain the i-th first feature, and the feature input to the feature fusion structure is convolved to obtain the i-th second feature. The i-th first feature and the i-th second feature have the same number of channels. Based on a dual attention network, the local features and global dependencies of the i-th first feature are combined to obtain the i-th third feature; The i-th second feature and the i-th third feature are concatenated to obtain the i-th concatenated feature, and the i-th concatenated feature is processed to obtain the i-th fused feature.

5. The method according to claim 4, characterized in that, The process of processing the i-th splicing feature to obtain the i-th fused feature includes: The i-th spliced ​​feature is subjected to cross-iterative batch normalization to obtain the i-th normalized feature; Based on the i-th normalized feature, the i-th fusion feature is obtained.

6. The method according to claim 1, characterized in that, The first network extracts features from the image to be detected in the following manner: The image to be detected is scaled according to a preset size to obtain an image of the preset size; The features of the image to be detected are obtained by sequentially performing feature extraction, feature scaling, and feature combination on the image of the preset size.

7. The method according to claim 1, characterized in that, The first target detection result is obtained by adjusting and selecting the original detection result.

8. The method according to claim 1, characterized in that, The method further includes: A second target detection result is obtained from the image to be detected, which is obtained by processing the image to be detected based on a mask region convolutional neural network; The first target detection result and the second target detection result are weighted and fused to determine the probability of the presence of the target in the image to be detected. In response to the probability satisfying the probability condition, it is determined that a target exists in the image to be detected; In response to the probability not satisfying the probability condition, it is determined that there is no target in the image to be detected.

9. The method according to claim 6, characterized in that, The process of sequentially extracting, scaling, and combining features from the image of the preset size to obtain the features of the image to be detected includes: Based on the first point convolutional layer, the image of the preset size is channel-expanded to obtain features including the first preset number of channels; Based on the deep convolutional layer, feature extraction is performed on the features of each channel in the first preset number of channels to obtain deep convolutional features; Based on the second point convolutional layer, the depth convolutional features are combined to obtain features with a second preset number of channels, and the features with the second preset number of channels are used as features of the image to be detected.

10. The method according to claim 3 or 9, characterized in that, After feature extraction, the method further includes: Based on the features obtained from the feature extraction, m iterations of attention calculation are performed to obtain the first target index of the feature in each iteration of attention calculation. The first target index is used to identify a preset number of positional features with the highest attention scores in the feature, where m is a positive integer greater than 1. Obtain the first target index obtained by the attention calculation in the j-th iteration, where j is a positive integer greater than or equal to 1 and less than or equal to m; In response to j being 1, the average of the first target index calculated based on the attention in the j-th iteration and the first target index calculated based on the (j+1)-th iteration is used as the second target index calculated based on the attention in the j-th iteration. In response to the condition that j is greater than 1 and less than m, the average of the first target index calculated based on the (j-1)th iteration attention calculation, the first target index calculated based on the jth iteration attention calculation, and the first target index calculated based on the (j+1)th iteration attention calculation is used as the second target index calculated based on the jth iteration attention calculation. In response to j being m, the average of the first target index calculated based on the attention in the j-th iteration and the first target index calculated based on the (j-1)-th iteration is used as the second target index calculated based on the attention in the j-th iteration. Based on the second target index, local context enhancement is performed on the features obtained from feature extraction.

11. A target detection device, characterized in that, include: The acquisition unit is used to acquire the image to be detected; The processing unit is used to obtain a first target detection result of the image to be detected based on a target detection model. The target detection model includes a first network, a second network, and a third network. The first network is used to extract features of the image to be detected. The second network is used to fuse the features extracted by the first network. The third network is used to generate an original detection result of the image to be detected based on the fused features obtained by the second network. The first target detection result is obtained by post-processing the original detection result. The second network includes n feature fusion structures, and the number of feature extraction modules included in different feature fusion structures is different, where n is a positive integer greater than 2.

12. The apparatus according to claim 11, characterized in that, The second network fuses the features extracted by the first network in the following manner: Obtain the features extracted from the first network; Based on the i-th feature fusion structure, feature extraction is performed on the (i-1)-th fusion feature to obtain the i-th extracted feature, and feature fusion is performed on the i-th extracted feature to obtain the i-th fusion feature. The i-th fusion feature is then input into the (i+1)-th feature fusion structure for feature extraction and feature fusion to obtain the (i+1)-th fusion feature, where i is a positive integer greater than or equal to 1 and less than n, and the 0th fusion feature is the feature extracted by the first network. Repeat the above steps of feature extraction and feature fusion until the nth fused feature is obtained, and use the nth fused feature as the fused feature obtained by the second network.

13. The apparatus according to claim 12, characterized in that, The number of feature extraction operations performed in the i-th feature fusion structure is less than the number of feature extraction operations performed in the (i+1)-th feature fusion structure.

14. The apparatus according to claim 12 or 13, characterized in that, The step of fusing the extracted i-th feature to obtain the fused i-th feature includes: The i-th extracted feature is convolved to obtain the i-th first feature, and the feature input to the feature fusion structure is convolved to obtain the i-th second feature. The i-th first feature and the i-th second feature have the same number of channels. Based on a dual attention network, the local features and global dependencies of the i-th first feature are combined to obtain the i-th third feature; The i-th second feature and the i-th third feature are concatenated to obtain the i-th concatenated feature, and the i-th concatenated feature is processed to obtain the i-th fused feature.

15. The apparatus according to claim 14, characterized in that, The process of processing the i-th splicing feature to obtain the i-th fused feature includes: The i-th spliced ​​feature is subjected to cross-iterative batch normalization to obtain the i-th normalized feature; Based on the i-th normalized feature, the i-th fusion feature is obtained.

16. The apparatus according to claim 11, characterized in that, The first network extracts features from the image to be detected in the following manner: The image to be detected is scaled according to a preset size to obtain an image of the preset size; The features of the image to be detected are obtained by sequentially performing feature extraction, feature scaling, and feature combination on the image of the preset size.

17. The apparatus according to claim 11, characterized in that, The first target detection result is obtained by adjusting and selecting the original detection result.

18. The apparatus according to claim 11, characterized in that, The processing unit is also used for: A second target detection result is obtained from the image to be detected, which is obtained by processing the image to be detected based on a mask region convolutional neural network; The first target detection result and the second target detection result are weighted and fused to determine the probability of the presence of the target in the image to be detected. In response to the probability satisfying the probability condition, it is determined that a target exists in the image to be detected; In response to the probability not satisfying the probability condition, it is determined that there is no target in the image to be detected.

19. The apparatus according to claim 16, characterized in that, The process of sequentially extracting, scaling, and combining features from the image of the preset size to obtain the features of the image to be detected includes: Based on the first point convolutional layer, the image of the preset size is channel-expanded to obtain features including the first preset number of channels; Based on the deep convolutional layer, feature extraction is performed on the features of each channel in the first preset number of channels to obtain deep convolutional features; Based on the second point convolutional layer, the depth convolutional features are combined to obtain features with a second preset number of channels, and the features with the second preset number of channels are used as features of the image to be detected.

20. The apparatus according to claim 13 or 19, characterized in that, After feature extraction, the processing unit is further configured to: Based on the features obtained from the feature extraction, m iterations of attention calculation are performed to obtain the first target index of the feature in each iteration of attention calculation. The first target index is used to identify a preset number of positional features with the highest attention scores in the feature, where m is a positive integer greater than 1. Obtain the first target index obtained by the attention calculation in the j-th iteration, where j is a positive integer greater than or equal to 1 and less than or equal to m; In response to j being 1, the average of the first target index calculated based on the attention in the j-th iteration and the first target index calculated based on the (j+1)-th iteration is used as the second target index calculated based on the attention in the j-th iteration. In response to the condition that j is greater than 1 and less than m, the average of the first target index calculated based on the (j-1)th iteration attention calculation, the first target index calculated based on the jth iteration attention calculation, and the first target index calculated based on the (j+1)th iteration attention calculation is used as the second target index calculated based on the jth iteration attention calculation. In response to j being m, the average of the first target index calculated based on the attention in the j-th iteration and the first target index calculated based on the (j-1)-th iteration is used as the second target index calculated based on the attention in the j-th iteration. Based on the second target index, local context enhancement is performed on the features obtained from feature extraction.

21. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to perform the method described in any one of claims 1 to 10.

22. A storage medium, characterized in that, The storage medium stores instructions that, when executed by the processor of the electronic device, enable the terminal to perform the method described in any one of claims 1 to 10.

23. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method as described in any one of claims 1 to 10.

Images