Image object detection method, device and storage medium

Through the combination of depth and non-deep feature extraction models, efficient image object detection on mobile terminals is achieved, solving the problems of slow operation speed and large resource requirements in the prior art.

CN110717929BActive Publication Date: 2024-12-03TENCENT TECHNOLOGY (SHENZHEN) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN201810754633.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2018-07-11
Publication Date
2024-12-03
Estimated Expiration
2038-07-11

AI Technical Summary

Technical Problem

The existing image object detection algorithms are slow to run and cannot be implemented on mobile terminals with smaller resource configurations, mainly due to inefficiency caused by excessive model size.

Method used

The deep feature extraction model and the non-deep feature extraction model are used to perform multi-level feature extraction on the detection image, large-size feature extraction is performed through the deep feature extraction model, small-size feature extraction is performed using the non-deep feature extraction model, and information regression is performed by combining the feature prediction model. , realize the identification of target types and locations.

Benefits of technology

The overall feature extraction speed of image object detection is improved, the demand for configuration resources is reduced, and the image object detection is more efficient on the mobile terminal.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN110717929B_ABST
    Figure CN110717929B_ABST
Patent Text Reader

Abstract

The present invention provides an image target detection method, which includes: obtaining a detection image, an n-level deep feature map framework and an m-level non-deep feature map framework, wherein the feature map framework includes an output feature size and dimension; based on a deep feature extraction model, using the i-th level deep feature map framework to perform deep feature extraction on the (i-1)-th level feature of the detection image to obtain the i-th level feature of the detection image; based on a non-deep feature extraction model, using the j-th level non-deep feature map framework to perform non-deep feature extraction on the (j-1+n)-th level feature of the detection image to obtain the (j+n)-th level feature of the detection image; and based on a feature prediction model, performing information regression operation on the a-th level feature to the (m+n)-th level feature of the detection image, thereby obtaining the target type and target position of the detection image. The present invention also provides an image target detection device, the overall feature extraction speed of the present invention is fast, and the demand for configuration resources is low.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to an image target detection method, device and storage medium. Background Art

[0002] With the development of science and technology, the object recognition technology in images has become an increasingly important problem in computer vision. That is, in a given picture, mark the objects to be recognized, such as marking people, cars, houses and other objects in the picture.

[0003] With the recent rise of deep learning, the application of deep learning in the field of image object detection has made great breakthroughs, and a series of image object learning methods based on deep learning algorithms have been proposed, such as Faster-RCNN (Fast Regional Convolutional Neural Networks features) and YOLO (You Only Look Once) and other deep learning algorithms.

[0004] However, the technical focus of existing image target detection algorithms is on the accuracy of detection results. Therefore, the running speed of existing image target detection algorithms does not meet the requirements of actual scenarios. The model size of some relatively efficient image target detection systems exceeds 100Mb, which makes the existing image target detection systems run slowly and cannot be implemented on mobile terminals with smaller resource configurations. Summary of the invention

[0005] The embodiments of the present invention provide an image target detection method and device with a relatively fast running speed and a relatively low requirement for configuration resources, so as to solve the technical problem that the existing image target detection method and device have a relatively slow running speed and cannot be implemented on a mobile terminal with a relatively small resource configuration.

[0006] An embodiment of the present invention provides an image target detection method, which includes:

[0007] Obtain a detection image, an n-level deep feature map framework, and an m-level non-depth feature map framework, where n is an integer greater than or equal to 2, and m is an integer greater than or equal to 1; wherein the feature map framework includes the output feature size and dimension;

[0008] Based on the deep feature extraction model, use the i-th level deep feature map framework to perform deep feature extraction on the (i-1)-th level features of the detection image to obtain the i-th level features of the detection image, where i is a positive integer less than or equal to n;

[0009] Based on the non-depth feature extraction model, using the j-th level non-depth feature map framework to perform non-depth feature extraction on the (j-1+n)-th level features of the detection image to obtain the (j+n)-th level features of the detection image, where j is a positive integer less than or equal to m; and

[0010] Based on the feature prediction model, information regression operation is performed on the ath level feature to the (m+n)th level feature of the detection image to obtain the target type and target position of the detection image, where a is an integer less than n and greater than or equal to 2.

[0011] The embodiment of the present invention further provides an image object detection device, which includes:

[0012] An image and frame acquisition module, used to acquire a detection image, an n-level deep feature map frame, and an m-level non-deep feature map frame, where n is an integer greater than or equal to 2, and m is an integer greater than or equal to 1; wherein the feature map frame includes an output feature size and dimension;

[0013] A deep feature extraction module, used to perform deep feature extraction on the (i-1)th level features of the detection image using the i-th level deep feature map framework based on the deep feature extraction model to obtain the i-th level features of the detection image, where i is a positive integer less than or equal to n;

[0014] A non-depth feature extraction module, used for performing non-depth feature extraction on the (j-1+n)th level features of the detection image using the jth level non-depth feature map framework based on the non-depth feature extraction model to obtain the (j+n)th level features of the detection image, where j is a positive integer less than or equal to m;

[0015] The target detection module is used to perform information regression operations on the ath level features to the (m+n)th level features of the detection image based on the feature prediction model, so as to obtain the target type and target position of the detection image, where a is an integer less than n and greater than or equal to 2.

[0016] An embodiment of the present invention further provides a storage medium storing processor-executable instructions, wherein the instructions are loaded by one or more processors to execute any of the above-mentioned image object detection methods.

[0017] Compared with the prior art, the image target detection method, device and storage medium of the present invention extract and identify features of multiple different sizes of the same detection image based on a deep feature extraction model and a non-deep feature extraction model. Since the small-size features of the detection image can be directly extracted based on the large-size features of the detection image, the overall feature extraction speed is faster and the demand for configuration resources is lower; it effectively solves the technical problems that the existing image target detection methods and devices have slow running speeds and cannot be implemented on mobile terminals with smaller resource configurations. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 is a flow chart of a first embodiment of the image object detection method of the present invention;

[0019] Figure 2 is a flow chart of a second embodiment of the image object detection method of the present invention;

[0020] Figure 3a This is a flow chart of step S202 of the second embodiment of the image object detection method of the present invention;

[0021] Figure 3b This is a schematic diagram of feature extraction in step S202 of the second embodiment of the image object detection method of the present invention;

[0022] Figure 4a This is a flow chart of step S203 of the second embodiment of the image object detection method of the present invention;

[0023] Figure 4b This is a schematic diagram of feature extraction in step S203 of the second embodiment of the image object detection method of the present invention;

[0024] Figure 5a This is a flow chart of step S204 of the second embodiment of the image object detection method of the present invention;

[0025] Figure 5b This is a schematic diagram of feature extraction in step S204 of the second embodiment of the image object detection method of the present invention;

[0026] Figure 6 is a schematic structural diagram of a first embodiment of an image object detection device of the present invention;

[0027] Figure 7 is a schematic structural diagram of a second embodiment of an image object detection device of the present invention;

[0028] Figure 8 A schematic diagram of the structure of a depth feature extraction module of a second embodiment of an image object detection device of the present invention;

[0029] Fig. 9 A schematic diagram of the structure of a non-depth feature extraction module of a second embodiment of an image object detection device of the present invention;

[0030] Fig.10 A schematic diagram of the structure of an object detection module of a second embodiment of an image object detection device of the present invention;

[0031] Fig.11 It is a schematic diagram of the use of a specific embodiment of the image object detection method and the image object detection device of the present invention;

[0032] Fig.12 The figure is a schematic diagram of the working environment structure of the electronic device where the image target detection device of the present invention is located. DETAILED DESCRIPTION

[0033] Please refer to the drawings, in which the same component symbols represent the same components, and the principle of the present invention is illustrated by implementing it in an appropriate computing environment. The following description is based on the illustrated specific embodiments of the present invention, which should not be considered as limiting other specific embodiments of the present invention not described in detail herein.

[0034] In the following description, specific embodiments of the present invention will be described with reference to steps and symbols of operations performed by one or more computers, unless otherwise stated. Therefore, it will be understood that these steps and operations, which are several times referred to as being performed by a computer, include manipulation by a computer processing unit of electronic signals representing data in a structured form. This manipulation transforms the data or maintains it at a location in the memory system of the computer, which can be reconfigured or otherwise change the operation of the computer in a manner familiar to those skilled in the art. The data structure in which the data is maintained is a physical location in the memory, which has specific characteristics defined by the data format. However, the principles of the present invention are described in the above text, which does not represent a limitation, and those skilled in the art will understand that the various steps and operations described below can also be implemented in hardware.

[0035] The image target detection method and the image target detection device of the present invention can be set in any electronic device, and are used to detect and identify target problems such as people, cars, houses, etc. in pictures or photos. The electronic device includes but is not limited to wearable devices, head-mounted devices, medical and health platforms, personal computers, server computers, handheld or laptop devices, mobile devices (such as mobile phones, personal digital assistants (PDAs), media players, etc.), multi-processor systems, consumer electronic devices, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like. The electronic device is preferably a mobile terminal equipped with an image target recognition application, which can quickly extract target features in an image, and has a low demand for configuration resources of the mobile terminal itself.

[0036] Please refer to Figure 1 , Figure 1 Flow chart of the first embodiment of the image target detection method of the present invention; the image target detection method of this embodiment can be implemented using the above electronic device, and the image target detection method of this embodiment includes:

[0037] Step S101, obtaining a detection image, an n-level depth feature map framework, and an m-level non-depth feature map framework, where n is an integer greater than or equal to 2, and m is an integer greater than or equal to 1; wherein the feature map framework includes an output feature size and dimension;

[0038] Step S102, based on the deep feature extraction model, using the i-th level deep feature map framework to perform deep feature extraction on the (i-1)-th level features of the detection image to obtain the i-th level features of the detection image, where i is a positive integer less than or equal to n;

[0039] Step S103, based on the non-depth feature extraction model, using the j-th level non-depth feature map framework to perform non-depth feature extraction on the (j-1+n)-th level features of the detection image to obtain the (j+n)-th level features of the detection image, where j is a positive integer less than or equal to m;

[0040] Step S104, based on the feature prediction model, perform information regression operation on the ath level feature to the (m+n)th level feature of the detection image to obtain the target type and target position of the detection image, where a is an integer less than n and greater than or equal to 2.

[0041] The image object detection process of the image object detection method of this embodiment is described in detail below.

[0042] In step S101, the image target detection device obtains a detection image that needs to be detected, as well as an n-level depth feature map framework and an m-level non-depth feature map framework for target detection on the detection image. In this embodiment, n is an integer greater than or equal to 2, and m is an integer greater than or equal to 1. That is, the detection image must undergo at least three feature extraction operations.

[0043] In order to perform accurate and comprehensive target detection on the detection image, it is necessary to perform multi-level feature extraction operations on the detection image, such as m+n levels. Since the feature size of the next level must be smaller than the feature size of the previous level, the feature extraction operation of the next level can be performed on the features output by the previous feature extraction operation. For large-size feature extraction operations, since fewer previous feature extraction operations have been performed, deep feature extraction models and deep feature map frameworks are required for feature extraction. For small-size feature extraction operations, since multiple previous feature extraction operations have been performed, only non-deep feature extraction models and non-deep feature map frameworks are required for feature extraction.

[0044] Here, the deep feature map framework is an identification parameter for feature recognition of the detection image or the lower-level features corresponding to the detection image, and the deep feature map framework may include the feature size and dimension output at each deep feature level. The non-depth feature framework is an identification parameter for feature recognition of the lower-level features corresponding to the detection image, and the non-depth feature map framework may include the feature size and dimension output at each non-depth feature level.

[0045] In step S102, the image target detection device performs deep feature extraction on the (i-1)th level features of the detection image based on a preset depth feature extraction model using the i-th level depth feature map framework obtained in step S101 to obtain the i-th level features of the detection image, where i is a positive integer less than or equal to n.

[0046] That is, the image target detection device performs deep feature extraction on the pixels of the detection image based on the preset deep feature extraction model to obtain the first-level features of the detection image corresponding to the first-level deep feature map framework; then the image target detection device performs deep feature extraction on the first-level features of the detection image to obtain the second-level features of the detection image corresponding to the second-level deep feature map framework...; finally, the image target detection device performs deep feature extraction on the (n-1)-level features of the detection image to obtain the n-level features of the detection image corresponding to the n-level deep feature map framework. In this way, the first-level features to the n-level features of the detection image are obtained.

[0047] In step S103, the image target detection device performs non-depth feature extraction on the (j-1+n)th level features of the detection image based on a preset non-depth feature extraction model and using the jth level non-depth feature map framework obtained in step S101 to obtain the (j+n)th level features of the detection image, where j is a positive integer less than or equal to m.

[0048] That is, the image target detection device performs non-depth feature extraction on the n-level features of the detection image based on the preset non-depth feature extraction model to obtain the (n+1)-level features of the detection image corresponding to the 1-level non-depth feature map framework; then the image target detection device performs non-depth feature extraction on the (n+1)-level features of the detection image to obtain the (n+2)-level features of the detection image corresponding to the 2-level non-depth feature map framework...; finally, the image target detection device performs depth feature extraction on the (n+m-1)-level features of the detection image to obtain the (m+n)-th level features of the detection image corresponding to the (m+n)-th level depth feature map framework. In this way, the (n+1)-th level features to the (m+n)-th level features of the detection image are obtained.

[0049] In step S104, the image target detection device performs information regression operations on the ath-level features to the (m+n)th-level features of the detection image obtained in step S102 and step S103 based on a preset feature prediction model, so as to obtain the target type and target position of the detection image, where a is an integer less than n and greater than or equal to 2.

[0050] Specifically, since the feature sizes of the level 1 features to the level (a-1) features of the detection image are relatively large, it is meaningless to perform feature classification and recognition, so the image object detection device directly discards the level 1 features to the level (a-1) features of the detection image. The image object detection device performs feature classification and recognition on the level a features to the level (m+n) features of the detection image, thereby obtaining the target type (such as a person, a car, a house, etc.) and the target position (such as the center coordinates of the target and the length and width of the target box, etc.) of the detection image corresponding to the feature.

[0051] This completes the image object detection process of the image object detection method of this embodiment.

[0052] The image target detection method of this embodiment is based on a deep feature extraction model and a non-deep feature extraction model, and extracts and recognizes features of multiple different sizes of the same detection image. Since the small-size features of the detection image can be directly extracted based on the large-size features of the detection image, the overall feature extraction speed is faster and the demand for configuration resources is lower.

[0053] Please refer to Figure 2 , Figure 2Flow chart of the second embodiment of the image target detection method of the present invention; the image target detection method of this embodiment can be implemented using the above electronic device, and the image target detection method of this embodiment includes:

[0054] Step S201, obtaining a detection image, an n-level depth feature map framework, and an m-level non-depth feature map framework, where n is an integer greater than or equal to 2, and m is an integer greater than or equal to 1; wherein the feature map framework includes an output feature size and dimension;

[0055] Step S202, based on the deep feature extraction model, using the i-th level deep feature map framework to perform deep feature extraction on the (i-1)-th level features of the detection image to obtain the i-th level features of the detection image, where i is a positive integer less than or equal to n;

[0056] Step S203, based on the non-depth feature extraction model, use the j-th level non-depth feature map framework to perform non-depth feature extraction on the (j-1+n)-th level features of the detection image to obtain the (j+n)-th level features of the detection image, where j is a positive integer less than or equal to m;

[0057] Step S204, based on the feature prediction model, perform information regression operation on the ath level feature to the (m+n)th level feature of the detection image to obtain the target type and target position of the detection image, where a is an integer less than n and greater than or equal to 2.

[0058] The image object detection process of the image object detection method of this embodiment is described in detail below.

[0059] In step S201, the image target detection device obtains a detection image that needs to be detected, as well as an n-level depth feature map framework and an m-level non-depth feature map framework for target detection on the detection image. In this embodiment, n is an integer greater than or equal to 2, and m is an integer greater than or equal to 1. That is, the detection image must be subjected to at least 3 feature extraction operations.

[0060] Here, the deep feature map framework is an identification parameter for feature recognition of the detection image or the lower-level features corresponding to the detection image, and the deep feature map framework may include the feature size and dimension output at each deep feature level. The non-depth feature framework is an identification parameter for feature recognition of the lower-level features corresponding to the detection image, and the non-depth feature map framework may include the feature size and dimension output at each non-depth feature level.

[0061] In step S202, the image target detection device performs deep feature extraction on the (i-1)th level features of the detection image based on a preset depth feature extraction model using the i-th level depth feature map framework obtained in step S101 to obtain the i-th level features of the detection image, where i is a positive integer less than or equal to n.

[0062] In this embodiment, the deep feature extraction model includes a deep input convolution layer, a deep first nonlinear transformation convolution layer, a deep second nonlinear transformation convolution layer and a deep output convolution layer.

[0063] Please refer to Figure 3a and Figure 3b , Figure 3a is a flowchart of step S202 of the second embodiment of the image object detection method of the present invention, Figure 3b This is a schematic diagram of feature extraction of step S202 of the second embodiment of the image object detection method of the present invention. Step S202 includes:

[0064] In step S301, the image object detection apparatus uses the deep input convolution layer of the deep feature extraction model to perform a dimensionality increase operation on the (i-1)th level features of the detection image to obtain the i-th level dimensionality increase feature of the detection image.

[0065] The deep input convolution layer is a standard convolution layer with a 1*1 convolution kernel size and a nonlinear activation function, and the deep input convolution layer can be set with a larger number of channels, such as 4-6. This can increase the dimension of the input feature while ensuring the feature size of the detection image, thereby solving the problem of feature loss of the detection image.

[0066] The number of channels of the deep input convolution layer is used to represent the number of feature extraction patterns for feature extraction from low-level features of the detection image, and the convolution kernel size of the deep input convolution layer is used to adjust the complexity of the deep neural network model.

[0067] For example, the (i-1)th level feature of the input detection image is a 32*32*3 feature point matrix, where 3 is the number of input channels of the detection image, such as the red pixel brightness value, the blue pixel brightness value, and the green pixel brightness value; if the convolution kernel size of the deep input convolution layer is set to 1*1, the output feature size of the deep input convolution layer is 32*32, that is, a 32*32 feature map can be obtained by traversing the 32*32 feature point matrix in sequence using a 1*1 convolution kernel. If the number of channels of the deep input convolution layer is 6, the output of the deep input convolution layer is a 32*32*18 i-th level dimension-raising feature. In this way, the dimension-raising feature of the detection image with a higher dimension is obtained without changing the output feature size.

[0068] The deep input convolution layer then uses a nonlinear activation function, such as a linear rectifier function (ReLU, Rectified Linear Unit), to perform nonlinear processing on the i-th level dimensional features of the output to ensure that the output of the deep input convolution layer is differentiable, thereby improving the accuracy of subsequent output features.

[0069] In step S302, the image target detection device uses the deep first nonlinear transformation convolution layer of the deep feature extraction model to perform a first feature extraction operation on the i-th level upscaled feature of the detection image obtained in step S301 to obtain the i-th level first convolution feature of the detection image.

[0070] The first nonlinear transformation convolution layer is a depth-wise separable convolution layer with a 3*3 convolution kernel size and a nonlinear activation function. The setting of the depth-wise separable convolution layer can greatly reduce the amount of computation of the first nonlinear transformation convolution layer, thereby greatly reducing the size of the deep feature extraction model.

[0071] Among them, the depthwise separable convolution layer can realize spatial convolution while maintaining channel separation. For example, for a standard convolution layer with a 3*3 convolution kernel size, the number of input channels is 16 and the number of output channels is 32. Then, 32 3*3 convolution kernels traverse each data in the 16 channels, and 16*32*3*3=4608 parameters need to be set for convolution operation. For example, for a depthwise separable convolution layer with a 3*3 convolution kernel size, a 3*3 convolution kernel is used to traverse the data of 16 channels to obtain 16 feature maps, and then 32 1*1 convolution kernels are used to traverse these 16 feature maps. In this way, only 16*3*3+16+32+1+1=656 parameters need to be set to complete the convolution operation.

[0072] In this step, the deep first nonlinear transformation convolution layer first performs the first feature extraction operation on the i-th level up-dimensional features of the detection image, and then the deep first nonlinear transformation convolution layer uses a nonlinear activation function, such as a linear rectification function (ReLU, Rectified Linear Unit), to perform nonlinear processing on the output i-th level first convolution features to ensure that the output of the deep first nonlinear transformation convolution layer is differentiable, thereby improving the accuracy of subsequent output features.

[0073] In step S303, the image target detection device uses the deep second nonlinear transformation convolution layer of the deep feature extraction model to perform a second feature extraction operation on the i-th level first convolution feature of the detection image obtained in step S302 to obtain the i-th level second convolution feature of the detection image.

[0074] The deep second nonlinear transformation convolution layer is a deep separable atrous convolution layer (atrous convolutions) with a 3*3 convolution kernel size and a nonlinear activation function. The setting of the deep separable atrous convolution layer can greatly reduce the amount of computation of the second nonlinear transformation convolution layer, while increasing the receptive field of each feature basic unit of the detection image, thereby further improving the accuracy of the i-th level second convolution feature output by the second nonlinear transformation convolution layer.

[0075] The dilated convolution can set a parameter called "dilation rate" in the convolution operation, which defines the spacing between each data when the convolution layer processes data. For example, for a standard convolution layer with a 5*5 convolution kernel size, 25 parameters need to be set; but if a dilated convolution layer with a 3*3 convolution kernel size and a dilation rate of 2 is set, only 9 parameters need to be set, that is, on the basis of a 5*5 convolution kernel, one row of data is deleted every other row and one column of data is deleted every other column. Therefore, under the same computing conditions, the dilated convolution layer can provide a larger receptive field without increasing the amount of computation.

[0076] Here, the hole convolution layer is set in the second depth nonlinear transformation convolution layer. On the basis of the preliminary depth feature extraction performed by the first depth nonlinear transformation convolution layer, the depth feature extraction can be performed again with fewer resources, which can better compensate for the problem of small receptive field in the first feature extraction operation.

[0077] In this step, the deep second nonlinear transformation convolution layer first performs a second feature extraction operation on the i-th level first convolution feature of the detection image, and then the deep second nonlinear transformation convolution layer uses a nonlinear activation function, such as a linear rectification function (ReLU, Rectified Linear Unit), to perform nonlinear processing on the output i-th level second convolution feature to ensure that the output of the deep second nonlinear transformation convolution layer is differentiable, thereby improving the accuracy of subsequent output features.

[0078] In step S304, the image object detection apparatus uses the deep output convolution layer of the deep feature extraction model to perform a dimensionality reduction operation on the i-th level second convolution feature of the detection image obtained in step S303 to obtain the i-th level feature of the detection image.

[0079] The depth output convolution layer is a standard convolution layer with a 1*1 convolution kernel size and no activation function. Here, the depth output convolution layer can restore the dimension added in step S301 to the dimension input to the depth input convolution layer; and no activation function is set in the depth output convolution layer to avoid the loss of output features caused by the activation function. The i-th level feature of the detection image output by the depth output convolution layer should be consistent with the i-th level depth feature map framework.

[0080] In this way, the deep feature extraction of the (i-1)th level feature of the detection image using the i-th level deep feature map framework is completed to obtain the i-th level feature of the detection image. Repeating steps S301 to S304 can obtain level 1 to level n features of the detection image.

[0081] In step S203, the image target detection device performs non-depth feature extraction on the (j-1+n)th level features of the detection image based on a preset non-depth feature extraction model using the j-th level non-depth feature map framework obtained in step S101 to obtain the (j+n)th level features of the detection image, where j is a positive integer less than or equal to m.

[0082] In this embodiment, the non-depth feature extraction model includes a non-depth input convolution layer, a non-depth nonlinear transformation convolution layer, and a non-depth output convolution layer.

[0083] Please refer to Figure 4a and Figure 4b , Figure 4a is a flowchart of step S203 of the second embodiment of the image object detection method of the present invention, Figure 4b This is a schematic diagram of feature extraction of step S203 of the second embodiment of the image object detection method of the present invention. Step S203 includes:

[0084] In step S401, the image object detection apparatus uses the non-depth input convolution layer of the non-depth feature extraction model to perform a dimensionality increase operation on the (j-1+n)th level features of the detection image to obtain the (j+n)th level dimensionality increase features of the detection image.

[0085] The non-depth input convolution layer is a standard convolution layer with a 1*1 convolution kernel size and a nonlinear activation function, and the depth input convolution layer can be set with a larger number of channels, such as 4-6, etc. This can increase the dimension of the input feature while ensuring the feature size of the detection image, thereby solving the problem of feature loss of the detection image.

[0086] The non-depth input convolution layer will then use non-linear activation functions, such as the linear rectification function (ReLU, Rectified Linear Unit), to perform non-linear processing on the output (j+n)th level dimensional features to ensure that the output of the non-depth input convolution layer is differentiable, thereby improving the accuracy of subsequent output features.

[0087] In step S402, the image target detection device uses the non-deep nonlinear transformation convolution layer of the non-deep feature extraction model to perform a feature extraction operation on the (j+n)th level upscaled features of the detection image obtained in step S401 to obtain the (j+n)th level convolution features of the detection image.

[0088] The non-depth non-linear conversion convolution layer is a depth-separable convolution layer with a 3*3 convolution kernel size and a non-linear activation function. The setting of the depth-separable convolution layer can greatly reduce the amount of computation of the non-depth non-linear conversion convolution layer, thereby greatly reducing the size of the deep feature extraction model. The non-depth non-linear conversion convolution layer here can also be a depth-separable hole convolution layer.

[0089] Since the non-depth non-linear conversion convolution layer directly inputs the features output by the depth non-linear conversion convolution layer, the non-depth feature extraction model here only needs to use one non-depth non-linear conversion convolution layer for feature extraction, without setting multiple non-linear conversion convolution layers for feature extraction.

[0090] In this step, the non-depth non-linear conversion convolution layer first performs feature extraction on the (j+n)th level up-dimensional features of the detection image, and then the non-depth non-linear conversion convolution layer uses a non-linear activation function, such as a linear rectification function (ReLU, Rectified Linear Unit), to perform non-linear processing on the output (j+n)th level convolution features to ensure that the output of the non-depth non-linear conversion convolution layer is differentiable, thereby improving the accuracy of subsequent output features.

[0091] In step S403, the image target detection device uses the non-depth output convolution layer of the non-depth feature extraction model to perform a dimensionality reduction operation on the (j+n)th level convolution feature of the detection image obtained in step S402 to obtain the (j+n)th level feature of the detection image.

[0092] The non-depth output convolution layer is a standard convolution layer with a 1*1 convolution kernel size and no activation function. Here, the non-depth output convolution layer can restore the dimension added in step S401 to the dimension input to the non-depth input convolution layer; and no activation function is set in the non-depth output convolution layer to avoid the loss of output features caused by the activation function. The (j+n)th level features of the detection image output by the non-depth output convolution layer should be consistent with the jth level non-depth feature map framework.

[0093] In this way, the non-depth feature extraction of the (j-1+n)th level features of the detection image using the jth level non-depth feature map framework is completed to obtain the (j+n)th level features of the detection image. Repeating steps S401 to S403 can obtain the (n+1)th level features to the (j+n)th level features of the detection image.

[0094] This embodiment uses a deep feature extraction model and a non-deep feature extraction model based on the feature extraction depth of the detected image, which can greatly reduce the computational complexity of the feature extraction operation. Here, n can be set according to the user's requirements. If the computational complexity of the feature extraction operation is large, the size of n can be appropriately reduced. If the accuracy of feature extraction needs to be improved, the size of n can be appropriately increased.

[0095] In step S204, the image target detection device performs information regression operation on the a-th level feature to the (m+n)-th level feature of the detection image obtained in step S202 and step S203 based on the preset feature prediction model, so as to obtain the target type and target position of the detection image, where a is an integer less than n and greater than or equal to 2. Here, the function of the feature prediction model is equivalent to a regressor, which is used to obtain the target type and target position of the target in the detection image. Here, the target type is identified using the classification probability, such as an 80% probability that a certain target is a cat, a 20% probability that a certain target is a dog, etc.

[0096] In this embodiment, the feature prediction model includes a feature classification convolution layer and a feature output convolution layer.

[0097] Please refer to Figure 5a and Figure 5b , Figure 5a is a flowchart of step S204 of the second embodiment of the image object detection method of the present invention, Figure 5b This is a schematic diagram of feature extraction of step S204 of the second embodiment of the image object detection method of the present invention. Step S204 includes:

[0098] In step S501, the image object detection apparatus uses the feature classification convolution layer of the feature prediction model to perform feature extraction operations on the ath level features to the (m+n)th level features of the detection image to obtain classification recognition features of the detection image.

[0099] The feature classification convolution layer is a depth-separable convolution layer with a 3*3 convolution kernel size and no activation function. Since the feature sizes of the level 1 feature to the (a-1) level feature of the detection image are large and generally will not become the image target of the detection, all the previous level features of the a-th level feature of the detection image are discarded here.

[0100] The image target detection device then uses the ath to (m+n)th level features of the detection image to perform feature extraction operations, thereby obtaining classification recognition features of the detection image, so as to perform subsequent prediction operations of the target type and target position of the detection image.

[0101] Specifically, here, according to user needs, some features from the ath level feature to the (m+n)th level feature can be selected to perform feature extraction operations, thereby further reducing the amount of calculation of the feature extraction operation.

[0102] In step S502, the image object detection apparatus uses the feature output convolution layer of the feature prediction model to perform a dimensionality reduction operation on the classification recognition features of the detection image obtained in step S501 to obtain the object type and the object position of the detection image.

[0103] The feature output convolution layer is a standard convolution layer with a 1*1 convolution kernel size and no activation function. No activation function is set in the feature output convolution layer to avoid the loss of output features caused by the activation function.

[0104] The target type output here can be objects such as people, cars, houses, etc., and the output target position can be the center coordinates of the target and parameters such as the length and width of the target box.

[0105] This completes the image object detection process of the image object detection method of this embodiment.

[0106] On the basis of the first embodiment, the deep feature extraction model and the non-deep feature extraction model in the image target detection method of this embodiment adopt different structures, and the first nonlinear transformation convolution layer and the second nonlinear transformation convolution layer in the deep feature extraction model also adopt different results. This can maximize the extraction speed of the target features of the detection image and further reduce the demand for configuration resources, thereby realizing the implementation of the target detection function on the mobile terminal.

[0107] The present invention also provides an image target detection device, please refer to Figure 6 , Figure 6 Schematic diagram of the structure of the first embodiment of the image target detection device of the present invention. The image target detection device of this embodiment can be implemented using the first embodiment of the above-mentioned image target detection method. The image target detection device 60 of this embodiment includes an image and frame acquisition module 61, a depth feature extraction module 62, a non-depth feature extraction module 63 and a target detection module 64.

[0108] The image and framework acquisition module 61 is used to acquire a detection image, an n-level deep feature map framework and an m-level non-depth feature map framework, where n is an integer greater than or equal to 2, and m is an integer greater than or equal to 1; wherein the feature map framework includes the output feature size and dimension; the depth feature extraction module 62 is used to perform deep feature extraction on the (i-1)-th level features of the detection image based on the depth feature extraction model using the i-th level deep feature map framework to obtain the i-th level features of the detection image, where i is a positive integer less than or equal to n; the non-depth feature extraction module 63 is used to perform non-depth feature extraction on the (j-1+n)-th level features of the detection image based on the non-depth feature extraction model using the j-th level non-depth feature map framework to obtain the (j+n)-th level features of the detection image, where j is a positive integer less than or equal to m; the target detection module 64 is used to perform information regression operations on the a-th level features to the (m+n)-th level features of the detection image based on the feature prediction model, so as to obtain the target type and target position of the detection image, where a is an integer less than n and greater than or equal to 2.

[0109] When the image target detection device 60 of this embodiment is used, first, the image and framework acquisition module 61 acquires the detection image that needs to be detected, as well as the n-level depth feature map framework and the m-level non-depth feature map framework for target detection on the detection image. In this embodiment, n is an integer greater than or equal to 2, and m is an integer greater than or equal to 1. That is, the detection image must be subjected to at least 3 feature extraction operations.

[0110] In order to perform accurate and comprehensive target detection on the detection image, it is necessary to perform multi-level feature extraction operations on the detection image, such as m+n levels. Since the feature size of the next level must be smaller than the feature size of the previous level, the feature extraction operation of the next level can be performed on the features output by the previous feature extraction operation. For large-size feature extraction operations, since fewer previous feature extraction operations have been performed, deep feature extraction models and deep feature map frameworks are required for feature extraction. For small-size feature extraction operations, since multiple previous feature extraction operations have been performed, only non-deep feature extraction models and non-deep feature map frameworks are required for feature extraction.

[0111] Here, the deep feature map framework is an identification parameter for feature recognition of the detection image or the lower-level features corresponding to the detection image, and the deep feature map framework may include the feature size and dimension output at each deep feature level. The non-depth feature framework is an identification parameter for feature recognition of the lower-level features corresponding to the detection image, and the non-depth feature map framework may include the feature size and dimension output at each non-depth feature level.

[0112] Then, the depth feature extraction module 62 performs depth feature extraction on the (i-1)th level features of the detection image based on a preset depth feature extraction model using the i-th level depth feature map framework to obtain the i-th level features of the detection image, where i is a positive integer less than or equal to n.

[0113] That is, the depth feature extraction module 62 performs depth feature extraction on the pixels of the detection image based on the preset depth feature extraction model to obtain the level 1 features of the detection image corresponding to the level 1 depth feature map framework; then the depth feature extraction module performs depth feature extraction on the level 1 features of the detection image to obtain the level 2 features of the detection image corresponding to the level 2 depth feature map framework...; finally, the image target detection device performs depth feature extraction on the (n-1) level features of the detection image to obtain the level n features of the detection image corresponding to the level n depth feature map framework. In this way, the level 1 features to the level n features of the detection image are obtained.

[0114] Then, the non-depth feature extraction module 63 performs non-depth feature extraction on the (j-1+n)th level features of the detection image based on a preset non-depth feature extraction model and uses the jth level non-depth feature map framework to obtain the (j+n)th level features of the detection image, where j is a positive integer less than or equal to m.

[0115] That is, the non-depth feature extraction module 63 performs non-depth feature extraction on the n-level features of the detection image based on the preset non-depth feature extraction model to obtain the (n+1)-level features of the detection image corresponding to the 1-level non-depth feature map framework; then the non-depth feature extraction module performs non-depth feature extraction on the (n+1)-level features of the detection image to obtain the (n+2)-level features of the detection image corresponding to the 2-level non-depth feature map framework...; finally, the image target detection device performs depth feature extraction on the (n+m-1)-level features of the detection image to obtain the (m+n)-th level features of the detection image corresponding to the (m+n)-th level depth feature map framework. In this way, the (n+1)-th level features to the (m+n)-th level features of the detection image are obtained.

[0116] Finally, the target detection module 64 performs information regression operations on the ath level features to the (m+n)th level features of the detection image based on a preset feature prediction model, thereby obtaining the target type and target position of the detection image, where a is an integer less than n and greater than or equal to 2.

[0117] Specifically, since the feature sizes of the level 1 features to the level (a-1) features of the detection image are relatively large, it is meaningless to perform feature classification and recognition, so the target detection module 64 directly discards the level 1 features to the level (a-1) features of the detection image. The target detection module 64 performs feature classification and recognition on the level a features to the level (m+n) features of the detection image, thereby obtaining the target type (such as a person, a car, a house, etc.) and the target position (such as the center coordinates of the target and the length and width of the target box, etc.) of the detection image corresponding to the feature.

[0118] This completes the image object detection process of the image object detection device 60 of this embodiment.

[0119] The image target detection device of this embodiment extracts and recognizes features of multiple different sizes of the same detection image based on a deep feature extraction model and a non-deep feature extraction model. Since the small-size features of the detection image can be directly extracted based on the large-size features of the detection image, the overall feature extraction speed is faster and the demand for configuration resources is lower.

[0120] Please refer to Figure 7 , Figure 7 Schematic diagram of the structure of the second embodiment of the image target detection device of the present invention. The image target detection device of this embodiment can be implemented using the second embodiment of the above-mentioned image target detection method. The image target detection device 70 of this embodiment includes an image and frame acquisition module 71, a depth feature extraction module 72, a non-depth feature extraction module 73 and a target detection module 74.

[0121] The image and framework acquisition module 71 is used to acquire a detection image, an n-level depth feature map framework and an m-level non-depth feature map framework, wherein the feature map framework includes an output feature size and dimension; the depth feature extraction module 72 is used to perform depth feature extraction on the (i-1)th level features of the detection image based on a depth feature extraction model using the i-th level depth feature map framework to obtain the i-th level features of the detection image; the non-depth feature extraction module 73 is used to perform non-depth feature extraction on the (j-1+n)th level features of the detection image based on a non-depth feature extraction model using the j-th level non-depth feature map framework to obtain the (j+n)th level features of the detection image; the target detection module 74 is used to perform information regression operations on the a-th level features to the (m+n)-th level features of the detection image based on a feature prediction model, thereby obtaining the target type and target position of the detection image.

[0122] Please refer to Figure 8 , Figure 8The structure diagram of the depth feature extraction module of the second embodiment of the image object detection device of the present invention is shown in FIG. The depth feature extraction module 72 includes a depth dimension increase operation unit 81 , a first depth feature extraction unit 82 , a second depth feature extraction unit 83 and a depth dimension reduction operation unit 84 .

[0123] The deep dimension increasing operation unit 81 is used to use the deep input convolution layer to perform a dimension increasing operation on the i-th level feature of the detection image to obtain the i-th level increased dimension feature of the detection image; the first deep feature extraction unit 82 is used to use the deep first nonlinear transformation convolution layer to perform a first feature extraction operation on the (i-1)-th level increased dimension feature of the detection image to obtain the i-th level first convolution feature of the detection image; the second deep feature extraction unit 83 uses the deep second nonlinear transformation convolution layer to perform a second feature extraction operation on the i-th level first convolution feature of the detection image to obtain the i-th level second convolution feature of the detection image; the deep dimension reduction operation unit 84 is used to use the deep output convolution layer to perform a dimension reduction operation on the i-th level second convolution feature of the detection image to obtain the i-th level feature of the detection image.

[0124] Please refer to Fig. 9 , Fig. 9 The non-depth feature extraction module 73 of the second embodiment of the image object detection device of the present invention comprises a non-depth dimension increasing operation unit 91, a non-depth feature extraction unit 92 and a non-depth dimension reducing operation unit 93.

[0125] The non-depth dimension increasing operation unit 91 is used to use the non-depth input convolution layer to perform a dimension increasing operation on the (j-1+n)th level features of the detection image to obtain the (j+n)th level dimension increasing features of the detection image; the non-depth feature extraction unit 92 is used to use the non-depth nonlinear transformation convolution layer to perform a feature extraction operation on the (j+n)th level dimension increasing features of the detection image to obtain the (j+n)th level convolution features of the detection image; the non-depth dimension reduction operation unit 93 is used to use the non-depth output convolution layer to perform a dimension reduction operation on the (j+n)th level convolution features of the detection image to obtain the (j+n)th level features of the detection image.

[0126] Please refer to Fig.10 , Fig.10 The target detection module 74 of the second embodiment of the image target detection device of the present invention comprises a feature classification unit 101 and a feature output unit 102 .

[0127] The feature classification unit 101 is used to use the feature classification convolution layer to perform feature extraction operations on the a-th level features to the (m+n)-th level features of the detection image to obtain classification recognition features of the detection image; the feature output unit 102 is used to use the feature output convolution layer to perform dimensionality reduction operations on the classification recognition features of the detection image to obtain the target type and position of the detection image.

[0128] When the image target detection device 70 of this embodiment is used, first, the image and framework acquisition module 71 acquires the detection image that needs to be detected, as well as the n-level depth feature map framework and the m-level non-depth feature map framework for target detection on the detection image. In this embodiment, n is an integer greater than or equal to 2, and m is an integer greater than or equal to 1. That is, the detection image must be subjected to at least 3 feature extraction operations.

[0129] Here, the deep feature map framework is an identification parameter for feature recognition of the detection image or the lower-level features corresponding to the detection image, and the deep feature map framework may include the feature size and dimension output at each deep feature level. The non-depth feature framework is an identification parameter for feature recognition of the lower-level features corresponding to the detection image, and the non-depth feature map framework may include the feature size and dimension output at each non-depth feature level.

[0130] Then, the depth feature extraction module 72 performs depth feature extraction on the (i-1)th level features of the detection image based on a preset depth feature extraction model and uses the i-th level depth feature map framework to obtain the i-th level features of the detection image, where i is a positive integer less than or equal to n.

[0131] In this embodiment, the deep feature extraction model includes a deep input convolution layer, a deep first nonlinear transformation convolution layer, a deep second nonlinear transformation convolution layer and a deep output convolution layer.

[0132] The specific deep feature extraction process includes:

[0133] The deep dimension upscaling operation unit 81 of the deep feature extraction module 72 uses the deep input convolution layer of the deep feature extraction model to perform a dimension upscaling operation on the (i-1)th level features of the detection image to obtain the i-th level upscaling feature of the detection image.

[0134] The deep input convolution layer is a standard convolution layer with a 1*1 convolution kernel size and a nonlinear activation function, and the deep input convolution layer can be set with a larger number of channels, such as 4-6. This can increase the dimension of the input feature while ensuring the feature size of the detection image, thereby solving the problem of feature loss of the detection image.

[0135] The number of channels of the deep input convolution layer is used to represent the number of feature extraction patterns for feature extraction from low-level features of the detection image, and the convolution kernel size of the deep input convolution layer is used to adjust the complexity of the deep neural network model.

[0136] For example, the (i-1)th level feature of the input detection image is a 32*32*3 feature point matrix, where 3 is the number of input channels of the detection image, such as the red pixel brightness value, the blue pixel brightness value, and the green pixel brightness value; if the convolution kernel size of the deep input convolution layer is set to 1*1, the output feature size of the deep input convolution layer is 32*32, that is, a 32*32 feature map can be obtained by traversing the 32*32 feature point matrix in sequence using a 1*1 convolution kernel. If the number of channels of the deep input convolution layer is 6, the output of the deep input convolution layer is a 32*32*18 i-th level dimension-raising feature. In this way, the dimension-raising feature of the detection image with a higher dimension is obtained without changing the output feature size.

[0137] The deep input convolution layer will then use nonlinear activation functions, such as the linear rectification function (ReLU, Rectified Linear Unit), to perform nonlinear processing on the i-th level dimensional features of the output to ensure that the output of the deep input convolution layer is differentiable, thereby improving the accuracy of subsequent output features.

[0138] The first deep feature extraction unit 82 of the deep feature extraction module 72 uses the deep first nonlinear transformation convolution layer of the deep feature extraction model to perform a first feature extraction operation on the i-th level up-dimensional feature of the detection image to obtain the i-th level first convolution feature of the detection image.

[0139] The first nonlinear transformation convolution layer is a depth-wise separable convolution layer with a 3*3 convolution kernel size and a nonlinear activation function. The setting of the depth-wise separable convolution layer can greatly reduce the amount of computation of the first nonlinear transformation convolution layer, thereby greatly reducing the size of the deep feature extraction model.

[0140] Among them, the depthwise separable convolution layer can realize spatial convolution while maintaining channel separation. For example, for a standard convolution layer with a 3*3 convolution kernel size, the number of input channels is 16 and the number of output channels is 32. Then, 32 3*3 convolution kernels traverse each data in the 16 channels, and 16*32*3*3=4608 parameters need to be set for convolution operation. For example, for a depthwise separable convolution layer with a 3*3 convolution kernel size, a 3*3 convolution kernel is used to traverse the data of 16 channels to obtain 16 feature maps, and then 32 1*1 convolution kernels are used to traverse these 16 feature maps. In this way, only 16*3*3+16+32+1+1=656 parameters need to be set to complete the convolution operation.

[0141] The deep first nonlinear transformation convolution layer first performs the first feature extraction operation on the i-th level up-dimensional features of the detection image. Then the deep first nonlinear transformation convolution layer uses a nonlinear activation function, such as a linear rectification function (ReLU, Rectified Linear Unit), to perform nonlinear processing on the output i-th level first convolution features to ensure that the output of the deep first nonlinear transformation convolution layer is differentiable, thereby improving the accuracy of subsequent output features.

[0142] The second deep feature extraction unit 83 of the deep feature extraction module 72 uses the deep second nonlinear transformation convolution layer of the deep feature extraction model to perform a second feature extraction operation on the i-th level first convolution feature of the detection image to obtain the i-th level second convolution feature of the detection image.

[0143] The deep second nonlinear transformation convolution layer is a deep separable atrous convolution layer (atrous convolutions) with a 3*3 convolution kernel size and a nonlinear activation function. The setting of the deep separable atrous convolution layer can greatly reduce the amount of computation of the second nonlinear transformation convolution layer, while increasing the receptive field of each feature basic unit of the detection image, thereby further improving the accuracy of the i-th level second convolution feature output by the second nonlinear transformation convolution layer.

[0144] The dilated convolution can set a parameter called "dilation rate" in the convolution operation, which defines the spacing between each data when the convolution layer processes data. For example, for a standard convolution layer with a 5*5 convolution kernel size, 25 parameters need to be set; but if a dilated convolution layer with a 3*3 convolution kernel size and a dilation rate of 2 is set, only 9 parameters need to be set, that is, on the basis of a 5*5 convolution kernel, one row of data is deleted every other row and one column of data is deleted every other column. Therefore, under the same computing conditions, the dilated convolution layer can provide a larger receptive field without increasing the amount of computation.

[0145] Here, the hole convolution layer is set in the second depth nonlinear transformation convolution layer. On the basis of the preliminary depth feature extraction performed by the first depth nonlinear transformation convolution layer, the depth feature extraction can be performed again with fewer resources, which can better compensate for the problem of small receptive field in the first feature extraction operation.

[0146] In this step, the deep second nonlinear transformation convolution layer first performs a second feature extraction operation on the i-th level first convolution feature of the detection image, and then the deep second nonlinear transformation convolution layer uses a nonlinear activation function, such as a linear rectification function (ReLU, Rectified Linear Unit), to perform nonlinear processing on the output i-th level second convolution feature to ensure that the output of the deep second nonlinear transformation convolution layer is differentiable, thereby improving the accuracy of subsequent output features.

[0147] The deep dimension reduction operation unit 84 of the deep feature extraction module 72 uses the deep output convolution layer of the deep feature extraction model to perform a dimension reduction operation on the i-th level second convolution feature of the detection image to obtain the i-th level feature of the detection image.

[0148] The depth output convolution layer is a standard convolution layer with a 1*1 convolution kernel size and no activation function. Here, the depth output convolution layer can restore the increased dimension to the dimension input to the depth input convolution layer; and no activation function is set in the depth output convolution layer to avoid the loss of output features caused by the activation function. The i-th level feature of the detection image output by the depth output convolution layer should be consistent with the i-th level deep feature map framework.

[0149] In this way, the deep feature extraction of the (i-1)th level feature of the detection image is completed using the i-th level deep feature map framework to obtain the i-th level feature of the detection image. Repeating the above-mentioned dimensionality increase operation, the first feature extraction operation, the second feature extraction operation and the dimensionality reduction operation, the level 1 feature to the level n feature of the detection image can be obtained.

[0150] Then, the non-depth feature extraction module 73 performs non-depth feature extraction on the (j-1+n)th level features of the detection image based on a preset non-depth feature extraction model and uses the jth level non-depth feature map framework to obtain the (j+n)th level features of the detection image, where j is a positive integer less than or equal to m.

[0151] In this embodiment, the non-depth feature extraction model includes a non-depth input convolution layer, a non-depth nonlinear transformation convolution layer, and a non-depth output convolution layer.

[0152] The specific non-deep feature extraction process includes:

[0153] The non-depth dimension upgrading operation unit 91 of the non-depth feature extraction module 73 uses the non-depth input convolution layer of the non-depth feature extraction model to perform a dimension upgrading operation on the (j-1+n)th level features of the detection image to obtain the (j+n)th level dimension upgrading features of the detection image.

[0154] The non-depth input convolution layer is a standard convolution layer with a 1*1 convolution kernel size and a nonlinear activation function, and the depth input convolution layer can be set with a larger number of channels, such as 4-6, etc. This can increase the dimension of the input feature while ensuring the feature size of the detection image, thereby solving the problem of feature loss of the detection image.

[0155] The non-depth input convolution layer will then use non-linear activation functions, such as the linear rectification function (ReLU, Rectified Linear Unit), to perform non-linear processing on the output (j+n)th level dimensional features to ensure that the output of the non-depth input convolution layer is differentiable, thereby improving the accuracy of subsequent output features.

[0156] The non-depth feature extraction unit 92 of the non-depth feature extraction module 73 uses the non-depth nonlinear transformation convolution layer of the non-depth feature extraction model to perform feature extraction operations on the (j+n)th level up-dimensional features of the detection image to obtain the (j+n)th level convolution features of the detection image.

[0157] The non-depth non-linear conversion convolution layer is a depth-separable convolution layer with a 3*3 convolution kernel size and a non-linear activation function. The setting of the depth-separable convolution layer can greatly reduce the amount of computation of the non-depth non-linear conversion convolution layer, thereby greatly reducing the size of the deep feature extraction model. The non-depth non-linear conversion convolution layer here can also be a depth-separable hole convolution layer.

[0158] Since the non-depth non-linear conversion convolution layer directly inputs the features output by the depth non-linear conversion convolution layer, the non-depth feature extraction model here only needs to use one non-depth non-linear conversion convolution layer for feature extraction, without setting multiple non-linear conversion convolution layers for feature extraction.

[0159] In this step, the non-depth non-linear conversion convolution layer first performs feature extraction on the (j+n)th level up-dimensional features of the detection image, and then the non-depth non-linear conversion convolution layer uses a non-linear activation function, such as a linear rectification function (ReLU, Rectified Linear Unit), to perform non-linear processing on the output (j+n)th level convolution features to ensure that the output of the non-depth non-linear conversion convolution layer is differentiable, thereby improving the accuracy of subsequent output features.

[0160] The non-depth dimension reduction operation unit 93 of the non-depth feature extraction module 71 uses the non-depth output convolution layer of the non-depth feature extraction model to perform a dimension reduction operation on the (j+n)th level convolution feature of the detection image to obtain the (j+n)th level feature of the detection image.

[0161] The non-depth output convolution layer is a standard convolution layer with a 1*1 convolution kernel size and no activation function. Here, the non-depth output convolution layer can restore the previously added dimension to the dimension input to the non-depth input convolution layer; and no activation function is set in the non-depth output convolution layer to avoid the loss of output features caused by the activation function. The (j+n)th level features of the detection image output by the non-depth output convolution layer should be consistent with the jth level non-depth feature map framework.

[0162] In this way, the non-depth feature extraction of the (j-1+n)th level features of the detection image is completed using the jth level non-depth feature map framework to obtain the (j+n)th level features of the detection image. Repeating the above-mentioned dimensionality increase operation, feature extraction operation and dimensionality reduction operation, the (n+1)th level features to the (j+n)th level features of the detection image can be obtained.

[0163] This embodiment uses a deep feature extraction model and a non-deep feature extraction model based on the feature extraction depth of the detected image, which can greatly reduce the computational complexity of the feature extraction operation. Here, n can be set according to the user's requirements. If the computational complexity of the feature extraction operation is large, the size of n can be appropriately reduced. If the accuracy of feature extraction needs to be improved, the size of n can be appropriately increased.

[0164] Finally, the target detection module 74 performs information regression operations on the a-th level features to the (m+n)-th level features of the detection image based on the preset feature prediction model, so as to obtain the target type and target position of the detection image, where a is an integer less than n and greater than or equal to 2. Here, the function of the feature prediction model is equivalent to a regressor, which is used to obtain the target type and target position of the target in the detection image. Here, the target type is identified by the classification probability, such as an 80% probability that a certain target is a cat, a 20% probability that it is a dog, etc.

[0165] In this embodiment, the feature prediction model includes a feature classification convolution layer and a feature output convolution layer.

[0166] The specific target detection process includes:

[0167] The feature classification unit 101 of the target detection module 74 uses the feature classification convolution layer of the feature prediction model to perform feature extraction operations on the ath level features to the (m+n)th level features of the detection image to obtain classification recognition features of the detection image.

[0168] The feature classification convolution layer is a depth-separable convolution layer with a 3*3 convolution kernel size and no activation function. Since the feature sizes of the level 1 feature to the (a-1) level feature of the detection image are large and generally will not become the image target of the detection, all the previous level features of the a-th level feature of the detection image are discarded here.

[0169] The feature classification unit 101 uses the ath to (m+n)th level features of the detection image to perform feature extraction operations, thereby obtaining classification recognition features of the detection image, so as to perform subsequent prediction operations of the target type and target position of the detection image.

[0170] Specifically, here the feature classification unit may select some features from the ath level feature to the (m+n)th level feature to perform feature extraction operation according to user needs, thereby further reducing the computational complexity of the feature extraction operation.

[0171] The feature output unit 102 of the target detection module 74 uses the feature output convolution layer of the feature prediction model to perform a dimensionality reduction operation on the classification recognition features of the detection image to obtain the target type and target position of the detection image.

[0172] The feature output convolution layer is a standard convolution layer with a 1*1 convolution kernel size and no activation function. No activation function is set in the feature output convolution layer to avoid the loss of output features caused by the activation function.

[0173] The target type output here can be objects such as people, cars, houses, etc., and the output target position can be the center coordinates of the target and parameters such as the length and width of the target box.

[0174] This completes the image object detection process of the image object detection device 70 of this embodiment.

[0175] On the basis of the first embodiment, the deep feature extraction model and the non-deep feature extraction model in the image target detection device of this embodiment adopt different structures, and the first nonlinear transformation convolution layer and the second nonlinear transformation convolution layer in the deep feature extraction model also adopt different results. This can maximize the extraction speed of the target features of the detection image and further reduce the demand for configuration resources, thereby realizing the implementation of the target detection function on the mobile terminal.

[0176] The following is a specific example to illustrate the working principle of the image target detection method and the image target detection device of the present invention. Fig.11 , Fig.11 It is a schematic diagram of the use of a specific embodiment of the image object detection method and the image object detection device of the present invention.

[0177] The image target detection device of this specific embodiment can be set in a mobile terminal installed with an image target recognition application. The mobile terminal can quickly extract target features in the image and has low requirements for the configuration resources of the mobile terminal itself. The steps of performing image target detection by the image target detection device of this specific embodiment include:

[0178] Step S1101, obtain the detection image, n-level depth feature map framework and m-level non-depth feature map framework. In this embodiment, the depth feature map framework includes feature sizes such as 128*128*12, 64*64*24, 32*32*48, 16*16*64 and different levels of depth feature map frameworks of latitude. Here, 128*128 refers to the feature size of the feature map framework, and 12 refers to the dimension of the feature map framework. Of course, more depth feature map frameworks can be included here, such as adding depth feature map frameworks such as 64*64*32 and 64*64*40 between 64*64*24 and 32*32*48. In this embodiment, 4-level depth feature map frameworks and 4-level non-depth feature map frameworks are obtained.

[0179] Step S1102, based on the deep feature extraction model, use the i-th level deep feature map framework to perform deep feature extraction on the (i-1)-th level features of the detection image to obtain the i-th level features of the detection image. For example, in this embodiment, 4 levels of deep features are set for the detection image. In actual use, the number of levels of deep features of the detection image should be greater than 4.

[0180] Based on the deep feature extraction model, the image object detection device performs deep feature extraction on the pixels of the detection image (the pixel size and dimension are 256*256*3) to obtain the level 1 features of the detection image corresponding to the level 1 deep feature map framework (the feature size and dimension are 128*128*12); then the image object detection device performs deep feature extraction on the level 1 features of the detection image to obtain the level 2 features of the detection image corresponding to the level 2 deep feature map framework (the feature size and dimension are 64*64*24); then the image object detection device performs deep feature extraction on the level 2 features of the detection image to obtain the level 3 features of the detection image corresponding to the level 3 deep feature map framework (the feature size and dimension are 32*32*48); finally, the image object detection device performs deep feature extraction on the level 3 features of the detection image to obtain the level 4 features of the detection image corresponding to the level 4 deep feature map framework (the feature size and dimension are 16*16*64).

[0181] Here, we take the three-level features of the detection image as an example to explain how to extract deep features from the detection image. The process of deep feature extraction includes:

[0182] The image object detection device uses a deep input convolution layer to perform a dimensionality increase operation on the 2nd level features of the detection image (the feature size and dimension is 64*64*24) to obtain the 3rd level dimensionality increase feature of the detection image (the feature size and dimension is 64*64*144). Here, the deep input convolution layer is a standard convolution layer Conv1 with a 1*1 convolution kernel size and a nonlinear activation function Relu.

[0183] The image target detection device uses the first nonlinear conversion convolution layer to perform the first feature extraction operation on the three-level dimensional features of the detection image to obtain the three-level first convolution features of the detection image (the feature size and dimension are 32*32*144). The first nonlinear conversion convolution layer here is a depth-separable standard convolution layer Dwise2 with a 3*3 convolution kernel size and a nonlinear activation function Relu. Since the size of the three-level first convolution feature is reduced, the convolution step stride of the depth-separable standard convolution layer Dwise2 here is 2.

[0184] The image object detection device uses the second nonlinear transformation convolution layer to perform a second feature extraction operation on the 3-level first convolution feature of the detection image to obtain the 3-level second convolution feature of the detection image (the feature size and dimension is 32*32*144). The second nonlinear transformation convolution layer here is a depth-separable dilated convolution layer Dwise3 with a 3*3 convolution kernel size and a nonlinear activation function Relu, wherein the expansion rate of the depth-separable dilated convolution layer Dwise3 is 2.

[0185] The image object detection device uses a deep output convolution layer to perform a dimensionality reduction operation on the 3rd level second convolution feature of the detection image to obtain the 3rd level feature of the detection image (the feature size and dimension is 32*32*48). The deep output convolution layer here is a standard convolution layer Conv4 with a 1*1 convolution kernel size and no activation function.

[0186] Step S1103, based on the non-depth feature extraction model, use the j-th level non-depth feature map framework to extract the (j-1+n)-th level features of the detection image to obtain the (j+n)-th level features of the detection image. For example, in this embodiment, the detection image is set with 4 levels of non-depth features, that is, the detection image is set with a total of 8 levels of features. In actual use, the number of non-depth feature levels of the detection image should be greater than 4.

[0187] Based on the non-depth feature extraction model, the image object detection device performs non-depth feature extraction on the 4-level features of the detection image to obtain the 5-level features of the detection image corresponding to the 1-level non-depth feature map framework (the feature size and dimension are 8*8*144); then the image object detection device performs non-depth feature extraction on the 5-level features of the detection image to obtain the 6-level features of the detection image corresponding to the 2-level non-depth feature map framework (the feature size and dimension are 4*4*256); then the image object detection device performs non-depth feature extraction on the 6-level features of the detection image to obtain the 7-level features of the detection image corresponding to the 3-level non-depth feature map framework (the feature size and dimension are 2*2*256); finally, the image object detection device performs non-depth feature extraction on the 7-level features of the detection image to obtain the 8-level features of the detection image corresponding to the 4-level non-depth feature map framework (the feature size and dimension are 1*1*256).

[0188] Here, we take the 7-level features of the detection image as an example to explain how to extract non-depth features from the detection image. The process of non-depth feature extraction includes:

[0189] The image object detection device uses a non-depth input convolution layer to perform a dimensionality increase operation on the 6-level features of the detection image (the feature size and dimension are 4*4*256) to obtain the 7-level dimensionality increase feature of the detection image (the feature size and dimension are 4*4*1536). The non-depth input convolution layer here is a standard convolution layer Conv5 with a 1*1 convolution kernel size and a non-linear activation function Relu.

[0190] The image target detection device uses a non-deep nonlinear conversion convolution layer to perform feature extraction operations on the 7-level dimensional features of the detection image to obtain the 7-level convolution features of the detection image (the feature size and dimension is 2*2*1536). The non-deep nonlinear conversion convolution layer here is a depth-separable dilated convolution layer Dwise6 with a 3*3 convolution kernel size and a nonlinear activation function Relu. Since the size of the 7-level convolution features is reduced, the convolution step stride of the depth-separable convolution layer Dwise6 here is 2, and the expansion rate of the depth-separable dilated convolution layer Dwise6 is 2.

[0191] The image object detection device uses a non-depth output convolution layer to perform a dimensionality reduction operation on the 7-level convolution features of the detection image to obtain the 7-level features of the detection image (the feature size and dimension is 2*2*256). The non-depth output convolution layer here is a standard convolution layer Conv7 with a 1*1 convolution kernel size and no activation function.

[0192] Step S1104: Based on the feature prediction model, information regression operation is performed on the level 3 to level 8 features of the detection image, so as to obtain the target type and target position of the detection image. The process of obtaining the target type and target position of the detection image includes:

[0193] The image object detection device uses a feature classification convolution layer to perform information regression operations on the level 3 to level 8 features of the detection image to obtain the classification recognition features of the detection image. The feature classification convolution layer is a depth-separable convolution layer with a 3*3 convolution kernel size and no activation function.

[0194] The image object detection device uses a feature output convolution layer to perform a dimensionality reduction operation on the classification recognition features of the detection image to obtain the target type and target position of the detection image. The feature output convolution layer is a standard convolution layer with a 1*1 convolution kernel size and no activation function.

[0195] This completes the output operation of the target type and target position in the image, such as Fig.11 As shown in 1101.

[0196] The target detection process of the image target detection method and the image target detection device of this specific embodiment simultaneously optimizes the three parts of the deep feature extraction model, the non-deep feature extraction model and the feature prediction model, so that the original feature extraction model is reduced from 100Mb to less than 1Mb, and the operating speed is also improved by more than 10 times.

[0197] The image target detection method, device and storage medium of the present invention extract and identify features of multiple different sizes of the same detection image based on a deep feature extraction model and a non-deep feature extraction model. Since the small-size features of the detection image can be directly extracted based on the large-size features of the detection image, the overall feature extraction speed is fast and the demand for configuration resources is low; the technical problems that the existing image target detection methods and devices have slow running speeds and cannot be implemented on mobile terminals with small resource configurations are effectively solved.

[0198] As used herein, the terms "component," "module," "system," "interface," "process," and the like are generally intended to refer to a computer-related entity: hardware, a combination of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable application, a thread of execution, a program, and / or a computer. By way of illustration, both an application running on a controller and the controller can be components. One or more components can be within a process and / or thread of execution, and a component can be located on one computer and / or distributed between two or more computers.

[0199] Fig.12 The following discussion provides a brief, general description of the working environment of the electronic device in which the image object detection apparatus described in the present invention is implemented. Figure 8 The working environment is merely one example of a suitable working environment and is not intended to suggest any limitation as to the scope of use or functionality of the working environment. Example electronic devices 1212 include, but are not limited to, wearable devices, head mounted devices, healthcare platforms, personal computers, server computers, handheld or laptop devices, mobile devices (such as mobile phones, personal digital assistants (PDAs), media players, etc.), multi-processor systems, consumer electronic devices, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like.

[0200] Although not required, embodiments are described in the general context of "computer readable instructions" being executed by one or more electronic devices. Computer readable instructions may be distributed via computer readable media (discussed below). Computer readable instructions may be implemented as program modules, such as functions, objects, application programming interfaces (APIs), data structures, etc. that perform specific tasks or implement specific abstract data types. Typically, the functionality of the computer readable instructions may be arbitrarily combined or distributed in a variety of environments.

[0201] Fig.12 An example of an electronic device 1212 including one or more embodiments of the image object detection apparatus of the present invention is illustrated. In one configuration, the electronic device 1212 includes at least one processing unit 1216 and a memory 1218. Depending on the exact configuration and type of the electronic device, the memory 1218 may be volatile (such as RAM), non-volatile (such as ROM, flash memory, etc.), or some combination of the two. Fig.12 This is illustrated by dashed line 1214.

[0202] In other embodiments, the electronic device 1212 may include additional features and / or functionality. For example, the device 1212 may also include additional storage devices (e.g., removable and / or non-removable), including but not limited to magnetic storage devices, optical storage devices, etc. Such additional storage devices may be stored in the form of a memory card or a memory card. Fig.12 12 is illustrated by storage device 1220. In one embodiment, computer readable instructions for implementing one or more embodiments provided herein may be in storage device 1220. Storage device 1220 may also store other computer readable instructions for implementing an operating system, application programs, etc. The computer readable instructions may be loaded into memory 1218 and executed by, for example, processing unit 1216.

[0203] The term "computer-readable media" as used herein includes computer storage media. Computer storage media include volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storing information such as computer-readable instructions or other data. Memory 1218 and storage device 1220 are examples of computer storage media. Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical storage device, cassette, magnetic tape, magnetic disk storage device or other magnetic storage device, or any other medium that can be used to store the desired information and can be accessed by the electronic device 1212. Any such computer storage media can be part of the electronic device 1212.

[0204] The electronic device 1212 may also include a communication connection 1226 that allows the electronic device 1212 to communicate with other devices. The communication connection 1226 may include, but is not limited to, a modem, a network interface card (NIC), an integrated network interface, a radio frequency transmitter / receiver, an infrared port, a USB connection, or other interfaces for connecting the electronic device 1212 to other electronic devices. The communication connection 1226 may include a wired connection or a wireless connection. The communication connection 1226 may transmit and / or receive communication media.

[0205] The term "computer-readable medium" may include communication media. Communication media typically embodies computer-readable instructions or other data in a "modulated data signal" such as a carrier wave or other transport mechanism, and includes any information delivery media. The term "modulated data signal" may include a signal that has one or more of its characteristics set or changed in such a manner as to encode information into the signal.

[0206] The electronic device 1212 may include an input device 1224, such as a keyboard, a mouse, a pen, a voice input device, a touch input device, an infrared camera, a video input device, and / or any other input device. The device 1212 may also include an output device 1222, such as one or more displays, speakers, a printer, and / or any other output device. The input device 1224 and the output device 1222 may be connected to the electronic device 1212 via a wired connection, a wireless connection, or any combination thereof. In one embodiment, an input device or an output device from another electronic device may be used as an input device 1224 or an output device 1222 of the electronic device 1212.

[0207] The components of the electronic device 1212 may be connected via various interconnections (e.g., buses). Such interconnections may include a peripheral component interconnect (PCI) (e.g., PCI Express), a universal serial bus (USB), FireWire (IEEE 1394), an optical bus structure, and the like. In another embodiment, the components of the electronic device 1212 may be interconnected via a network. For example, the memory 1218 may be composed of multiple physical memory units located in different physical locations and interconnected via a network.

[0208] Those skilled in the art will recognize that storage devices for storing computer-readable instructions can be distributed across a network. For example, an electronic device 1230 accessible via network 1228 can store computer-readable instructions for implementing one or more embodiments provided by the present invention. Electronic device 1212 can access electronic device 1230 and download a portion or all of the computer-readable instructions for execution. Alternatively, electronic device 1212 can download multiple computer-readable instructions as needed, or some instructions can be executed at electronic device 1212 and some instructions can be executed at electronic device 1230.

[0209] Various operations of the embodiments are provided herein. In one embodiment, the one or more operations described may constitute computer-readable instructions stored on one or more computer-readable media, which, when executed by an electronic device, will cause a computing device to perform the operations. The order in which some or all operations are described should not be interpreted as implying that these operations must be sequence-dependent. Those skilled in the art will appreciate alternative sequencing with the benefit of this specification. Moreover, it should be understood that not all operations must exist in each embodiment provided herein.

[0210] Moreover, although the present disclosure has been shown and described with respect to one or more implementations, those skilled in the art will think of equivalent variations and modifications based on the reading and understanding of this specification and the accompanying drawings. The present disclosure includes all such modifications and variations, and is limited only by the scope of the appended claims. In particular, with respect to the various functions performed by the above-mentioned components (such as elements, resources, etc.), the terms used to describe such components are intended to correspond to any component (unless otherwise indicated) that performs the specified function of the component (such as it is functionally equivalent), even if the structure is not equivalent to the disclosed structure of the function in the exemplary implementation of the present disclosure shown herein. In addition, although the specific features of the present disclosure have been disclosed with respect to only one of several implementations, such features can be combined with one or more other features of other implementations that may be desired and advantageous for a given or specific application. Moreover, insofar as the terms "including", "having", "containing" or their variations are used in specific embodiments or claims, such terms are intended to be included in a manner similar to the term "comprising".

[0211] The functional units in the embodiments of the present invention may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium. The above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc. The above-mentioned devices or systems may execute the methods in the corresponding method embodiments.

[0212] In summary, although the present invention has been disclosed as above by embodiments, the serial numbers before the embodiments are only used for convenience of description and do not limit the order of the embodiments of the present invention. Moreover, the above embodiments are not used to limit the present invention. A person skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the scope defined by the claims.

Claims

1. A method for detecting an image target, characterized in that: include: Acquire a detection image, an n-level deep feature map framework, and an m-level non-depth feature map framework, where n is an integer greater than or equal to 2, and m is an integer greater than or equal to 1; wherein the n-level deep feature map framework corresponds to an n-layer network of a deep feature extraction model, and each level of the deep feature map framework in the n-level deep feature map framework is used to reflect the feature size and dimension output by the corresponding layer network in the deep feature extraction model, and the deep feature extraction model is a model that performs multiple feature extractions on the detection image; The m-level non-depth feature map framework corresponds to the m-layer network of the non-depth feature extraction model, and each level of the non-depth feature map framework in the m-level non-depth feature map framework is used to reflect the feature size and dimension output by the corresponding layer network in the non-depth feature extraction model; the non-depth feature extraction model is a model that continues to perform multiple feature extractions on the output result of the depth feature extraction model; Based on the deep feature extraction model, use the i-th level deep feature map framework to perform deep feature extraction on the (i-1)-th level feature of the detection image to obtain the i-th level feature of the detection image, wherein i is a positive integer less than or equal to n, the i-th level deep feature map framework is any level of the n-level deep feature map framework, and the deep feature extraction is a feature extraction operation of the deep feature extraction model; Based on the non-depth feature extraction model, use the j-th level non-depth feature map framework to perform non-depth feature extraction on the (j-1+n)-th level features of the detection image to obtain the (j+n)-th level features of the detection image, wherein j is a positive integer less than or equal to m, the j-th level non-depth feature map framework is any level of the m-level depth feature map framework, and the non-depth feature extraction is a feature extraction operation of the non-depth feature extraction model; and Based on the feature prediction model, an information regression operation is performed on multiple features of the detection image: the ath level feature to the (m+n)th level feature, so as to obtain the target type and target position of the detection image, wherein a is an integer less than n and greater than or equal to 2; the information regression operation is performed on multiple features of the detection image: the ath level feature to the (m+n)th level feature, so as to obtain the target type and target position of the detection image, including: Using a feature classification convolution layer, performing feature extraction operations on the a-th level features to the (m+n)-th level features of the detection image to obtain classification recognition features of the detection image; and A feature output convolution layer is used to perform a dimensionality reduction operation on the classification recognition features of the detection image to obtain the target type and position of the detection image.

2. The image target detection method according to claim 1, characterized in that: The deep feature extraction model includes a deep input convolution layer, a deep first nonlinear transformation convolution layer, a deep second nonlinear transformation convolution layer and a deep output convolution layer; The step of performing deep feature extraction on the (i-1)th level features of the detection image using the i-th level deep feature map framework based on the deep feature extraction model to obtain the i-th level features of the detection image comprises: Using the deep input convolutional layer, performing a dimensionality increase operation on the (i-1)th level feature of the detection image to obtain the i-th level dimensionality increase feature of the detection image; Using the deep first nonlinear transformation convolutional layer, performing a first feature extraction operation on the i-th level up-dimensional features of the detection image to obtain the i-th level first convolutional features of the detection image; Using the deep second nonlinear transformation convolutional layer, performing a second feature extraction operation on the i-th level first convolutional feature of the detection image to obtain the i-th level second convolutional feature of the detection image; and Using the deep output convolution layer, a dimensionality reduction operation is performed on the i-th level second convolution feature of the detection image to obtain the i-th level feature of the detection image.

3. The image target detection method according to claim 2, characterized in that: The convolution kernel size of the depth input convolution layer is 1*1, the convolution kernel size of the depth first nonlinear conversion convolution layer is 3*3, the convolution kernel size of the depth second nonlinear conversion convolution layer is 3*3, and the convolution kernel size of the depth output convolution layer is 1*1; The depth input convolution layer is a standard convolution layer with a nonlinear activation function, the depth first nonlinear transformation convolution layer is a depth separable convolution layer with a nonlinear activation function, the depth second nonlinear transformation convolution layer is a depth separable convolution layer with a nonlinear activation function, and the depth output convolution layer is a standard convolution layer without an activation function.

4. The image target detection method according to claim 3, characterized in that: The depth second nonlinear transformation convolution layer is a depth separable hole convolution layer with a nonlinear activation function.

5. The image target detection method according to claim 1, characterized in that: The non-depth feature extraction model includes a non-depth input convolution layer, a non-depth nonlinear conversion convolution layer and a non-depth output convolution layer; The step of extracting non-depth features from the (j-1+n)th level features of the detection image using the jth level non-depth feature map framework based on the non-depth feature extraction model to obtain the (j+n)th level features of the detection image comprises: Using the non-depth input convolutional layer, performing a dimensionality increase operation on the (j-1+n)th level feature of the detection image to obtain the (j+n)th level dimensionality increase feature of the detection image; Using the non-depth nonlinear conversion convolution layer, performing a feature extraction operation on the (j+n)th level up-dimensional features of the detection image to obtain the (j+n)th level convolution features of the detection image; and Using the non-depth output convolution layer, a dimensionality reduction operation is performed on the (j+n)th level convolution features of the detection image to obtain the (j+n)th level features of the detection image.

6. The image target detection method according to claim 5, characterized in that: The convolution kernel size of the non-depth input convolution layer is 1*1, the convolution kernel size of the non-depth nonlinear conversion convolution layer is 3*3, and the convolution kernel size of the non-depth output convolution layer is 1*1; The non-depth input convolution layer is a standard convolution layer with a non-linear activation function, the non-depth non-linear conversion convolution layer is a depth-separable convolution layer with a non-linear activation function, and the non-depth output convolution layer is a standard convolution layer without an activation function.

7. The image target detection method according to claim 6, characterized in that: The non-depth non-linear transformation convolution layer is a depth-separable hole convolution layer with a non-linear activation function.

8. The image target detection method according to claim 1, characterized in that: The convolution kernel size of the feature classification convolution layer is 3*3, and the convolution kernel size of the feature output convolution layer is 1*1; The feature classification convolution layer is a depth-separable convolution layer without an activation function, and the feature output convolution layer is a standard convolution layer without an activation function.

9. An image target detection device, characterized in that: include: An image and framework acquisition module, used to acquire a detection image, an n-level deep feature map framework and an m-level non-deep feature map framework, where n is an integer greater than or equal to 2, and m is an integer greater than or equal to 1; wherein the n-level deep feature map framework corresponds to an n-layer network of a deep feature extraction model, and each level of the deep feature map framework in the n-level deep feature map framework is used to reflect the feature size and dimension output by the corresponding layer network in the deep feature extraction model, and the deep feature extraction model is a model that performs multiple feature extractions on the detection image; The m-level non-depth feature map framework corresponds to the m-layer network of the non-depth feature extraction model, and each level of the non-depth feature map framework in the m-level non-depth feature map framework is used to reflect the feature size and dimension output by the corresponding layer network in the non-depth feature extraction model; the non-depth feature extraction model is a model that continues to perform multiple feature extractions on the output result of the depth feature extraction model; a deep feature extraction module, configured to perform deep feature extraction on the (i-1)th level features of the detection image using the i-th level deep feature map framework based on the deep feature extraction model to obtain the i-th level features of the detection image, wherein i is a positive integer less than or equal to n, the i-th level deep feature map framework is any level of the n-level deep feature map framework, and the deep feature extraction is a feature extraction operation of the deep feature extraction model; a non-depth feature extraction module, configured to perform non-depth feature extraction on the (j-1+n)th level features of the detection image using the jth level non-depth feature map framework based on the non-depth feature extraction model to obtain the (j+n)th level features of the detection image, wherein j is a positive integer less than or equal to m, the jth level non-depth feature map framework is any level of the m-level depth feature map framework, and the non-depth feature extraction is a feature extraction operation of the non-depth feature extraction model; and The target detection module is used to perform information regression operation on multiple features of the detection image: level a features to (m+n) level features based on the feature prediction model, so as to obtain the target type and target position of the detection image, wherein a is an integer less than n and greater than or equal to 2; the information regression operation is performed on multiple features of the detection image: level a features to (m+n) level features, so as to obtain the target type and target position of the detection image, including: Using a feature classification convolution layer, performing feature extraction operations on the a-th level features to the (m+n)-th level features of the detection image to obtain classification recognition features of the detection image; and A feature output convolution layer is used to perform a dimensionality reduction operation on the classification recognition features of the detection image to obtain the target type and position of the detection image.

10. The image object detection device according to claim 9, characterized in that: The deep feature extraction model includes a deep input convolution layer, a deep first nonlinear transformation convolution layer, a deep second nonlinear transformation convolution layer and a deep output convolution layer; The depth feature extraction module comprises: A deep dimension-upgrading operation unit, configured to use the deep input convolution layer to perform a dimension-upgrading operation on the i-th level feature of the detection image to obtain the i-th level dimension-upgrading feature of the detection image; A first deep feature extraction unit, configured to use the deep first nonlinear transformation convolution layer to perform a first feature extraction operation on the (i-1)th level up-dimensional feature of the detection image to obtain an i-th level first convolution feature of the detection image; a second deep feature extraction unit, using the deep second nonlinear transformation convolution layer, to perform a second feature extraction operation on the i-th level first convolution feature of the detection image to obtain the i-th level second convolution feature of the detection image; and A deep dimension reduction operation unit is used to use the deep output convolution layer to perform a dimension reduction operation on the i-th level second convolution feature of the detection image to obtain the i-th level feature of the detection image.

11. The image object detection device according to claim 9, characterized in that: The non-depth feature extraction model includes a non-depth input convolution layer, a non-depth nonlinear conversion convolution layer and a non-depth output convolution layer; The non-depth feature extraction module comprises: a non-depth dimension-raising operation unit, configured to use the non-depth input convolution layer to perform a dimension-raising operation on the (j-1+n)th level feature of the detection image to obtain the (j+n)th level dimension-raising feature of the detection image; a non-depth feature extraction unit, configured to use the non-depth non-linear conversion convolution layer to perform a feature extraction operation on the (j+n)th level up-dimensional features of the detection image to obtain the (j+n)th level convolution features of the detection image; and A non-depth dimensionality reduction operation unit is used to use the non-depth output convolution layer to perform a dimensionality reduction operation on the (j+n)th level convolution feature of the detection image to obtain the (j+n)th level feature of the detection image.

12. The image object detection device according to claim 9, characterized in that: The feature prediction model includes a feature classification convolution layer and a feature output convolution layer; The target detection module comprises: a feature classification unit, configured to use the feature classification convolution layer to perform a feature extraction operation on the ath-level features to the (m+n)th-level features of the detection image to obtain classification recognition features of the detection image; and The feature output unit is used to use the feature output convolution layer to perform a dimensionality reduction operation on the classification recognition features of the detection image to obtain the target type and position of the detection image.

13. The image target detection device according to claim 12, characterized in that: The convolution kernel size of the feature classification convolution layer is 3*3, and the convolution kernel size of the feature output convolution layer is 1*1; The feature classification convolution layer is a depth-separable convolution layer without an activation function, and the feature output convolution layer is a standard convolution layer without an activation function.

14. A storage medium storing processor-executable instructions, wherein the instructions are loaded by one or more processors to execute the image object detection method according to any one of claims 1 to 8.