Target detection method, device and electronic equipment

Abstract

This invention provides a target detection method, apparatus, and electronic device. The method includes: acquiring image data of a target; inputting the image data into a pre-built deep lightweight network to output prediction heads at different scales; and decoding the prediction heads to obtain the target's category, bounding box, and category confidence. This approach can significantly improve the accuracy and inference speed of target detection.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision and intelligent transportation technology, and in particular relates to a target detection method, device and electronic equipment. Background Art

[0002] At present, the new generation of information communication and artificial intelligence technology is developing rapidly. As an important carrier of these new technologies, automobiles are accelerating their transformation to intelligence and networking, and intelligent networked vehicles will become an important symbol and support for a new round of industrial transformation and upgrading. Automobiles are not only a traditional mechanical means of transportation, but in fact, they are also a carrier of modern science and technology. Today, what people call intelligent networked vehicles, mobile robots, mobile information platforms, including electronic line control devices and vehicle networks are all products of the combination of modern technology and automobiles. In the development of intelligent transportation, environmental perception, as a key technology for obtaining real-time road traffic information, is related to whether the intelligent transportation system can make accurate decisions based on current traffic information, and has a great role in alleviating traffic congestion and improving road safety.

[0003] Visual sensors are widely used in the field of intelligent transportation due to their simple structure, easy installation and low cost. There have been many studies on vision-based target detection and tracking technology at home and abroad. In recent years, due to the rapid development of artificial intelligence technology, deep learning-based target detection and tracking technology has become a research hotspot and mainstream direction, which can be mainly divided into two technical routes: CNN-based and Transformer-based. The Transformer-based target detector has high accuracy, but its structure is complex, the computational cost is high, and the hardware requirements are high. There are many challenges in the process of algorithm application, and it is difficult to use it in engineering applications on a large scale at this stage; the CNN-based target detector model has a simple structure and good portability, but its accuracy and reasoning speed are still difficult to meet the requirements of technical implementation, and the target detector needs to be optimized.

[0004] CNN-based object detectors can be divided into one-stage and two-stage. As one of the outstanding one-stage object detectors, algorithms such as the YOLO series are widely used in autonomous driving, smart agriculture, smart security and other fields, and have achieved good results. However, in terms of large-scale application of algorithms, the performance of existing object detection algorithms still cannot meet the real-time and accuracy requirements, and the overall performance of the algorithms needs to be improved. Summary of the invention

[0005] The purpose of the embodiments of this specification is to provide a target detection method, device and electronic device.

[0006] To solve the above technical problems, the embodiments of the present application are implemented in the following ways:

[0007] In a first aspect, the present application provides a target detection method, the method comprising:

[0008] Acquire image data of the target;

[0009] Input image data into a pre-built deep lightweight network and output prediction heads of different scales;

[0010] Decode the prediction head to obtain the target category, positioning box and category confidence.

[0011] In one of the embodiments, the pre-built deep lightweight network includes a backbone feature extraction network and an enhanced feature extraction network;

[0012] Input image data into a pre-built deep lightweight network and output prediction heads of different scales, including:

[0013] Extract features from image data based on the backbone feature extraction network to obtain feature maps of different scales;

[0014] Extracting a preset number of feature maps of different scales from the feature maps of different scales;

[0015] A preset number of feature maps of different scales are input into the enhanced feature extraction network, and prediction heads of different scales are output.

[0016] In one of the embodiments, the backbone feature extraction network includes a common convolutional layer and a lightweight linear residual feature extraction module;

[0017] Based on the backbone feature extraction network, features are extracted from image data to obtain feature maps of different scales, including:

[0018] The ordinary convolution layer processes the image data to obtain the first feature map;

[0019] The lightweight linear residual feature extraction module performs feature extraction on the first feature map to obtain feature maps of different scales; the lightweight linear residual feature extraction module includes extraction modules with different sliding step sizes.

[0020] In one embodiment, the enhanced feature extraction network includes top-down and bottom-up attention pyramid networks;

[0021] A preset number of feature maps of different scales are input into the enhanced feature extraction network, and prediction heads of different scales are output, including:

[0022] A preset number of feature maps of different scales are sequentially input into the attention pyramid network to output prediction heads of different scales.

[0023] In one of the embodiments, the pre-built deep lightweight network training phase uses an IoU-aware loss function to supervise the training, wherein the IoU-aware loss function includes a category loss function, a target loss function, and a regression loss function.

[0024] In one embodiment, the class loss function L cls Calculated by the following formula:

[0025] L cls (p,c)=-(tp c ) V log(p c )·μ

[0026] in,

[0027] Among them, α and γ are adjustment coefficients, μ is the intersection-over-union perceptual coefficient, CIoU is the intersection-over-union loss function considering the Euclidean distance of the center point and the aspect ratio, p is the probability of belonging to a certain category, and c is the category it belongs to.

[0028] In one embodiment, image data is input into a pre-built deep lightweight network, and prediction heads of different scales are output, including:

[0029] Preprocessing the image data to obtain preprocessed image data;

[0030] The preprocessed image data is input into a pre-built deep lightweight network to output prediction heads of different scales.

[0031] In one embodiment, preprocessing the image data to obtain the preprocessed image data includes:

[0032] Adjusting the resolution of the image data to obtain adjusted image data;

[0033] The adjusted image data is subjected to bilateral filtering and denoising processing to obtain preprocessed image data.

[0034] In a second aspect, the present application provides a target detection device, the device comprising:

[0035] An acquisition module, used for acquiring image data of a target;

[0036] The processing module is used to input image data into a pre-built deep lightweight network and output prediction heads of different scales;

[0037] The decoding module is used to decode the prediction head to obtain the target category, positioning box and category confidence.

[0038] In a third aspect, the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the target detection method of the first aspect is implemented.

[0039] It can be seen from the technical solutions provided in the above embodiments of this specification that this solution can greatly improve the accuracy and reasoning speed of target detection by extracting feature maps through a pre-built deep lightweight network, and obtaining the target category, positioning box and category confidence according to the feature map. It can provide a new solution for deploying target detection algorithms to mobile terminals and large-scale engineering applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of this specification or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0041] Figure 1 A schematic diagram of the process of the target detection method provided in this application;

[0042] Figure 2 This is a diagram showing the verification results of the target detection method of this application on a public dataset;

[0043] Figure 3 This is the verification result diagram of the YOLOv4 algorithm on a public dataset;

[0044] Figure 4 This is the verification result diagram of the YOLOv4-Mobilenet V3 model algorithm on a public dataset;

[0045] Figure 5 A schematic diagram of the structure of the target detection device provided in this application;

[0046] Figure 6 A schematic diagram of the structure of the electronic device provided in this application. DETAILED DESCRIPTION

[0047] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this specification.

[0048] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0049] It will be apparent to those skilled in the art that various modifications and variations may be made to the specific embodiments of the present application description without departing from the scope or spirit of the present application. Other embodiments derived from the present application description will be apparent to those skilled in the art. The present application description and examples are exemplary only.

[0050] The words “include,” “including,” “have,” “contain,” etc. used in this document are open-ended terms, meaning including but not limited to.

[0051] Unless otherwise specified, "parts" in this application are all calculated by mass.

[0052] The present invention is further described in detail below with reference to the accompanying drawings and embodiments.

[0053] Reference Figure 1 , which shows a flow chart applicable to the target detection method provided in the embodiment of the present application. It is understandable that the target detection method can be used for vehicle detection, applied to intelligent transportation, vehicle networking technology, etc.; it can also be used for other detections. In the following embodiments, the target is shown as a vehicle. It is also understandable that the target detection method can be executed by an electronic device. Exemplarily, the electronic device can be a mobile phone, a computer, a wearable device, etc. In the following embodiments, a computer device is used as an example. The computer device can be composed of a high-performance graphics card, a processor, and a software system, and the hardware computing power must meet the computing and processing requirements.

[0054] like Figure 1 As shown, the target detection method may include:

[0055] S110, acquiring image data of the target.

[0056] Specifically, the image data of the vehicle may be image data of a moving vehicle acquired in real time, image data of a moving vehicle acquired in a stored video, or image data of a stored vehicle acquired, which is not limited here. In the embodiment of the present application, the image data of a moving vehicle acquired in real time is taken as an example.

[0057] The image data of the moving vehicle can be collected by a visual sensor (such as an industrial-grade visual sensor), and then the collected vehicle data is sent to a computer device. The visual sensor can be arranged in the best field of view, covering the entire lane to be identified as much as possible within the performance range of the visual sensor, and the arrangement angle should be such that the visual sensor is not easily affected by other conditions such as lighting and weather.

[0058] In one embodiment, the image data is input into a pre-built deep lightweight network, and prediction heads of different scales are output, including:

[0059] Preprocessing the image data to obtain preprocessed image data;

[0060] The preprocessed image data is input into a pre-built deep lightweight network to output prediction heads of different scales.

[0061] Preprocess the image data, and obtain the preprocessed image data including:

[0062] Adjusting the resolution of the image data to obtain adjusted image data;

[0063] The adjusted image data is subjected to bilateral filtering and denoising processing to obtain preprocessed image data.

[0064] Specifically, in order to achieve a balance between computational efficiency and image quality, the image resolution of the image data collected in real time is adjusted to 640×640 to obtain adjusted image data.

[0065] The adjusted image data is preprocessed by bilateral filtering and denoising. The filtering formula is as follows:

[0066]

[0067] The pixel value g(k,l) after filtering is:

[0068]

[0069] Among them, w(i, j, k, l) is the weight of the spatial domain kernel template, w d (i, j, k, l) is the value range kernel template weight, w r (i, j, k, l) is the weight of the bilateral filter template; (i, j) is the position of the relevant pixels around the target pixel, f(i, j) is the pixel value; (k, l) is the position of the target pixel, f(k, l) is the pixel value of the target pixel, and g(k, l) is the pixel value after filtering; σ d is the distance standard deviation of the Gaussian function, σ r is the grayscale standard deviation of the Gaussian function.

[0070] In this embodiment, image data is preprocessed to obtain image data of higher quality.

[0071] S120, input the image data into a pre-built deep lightweight network, and output prediction heads of different scales.

[0072] In one embodiment, the pre-built deep lightweight network includes a backbone feature extraction network and an enhanced feature extraction network;

[0073] Input image data into a pre-built deep lightweight network and output prediction heads of different scales, including:

[0074] Extract features from image data based on the backbone feature extraction network to obtain feature maps of different scales;

[0075] Extracting a preset number of feature maps of different scales from the feature maps of different scales;

[0076] A preset number of feature maps of different scales are input into the enhanced feature extraction network, and prediction heads of different scales are output.

[0077] Optionally, the backbone feature extraction network includes a common convolutional layer and a lightweight linear residual feature extraction module;

[0078] Based on the backbone feature extraction network, features are extracted from image data to obtain feature maps of different scales, including:

[0079] The ordinary convolution layer processes the image data to obtain the first feature map;

[0080] The lightweight linear residual feature extraction module performs feature extraction on the first feature map to obtain feature maps of different scales; the lightweight linear residual feature extraction module includes extraction modules with different sliding step sizes.

[0081] Specifically, the first layer is a common convolution layer; the feature map size of the first layer input is 3×640×640, and the input feature map is processed by 16 3×3 common convolution kernels with a sliding step size of 2 to obtain a feature map of 16×320×320;

[0082] Starting from the second layer, there is a lightweight linear residual feature extraction module. The extraction modules in this lightweight linear residual feature extraction module can be divided into two types according to the sliding step size, one with a sliding step size s of 1 and the other with a sliding step size s of 2. The extraction module with a sliding step size s of 1 does not change the scale of the feature map, and the extraction module with a sliding step size s of 2 uses depthwise separable convolution to expand the feature map channel and reduce the size of the feature map;

[0083] The second layer input feature map size is 16×320×320, which is input into the extraction module with a step size s of 1. The original feature map size is not changed, and a feature map of 16×320×320 is obtained.

[0084] The size of the input feature map of the third layer is 16×320×320. The input feature map is processed by an extraction module with a sliding step size s of 2. The feature channel is expanded and the size is reduced to obtain a feature map of 24×160×160.

[0085] The fourth, fifth, sixth, and seventh layers are similar to the above operations, and are extraction modules with sliding steps s of 1, 2, 1, and 2, respectively, outputting feature maps of 24×160×160, 40×80×80, 40×80×80, and 80×40×40, respectively.

[0086] The eighth to tenth layers are all extraction modules with a sliding step size s of 1. The purpose is to obtain higher quality feature maps of the same level. The output feature map size is 80×40×40.

[0087] The eleventh layer is a dimensionality increase layer. The scale of the feature map remains unchanged, the number of channels increases to 120, and the final output feature map is 120×40×40.

[0088] The twelfth layer uses an extraction module with a sliding step size s of 2 to process the input feature map, and the number of channels increases to 160, resulting in a feature map of 160×20×20.

[0089] The thirteenth, fourteenth, and fifteenth layers are all extraction modules with a sliding step size s of 1, and the output feature map size is 160×20×20.

[0090] The sixteenth layer is an extraction module with a sliding step size s of 2. After processing, it outputs a feature map of size 180×10×10.

[0091] In order to obtain a feature map containing accurate semantic-level information and physical-level information, the feature map processed by the backbone feature extraction network can be input into the enhanced feature extraction network for further extraction. A preset number (e.g., four) of feature maps of different scales can be extracted from the feature maps of several different scales output by the backbone feature extraction network. For example, feature maps of four different scales of 180×10×10, 160×20×20, 120×40×40, and 40×80×80 are taken.

[0092] Optionally, the enhanced feature extraction network includes top-down and bottom-up attention pyramid networks;

[0093] A preset number of feature maps of different scales are input into the enhanced feature extraction network, and prediction heads of different scales are output, including:

[0094] A preset number of feature maps of different scales are sequentially input into the attention pyramid network to output prediction heads of different scales.

[0095] The four feature maps of different scales, 180×10×10, 160×20×20, 120×40×40, and 40×80×80, are sequentially input into the attention pyramid network. The enhanced feature extraction can be completed through bilinear interpolation upsampling technology and tensor splicing technology to obtain a more accurate feature layer.

[0096] The principle of bilinear interpolation is to find the four pixel points closest to the target coordinates and infer the value of the point by interpolating in the x and y directions.

[0097] The bilinear interpolation calculation formula is:

[0098]

[0099]

[0100]

[0101]

[0102] Among them, Q 11 , Q 21 , Q 12 , Q 22 are the four nearest pixel points of the target coordinates,

[0103] The attention pyramid network consists of a pyramid structure with a network weight distribution module added.

[0104] The network weight distribution module consists of a global average pooling layer, a fully connected layer, a convolutional layer, etc. The size of the feature map does not change before and after the network weight distribution process. The specific steps are:

[0105] ① Perform global average pooling on the obtained feature map to convert the two-dimensional features on each channel into a number;

[0106] ② The correlation between channels is modeled through the fully connected layer to obtain the weight value of each feature channel;

[0107] ③Add the obtained weight value to the features of each channel, and finally output the weighted feature map.

[0108] After completing the enhanced extraction, four scales of prediction heads are output (the prediction head is the weighted feature map of the final output, which is a multidimensional vector), with sizes of C×10×10, C×20×20, C×40×40, and C×80×80 respectively; where C represents the number of channels of the final feature map, which is determined by the prediction category and positioning box.

[0109] It can be understood that in the above embodiment, if the image data is preprocessed, the input of the pre-built deep lightweight network is the pre-processed image data, otherwise, the input of the pre-built deep lightweight network is the image data.

[0110] In one embodiment, the pre-built deep lightweight network training phase uses an IoU-perceived loss function to supervise the training, wherein the IoU-perceived loss function Loss includes a category loss function L cls 、Objective loss function L obj , regression loss function L reg :

[0111] Loss = L cls +L obj +L reg

[0112] Among them, the category loss function L cls The cross entropy loss function with the intersection-over-union perception coefficient μ is adopted:

[0113] L cls (p, c) = -(1-p c ) V log(p c )·μ

[0114] The calculation formula of the intersection-over-combination ratio perception coefficient μ is as follows:

[0115]

[0116] Among them, α and γ are adjustment coefficients, μ is the intersection-over-union perceptual coefficient, CIoU is the intersection-over-union loss function considering the Euclidean distance of the center point and the aspect ratio, p is the probability of belonging to a certain category, and c is the category it belongs to.

[0117] Objective loss function L obj Using the binary cross entropy loss function:

[0118]

[0119] Among them, y is the sample label, is the probability of predicting a positive sample.

[0120] Regression loss function L reg Using CIoU loss function:

[0121]

[0122] Among them, ρ is the Euclidean distance between the center points of the real box and the predicted box, A and B represent the real box and the predicted box respectively, and c represents the diagonal distance of the smallest rectangular box that can enclose the real box and the predicted box.

[0123] α is the weight function, and v is used to measure the consistency of the aspect ratio. The expressions are:

[0124]

[0125]

[0126] Among them, IoU represents the ratio of the intersection area and the union area of ​​the real box and the predicted box, w1 and w2 represent the width of the real box and the predicted box, and h1 and h2 represent the height of the real box and the predicted box.

[0127] The final expression of CIoU after integration and calculation is:

[0128]

[0129] Assign weights to the three parts of the loss function respectively. Finally, the expression of the intersection-over-union-perceptual loss function is:

[0130]

[0131] Among them, θ1, θ2, θ3 are weight coefficients, and satisfy θ1+θ2+θ3=1.

[0132] S130 , decoding the prediction head to obtain the target category, positioning box, and category confidence.

[0133] Specifically, the network decoder can be used to decode the prediction head to obtain the target category, positioning box and category confidence.

[0134] The proposed target detection method is verified on a public dataset. The verification results are as follows: Figure 2 As shown, the verification results show that the average accuracy of vehicle detection is >90%, which meets the classification accuracy requirements of the "White Paper on Autonomous Driving Data Security" (2020) issued by the Ministry of Industry and Information Technology.

[0135] like Figure 3 , 4 The vehicle detection was performed using the YOLOv4 and YOLOv4-Mobilenet V3 models, with average accuracy rates of 39.04% and 88.22% respectively, which are significantly lower than the detection accuracy rate of the target detection method proposed in the embodiment of the present application.

[0136] The target detection method proposed in the embodiment of the present application can greatly improve the accuracy and reasoning speed of target detection by extracting feature maps through a pre-built deep lightweight network, and obtaining the target category, positioning box and category confidence according to the feature map. It can provide a new solution for the deployment of target detection algorithms on mobile terminals and large-scale engineering applications.

[0137] Reference Figure 5 , which shows a schematic structural diagram of a target detection device described according to an embodiment of the present application.

[0138] like Figure 5 As shown, the target detection device may include:

[0139] An acquisition module 510 is used to acquire image data of a target;

[0140] Processing module 520, used to input image data into a pre-built deep lightweight network and output prediction heads of different scales;

[0141] The decoding module 530 is used to decode the prediction head to obtain the target category, positioning box and category confidence.

[0142] Optionally, the processing module 520 is further configured to:

[0143] Extract features from image data based on the backbone feature extraction network to obtain feature maps of different scales;

[0144] Extracting a preset number of feature maps of different scales from the feature maps of different scales;

[0145] A preset number of feature maps of different scales are input into the enhanced feature extraction network, and prediction heads of different scales are output.

[0146] Optionally, the processing module 520 is further configured to:

[0147] The ordinary convolution layer processes the image data to obtain the first feature map;

[0148] The lightweight linear residual feature extraction module performs feature extraction on the first feature map to obtain feature maps of different scales; the lightweight linear residual feature extraction module includes extraction modules with different sliding step sizes.

[0149] Optionally, the processing module 520 is further configured to:

[0150] A preset number of feature maps of different scales are sequentially input into the attention pyramid network to output prediction heads of different scales.

[0151] Optionally, the pre-built deep lightweight network training phase uses an intersection-over-union (IoU)-aware loss function to supervise the training, where the IoU-aware loss function includes a category loss function, a target loss function, and a regression loss function.

[0152] Optional, class loss function L cls Calculated by the following formula:

[0153] L cls (p, c) = -(1-p c ) V log(pc )·μ

[0154] in,

[0155] Among them, α and γ are adjustment coefficients, μ is the intersection-over-union perceptual coefficient, CIoU is the intersection-over-union loss function considering the Euclidean distance of the center point and the aspect ratio, p is the probability of belonging to a certain category, and c is the category it belongs to.

[0156] Optionally, the processing module 520 is further configured to:

[0157] Preprocessing the image data to obtain preprocessed image data;

[0158] The preprocessed image data is input into a pre-built deep lightweight network to output prediction heads of different scales.

[0159] Optionally, the processing module 520 is further configured to:

[0160] Adjusting the resolution of the image data to obtain adjusted image data;

[0161] The adjusted image data is subjected to bilateral filtering and denoising processing to obtain preprocessed image data.

[0162] The target detection device provided in this embodiment can execute the embodiment of the above method. Its implementation principle and technical effect are similar and will not be described in detail here.

[0163] Figure 6 FIG. 1 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. Figure 6 As shown, a structural schematic diagram of an electronic device 300 suitable for implementing an embodiment of the present application is shown.

[0164] like Figure 6 As shown, the electronic device 300 includes a central processing unit (CPU) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage part 308 into a random access memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the device 300 are also stored. The CPU 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0165] The following components are connected to the I / O interface 305: an input section 306 including a keyboard, a mouse, etc.; an output section 307 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 308 including a hard disk, etc.; and a communication section 309 including a network interface card such as a LAN card, a modem, etc. The communication section 309 performs communication processing via a network such as the Internet. A drive 310 is also connected to the I / O interface 306 as needed. A removable medium 311, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 310 as needed, so that a computer program read therefrom is installed into the storage section 308 as needed.

[0166] In particular, according to the embodiments of the present disclosure, the above reference Figure 1 The described process can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program tangibly contained on a machine-readable medium, and the computer program includes program code for performing the above-mentioned target detection method. In such an embodiment, the computer program can be downloaded and installed from a network through the communication part 309, and / or installed from the removable medium 311.

[0167] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of the code, and the aforementioned module, program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0168] The units or modules involved in the embodiments described in the present application may be implemented by software or hardware. The units or modules described may also be arranged in a processor. The names of these units or modules do not constitute limitations on the units or modules themselves in certain circumstances.

[0169] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop, a mobile phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0170] As another aspect, the present application further provides a storage medium, which may be the storage medium included in the aforementioned device in the above embodiment; or may be a storage medium that exists independently and is not assembled into the device. The storage medium stores one or more programs, and the aforementioned programs are used by one or more processors to execute the target detection method described in the present application.

[0171] Storage media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0172] It should be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of further restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0173] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

Claims

1. A target detection method, characterized in that: The method comprises: Acquiring image data of the target; Input the image data into a pre-built deep lightweight network and output prediction heads of different scales; Decoding the prediction head to obtain the category, positioning box and category confidence of the target; The pre-built deep lightweight network includes a backbone feature extraction network and an enhanced feature extraction network; The step of inputting the image data into a pre-built deep lightweight network and outputting prediction heads of different scales includes: Extracting features from the image data based on the backbone feature extraction network to obtain feature maps of different scales; Extracting a preset number of feature maps of different scales from the feature maps of different scales; Inputting the preset number of feature maps of different scales into the enhanced feature extraction network, and outputting the prediction heads of different scales; Wherein, the backbone feature extraction network includes a common convolutional layer and a lightweight linear residual feature extraction module; The extracting features from the image data based on the backbone feature extraction network to obtain feature maps of different scales includes: The common convolution layer processes the image data to obtain a first feature map; The lightweight linear residual feature extraction module performs feature extraction on the first feature map to obtain feature maps of different scales; The lightweight linear residual feature extraction module includes extraction modules with different sliding steps; The first layer of the backbone feature extraction network is the common convolution layer, the feature map size of the first layer input is 3×640×640, 16 3×3 common convolution kernels are used to process the input feature map, the sliding step size is 2, and a feature map of 16×320×320 is obtained; Starting from the second layer, there is the lightweight linear residual feature extraction module. The extraction modules in the lightweight linear residual feature extraction module can be divided into two types according to the sliding step size, one with a sliding step size s of 1 and the other with a sliding step size s of 2. The extraction module with a sliding step size s of 1 does not change the scale of the feature map, and the extraction module with a sliding step size s of 2 uses a depthwise separable convolution to expand the feature map channel and reduce the size of the feature map. The second layer input feature map size is 16×320×320, which is input into the extraction module with a step size s of 1. The original feature map size is not changed, and a feature map of 16×320×320 is obtained. The third layer input feature map size is 16×320×320, which is processed by the extraction module with a sliding step size s of 2. The feature channel is expanded and the size is reduced to obtain a feature map of 24×160×160. The fourth, fifth, sixth, and seventh layers are respectively The extraction modules with sliding step s of 1, 2, 1, and 2 output feature maps of 24×160×160, 40×80×80, 40×80×80, and 80×40×40, respectively. The eighth to tenth layers are all extraction modules with sliding step s of 1. The purpose is to obtain higher quality feature maps of the same level. The output feature map size is 80×40×40. The eleventh layer is a dimensionality increase layer. The scale of the feature map remains unchanged, the number of channels increases to 120, and the final output feature map is 120×40×40. The twelfth layer uses the extraction module with sliding step s of 2 to process the input feature map, and the number of channels increases to 160 to obtain a feature map of 160×20×20. The thirteenth, fourteenth, and fifteenth layers are all extraction modules with sliding step s of 1, and the output feature map size is 160×20×20. The sixteenth layer is an extraction module with sliding step s of 2. After processing, the output feature map size is 180×10×10. The pre-built deep lightweight network training phase uses an intersection-over-union (IoU)-aware loss function to supervise the training, wherein the IoU-aware loss function includes a category loss function, a target loss function, and a regression loss function; The category loss function Calculated by the following formula: in, in, , is the adjustment coefficient, is the intersection-over-union perceptual coefficient, To consider the intersection-over-union loss function of the Euclidean distance of the center point and the aspect ratio, is the probability of belonging to a certain category, To belong to a category.

2. The method according to claim 1, characterized in that: The enhanced feature extraction network includes a top-down and bottom-up attention pyramid network; The step of inputting the preset number of feature maps of different scales into the enhanced feature extraction network and outputting the prediction heads of different scales comprises: The preset number of feature maps of different scales are sequentially input into the attention pyramid network, and the prediction heads of different scales are output.

3. The method according to any one of claims 1 to 2, characterized in that: The step of inputting the image data into a pre-built deep lightweight network and outputting prediction heads of different scales includes: Preprocessing the image data to obtain preprocessed image data; The preprocessed image data is input into the pre-built deep lightweight network, and the prediction heads of different scales are output.

4. The method according to claim 3, characterized in that The preprocessing of the image data to obtain the preprocessed image data comprises: Adjusting the resolution of the image data to obtain adjusted image data; The adjusted image data is subjected to bilateral filtering and denoising processing to obtain the preprocessed image data.

5. A target detection device, characterized in that: The device comprises: An acquisition module, used for acquiring image data of the target; A processing module, used for inputting the image data into a pre-built deep lightweight network and outputting prediction heads of different scales; A decoding module, used to decode the prediction head to obtain the category, positioning box and category confidence of the target; The pre-built deep lightweight network includes a backbone feature extraction network and an enhanced feature extraction network; The step of inputting the image data into a pre-built deep lightweight network and outputting prediction heads of different scales includes: Extracting features from the image data based on the backbone feature extraction network to obtain feature maps of different scales; Extracting a preset number of feature maps of different scales from the feature maps of different scales; Inputting the preset number of feature maps of different scales into the enhanced feature extraction network, and outputting the prediction heads of different scales; Wherein, the backbone feature extraction network includes a common convolutional layer and a lightweight linear residual feature extraction module; The extracting features from the image data based on the backbone feature extraction network to obtain feature maps of different scales includes: The common convolution layer processes the image data to obtain a first feature map; The lightweight linear residual feature extraction module performs feature extraction on the first feature map to obtain feature maps of different scales; The lightweight linear residual feature extraction module includes extraction modules with different sliding steps; The first layer of the backbone feature extraction network is the common convolution layer, the feature map size of the first layer input is 3×640×640, 16 3×3 common convolution kernels are used to process the input feature map, the sliding step size is 2, and a feature map of 16×320×320 is obtained; Starting from the second layer, there is the lightweight linear residual feature extraction module. The extraction modules in the lightweight linear residual feature extraction module can be divided into two types according to the sliding step size, one with a sliding step size s of 1 and the other with a sliding step size s of 2. The extraction module with a sliding step size s of 1 does not change the scale of the feature map, and the extraction module with a sliding step size s of 2 uses a depthwise separable convolution to expand the feature map channel and reduce the size of the feature map. The second layer input feature map size is 16×320×320, which is input into the extraction module with a step size s of 1. The original feature map size is not changed, and a feature map of 16×320×320 is obtained. The third layer input feature map size is 16×320×320, which is processed by the extraction module with a sliding step size s of 2. The feature channel is expanded and the size is reduced to obtain a feature map of 24×160×160. The fourth, fifth, sixth, and seventh layers are respectively The extraction modules with sliding step s of 1, 2, 1, and 2 output feature maps of 24×160×160, 40×80×80, 40×80×80, and 80×40×40, respectively. The eighth to tenth layers are all extraction modules with sliding step s of 1. The purpose is to obtain higher quality feature maps of the same level. The output feature map size is 80×40×40. The eleventh layer is a dimensionality increase layer. The scale of the feature map remains unchanged, the number of channels increases to 120, and the final output feature map is 120×40×40. The twelfth layer uses the extraction module with sliding step s of 2 to process the input feature map, and the number of channels increases to 160 to obtain a feature map of 160×20×20. The thirteenth, fourteenth, and fifteenth layers are all extraction modules with sliding step s of 1, and the output feature map size is 160×20×20. The sixteenth layer is an extraction module with sliding step s of 2. After processing, the output feature map size is 180×10×10. The pre-built deep lightweight network training phase uses an intersection-over-union (IoU)-aware loss function to supervise the training, wherein the IoU-aware loss function includes a category loss function, a target loss function, and a regression loss function; The category loss function Calculated by the following formula: in, in, , is the adjustment coefficient, is the intersection-over-union perceptual coefficient, To consider the intersection-over-union loss function of the Euclidean distance of the center point and the aspect ratio, is the probability of belonging to a certain category, To belong to a category.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the target detection method as described in any one of claims 1 to 4 is implemented.

Images