Target detection method, device, storage medium, equipment and program product

Abstract

The present application provides a target detection method, device, storage medium, equipment and program product, which can be applied to various scenarios such as artificial intelligence, computer vision, image processing, etc. The method includes: obtaining a picture to be detected, and performing feature extraction on the picture to be detected to obtain a feature map of the picture to be detected; determining multiple first intermediate layer feature maps according to the feature map; grouping and splicing multiple first intermediate layer feature maps to obtain multiple grouped feature maps, wherein each grouped feature map contains at least two first intermediate layer feature maps; determining multiple second intermediate layer feature maps according to the grouped feature map; merging multiple second intermediate layer feature maps to obtain a target feature map; performing target detection based on the target feature map, and outputting the detection result, thereby improving the utilization rate of the feature, enabling the network to better detect targets of different scales, and improving the accuracy of target detection.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and more specifically, to a target detection method, device, storage medium, equipment and program product. Background Art

[0002] With the development of artificial intelligence technology and computer vision technology, target detection, as an important task in computer vision technology, is widely used in many fields such as autonomous driving, robot navigation, industrial inspection, etc.

[0003] At present, the target detection methods based on artificial intelligence technology are usually based on deep learning convolutional neural networks. However, the current target detection methods cannot handle the scale changes of objects well, and there are problems such as poor detection performance for small-scale targets and low feature utilization. Summary of the invention

[0004] The present application provides a target detection method, apparatus, storage medium, device and program product, which can improve the utilization rate of features, enable the network to better detect targets of different scales, and improve the accuracy of target detection.

[0005] In a first aspect, a target detection method is provided, comprising:

[0006] Acquire a picture to be detected, and perform feature extraction on the picture to be detected to obtain a feature map of the picture to be detected;

[0007] Determine a plurality of first intermediate layer feature maps according to the feature map;

[0008] Grouping and splicing the multiple first intermediate layer feature maps to obtain multiple grouped feature maps, wherein each of the grouped feature maps contains at least two of the first intermediate layer feature maps;

[0009] Determine a plurality of second intermediate layer feature maps according to the grouped feature maps;

[0010] Merging the multiple second intermediate layer feature maps to obtain a target feature map;

[0011] Target detection is performed based on the target feature map, and the detection result is output.

[0012] In another aspect, a target detection device is provided, comprising:

[0013] An acquisition module is used to acquire a picture to be detected, and perform feature extraction on the picture to be detected to obtain a feature map of the picture to be detected;

[0014] A first determining module, configured to determine a plurality of first intermediate layer feature maps according to the feature map;

[0015] A splicing module, used for grouping and splicing the plurality of first intermediate layer feature maps to obtain a plurality of grouped feature maps, wherein each of the grouped feature maps contains at least two of the first intermediate layer feature maps;

[0016] A second determination module, used to determine a plurality of second intermediate layer feature maps according to the grouping feature map;

[0017] A merging module, used for merging the plurality of second intermediate layer feature maps to obtain a target feature map;

[0018] The detection module is used to perform target detection based on the target feature map and output the detection result.

[0019] On the other hand, a computer-readable storage medium is provided, comprising instructions, which, when executed on a computer device, enable the computer device to execute the method as described in the first aspect above.

[0020] On the other hand, a computer device is provided, comprising a processor and a memory, wherein the memory stores a computer program, and the processor executes the steps of the target detection method as described in the first aspect above by calling the computer program stored in the memory.

[0021] On the other hand, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the steps in the target detection method described in the first aspect above.

[0022] The embodiment of the present application provides a target detection method, including: obtaining a picture to be detected, and performing feature extraction on the picture to be detected to obtain a feature map of the picture to be detected; determining multiple first intermediate layer feature maps according to the feature map; grouping and splicing the multiple first intermediate layer feature maps to obtain multiple grouped feature maps, wherein each of the grouped feature maps contains at least two of the first intermediate layer feature maps; determining multiple second intermediate layer feature maps according to the grouped feature maps; merging the multiple second intermediate layer feature maps to obtain a target feature map; performing target detection based on the target feature map, and outputting the detection result. The present application divides the picture to be detected into equal parts, and groups and splices them to obtain a grouped feature map, and then fuses the grouped feature maps to obtain a target feature map, thereby effectively improving the utilization rate of features, enhancing the network topology, and achieving multi-scale feature fusion. Then, target detection is performed based on the target feature map, so that the network can better detect targets of different scales and improve the accuracy of target detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is a schematic diagram of the structure of the target detection system provided in the embodiment of the present application.

[0024] Figure 2 It is a flowchart of the target detection method provided in the embodiment of the present application.

[0025] Figure 3 It is a schematic diagram of an application scenario of the target detection method provided in an embodiment of the present application.

[0026] Figure 4 It is another flowchart of the target detection method provided in an embodiment of the present application.

[0027] Figure 5 It is a schematic diagram of another application scenario of the target detection method provided in an embodiment of the present application.

[0028] Figure 6 It is a schematic block diagram of a target detection device provided in an embodiment of the present application.

[0029] Figure 7 It is a schematic block diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0030] The following will describe the technical solutions in the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. For the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0031] The embodiments of the present application provide a target detection method, apparatus, storage medium, device and program product. Specifically, the target detection method of the embodiments of the present application can be executed by a computer device, wherein the computer device can be a terminal or a server. The embodiments of the present application can be applied to various scenarios such as smart transportation, assisted driving, artificial intelligence, computer vision technology, image processing, etc.

[0032] First, some nouns or terms that appear in the description of the embodiments of the present application are explained as follows:

[0033] Intelligent Traffic System (ITS), also known as Intelligent Transport System (ITS), effectively and comprehensively applies advanced science and technology (information technology, computer technology, data communication technology, sensor technology, electronic control technology, automatic control theory, operations research, artificial intelligence, etc.) to transportation, service control and vehicle manufacturing, strengthens the connection between vehicles, roads and users, and thus forms a comprehensive transportation system that ensures safety, improves efficiency, improves the environment and saves energy.

[0034] Intelligent Vehicle Infrastructure Cooperative Systems (IVICS), referred to as IVICS, is a development direction of Intelligent Transportation System (ITS). IVICS uses advanced wireless communication and new generation Internet technologies to implement dynamic real-time information interaction between vehicles and roads in all directions, and conducts active vehicle safety control and road cooperative management based on the collection and integration of dynamic traffic information in all time and space, fully realizing the effective coordination of people, vehicles and roads, ensuring traffic safety, and improving traffic efficiency, thus forming a safe, efficient and environmentally friendly road traffic system.

[0035] Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0036] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. The basic technologies of artificial intelligence generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0037] Computer Vision Technology (CV) Computer vision is a science that studies how to make machines "see". To put it more specifically, it refers to machine vision such as using cameras and computers to replace human eyes to identify and measure targets, and further perform graphic processing so that computer processing becomes an image that is more suitable for human eye observation or transmission to instrument detection. As a scientific discipline, computer vision studies related theories and technologies, and attempts to establish an artificial intelligence system that can obtain information from images or multi-dimensional data. Computer vision technology usually includes image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous positioning and map construction, and other technologies, as well as common biometric recognition technologies such as face recognition and fingerprint recognition.

[0038] Cloud technology: refers to a hosting technology that unifies hardware, software, network and other resources within a wide area network or local area network to achieve data computing, storage, processing and sharing. Cloud technology is a general term for network technology, information technology, integration technology, management platform technology, application technology, etc. based on the cloud computing business model. It can form a resource pool that is used on demand and is flexible and convenient. Cloud computing technology will become an important support. The background services of the technical network system require a large amount of computing and storage resources, such as video websites, picture websites and more portal websites. With the high development and application of the Internet industry, in the future, each item may have its own identification mark, and all need to be transmitted to the background system for logical processing. Data of different levels will be processed separately. All kinds of industry data require strong system backing support, which can only be achieved through cloud computing.

[0039] Machine Learning (ML) is a multi-disciplinary subject involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and self-learning.

[0040] Deep learning (DL) is a new research direction in the field of machine learning. It is introduced into machine learning to make it closer to its original goal - artificial intelligence.

[0041] Convolutional Neural Networks (CNN) is a type of feedforward neural network that includes convolution calculations and has a deep structure. It is one of the representative algorithms of deep learning. Convolutional neural networks have representation learning capabilities and can perform shift-invariant classification on input information according to their hierarchical structure. Therefore, they are also called "Shift-Invariant Artificial Neural Networks (SIANN)".

[0042] In convolutional neural networks, the definition of receptive field is the size of the area mapped on the original image by each pixel on the feature map output by each layer of the convolutional neural network. The original image here refers to the input image of the network, which is a preprocessed image.

[0043] In the context of machine learning, hyperparameters are parameters whose values ​​are set before starting the learning process, rather than parameter data obtained through training. Typically, hyperparameters need to be optimized to select a set of optimal hyperparameters for the learning machine to improve learning performance and results.

[0044] Object detection, also known as object extraction, is a kind of image segmentation based on the geometric and statistical features of the object. It combines the segmentation and recognition of the object into one, and its accuracy and real-time performance are important capabilities of the entire system.

[0045] Image object detection aims to identify and locate object instances of predefined categories through network calculations given an input image, and output the object's minimum rectangular bounding box and category. Object detection is an important task in computer vision because of its prediction of position and category. It is widely used in many fields such as autonomous driving, robot navigation, and industrial inspection. It has important practical significance for reducing labor costs through computer vision, and is also a hot research topic in theory and application in recent years. For the task of image object detection, the main difficulties include the following two aspects: On the one hand, the target detection scene is complex, the object morphology varies, and the distribution is dense. On the other hand, the scale of the target detection objects varies, and the detector needs to adapt well to both large-scale and small-scale objects. Therefore, image object detection is very challenging.

[0046] At present, most of the image target detection methods based on artificial intelligence technology are based on deep learning convolutional neural network methods. The mainstream methods generally adopt the structure of a two-stage target detection model or a one-stage target detection model. Usually, convolutional neural networks require the stacking of convolutional layers to enhance the ability to extract complex features, which makes the low-level features have higher resolution information and contain more position and detail information. However, due to fewer convolutional layers, the semantics of the underlying features is lower and there is more noise. However, although high-level features have stronger semantic information, with the increase in the number of layers and the downsampling operation, the geometric information of the extracted features may be destroyed, resulting in low resolution of high-level features and poor perception of details. In addition, since the current network design usually lacks the topological structure design for multi-scale feature fusion and lacks an explicit mechanism for fusion of deep features with shallow network features, it leads to the lack of multi-scale features, which makes the network unable to handle targets of different scales well. Therefore, the embodiments of the present application provide a target detection method, apparatus, storage medium, device and program product, which involve artificial intelligence computer vision technology and machine learning technology, introduce feature reuse mechanism and receptive field augmentation mechanism, and effectively improve feature utilization and enhance network topology through the fusion of grouped features, so as to achieve the effect of multi-scale feature fusion, so that the network can better handle targets of different scales and improve the accuracy of target detection.

[0047] Please refer to Figure 1 , Figure 1 A schematic diagram of the structure of the target detection system provided for the embodiment of the present application. The target detection system includes a terminal 100 and a server 200, etc.; the terminal 100 and the server 200 are connected via a network, for example, via a wired or wireless network connection, etc. The terminal 100 can be a smart phone, a tablet computer, a laptop computer, a smart TV, a smart speaker, a wearable smart device, a smart car terminal, an aircraft, etc. The terminal 100 can also include a client, which can be an application client, a browser client, or an instant messaging client, etc. The server 200 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks (Content Delivery Network, CDN), and big data and artificial intelligence platforms.

[0048] Among them, the terminal 100 can be used to display a graphical user interface. Among them, the terminal is used to interact with the user through a graphical user interface, for example, by downloading and installing the corresponding client through the terminal and running it, for example, by calling the corresponding applet and running it, for example, by logging into the website to present the corresponding graphical user interface, etc. In an embodiment of the present application, the terminal 100 can be a terminal device for users to upload pictures to be detected. Among them, when performing target detection, the picture to be detected is uploaded through the terminal device used by the user, and the picture to be detected is sent to the server 200 for target detection. The server 200 will send the detection result to the terminal 100.

[0049] Among them, in an embodiment of the present application, when performing target detection on a picture to be detected, the server 200 can be specifically used to: obtain the picture to be detected, and perform feature extraction on the picture to be detected to obtain a feature map of the picture to be detected; determine multiple first intermediate layer feature maps based on the feature map; group and splice the multiple first intermediate layer feature maps to obtain multiple grouped feature maps; determine multiple second intermediate layer feature maps based on the grouped feature maps; merge the multiple second intermediate layer feature maps to obtain a target feature map; perform target detection based on the target feature map, and output the detection result; and send the detection result to the terminal 100.

[0050] It should be noted that the order of description of the following embodiments is not intended to limit the priority order of the embodiments.

[0051] See also Figure 2 , Figure 2 The flowchart of the target detection method described in the embodiment of the present application is shown. The target detection method is applied to a computer device. The method mainly includes steps 101 to 106, which are described as follows:

[0052] Step 101, obtain a picture to be detected, and perform feature extraction on the picture to be detected to obtain a feature map of the picture to be detected.

[0053] For example, the image to be detected can be specifically understood as an image with detection requirements. The image to be detected can include a target to be detected, and the image to be detected can be an image captured in real time by an image acquisition device, such as an image captured in real time by a camera, or an image uploaded by a user through a user terminal. For example, the image to be detected can be a corresponding image in each detection scene, and the detection scene can be understood as a scene used by a target detection model, such as a detection scene that can be a vehicle detection scene, a pedestrian detection scene, an object detection scene, a target action detection scene in a video, and other detection scenes. Among them, the target to be detected can correspond to the detection scene, for example, in a vehicle detection scene, the target to be detected can be a vehicle; in a pedestrian detection scene, the target to be detected can be a pedestrian; in an object detection scene, the target to be detected can be an object; in a target action detection scene in a video, the target to be detected can be an object action.

[0054] For example, taking autonomous driving as an example, the image acquisition device on the vehicle collects images of the environment around the vehicle in real time during the driving process. The images may include buildings, pedestrians, motor vehicles and non-motor vehicles, etc. The collected images are used as images to be detected for target detection, and pedestrians, motor vehicles and non-motor vehicles can be identified, providing a basis for autonomous driving, for example, route planning for autonomous driving is performed based on the recognition results.

[0055] Wherein, the computer device may be installed with a target detection model, and the target detection model may be implemented based on a one-stage target detection model. Specifically, the target detection model may include a backbone network and a feature pyramid network (Feature Pyramid Network, FPN), and the backbone network may include modules such as convolution modules and backbone modules, wherein the backbone network may include multiple cascaded backbone modules, and one or more backbone modules may be used to process the feature map. Wherein, the image to be detected may be an RGB image or a grayscale image. Specifically, the image to be detected may be input into the backbone network, and through the convolution module, the feature map of the image to be detected may be obtained. Wherein, the feature map of the image to be detected may characterize the global features of the image to be detected.

[0056] Step 102, determining a plurality of first intermediate layer feature maps according to the feature map.

[0057] In this embodiment, step 102 may mainly include: dividing the feature map equally into multiple parts, and determining each part as a first intermediate layer feature map.

[0058] Among them, the first intermediate layer feature map can represent the local features of the image to be detected or represent the fine-grained local features of the image to be detected.

[0059] Specifically, the feature map can be equally divided into a preset number of parts to obtain multiple first intermediate layer features. Then, in the subsequent steps, the multiple first intermediate layer features are combined and spliced ​​in pairs, and then the grouped features are fused, which can effectively improve the utilization rate of the features, enhance the network topology, and achieve the effect of multi-scale feature fusion.

[0060] Among them, this application does not limit the preset number of copies, and different preset numbers can be determined according to different tasks. Specifically, the preset score is a hyperparameter learned by the target detection model. For example, for classification tasks, the preset number of copies can be s1, and for target detection tasks, the preset number of copies can be s2. Among them, different preset numbers can also be set according to the size of the required receptive field. The larger the required receptive field, the larger the preset number.

[0061] Specifically, this embodiment uses represents the feature map, where c is the number of input channels, h and w are the height and width of the feature map respectively. For example, if the image to be detected is an RGB image, the number of input channels is 3. Specifically, when object detection is to be performed, the image to be detected can be input into the backbone network first, and then the convolution module extracts features based on the image to be detected to obtain a feature map. Then, the backbone module can convert the feature map The equal parts are s parts, each part is recorded as Where i∈{1,2,…,s}, the number of channels in each part is For example, see Figure 3 In this embodiment, s=4 is used as an example, and the feature map R in It is divided into 4 equal parts, namely The number of channels in each part is

[0062] Step 103, grouping and splicing the multiple first intermediate layer feature maps to obtain multiple grouped feature maps, wherein each grouped feature map contains at least two first intermediate layer feature maps.

[0063] In this embodiment, step 103 may mainly include: splicing every two first intermediate layer feature maps to obtain multiple grouping feature maps.

[0064] Specifically, each first intermediate layer feature map can be merged and concatenated in pairs through multiplexing transformation to obtain multiple group feature maps. Represents that, where each group feature map The number of channels is For example, see Figure 3 ,Will Merge two by two to obtain multiple group feature maps In this embodiment, the number of grouped feature maps is determined by the number of first intermediate layer feature maps. Assuming there are s first intermediate layer feature maps, the number of grouped feature maps is (s*(s-1)) / 2.

[0065] Step 104: Obtain multiple second intermediate layer feature maps based on the grouped feature maps.

[0066] Specifically, step 104 may include: sorting the group feature maps; performing convolution processing on the first group feature map in the sorted order according to a preset convolution kernel to obtain the first second intermediate layer feature map; concatenating the i-th group feature map in the sorted order with the i-1-th second intermediate layer feature map, and performing convolution processing according to a preset convolution kernel to obtain the i-th second intermediate layer feature map, where i is an integer greater than 1.

[0067] Specifically, concatenating multiple second intermediate layer feature maps can better encode the semantic and geometric detail synergy between groups. represents the second intermediate layer feature map, where It can be expressed as:

[0068]

[0069] in, is the i-th group feature map in the sorting order, through By performing convolution, we can get the first second intermediate layer feature map Then, the i-th group feature map in the sorted order and the i-1th second intermediate layer feature map The concatenation is performed and convolution is performed according to the preset convolution kernel to obtain the feature map of the second intermediate layer of the ith layer. i′ and b i′ They are the convolution kernel and bias term of size k×k corresponding to the i-th group feature map, and cat represents the concatenation operation on the channel dimension.

[0070] For details, please continue to see Figure 3 For example, the preset convolution kernel can be a 3×3 convolution kernel, such as After convolution with a 3×3 convolution kernel, we get Then, and Perform the splicing operation on the channel dimension, and then convolve it with a 3×3 convolution kernel to get Similarly, all group feature maps are processed to obtain multiple second intermediate layer feature maps.

[0071] It is easy to understand that, except for the first 3X3 convolution operation, all other 3X3 convolution operations potentially accept all the feature information on the left, so that each output from left to right The receptive fields of the images are increasing. Therefore, this split hybrid connection structure can obtain a combination of different receptive field sizes, which is conducive to extracting global and local information from the image to be detected. Therefore, the preset convolution kernel can be set according to the required receptive field size.

[0072] Step 105, merging multiple second intermediate layer feature maps to obtain a target feature map.

[0073] Specifically, by merging the feature maps of the second intermediate layer based on the merging transformation of the receptive field augmentation, the multi-scale feature representation capability of the network can be enhanced, so that the target detection model can handle targets of different scales well.

[0074] In some embodiments, step 105 may include: performing a splicing operation on multiple second intermediate layer feature maps in the channel dimension to obtain a target feature map.

[0075] Specifically, this embodiment uses R′ out represents the target feature map, where Among them, Cat represents the splicing operation on the channel dimension, It is the feature map of the second intermediate layer.

[0076] For details, please continue to see Figure 3 , by and Perform splicing operation on the channel dimension to obtain the target feature map in, and The number of channels can be It is worth noting that the number of channels is artificially set to ensure that the number of input channels and the number of output channels are the same, that is, c.

[0077] In some embodiments, step 105 may include: performing weighted addition on multiple second intermediate layer feature maps to obtain a target feature map.

[0078] Specifically, a target feature map can be obtained by weighted addition of multiple second intermediate layer feature maps, so as to construct an adaptive feature reuse mechanism through an adaptive weighting strategy.

[0079] In this embodiment, in order to ensure that the number of input channels is the same as the number of output channels, each The output channel is c, R′ out It can be expressed as: Among them, αi For each The weight of i These are also the hyperparameters of the backbone network.

[0080] In some embodiments, after obtaining the target feature map, the target feature map can be input into a feature pyramid network, and then a multi-scale feature can be constructed through a feature pyramid network (FPN). Alternatively, the target feature map can be input into a next-level backbone module, and the next-level backbone module can continue to split, splice, and merge the target feature map.

[0081] Step 106, performing target detection based on the target feature map and outputting the detection result.

[0082] Specifically, step 106 may mainly include: constructing a multi-scale feature according to the target feature map; performing target prediction according to the multi-scale feature, and outputting a detection result.

[0083] Among them, the step of "constructing multi-scale features according to the target feature map" can mainly include: down-sampling according to the target feature map to obtain a first multi-level feature map; up-sampling according to the target feature map to obtain a second multi-level feature map; fusing the corresponding feature maps in the first multi-level feature map and the second multi-level feature map to obtain multi-scale features.

[0084] Specifically, the target detection model also includes a feature pyramid network, and the target feature map can be input into the feature pyramid network to construct multi-scale features. Among them, the feature pyramid network mainly takes the target feature map output by the backbone module as input, and outputs a multi-level feature map of proportional size. Among them, the feature pyramid network can make full use of all the feature maps obtained by the backbone module, and enable the backbone network to better detect objects. The feature pyramid network extracts features of different scales at different scales and fuses them. With the multi-level feature map, the information extracted by the backbone network can be more fully utilized, so that the target detection algorithm can better cope with multi-scale situations. For example, when the scales of the targets to be detected in the image to be detected are different, the multi-level feature map can be a picture feature structure with different levels composed of the features of the targets to be detected of different scales.

[0085] Among them, Feature Pyramid Networks (FPN) consists of bottom-up (downsampling) and top-down (upsampling) paths and side connections.

[0086] Among them, bottom-up (downsampling) is the forward propagation process of network training, the size of the feature map is getting smaller and smaller, and the stride is scaled, where the stride can represent the space occupied by each row of pixels in the memory. During the forward propagation process, the size of the feature map will change after passing through certain layers, and will not change when passing through other layers. The layers that do not change the size of the feature map can be classified into a stage, and the size of the feature map in each stage is basically the same. Therefore, the output of the last layer of each stage can be selected as the reference feature map for classification and regression, so that a feature pyramid can be formed. Among them, the features extracted by the last layer of each stage are optimal.

[0087] Among them, top-down (upsampling) is to upsample the high-level feature map (such as nearest neighbor upsampling). By upsampling the feature map, the upsampled feature map can have the same size as the feature map of the next layer.

[0088] The side connection means fusing the downsampled feature map with the upsampled feature map. The downsampled feature map can be convolved with the upsampled feature map to obtain a new feature map. The 1×1 convolution operation can change the number of channels to the same number of channels as the next layer, which is convenient for the subsequent classification of the fused features.

[0089] Each layer of the feature pyramid network combines features with different resolutions and different semantic information to obtain a more powerful representation. Upsampling can transfer high-semantic information to shallow features while retaining the high resolution (small receptive field) characteristics of shallow features, which can better balance semantic information and location information. Among them, multi-scale can refer to multiple spatial resolutions, so it can be seen that multi-scale features contain features with multiple different spatial resolutions. The multi-scale features of the multi-resolution output of FPN replace the feature map of a single resolution, and the same size of anchor box is used at each level. The different scales of the feature map achieve the effect of multi-scale anchor box, and each level of the pyramid shares a similar semantic level.

[0090] In some embodiments, the step of "performing target detection based on multi-scale features" may mainly include: determining the target confidence of the target to be detected in the image to be detected based on the multi-scale features; determining the target category and target position of the target to be detected based on the target confidence, and obtaining the detection result.

[0091] Among them, the target confidence may include category confidence and position confidence, and the step of "determining the target category and target position of the target to be detected according to the target confidence" may include: determining the preset category with the highest category confidence as the target category; determining the position with the highest position confidence as the target position.

[0092] For example, the target confidence may include category confidence, that is, the confidence that the target to be detected belongs to a certain category. The category confidence of a certain target, that is, the probability that the true category of the target belongs to A and the probability that it belongs to B. The category of the target to be detected can be determined according to the category confidence of the target to be detected, and the category with the highest category confidence score is determined as the target category of the target to be detected.

[0093] For example, the target confidence may include position confidence, that is, the confidence of the position of the target to be detected. The target position may include the center and length and width offset of the anchor frame. The position confidence of a target, that is, the probability that the true position of the target is X and the probability that the true position is Y. The position of the target to be detected can be determined based on the position confidence of the target to be detected, and the category with the highest position confidence score is determined as the target position of the target to be detected.

[0094] Specifically, a prediction head network (Head) can be used to perform target detection based on multi-scale features. For example, the target detection model may also include a prediction head network, which can be used for target classification and position regression. In this embodiment, the prediction head network can be used to predict the target category, target position, and target confidence of the target to be detected, and generate a detection result based on the target category, target position, and target confidence.

[0095] For example, the prediction head network may include a target classification branch and a target position regression branch, wherein the classification branch is used to identify the target category. The target position regression branch can be used to identify the target position, and the target position regression branch can be implemented by ScopeHead, which has two branches: a bin classification branch and a border regression branch. Among them, the bin classification score performs N-class classification on the anchor selection in four directions, and the border regression score performs boundary prediction on the anchor selected from the bin classification branch. Specifically, ScopeHead considers N candidate anchors in each of the four directions (i.e., left, right, top, and bottom). The bin classification branch aims to perform N-class classification to learn the selection of anchors. For these candidate anchors, the boundary regression branch performs regression to locate the boundary along each direction. Specifically, for each direction, the bin classification branch selects the anchor with the highest position confidence score, and decodes its corresponding regression prediction to obtain the boundary position of the target in each direction; then the target bounding box is determined by collecting the boundary position along the four directions to regress the target position, and the target position with the highest position confidence score is determined as the position coordinate of the target to be detected.

[0096] For example, the detection result may include the category of the target to be detected and the position coordinates of the target to be detected.

[0097] In this embodiment, the step of “outputting the detection result” may include: generating an output image according to the detection result, where the output image is an image to be detected marked with a target category and a target position of the target to be detected.

[0098] Specifically, the output image can be generated by marking the position of each target to be detected on the image to be detected with an anchor frame, and marking the target category of the target to be detected around the anchor frame, as well as the position confidence and category confidence. Figure 4 and Figure 5 The process of the target detection method provided in the embodiment of the present application can be summarized into the following steps:

[0099] Step 201: Obtain a picture to be detected.

[0100] For example, the image to be detected can be specifically understood as an image with detection requirements. The image to be detected can include a target to be detected, and the image to be detected can be an image captured in real time by an image acquisition device, such as an image captured in real time by a camera, or an image uploaded by a user through a user terminal. For example, the image to be detected can be a corresponding image in each detection scene, and the detection scene can be understood as a scene used by a target detection model, such as a detection scene that can be a vehicle detection scene, a pedestrian detection scene, an object detection scene, a target action detection scene in a video, and other detection scenes. Among them, the target to be detected can correspond to the detection scene, for example, in a vehicle detection scene, the target to be detected can be a vehicle; in a pedestrian detection scene, the target to be detected can be a pedestrian; in an object detection scene, the target to be detected can be an object; in a target action detection scene in a video, the target to be detected can be an object action.

[0101] For example, taking autonomous driving as an example, the image acquisition device on the vehicle collects images of the environment around the vehicle in real time during the driving process. The images may include buildings, pedestrians, motor vehicles and non-motor vehicles, etc. The collected images are used as images to be detected for target detection, and pedestrians, motor vehicles and non-motor vehicles can be identified, providing a basis for autonomous driving, for example, route planning for autonomous driving is performed based on the recognition results.

[0102] Step 202: extracting a feature map based on the image to be detected.

[0103] The convolution module of the backbone network can be used to extract features of the image to be detected to obtain a feature map of the image to be detected, wherein the feature map of the image to be detected can represent the global features of the image to be detected.

[0104] Step 203: Divide the feature map into multiple equal parts to obtain multiple first intermediate layer feature maps.

[0105] Among them, the first intermediate layer feature map can represent the local features of the image to be detected or represent the fine-grained local features of the image to be detected.

[0106] Specifically, the feature map can be equally divided into a preset number of parts to obtain multiple first intermediate layer features. Then, in the subsequent steps, the multiple first intermediate layer features are combined and spliced ​​in pairs, and then the grouped features are fused, which can effectively improve the utilization rate of the features, enhance the network topology, and achieve the effect of multi-scale feature fusion.

[0107] Among them, this application does not limit the preset number of copies, and different preset numbers can be determined according to different tasks. Specifically, the preset score is a hyperparameter learned by the target detection model. For example, for classification tasks, the preset number of copies can be s1, and for target detection tasks, the preset number of copies can be s2. Among them, different preset numbers can also be set according to the size of the required receptive field. The larger the required receptive field, the larger the preset number.

[0108] Specifically, this embodiment uses represents the feature map, where c is the number of input channels, h and w are the height and width of the feature map respectively. For example, if the image to be detected is an RGB image, the number of input channels is 3. Specifically, when object detection is to be performed, the image to be detected can be input into the backbone network first, and then the convolution module extracts features based on the image to be detected to obtain a feature map. Then, the backbone module can convert the feature map The equal parts are s parts, each part is recorded as Where i∈{1,2,…,s}, the number of channels in each part is For example, see Figure 3 In this embodiment, s=4 is used as an example, and the feature map R in It is divided into 4 equal parts, namely The number of channels in each part is

[0109] Step 204: concatenate every two first intermediate layer feature maps to obtain multiple grouped feature maps.

[0110] Specifically, each first intermediate layer feature map can be merged and concatenated in pairs through multiplexing transformation to obtain multiple group feature maps. Represents that, where each group feature map The number of channels is For example, see Figure 3 ,Will Merge two by two to obtain multiple group feature maps In this embodiment, the number of grouped feature maps is determined by the number of first intermediate layer feature maps. Assuming there are s first intermediate layer feature maps, the number of grouped feature maps is (s*(s-1)) / 2.

[0111] Step 205: Sort the multiple grouping feature maps, and determine multiple second intermediate layer feature maps based on the first preset formula and the grouping feature maps.

[0112] Among them, the first preset formula is:

[0113]

[0114] in, is the i-th second intermediate layer feature map, is the i-th group feature map in the sorted order, is the i-1th second intermediate layer feature map, f i′ and b i′ They are the convolution kernel and bias term of size k×k corresponding to the i-th group feature map, and cat represents the concatenation operation on the channel dimension.

[0115] Step 206: Concatenate multiple second intermediate layer feature maps in the channel dimension to obtain a target feature map.

[0116] Specifically, by merging the feature maps of the second intermediate layer, the multi-scale feature representation capability of the network can be enhanced, so that the target detection model can handle targets of different scales well.

[0117] In some embodiments, step 105 may include: performing a splicing operation on multiple second intermediate layer feature maps in the channel dimension to obtain a target feature map.

[0118] Specifically, this embodiment uses R′ out represents the target feature map, where Among them, Cat represents the splicing operation on the channel dimension, It is the feature map of the second intermediate layer.

[0119] For details, please continue to see Figure 3 , by and Perform splicing operation on the channel dimension to obtain the target feature map in, and The number of channels can be It is worth noting that the number of channels is artificially set to ensure that the number of input channels and the number of output channels are the same, that is, c.

[0120] Step 207: Obtain a target feature map through a feature pyramid network, and construct multi-scale features based on the target feature map.

[0121] Among them, step 207 can mainly include downsampling according to the target feature map to obtain a first multi-level feature map; upsampling according to the target feature map to obtain a second multi-level feature map; fusing the corresponding feature maps in the first multi-level feature map and the second multi-level feature map to obtain a multi-scale feature.

[0122] Among them, Feature Pyramid Networks (FPN) consists of bottom-up (downsampling) and top-down (upsampling) paths and side connections.

[0123] Among them, bottom-up (downsampling) is the forward propagation process of network training, the size of the feature map is getting smaller and smaller, and the stride is scaled, where the stride can represent the space occupied by each row of pixels in the memory. During the forward propagation process, the size of the feature map will change after passing through certain layers, and will not change when passing through other layers. The layers that do not change the size of the feature map can be classified into a stage, and the size of the feature map in each stage is basically the same. Therefore, the output of the last layer of each stage can be selected as the reference feature map for classification and regression, so that a feature pyramid can be formed. Among them, the features extracted by the last layer of each stage are optimal.

[0124] Among them, top-down (upsampling) is to upsample the high-level feature map (such as nearest neighbor upsampling). By upsampling the feature map, the upsampled feature map can have the same size as the feature map of the next layer.

[0125] The side connection means fusing the downsampled feature map with the upsampled feature map. The downsampled feature map can be convolved with the upsampled feature map to obtain a new feature map. The 1×1 convolution operation can change the number of channels to the same number of channels as the next layer, which is convenient for the subsequent classification of the fused features.

[0126] Each layer of the feature pyramid network combines features with different resolutions and different semantic information to obtain a more powerful representation. Upsampling can transfer high-semantic information to shallow features while retaining the high resolution (small receptive field) characteristics of shallow features, which can better balance semantic information and location information. Among them, multi-scale can refer to multiple spatial resolutions. It can be seen that multi-scale features contain features with multiple different spatial resolutions. The multi-resolution multi-scale features output by FPN replace the single-resolution feature map, and the same size anchor box is used at each level. The different scales of the feature map achieve the effect of multi-scale anchor boxes, and each level of the pyramid shares a similar semantic level.

[0127] Step 208: Predict the target category, target position, and target confidence based on the multi-scale features through the prediction head network.

[0128] Among them, step 208 may mainly include: determining the target confidence of the target to be detected in the image to be detected according to the multi-scale features; determining the target category and target position of the target to be detected according to the target confidence, and obtaining the detection result.

[0129] Among them, the target confidence may include category confidence and position confidence, and the step of "determining the target category and target position of the target to be detected according to the target confidence" may include: determining the preset category with the highest category confidence as the target category; determining the position with the highest position confidence as the target position.

[0130] For example, the target confidence may include category confidence, that is, the confidence that the target to be detected belongs to a certain category. The category confidence of a certain target, that is, the probability that the true category of the target belongs to A and the probability that it belongs to B. The category of the target to be detected can be determined according to the category confidence of the target to be detected, and the category with the highest category confidence score is determined as the target category of the target to be detected.

[0131] For example, the target confidence may include position confidence, that is, the confidence of the position of the target to be detected. The target position may include the center and length and width offset of the anchor frame. The position confidence of a target, that is, the probability that the true position of the target is X and the probability that the true position is Y. The position of the target to be detected can be determined based on the position confidence of the target to be detected, and the category with the highest position confidence score is determined as the target position of the target to be detected.

[0132] For example, the target detection model may also include a prediction head network (Head), which may be used for target classification and position regression. In this embodiment, the prediction head network may be used to predict target categories, target locations, and target confidences, and generate detection results based on the target categories, target locations, and target confidences.

[0133] For example, the prediction head network may include a target classification branch and a target position regression branch, wherein the classification branch is used to identify the target category. The target position regression branch can be used to identify the target position, which can be implemented by ScopeHead and is also divided into two parts: bin classification and border regression. The bin classification performs N-class classification on the anchor selection in four directions. Border regression performs boundary prediction on the anchor selected from the bin classification branch. Specifically, ScopeHead considers N candidate anchors in each of the four directions (i.e., left, right, top, and bottom). ScopeHead has two branches: a bin classification branch and a border regression branch. The bin classification branch aims to perform N-class classification to learn the selection of anchors. For these candidate anchors, the boundary regression branch performs regression to locate the boundary along each direction. Specifically, for each direction, the bin classification branch selects the anchor with the highest confidence and decodes its corresponding regression prediction to obtain the boundary position of the target in each direction; then the target bounding box (target position) is determined by collecting the boundary positions along the four directions.

[0134] The target position includes the center and length and width offsets of the anchor box.

[0135] Step 209: Determine the detection result according to the target category, target position and target confidence, and output the detection result.

[0136] Among them, the step of "outputting the detection results" may include: generating an output image according to the detection results, and the output image is a to-be-detected image marked with the target category and target position of the to-be-detected target.

[0137] Specifically, the output image can be generated by marking the position of each target to be detected on the image to be detected using an anchor frame, and marking the target category of the target to be detected around the anchor frame, as well as the position confidence and category confidence.

[0138] The detection result can be displayed through a graphical user interface.

[0139] Specifically, see Figure 5As can be seen from the above, when performing image target detection, first, the image to be detected is input into the backbone network. Among them, the backbone network may include a convolution module and a backbone module, and the convolution module may perform feature extraction on the image to be detected to obtain a feature map, and then the backbone module may obtain a target feature map according to the feature map. Among them, the target feature map is a feature map obtained by averaging, grouping, splicing, and fusing the feature map. Then, the feature pyramid network may construct a multi-scale feature according to the target feature map, and the multi-scale feature may be a multi-level feature map with proportional size. With the multi-level feature map, the information extracted by the backbone network can be more fully utilized, so that the target detection algorithm can better cope with multi-scale situations. Then, the target category, target position, and target confidence of the target to be detected in the image to be detected may be determined by the prediction head network, and then, the target detection result may be generated according to the target category, target position, and target confidence. The embodiment of the present application effectively improves the utilization rate of the feature by averaging the feature map and fusing the grouped features, enhances the network topology, and achieves the effect of multi-scale fusion, thereby enhancing the learning ability of the network and improving the accuracy of target detection.

[0140] All of the above technical solutions can be arbitrarily combined to form optional embodiments of the present application, which will not be described in detail here.

[0141] The embodiment of the present application provides a target detection method, including: obtaining a picture to be detected and performing feature extraction on the picture to be detected to obtain a feature map of the picture to be detected; determining multiple first intermediate layer feature maps according to the feature map; grouping and splicing the multiple first intermediate layer feature maps to obtain multiple grouped feature maps, wherein each grouped feature map contains at least two first intermediate layer feature maps; determining multiple second intermediate layer feature maps according to the grouped feature maps; merging the multiple second intermediate layer feature maps to obtain a target feature map; performing target detection based on the target feature map, and outputting the detection result. The present application divides the picture to be detected into equal parts, and groups and splices them to obtain a grouped feature map, and then fuses the grouped feature maps to obtain a target feature map, thereby effectively improving the utilization rate of features, enhancing the network topology, and achieving multi-scale feature fusion. Then, target detection is performed based on the target feature map, so that the network can better detect targets of different scales and improve the accuracy of target detection.

[0142] The above describes the method embodiment of the present application in detail. Figure 6 , the device embodiments of the present application are described in detail. It should be understood that the device embodiments and the method embodiments correspond to each other, and similar descriptions can refer to the method embodiments.

[0143] Figure 6 is a schematic structural diagram of a target detection device 10 according to an embodiment of the present application, such as Figure 6As shown, the target detection device 10 may include:

[0144] The acquisition module 11 is used to acquire the image to be detected and perform feature extraction on the image to be detected to obtain a feature map of the image to be detected;

[0145] A first determination module 12, configured to determine a plurality of first intermediate layer feature maps according to the feature map;

[0146] A splicing module 13 is used to group and splice the multiple first intermediate layer feature maps to obtain multiple grouped feature maps, wherein each grouped feature map contains at least two first intermediate layer feature maps;

[0147] A second determination module 14, configured to determine a plurality of second intermediate layer feature maps according to the grouped feature maps;

[0148] A merging module 15, used for merging multiple second intermediate layer feature maps to obtain a target feature map;

[0149] The detection module 16 is used to perform target detection based on the target feature map and output the detection result.

[0150] Optionally, the first determination module 12 may be used to: divide the feature map into multiple parts in the channel dimension, and determine each part as a first intermediate layer feature map.

[0151] Optionally, the splicing module 13 may be used to splice every two first intermediate layer feature maps to obtain a plurality of grouped feature maps.

[0152] Optionally, the second determination module 14 can be used to: sort the group feature maps; convolve the first group feature map in the sorted order according to a preset convolution kernel to obtain the first second intermediate layer feature map; concatenate the i-th group feature map in the sorted order with the i-1-th second intermediate layer feature map, and convolve them according to a preset convolution kernel to obtain the i-th second intermediate layer feature map, where i is an integer greater than 1.

[0153] Optionally, the merging module 15 may be used to: perform a splicing operation on multiple second intermediate layer feature maps in the channel dimension to obtain a target feature map.

[0154] Optionally, the merging module 15 may also be used to perform weighted addition on multiple second intermediate layer feature maps to obtain a target feature map.

[0155] Optionally, the detection module 16 can be used to: construct a multi-scale feature according to the target feature map; perform target prediction according to the multi-scale feature, and output a detection result.

[0156] Optionally, the detection module 16 can be mainly used to: downsample according to the target feature map to obtain a first multi-level feature map; upsample according to the target feature map to obtain a second multi-level feature map; merge the corresponding feature maps in the first multi-level feature map and the second multi-level feature map to obtain multi-scale features.

[0157] Optionally, the detection module 16 may also be used to: determine the target confidence of the target to be detected in the image to be detected according to the multi-scale features; determine the target category and target position of the target to be detected according to the target confidence, and obtain a detection result.

[0158] Optionally, the target confidence includes category confidence and position confidence, and the detection module 16 may also be used to: determine a preset category with the highest category confidence as the target category; and determine a position with the highest position confidence as the target position.

[0159] Optionally, the detection module 16 may also be used to: generate an output image according to the detection result, where the output image is the image to be detected marked with the target category and target position of the target to be detected.

[0160] It should be noted that the functions of each module in the target detection device 10 in the embodiment of the present application can correspond to the specific implementation methods in the above-mentioned method embodiments, which will not be repeated here.

[0161] Each module in the above-mentioned target detection device 10 can be implemented in whole or in part by software, hardware or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each of the above-mentioned modules.

[0162] The target detection device 10 provided in the embodiment of the present application obtains the image to be detected through the acquisition module 11, and extracts features of the image to be detected to obtain a feature map of the image to be detected. Then the first determination module 12 determines multiple first intermediate layer feature maps according to the feature map, and then the splicing module 13 groups and splices the multiple first intermediate layer feature maps to obtain multiple grouped feature maps, wherein each grouped feature map contains at least two first intermediate layer feature maps. Then the second determination module 14 determines multiple second intermediate layer feature maps according to the grouped feature map, and then the merging module 15 merges the multiple second intermediate layer feature maps to obtain a target feature map. Finally, the detection module 16 performs target detection based on the target feature map and outputs the detection result, thereby improving the utilization rate of the features, enabling the network to better detect targets of different scales and improving the accuracy of target detection.

[0163] In some embodiments, the present application further provides a computer device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0164] Figure 7 A schematic structural diagram of a computer device provided in an embodiment of the present application, which may be Figure 1 The terminal or server shown. Figure 7 As shown, the computer device 30 may include: a communication interface 31, a memory 32, a processor 33 and a communication bus 34. The communication interface 31, the memory 32, and the processor 33 communicate with each other through the communication bus 34. The communication interface 31 is used for the computer device 30 to communicate data with external devices. The memory 32 can be used to store software programs and modules, and the processor 33 runs the software programs and modules stored in the memory 32, such as the software programs of the corresponding operations in the aforementioned method embodiments.

[0165] In some embodiments, the processor 33 may call software programs and modules stored in the memory 32 to perform the following operations:

[0166] Acquire a picture to be detected, and perform feature extraction on the picture to be detected to obtain a feature map of the picture to be detected; determine multiple first intermediate layer feature maps based on the feature map; group and splice the multiple first intermediate layer feature maps to obtain multiple grouped feature maps, wherein each grouped feature map contains at least two first intermediate layer feature maps; determine multiple second intermediate layer feature maps based on the grouped feature maps; merge the multiple second intermediate layer feature maps to obtain a target feature map; perform target detection based on the target feature map, and output the detection result.

[0167] The embodiment of the present application also provides a computer-readable storage medium for storing a computer program. The computer-readable storage medium can be applied to a computer device, and the computer program enables the computer device to execute the corresponding process in the video preview method in the embodiment of the present application, which will not be described here for brevity.

[0168] The embodiment of the present application also provides a computer program product, which includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the corresponding process in the video preview method in the embodiment of the present application, which will not be described here for brevity.

[0169] The embodiment of the present application also provides a computer program, which includes computer instructions, which are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the corresponding process in the video preview method in the embodiment of the present application, which will not be described here for the sake of brevity.

[0170] It should be understood that the processor of the embodiment of the present application may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method embodiment can be completed by an integrated logic circuit of hardware in the processor or an instruction in the form of software. The above processor may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit, or a processor that can process signals.

[0171] (Application Specific Integrated Circuit, ASIC), off-the-shelf programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor to be executed, or a combination of hardware and software modules in the decoding processor to be executed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0172] It can be understood that the memory in the embodiments of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus random access memory (DRRAM). It should be noted that the memory of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0173] It should be understood that the above-mentioned memory is exemplary but not restrictive. For example, the memory in the embodiments of the present application may also be static random access memory (static RAM, SRAM), dynamic random access memory (dynamic RAM, DRAM), synchronous dynamic random access memory (synchronous DRAM, SDRAM), double data rate synchronous dynamic random access memory (double data rate SDRAM, DDR SDRAM), enhanced synchronous dynamic random access memory (enhanced SDRAM, ESDRAM), synchronous link dynamic random access memory (synch link DRAM, SLDRAM) and direct memory bus random access memory (Direct Rambus RAM, DRRAM), etc. That is to say, the memory in the embodiments of the present application is intended to include but not limited to these and any other suitable types of memory.

[0174] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0175] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0176] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0177] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0178] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0179] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer or a server) to perform all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage media include various media that can store program codes, such as USB flash drives, mobile hard drives, ROM, RAM, magnetic disks, or optical disks.

[0180] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A target detection method, characterized in that: include: Acquire a picture to be detected, and perform feature extraction on the picture to be detected to obtain a feature map of the picture to be detected, wherein the feature map represents the global features of the picture to be detected; Determine a plurality of first intermediate-layer feature maps according to the feature map, where the first intermediate-layer feature maps represent local features or fine-grained local features of the image to be detected; Grouping and splicing the multiple first intermediate layer feature maps to obtain multiple grouped feature maps, wherein each of the grouped feature maps contains at least two of the first intermediate layer feature maps; Determine a plurality of second intermediate layer feature maps according to the grouped feature maps, each of the second intermediate layer feature maps representing combined feature information of different receptive field sizes; Merging the plurality of second intermediate layer feature maps through a merging transformation based on receptive field augmentation to obtain a target feature map; The target feature map is processed through a feature pyramid network to construct a multi-scale feature, wherein the feature pyramid network includes a bottom-up path, a top-down path and a side connection, including: downsampling the target feature map through the bottom-up path in the feature pyramid network to obtain a first multi-level feature map; upsampling the target feature map through the top-down path in the feature pyramid network to obtain a second multi-level feature map; and merging the corresponding feature maps in the first multi-level feature map and the second multi-level feature map through the side connection in the feature pyramid network to obtain a multi-scale feature; Performing target detection according to the multi-scale features through a prediction head network to predict the target category, target position and target confidence of the target to be detected in the image to be detected, wherein the prediction head network includes a target classification branch and a target position regression branch, which are used to predict the target category, target position and target confidence of the target to be detected in the image to be detected; Output a detection result according to the target category, the target position and the target confidence.

2. The target detection method according to claim 1, characterized in that: The determining a plurality of first intermediate layer feature maps according to the feature map comprises: In the channel dimension, the feature map is equally divided into a plurality of parts, and each part is determined as a first intermediate layer feature map.

3. The target detection method according to claim 2, characterized in that: The step of grouping and splicing the plurality of first intermediate layer feature maps to obtain a plurality of grouped feature maps includes: Every two of the first intermediate layer feature maps are concatenated to obtain multiple grouped feature maps.

4. The target detection method according to claim 1, characterized in that: The determining a plurality of second intermediate layer feature maps according to the grouped feature maps comprises: Sorting the grouping feature graphs; The first group feature map in the sorting order is convolved according to the preset convolution kernel to obtain the first second intermediate layer feature map; The i-th grouping feature map in the sorting order is concatenated with the i-1-th second intermediate layer feature map, and convolution processing is performed according to the preset convolution kernel to obtain the i-th second intermediate layer feature map, where i is an integer greater than 1.

5. The target detection method according to claim 4, characterized in that: The step of merging the plurality of second intermediate layer feature maps to obtain a target feature map includes: The multiple second intermediate layer feature maps are concatenated in the channel dimension to obtain a target feature map.

6. The target detection method according to claim 4, characterized in that: The step of merging the plurality of second intermediate layer feature maps to obtain a target feature map includes: The multiple second intermediate layer feature maps are weighted added to obtain a target feature map.

7. The target detection method according to claim 1, characterized in that: The performing target detection according to the multi-scale features comprises: Determining the target confidence of the target to be detected in the image to be detected according to the multi-scale features; The target category and target position of the target to be detected are determined according to the target confidence level to obtain a detection result.

8. The target detection method according to claim 7, characterized in that: The target confidence includes category confidence and position confidence, and determining the target category and target position of the target to be detected according to the target confidence to obtain the detection result includes: Determine the preset category with the highest category confidence as the target category; The position with the highest position confidence is determined as the target position.

9. The target detection method according to claim 7 or 8, characterized in that: The output detection result includes: An output image is generated according to the detection result, where the output image is the image to be detected marked with the target category and the target position of the target to be detected.

10. A target detection device, characterized in that: include: An acquisition module is used to acquire a picture to be detected, and perform feature extraction on the picture to be detected to obtain a feature map of the picture to be detected, wherein the feature map represents the global features of the picture to be detected; A first determination module, configured to determine a plurality of first intermediate-layer feature maps according to the feature map, wherein the first intermediate-layer feature maps represent local features or fine-grained local features of the image to be detected; A splicing module, used for grouping and splicing the plurality of first intermediate layer feature maps to obtain a plurality of grouped feature maps, wherein each of the grouped feature maps contains at least two of the first intermediate layer feature maps; A second determination module is used to determine a plurality of second intermediate layer feature maps according to the grouped feature map, each of which represents combined feature information of different receptive field sizes; A merging module, used for merging the plurality of second intermediate layer feature maps through a merging transformation based on receptive field augmentation to obtain a target feature map; The detection module is used to process the target feature map through a feature pyramid network to construct a multi-scale feature, wherein the feature pyramid network includes a bottom-up path, a top-down path and a side connection, including: downsampling the target feature map through the bottom-up path in the feature pyramid network to obtain a first multi-level feature map; upsampling the target feature map through the top-down path in the feature pyramid network to obtain a second multi-level feature map; fusing the corresponding feature maps in the first multi-level feature map and the second multi-level feature map through the side connection in the feature pyramid network to obtain a multi-scale feature; performing target detection according to the multi-scale feature through a prediction head network to predict the target category, target position and target confidence of the target to be detected in the image to be detected, wherein the prediction head network includes a target classification branch and a target position regression branch, which are used to predict the target category, target position and target confidence of the target to be detected in the image to be detected; and outputting a detection result according to the target category, the target position and the target confidence.

11. A computer-readable storage medium comprising instructions, characterized in that: When the instructions are executed on a computer device, the computer device is enabled to execute the target detection method according to any one of claims 1 to 9.

12. A computer device, characterized in that: The computer device includes a processor and a memory, wherein a computer program is stored in the memory, and the processor is used to execute the steps in the target detection method described in any one of claims 1 to 9 by calling the computer program stored in the memory.

13. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the target detection method described in any one of claims 1 to 9 are implemented.

Images