A small target image detection method and related apparatus

Abstract

Embodiments of the present application provide a small target image detection method and related device, which are used to improve the detection accuracy of small target images in an image. The method comprises: obtaining an original feature map of a to-be-detected image comprising a small target image; segmenting the original feature map to obtain M sub-maps, and performing context feature extraction and fusion on the M sub-maps to obtain a first feature map, the first feature map having spatial information of the small target image, M being an integer greater than 1; performing down-sampling on the first feature map to obtain N first-type feature maps, the N first-type feature maps having different resolutions, the N first-type feature maps comprising semantic information of the small target image at different resolutions, N being an integer greater than 1; fusing the N first-type feature maps and the first feature map to obtain a second feature map; and performing detection on the N first-type feature maps and the second feature map to obtain the small target image. The present application can be applied to computer vision, image processing, and artificial intelligence.

Description

Technical Field

[0001] This application relates to the fields of computer vision, image processing, artificial intelligence, etc., and in particular to a method and related apparatus for small target image detection. Background Technology

[0002] With the development of technology, computer vision technology is being used more and more in game scenarios. For example, in the testing process of complex large-scale games, it is often necessary to detect corresponding objects in the screen to realize the automatic operation of game objects. As game scenes change and versions are updated, more and more categories of objects of different sizes will appear.

[0003] In related technologies, the object detection process may include: inputting an image of the game interface into a neural network for detection, and outputting position indicators of each game object in the image of the game interface. However, neural networks are usually unable to meet the requirements for detecting small game objects. Summary of the Invention

[0004] This application provides a method and related apparatus for detecting small targets in images, which can improve the detection accuracy of small targets in images.

[0005] In view of this, this application provides a method and related apparatus for detecting small target images, comprising: acquiring an original feature map of a target image including a small target image; segmenting the original feature map into M sub-maps through a first network layer of a target model, and extracting and fusing contextual features from the M sub-maps to obtain a first feature map, wherein the first feature map contains spatial information of the small target image, and M is an integer greater than 1; downsampling the first feature map through a second network layer of the target model to obtain N first-class feature maps, wherein the N first-class feature maps have different resolutions, and the N first-class feature maps include semantic information of the small target image at different resolutions, wherein N is an integer greater than 1; fusing the N first-class feature maps and the first feature map through a third network layer of the target model to obtain a second feature map; and detecting the N first-class feature maps and the second feature map through a detection layer of the target model to obtain the small target image.

[0006] This application also provides a detection apparatus, including: an acquisition module for acquiring an original feature map of an image to be detected, including a small target image;

[0007] The first feature extraction module is used to segment the original feature map into M sub-maps through the first network layer of the target model, and to extract and fuse the context features of the M sub-maps to obtain a first feature map. The first feature map has the spatial information of the small target image, and M is an integer greater than 1.

[0008] The second feature extraction module is used to downsample the first feature map through the second network layer of the target model to obtain N first-class feature maps. The N first-class feature maps have different resolutions and include the semantic information of the small target image at different resolutions. N is an integer greater than 1.

[0009] The fusion module is used to fuse the N first-class feature maps and the first feature map through the third network layer of the target model to obtain a second feature map;

[0010] The detection module is used to detect the N first-class feature maps and the second feature map through the detection layer of the target model to obtain the small target image.

[0011] In one possible design, in another implementation of another aspect of the embodiments of this application, the first network layer is an adaptive spatial parallel convolutional layer. The first feature extraction module is used to segment the original feature map into M sub-maps through the M channels of the adaptive spatial parallel convolutional layer, and to perform parallel convolution on the M sub-maps to obtain M sub-feature maps, wherein the convolutional layers corresponding to the M sub-maps are different, and M is an integer greater than 1; to connect the M sub-feature maps and perform feature batch normalization processing to obtain a third feature map; to connect and fuse the third feature map with the original feature map to obtain a fourth feature map; and to downsample and extract features from the fourth feature map to obtain the first feature map.

[0012] In one possible design, in another implementation of another aspect of the embodiments of this application, the first feature extraction module is specifically used to divide the original feature map into two sub-maps through the two channels of the adaptive spatial parallel convolutional layer. One of the two sub-maps corresponds to a standard convolutional layer Conv2d, and the convolutional layer corresponding to the other sub-map includes a single-layer convolutional layer Conv and a standard convolutional layer Conv2d.

[0013] In one possible design, in another implementation of another aspect of the embodiments of this application, the convolution kernels of the standard convolutional layer Conv2d and the single-layer convolutional layer Conv are 3×3 or 1×1, and the stride is 1.

[0014] In one possible design, in another implementation of another aspect of the embodiments of this application, the second network layer includes a sub-backbone network and a feature map pyramid network. The second feature extraction module is specifically used to process the first feature map through X cross-stage double convolution transformation operators and 1 cross-stage double convolution transformation combination operator in the sub-backbone network to obtain Q second-class feature maps, wherein X and Q are positive integers, the cross-stage double convolution transformation operator includes a convolution unit and a feature extraction unit, the cross-stage double convolution transformation combination operator includes a convolution unit and a spatial pyramid pooling unit, and the Q second-class feature maps include the first feature map; the Q second-class feature maps are then connected by a matrix in the feature map pyramid network to obtain the N first-class feature maps.

[0015] In one possible design, in another implementation of another aspect of the embodiments of this application, the second feature extraction module is specifically used to obtain the first first-class feature map in the N first-class feature maps by passing the last output Q-th second-class feature map in the Q second-class feature maps through a feature map pyramid network according to the output timing.

[0016] The first first-type feature map is connected to the (Q-1)th second-type feature map among the Q second-type feature maps, and the second first-type feature map among the N first-type feature maps is obtained through the cross-stage double convolution transformation and combination operator in the feature map pyramid network; and so on, until the second second-type feature map among the Q second-type feature maps is connected to the (N-1)th first-type feature map, and the Nth first-type feature map among the N first-type feature maps is obtained through the cross-stage double convolution transformation and combination operator in the feature map pyramid network. The N first-type feature maps include the first first-type feature map, the second first-type feature map, and the Nth first-type feature map.

[0017] In one possible design, in another implementation of another aspect of the embodiments of this application, the third network layer is a multi-scale fusion network, and the fusion module is specifically used to map the N first-class feature maps through an ascending filter array of sub-pixel convolutional layers to generate a fifth feature map, the resolution of the fifth feature map being greater than the resolution of the N first-class feature maps; and to connect and fuse the fifth feature map with the first feature map to obtain the second feature map.

[0018] In one possible design, in another implementation of another aspect of the embodiments of this application, the fusion module is specifically used to generate a sixth feature map by mapping the (N-1)th first-class feature map of the N first-class feature maps through the ascending filter array of the sub-pixel convolutional layer according to the output timing; to connect and fuse the sixth feature map with the Nth first-class feature map of the N first-class feature maps, and to upsample it to generate the fifth feature map.

[0019] In one possible design, in another implementation of another aspect of the embodiments of this application, the fusion module is specifically used to extract features from the first feature map through the convolutional layer of the third network layer to obtain a seventh feature map; and to connect and fuse the seventh feature map with the fifth feature map to obtain the second feature map.

[0020] In one possible design, in another implementation of another aspect of the embodiments of this application, the detection module is specifically used to pass the N first-class feature maps and the second feature maps through the detection layer to obtain a set of classification and localization results; and to obtain the small target image by filtering the classification and localization results in the set of classification and localization results through confidence threshold and non-maximum suppression processing.

[0021] In one possible design, in another implementation of another aspect of the embodiments of this application, the acquisition module is further configured to acquire a sample set and an initial model, the initial model including an initial network layer and an initial detection layer, and the sample set including sample images of sample objects, the true location of the sample objects in the sample images, and the true classification of the sample objects in the sample images;

[0022] The detection device further includes a processing module, used to input the sample set into the initial model to obtain sample detection locations and sample detection classifications; based on the sample detection locations, sample detection classifications, and the true locations and true classifications, using a joint loss function, to determine the joint difference between the sample detection locations and the true locations, as well as the sample detection classifications and the true classifications output by the initial model; based on the joint difference, to adjust the model parameters of the initial model until the target conditions are met, and then stop adjusting to obtain the target model.

[0023] In one possible design, in another implementation of another aspect of the embodiments of this application, the joint loss function includes a location loss, a confidence loss, and a classification loss, wherein the location loss is the error between the detected sample location and the true location, the confidence loss is used to indicate the relative error of the predicted confidence score of the sample object, and the classification loss is used to indicate the error between the detected sample classification and the true classification.

[0024] This application also provides a computer device, including: a memory, a processor, and a bus system;

[0025] The memory is used to store programs;

[0026] The processor is used to execute programs in memory, and the processor is used to execute the methods mentioned above according to the instructions in the program code;

[0027] Bus systems are used to connect memory and processor to enable communication between them.

[0028] Another aspect of this application provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the methods described above.

[0029] Another aspect of this application provides a computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in the above aspects.

[0030] As can be seen from the above technical solutions, the embodiments of this application have the following advantages: when extracting features from the image to be detected, spatial information and semantic information of the small target image are extracted, and the spatial information and semantic information are fused together, so that the spatial information and semantic information of the image to be detected can be richly represented, thereby improving the detection accuracy of the small target image. Attached Figure Description

[0031] Figure 1 This is a schematic diagram of an implementation environment for the small target image detection method in this application embodiment;

[0032] Figure 2 This is a schematic diagram of another implementation environment for the small target image detection method in this application;

[0033] Figure 3 This is an exemplary result diagram of small target image detection in an embodiment of this application;

[0034] Figure 4 This is an exemplary network structure diagram of a target model used for small target image detection in the embodiments of this application;

[0035] Figure 5 This is a schematic diagram of one embodiment of the small target image detection method in this application;

[0036] Figure 6 This is a schematic diagram of an exemplary network structure of the adaptive spatial parallel convolution module in the target model of this application embodiment, and a schematic diagram of an exemplary process for obtaining the first feature map;

[0037] Figure 7 This is a schematic diagram of the network structure of the sub-backbone module and the feature pyramid network in the target model of this application embodiment, and an exemplary flowchart for obtaining the first type of feature map;

[0038] Figure 8a This is a schematic diagram of an exemplary network structure of the cross-stage double convolution transformation operator 1 in the sub-backbone structure of this application embodiment;

[0039] Figure 8b This is a schematic diagram of an exemplary network structure of the adaptive double convolution transformation operator 1 (ABT1) in the cross-stage double convolution transformation operator 1 in the embodiments of this application;

[0040] Figure 8c This is a schematic diagram of an exemplary network structure for a cross-stage double convolutional transformation combination operator in the sub-backbone structure of this application.

[0041] Figure 8d This is a schematic diagram of an exemplary network structure for the cross-stage double convolution transformation operator 2 in the feature pyramid network of this application embodiment;

[0042] Figure 8e This is a schematic diagram of an exemplary network structure of the adaptive double convolution transformation operator 2 (ABT2) in the cross-stage double convolution transformation operator 2 in the embodiments of this application;

[0043] Figure 8f This is a schematic diagram of an exemplary network structure for the adaptive double convolutional transformation combination operator i (ABTCi) between the sub-backbone module and the feature pyramid network in this application.

[0044] Figure 8g This is a schematic diagram of an exemplary network structure of Conv in an embodiment of this application;

[0045] Figure 9 This is a schematic diagram of an exemplary network structure of the fast multi-scale fusion module in the target model of this application embodiment and a schematic diagram of an exemplary process for obtaining the second feature map;

[0046] Figure 10 This is a schematic diagram of one embodiment of the detection device in this application;

[0047] Figure 11 This is a schematic diagram of another embodiment of the detection device in this application;

[0048] Figure 12 This is a schematic diagram of another embodiment of the detection device in this application;

[0049] Figure 13 This is a schematic diagram of another embodiment of the detection device in this application. Detailed Implementation

[0050] This application provides a method and related apparatus for detecting small targets in images, which can improve the detection accuracy of small targets in images.

[0051] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the application described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “corresponding to,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus. Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the terms "comprising" and "including" as used in the embodiments of this application mean that the corresponding feature can be implemented as the presented feature, information, data, step, operation, element, and / or component, but do not exclude implementation as other features, information, data, step, operation, element, component, and / or combinations thereof supported by the art. It should be understood that when we say that an element is "connected" or "coupled" to another element, the element can be directly connected or coupled to the other element, or it can mean that the element and the other element establish a connection relationship through an intermediate element. Furthermore, "connected" or "coupled" as used herein can include wireless connection or wireless coupling. The term "and / or" as used herein indicates at least one of the items defined by the term; for example, "A and / or B" indicates implementation as "A," or implementation as "A," or implementation as "A and B."

[0052] For ease of understanding, some technical terms in the embodiments of this application will be explained below.

[0053] The receptive field refers to the region of the input image that a point on the feature map can see. In other words, the point on the feature map is calculated from the receptive field size of the input image. A larger receptive field value indicates a wider range of original image information that the neuron can access, meaning it may contain more global and semantically higher-level features; conversely, a smaller value indicates that the features it contains are more local and detailed. Therefore, the receptive field value can be used to roughly determine the abstraction level of each layer.

[0054] Small target images: In this application, target images with an image resolution of 50 × 50 or lower are generally referred to as small target images.

[0055] Feature Pyramid Networks (FPNs) are an enhancement of the traditional Convolutional Neural Network (CNN) for representing image information. Based on the backbone network and built upon it with deeper convolutional layers, FPNs enhance the features extracted by the backbone network at multiple scales. Specifically, they construct feature pyramids of different scales based on the features extracted by the backbone network, extracting features for each scale of the image. This results in multi-scale feature representations, and all levels of feature maps possess strong semantic information, including high-resolution shallow features. The aim is to improve the feature extraction method of CNN networks, enabling the final output features to better represent the information of the input image across various dimensions. Its basic process consists of three parts: a bottom-up path (generating features of different dimensions from bottom to top); a top-down path (feature supplementation and enhancement from top to bottom); and the relationship between the CNN network layer features and the final output features across various dimensions.

[0056] Spatial information: Spatial information refers to the spatial position or relative orientation relationship between multiple targets segmented from an image. These relationships can be categorized into connectivity / adjacency, overlap / intersection, and containment / enclosure relationships. Spatial position information is generally divided into two categories: relative spatial position information and absolute spatial position information. The former emphasizes the relative situation between targets, such as up / down and left / right relationships, while the latter emphasizes the distance and orientation between targets. Obviously, relative spatial position can be derived from absolute spatial position, but expressing relative spatial position information is often simpler. Using spatial relationship features can enhance the ability to describe and distinguish image content, but spatial relationship features are often sensitive to image or target rotation, inversion, and scale changes.

[0057] Semantic information: Image semantics is divided into visual, object, and conceptual layers. The visual layer, commonly understood as the bottom layer, includes features such as color, texture, and shape; these features are called low-level semantic features. The object layer, the intermediate layer, typically contains attribute features, representing the state of an object at a given moment. The conceptual layer is the high layer, representing what the image expresses that is closest to human understanding. Low-level image features refer to contours, edges, color, texture, and shape features, with edges and contours reflecting the image content. High-level semantic features indicate what humans can see. For example, extracting low-level features from a face allows us to extract the face's contour, nose, and eyes; high-level features indicate that the image represents a face. High-level features provide richer semantic information, but the target location is relatively coarse. Deeper features contain stronger high-level semantics and have greater discriminative power. We call the visual features of an image the visual space, and the semantic information of the categories the semantic space.

[0058] Artificial Intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to have perception, reasoning, and decision-making capabilities. AI technology is a comprehensive discipline involving a wide range of fields, encompassing both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, mechatronics, and blockchain. AI software technologies mainly include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0059] Computer Vision (CV) is a science that studies how to enable machines to "see." More specifically, it refers to machine vision, which uses cameras and computers to replace human eyes in tasks such as target recognition, tracking, and measurement, and further performs image processing to create images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision studies related theories and technologies, attempting to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include image processing, image recognition, image semantic understanding, image retrieval, optical character recognition (OCR), video processing, video semantic understanding, video content / behavior recognition, 3D object reconstruction, three-dimensional (3D) technology, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), and common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0060] In the field of small object image detection, current methods typically employ relatively large receptive fields to capture local contextual information of objects in the image, attempting to obtain rich spatial information about the objects. However, excessive use of large-scale convolutional kernels increases the time and space complexity of the detection model, leading to poor detection performance and reduced detection results. Secondly, in neural networks, image features are gradually transferred to deeper layers, resulting in a corresponding decrease in image size and spatial information. Furthermore, due to the small size of small object images, the spatial information of small objects often disappears during feature transfer. Therefore, current neural networks cannot meet the requirements for detecting small object images. To address this, current optimizations typically focus on feature extraction, increasing image resolution, and multi-scale feature fusion. However, current optimization schemes are insufficient for practical applications.

[0061] To address the aforementioned technical problems, this application provides the following technical solution: Obtaining an original feature map of a target image including a small target image; segmenting the original feature map into M sub-maps using a first network layer of the target model, and extracting and fusing contextual features from the M sub-maps to obtain a first feature map, wherein the first feature map contains spatial information of the small target image, and M is an integer greater than 1; downsampling the first feature map using a second network layer of the target model to obtain N first-class feature maps, wherein the N first-class feature maps have different resolutions and include semantic information of the small target image at different resolutions, and N is an integer greater than 1; fusing the N first-class feature maps and the first feature map using a third network layer of the target model to obtain a second feature map; and detecting the N first-class feature maps and the second feature map using a detection layer of the target model to obtain the small target image.

[0062] Figures 1 to 2 This is a schematic diagram illustrating the implementation environment of the small target image detection method provided in this application. For example... Figure 1 As shown, the implementation environment may include a computer device 101. In one exemplary embodiment, the computer device 101 may be pre-configured with a trained target model and use the target model to perform target detection on the image to be detected; alternatively, the computer device 101 may also train the target model using a large number of samples and use the target model to perform target detection on the image to be detected. In this case, the image to be detected may be an image stored by the computer device 101, or it may be obtained by the computer device 101 taking a frame from the display interface of the application client installed on the computer device in response to an instruction.

[0063] In an exemplary application scenario, such as Figure 2As shown, the implementation environment may further include: a computer device 101 and an image acquisition device 102. The image acquisition device 102 can acquire a target image of a small target to be detected and send the target image to the computer device 101. The computer device 101 is pre-configured with a trained target model. The computer device 101 uses the target model to perform target detection on the target image and sends the detection result to the image acquisition device 102. For example, the image acquisition device 102 can be a smartphone, which can have an application (such as game testing software) with target image detection function installed. The image acquisition device 102 can be the backend server of the application, specifically manifested as a desktop computer. The smartphone and the desktop computer can interact with each other based on the application to realize real-time data transmission of the target image and the detection result. In an exemplary scenario, the desktop computer and the smartphone are connected via a Universal Serial Bus (USB) interface for game testing. During game testing, the desktop computer sends a test command to the smartphone. The smartphone responds by taking a screenshot of the image to be detected and then sends it to the desktop computer via USB. The desktop computer reads the image and performs target detection, identifying each target within the image. In one exemplary solution, the identification result can be as follows: Figure 3 As shown, each minion and hero in the game interface is displayed using borders. After identifying each target in the current game interface, instructions are issued through a pre-configured test rule engine to make the identified targets perform game operations. For example, based on the identified hero's position and skill information, the system can control the hero to perform corresponding movements and release skills, thereby enabling artificial intelligence to automatically play the game and conduct automated game testing.

[0064] Figure 1 and Figure 2 In the scenario shown, the computer equipment in the above small target image detection method can be a server, terminal, server cluster, or cloud computing service cluster, etc. It only needs to achieve the functions of acquiring the image to be detected and detecting small target images within that image. For example, the server or terminal can have both image acquisition and target detection functions; for instance, the server acquires the image to be detected and performs target detection based on the image and a target model.

[0065] Servers can be independent physical servers, server clusters or distributed systems composed of multiple physical servers, or cloud servers or server clusters providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. The aforementioned networks can include, but are not limited to, wired networks and wireless networks. Wired networks include local area networks (LANs), metropolitan area networks (MANs), and wide area networks (WANs). Wireless networks include Bluetooth, Wi-Fi, and other networks that enable wireless communication. Terminals can be smartphones (such as Android phones, Apple iOS phones, etc.), tablets, laptops, digital radio receivers, mobile internet devices (MIDs), personal digital assistants (PDAs), desktop computers, in-vehicle terminals (such as in-vehicle navigation terminals, in-vehicle computers, etc.), smart speakers, smartwatches, etc. Terminals and servers can be connected directly or indirectly through wired or wireless communication methods, but are not limited to these. Specific details can be determined based on the actual application scenario requirements and are not limited here.

[0066] Figure 4 This is a schematic diagram of the network structure of the target model in this application. Figure 4 As shown, the target model consists of five modules: an adaptive spatial parallel convolution module, a sub-backbone module, a feature pyramid network, a fast multi-scale fusion module, and a prediction module. The adaptive spatial parallel convolution module is used to segment the original feature map of the input image to be detected into sub-maps, and then extract context features from each sub-map and fuse them to obtain a first feature map. The sub-backbone module is used to extract features from the first feature map by downsampling and output Q second-class feature maps, each of which has a different resolution. The feature pyramid network is used to fuse the Q second-class feature maps to obtain N first-class feature maps, which have different resolutions and include semantic information of the small target image at different resolutions. Finally, the fast multi-scale fusion module is used to obtain a second feature map based on the N first-class feature maps and the first feature map. The second feature map includes the spatial and semantic information of the small target image in the image to be detected. Finally, the prediction module predicts all the feature maps output by the fast multi-scale fusion module and outputs prediction results. Then, based on the multiple prediction results, the detection result of the small target image in the image to be detected is output.

[0067] It is understood that in the specific embodiments of this application, sample data and other related data are involved. When the above embodiments of this application are applied to specific products or technologies, full permission or consent from the user is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0068] Based on the above introduction, the small target image detection method in this application will be described below. Please refer to [link / reference]. Figure 5 One embodiment of the small target image detection method in this application includes:

[0069] 501. Obtain the original feature map of the image to be detected, including the small target image.

[0070] The small target image detection method provided in this application embodiment can be a detection method that targets small target images in an image to be detected. The image to be detected may include multiple objects to be identified, and the small target image is the target of the small target image detection method provided in this application embodiment. In one exemplary solution, the detection device (which can be...) Figure 4 The server in the image acquisition device can obtain the image to be detected from the image acquisition device. Alternatively, the detection device can also pre-store the image to be detected. For example, the image to be detected can be an image captured by the image acquisition device or the detection device of the scene to be detected; or the image to be detected can also be an image obtained by taking a screenshot of the scene to be detected by the image acquisition device or the detection device.

[0071] The small target image detection process in this application embodiment can refer to detecting target images with small or relatively small image resolutions in the image to be detected. In one exemplary scheme, the image resolution of the small target image is 50 × 50 or less. In another exemplary scheme, the image resolution of the small target image can also refer to being relatively less than a certain threshold; for example, an image with an image resolution less than one-tenth of the image to be detected is considered a small target image. In one exemplary scheme, the small target image can be as follows: Figure 3 The game screen shown features minions or hero characters.

[0072] It is understood that the image to be detected can be an image with a high image resolution. That is, the image to be detected can be an image whose image resolution exceeds a target resolution threshold and whose size exceeds a target size threshold. For example, the image to be detected can be a high-resolution, large-size original image captured by a high-definition camera; or it can be a low-resolution, small-size original image after being captured and transmitted. In one exemplary scheme, such as... Figure 3As shown, the image to be detected can be a screenshot of a game screen taken by a smartphone, and the small target image in the image to be detected can be, for example, a small target image. Figure 3 The game screen shown depicts minions and hero characters.

[0073] In this embodiment, the original feature map of the image to be detected can be the original feature map obtained after global feature extraction of the image to be detected. It is understood that the original feature map can be obtained after being input into the adaptive spatial parallel convolution module in the target model, or it can be obtained before being input into the adaptive spatial parallel convolution module. No specific limitation is made here.

[0074] 502. The original feature map is segmented into M sub-maps through the first network layer of the target model, and the context features of the M sub-maps are extracted and fused to obtain the first feature map. The first feature map is used to indicate the spatial information of the small target image, where M is an integer greater than 1.

[0075] After obtaining the original feature map, the detection device inputs the original feature map into the first network layer (based on) of the target model. Figure 4 As shown in the network structure, the first network layer can be an adaptive spatial parallel convolutional module. When the original feature map passes through the adaptive spatial parallel convolutional module, the original feature map is divided into M sub-maps according to the output channels of the adaptive spatial parallel convolutional module; then the adaptive spatial parallel convolutional module designs a corresponding convolutional layer for each sub-map corresponding to the output channel, and performs context feature extraction on the M sub-maps to obtain M first intermediate feature maps. Finally, a second intermediate feature map is obtained by concatenating the M first intermediate feature maps through channels; finally, the original feature map is element-wise summed to obtain a third intermediate feature map, and finally, feature extraction is performed again on the third intermediate feature map to output the first feature map.

[0076] In one exemplary scheme, based on Figure 6 The structure of the adaptive spatial parallel convolution module shown illustrates the output process of the first feature map. In this diagram, the original feature map is named the feature map. Figure X (this feature) Figure X The corresponding specifications are H×W×C, where H indicates this feature. Figure X The image height, where W is used to indicate the feature. Figure X The image width, where C is used to indicate the feature. Figure X (The number of image channels). This adaptive spatial parallel convolution module includes two output channels, which will store the features. Figure XThe image is divided into two sub-images: feature map X1 (with dimensions H×W×C / 2) and feature map X2 (with dimensions H×W×C / 2). Feature map X1 is then processed through a single convolutional layer Conv and a standard convolutional layer Conv2d for feature extraction. The kernel of Conv is 3×3, and the kernel of Conv2d can be either 3×3 or 1×1 by default. Feature map X2 is then processed through a standard convolutional layer Conv2d for feature extraction. The kernel of Conv2d can be either 3×3 or 1×1 by default. After feature extraction from feature maps X1 and X2, feature map X3 is obtained by concatenating the channels (the size of feature map X3 is H×W×C). Feature map X3 is then fused with feature map X1 through element-wise addition and feature extraction to obtain feature map X4 (the size of feature map X4 is H / 2×W / 2×2C). Feature extraction is then performed on feature map X4, and it is then added element-wise to obtain the first feature map C1. It can be understood that the convolutional layers used for feature extraction of feature map X4 can be designed as follows: two convolutional layers, both single-layer convolutional layers Conv (i.e., Conv), with kernel sizes that can be defaulted to 3×3 or 1×1.

[0077] It is understandable that, after the above scheme, the original feature map (i.e., features) Figure X By splitting, fusing and feature extraction operations, multi-scale local contextual information based on the original feature map is obtained, and the resulting first feature map has rich spatial information, which can help improve the detection performance of the small target image.

[0078] 503. Through the second network layer of the target model, the first feature map is downsampled to obtain N first-class feature maps. The N first-class feature maps have different resolutions and include the semantic information of the small target image at different resolutions. N is an integer greater than 1.

[0079] After the detection device acquires a first feature map with rich spatial information, it downsamples the first feature map to obtain feature maps of multiple scales and multiple image resolutions, thereby obtaining rich semantic information of the small target image.

[0080] Optionally, in the second network layer (i.e., the sub-backbone module and the feature pyramid network), the process of obtaining the first type of feature map based on the first feature map can be specifically as follows: The first feature map is transformed into Q second type feature maps by X cross-stage double convolution transformation operators and 1 cross-stage double convolution transformation combination operator in the sub-backbone network, where X and Q are positive integers. The cross-stage double convolution transformation operator includes a convolution unit and a feature extraction unit, and the cross-stage double convolution transformation combination operator includes a convolution unit and a spatial pyramid pooling unit. The Q second type feature maps include the first feature map. The Q second type feature maps are then connected by a matrix in the feature map pyramid network to obtain the N first type feature maps.

[0081] The specific process of obtaining the N first-class feature maps by connecting the Q second-class feature maps through the feature map pyramid network matrix can also be as follows: According to the output sequence, the Qth second-class feature map of the last output of the Q second-class feature maps is passed through the feature map pyramid network to obtain the first first-class feature map in the N first-class feature maps; the first first-class feature map is connected with the (Q-1)th second-class feature map in the Q second-class feature maps, and the second first-class feature map in the N first-class feature maps is obtained through the cross-stage double convolution transformation and combination operator in the feature map pyramid network; and so on, until the second second-class feature map in the Q second-class feature maps is connected with the (N-1)th first-class feature map, and the Nth first-class feature map in the N first-class feature maps is obtained through the cross-stage double convolution transformation and combination operator in the feature map pyramid network. The N first-class feature maps include the first first-class feature map, the second first-class feature map, and the Nth first-class feature map. In this scheme, multi-scale feature maps are extracted and fused from the first feature map to enrich the semantic information of the small target image, thereby improving the detection performance of small target image detection.

[0082] In one exemplary solution, combined with Figure 7 The sub-backbone modules and feature pyramid network shown illustrate the specific process of obtaining N first-class feature maps based on the first feature map:

[0083] like Figure 7As shown, the sub-backbone module design includes two cross-stage double convolution transformation operators and one cross-stage double convolution transformation combination operator. The sub-backbone network and the feature pyramid network are cross-stage via ABTC2. The feature pyramid network design includes two cross-stage double convolution transformation operators. After the first feature map (i.e., feature map C1, in this embodiment, it is assumed that the image resolution of C1 is 160×160) is input into the sub-backbone module, a feature map C2 with an image resolution of 80×80 is obtained through downsampling and feature extraction using the cross-stage double convolution transformation operator; the feature map C2 is then downsampled and feature extracted again using the cross-stage double convolution transformation operator to obtain a feature map C3 with an image resolution of 40×40; the feature map C3 is then downsampled and feature extracted again using the cross-stage double convolution transformation combination operator to obtain a feature map C4 with an image resolution of 20×20; the feature map C4 is then processed... Feature extraction using ABTC2 yields a feature map P4 with an image resolution of 20×20. Feature map P4 is then upsampled by a factor of two to achieve a resolution of 40×40, and concatenated with feature map C3 via a channel to generate an intermediate feature map. This intermediate feature map is then subjected to feature extraction using a cross-stage double convolution operator to obtain feature map P3. Feature map P3 is then upsampled by a factor of two to achieve a resolution of 80×80, and concatenated with feature map C2 via a channel to generate an intermediate feature map. This intermediate feature map is then subjected to feature extraction using a cross-stage double convolution operator to obtain feature map P2. Following this process, the first type of feature maps includes feature maps P4, P3, and P2.

[0084] Understandable Figure 7 Only an exemplary structure of the sub-backbone module and the feature pyramid network is provided. Depending on the actual needs, the number of the first type of feature maps obtained through the sub-backbone module and the feature pyramid network can be other possible values, and the specific situation is not limited here.

[0085] In this embodiment, the sub-backbone structure, the cross-stage double convolution transformation operator, the cross-stage double convolution transformation combination operator, and ABTCi in the feature pyramid network can be designed as follows:

[0086] The cross-stage double convolution transformation operator in this sub-backbone structure can also be called cross-stage double convolution transformation operator 1, which includes convolution units and feature extraction units. Convolution is an integral operation used to calculate the area of ​​the overlapping region of two curves, and can also be seen as a weighted summation, which can be used to eliminate noise and enhance features. In image processing, convolution is the operation of the kernel on each pixel of the image. The kernel is essentially a fixed-size matrix array, whose center point is called the anchor point. In an exemplary scheme, the cross-stage double convolution transformation operator 1 can be designed as follows: Figure 8aAs shown, it includes a convolutional layer and an adaptive double convolutional transformation operator 1 (ABT1). In this embodiment, the convolutional layer of the cross-stage double convolutional transformation operator i can be designed as Conv, with its kernel size and stride being default values, and its corresponding number of channels being the same as the number of input channels. In this embodiment, the adaptive double convolutional transformation operator 1 (ABT1) includes a convolutional unit and a feature extraction unit. In an exemplary scheme, the ABT1 can be designed as follows: Figure 8b As shown, it includes two convolutional layers. Both of these convolutional layers are Conv, with the first convolutional layer having a 1×1 kernel and the second convolutional layer having a 3×3 kernel. The number of image channels corresponding to the two convolutional layers is the same as the number of input channels.

[0087] The cross-stage dual convolutional transformation combination operator in this sub-backbone structure includes convolutional units and Spatial Pyramid Pooling (SPP). Convolution is an integral operation used to calculate the area of ​​the overlapping region of two curves; it can also be viewed as a weighted sum and can be used to eliminate noise and enhance features. In image processing, convolution is the operation of the kernel on each pixel of the image. The kernel is essentially a fixed-size matrix array, with its center point called the anchor point. The SPP is used to enable feature maps of arbitrary size to be transformed into fixed-size feature vectors (e.g., extracting fixed-size feature vectors from multi-scale features). In one exemplary scheme, this cross-stage dual convolutional transformation combination operator can be designed as follows: Figure 8c As shown, it includes a convolutional layer, an SPP layer, and an adaptive double convolutional transformation operator 1 (ABT1). In this embodiment, the convolutional layer of this cross-stage double convolutional transformation combination operator can be designed as Conv, with a kernel size of 3×3, a stride of 2, and the corresponding number of channels being the same as the number of input channels. The design of this adaptive double convolutional transformation operator 1 (ABT1) can be found in [reference needed]. Figure 8b As shown, it includes two convolutional layers. Both convolutional layers are Conv layers, with the first layer having a 1×1 kernel and the second layer having a 3×3 kernel. The number of image channels corresponding to the two convolutional layers is the same as the number of input channels. In the specific feature extraction process, the feature map input to this ABT1 is concatenated (added) with the feature map after passing through the first and second convolutional layers before being output.

[0088] The cross-stage double convolution transformation operator in this feature pyramid network can also be called cross-stage double convolution transformation operator 2, which includes convolution units and feature extraction units. Convolution is an integral operation used to calculate the area of ​​the overlapping region of two curves, and can also be viewed as a weighted sum, which can be used to eliminate noise and enhance features. In image processing, convolution is the operation of the kernel on each pixel of the image. The kernel is essentially a fixed-size matrix array, whose center point is called the anchor point. In an exemplary scheme, the cross-stage double convolution transformation operator 2 can be designed as follows: Figure 8d As shown, it includes a convolutional layer and an adaptive double convolutional transformation operator 2 (ABT2). In this embodiment, the convolutional layer of the cross-stage double convolutional transformation operator i can be designed as Conv, with its kernel size and stride being default values, and its corresponding number of channels being the same as the number of input channels. In this embodiment, the adaptive double convolutional transformation operator 2 (ABT2) includes a convolutional unit and a feature extraction unit. In an exemplary scheme, the ABT2 can be designed as follows: Figure 8e As shown, it includes two convolutional layers. Both of these convolutional layers are Conv layers. The first convolutional layer has a 1×1 kernel, and its corresponding image channel number is half the number of input channels. The second convolutional layer has a 3×3 kernel, and its corresponding image channel number is the same as the number of input channels.

[0089] The cross-stage feature extraction between the sub-backbone module and the feature pyramid network utilizes an adaptive double convolution transformation combination operator i (ABTCi, i=1 or 2), which can be understood as a combination of the aforementioned ABT1 or ABT2 with convolutional layers. In one exemplary scheme, the ABTCi can be designed as follows: Figure 8f As shown, it includes four convolutional layers and the ABTi. The design of the ABTi can be found in the above description. Figure 8b and Figure 8eAs shown. The four convolutional layers can be designed as follows: the first convolutional layer is Conv, with a 1×1 kernel, and its corresponding feature map channel number is half that of the input channels; the second convolutional layer is Conv2d, with a 1×1 kernel, and its corresponding feature map channel number is half that of the input channels; the third convolutional layer is Conv, with a 1×1 kernel, and its corresponding feature map channel number is half that of the input channels; the fourth convolutional layer is Conv2d, with a 1×1 kernel, and its corresponding feature map channel number is half that of the input channels. Based on the above structure, the feature extraction process can be as follows: The ABTCi acquires the input feature map, and then the input feature map is divided into a first sub-map and a second sub-map through channels. The first sub-map is processed through the first convolutional layer, ABTi, and the second convolutional layer to obtain a first intermediate feature map; the second sub-map is processed through the fourth convolutional layer to obtain a second intermediate feature map; the first intermediate feature map and the second intermediate feature map are concatenated through channels (i.e., concat), and then processed through the third convolutional layer for feature extraction before outputting the result.

[0090] In this embodiment, the structure of Conv is as follows: Figure 8g As shown, it consists of three layers: Conv2d, a batch normalization layer (BatchNorm), and an activation function (HardSwish). The Conv2d convolutional kernel is typically 1×1 by default. The batch normalization layer (BatchNorm) normalizes the output of Conv2d, forcing all batches of data into a uniform data distribution before inputting them into the activation function layer. This makes the output values ​​of the target model more stable across layers, thus making the deep neural network more likely to converge and reducing the risk of overfitting. The activation function (HardSwish) introduces non-linearity. It is understood that this activation function can be chosen based on the specific application and is not limited to any particular type. Figure 8g The example scheme shown.

[0091] 504. Through the third network layer of the target model, the N first-class feature maps and the first feature map are fused to obtain the second feature map.

[0092] After the detection device acquires the first type of feature map, it can fuse the Nth first type of feature map with the first feature map through the third network layer to obtain a second feature map that simultaneously possesses rich spatial and semantic information. In this embodiment, an exemplary scheme for the detection device to fuse the N first type of feature maps and the first feature map to obtain the second feature map can be as follows: the N first type of feature maps are mapped to an ascending filter array of a sub-pixel convolutional layer to generate a fifth feature map, the resolution of which is greater than the resolution of the N first type of feature maps; the fifth feature map is then fused with the first feature map through channel concatenation to obtain the second feature map.

[0093] Optionally, the specific operation of mapping the N first-class feature maps to generate the fifth feature map through the ascending filter array of the sub-pixel convolutional layer can be as follows: according to the output timing, the (N-1)th first-class feature map of the N first-class feature maps is mapped to generate the sixth feature map through the ascending filter array of the sub-pixel convolutional layer; the sixth feature map is fused with the Nth first-class feature map of the N first-class feature maps through channel concatenation, and upsampling is performed to generate the fifth feature map.

[0094] Based on the above scheme, the specific operation of fusing the fifth feature map and the first feature map through channel connection to obtain the second feature map can be as follows: extract the features of the first feature map through the convolutional layer of the third network layer to obtain the seventh feature map; and fuse the seventh feature map with the fifth feature map to obtain the second feature map.

[0095] In one exemplary solution, combined with Figure 7 The scheme shown and Figure 9 The fast multi-scale fusion module shown illustrates the specific process of fusing the N first-class feature maps and the first feature map to obtain the second feature map:

[0096] like Figure 9 As shown, the detection device will Figure 7 Feature maps P2, P3, and P4 are obtained. To preserve richer spatial and semantic information in the second feature map, this embodiment maps feature map P3 to a high-resolution space to obtain a feature map P3' (at this point, the resolution of feature map P3' is the same as that of feature map P2). Then, feature map P3' and feature map P2 are concatenated via a channel to obtain a fourth intermediate feature map P2'. This fourth intermediate feature map is then processed by the ABTC2 module for feature extraction, and then element-wise added to feature map P3' and upsampled to obtain a fifth intermediate feature map. It is understood that the resolution of this fifth intermediate feature map should be the same as that of feature map C1, and higher than the resolutions of feature maps P2 and P3. Figure 9 As shown, the feature map C1 is processed by a convolutional layer to extract features to obtain the sixth intermediate feature map (i.e., the seventh feature map mentioned above); then the fifth intermediate feature map and the sixth intermediate feature map are connected by channels to generate the seventh intermediate feature map C1'; finally, the seventh intermediate feature map is processed by the ABTC2 module to extract features to obtain the feature map P1 (i.e., the second feature map mentioned above).

[0097] Understandably, in Figure 6 , Figure 7 as well as Figure 9In the constructed target model, feature map P2, feature map P3, and feature map P4 can all be selected as initial feature maps for mapping to the high-resolution space, and then channel-connected with feature map C1 to obtain feature map P1. For example, feature map P4 is mapped to a high-resolution space to obtain feature map P4' (at this time, the resolution of feature map P4' is the same as that of feature map P2). Then, feature map P4' and feature map P2 are concatenated through channels to obtain a fourth intermediate feature map. After feature extraction of the fourth intermediate feature map through the ABTC2 module, it is added element-wise to feature map P4' and upsampled to obtain an eighth intermediate feature map. It can be understood that the resolution of the eighth intermediate feature map should be the same as that of feature map C1, and higher than the resolution of feature maps P2 and P4. Feature map C1 is used to extract features through a convolutional layer to obtain a sixth intermediate feature map (i.e., the seventh feature map mentioned above). Then, the eighth intermediate feature map and the sixth intermediate feature map are concatenated through channels to generate a ninth intermediate feature map. Finally, feature extraction of the ninth intermediate feature map is performed through the ABTC2 module to obtain feature map P1' (also called the second feature map). Similarly, the feature map P4 can be mapped to a high-resolution space to obtain an intermediate feature map with the same resolution as the feature map P3. The fusion process is the same as described above, and will not be repeated here.

[0098] 505. The small target image is obtained by detecting the N first-class feature maps and the second-class feature map through the detection layer of the target model.

[0099] After acquiring the N first-class feature maps and the second-class feature map, the detection device uses the detection layer of the target model to predict the N first-class feature maps and the second-class feature map respectively to obtain N+1 classification and localization results. These N+1 classification and localization results are then defined as a set of classification and localization results. Finally, based on these N+1 classification and localization results, the final classification and localization result is obtained. This final classification and localization result can be an annotation of small target images in the image to be detected, such as... Figure 3 The detection result can be displayed as a textual description of the location information of the small target image in the image to be detected, such as outputting the pixel coordinates of the small target image in the image to be detected.

[0100] It is understandable that the specific operation of the detection device to obtain the final detection result based on the N+1 detection results can be as follows: the classification and localization results in the classification and localization result set are filtered by confidence threshold and non-maximum suppression processing to obtain the small target image.

[0101] In one exemplary scheme, with Figure 9The first type of feature maps (feature maps P2, P3, and P4) and feature map C1 are shown to illustrate the prediction process in this embodiment. The detection device can generate classification and localization results corresponding to feature maps P2, P3, P4, and C1 through this prediction layer. For example, feature map P2 corresponds to classification and localization result 1; feature map P3 corresponds to classification and localization result 2; feature map P4 corresponds to classification and localization result 3; and feature map C1 corresponds to classification and localization result 4. Then, confidence threshold filtering and non-maximum suppression processing are applied to classification and localization results 1 to 4 to obtain the final classification and localization result, which will display the final position of the small target image.

[0102] It is understood that the above process describes an embodiment of the method for small target image detection using the target model by the detection device. The training process of the target model is described below: A sample set and an initial model are obtained. The initial model includes an initial network layer and an initial detection layer. The sample set includes sample images of sample objects, the true location of the sample object in the sample image, and the true classification of the sample object in the sample image. The sample set is input into the initial model to obtain the sample detection location and sample detection classification. Based on the sample detection location, the sample detection classification, and the true location and true classification, a joint loss function is used to determine the joint difference between the sample detection location and the true location, and between the sample detection classification and the true classification output by the initial model. Based on the joint difference, the model parameters of the initial model are adjusted until the target condition is met, at which point the adjustment stops, resulting in the target model. It is understood that the target condition may be that the joint difference is less than a target difference threshold, or that the number of iterations based on the joint difference exceeds a target number threshold, etc.

[0103] In one exemplary scheme, the joint loss function includes a location loss, a confidence loss, and a classification loss, wherein the location loss is the error between the detected location of the sample and the true location, the confidence loss is used to indicate the relative error of the predicted confidence score of the sample object, and the classification loss is used to indicate the error between the detected classification of the sample and the true classification.

[0104] It is understandable that the above-mentioned sample images of the sample objects can be used in the feature extraction stage. Figure 5 The feature extraction schemes for steps 501 to 504 shown are not detailed here.

[0105] The detection device in this application is described in detail below. Please refer to [link / reference]. Figure 10 , Figure 10 This is a schematic diagram of one embodiment of the detection device in this application. The detection device 20 includes:

[0106] The acquisition module 201 is used to acquire the original feature map of the image to be detected, including the small target image;

[0107] The first feature extraction module 202 is used to segment the original feature map into M sub-maps through the first network layer of the target model, and to extract and fuse the context features of the M sub-maps to obtain a first feature map. The first feature map has the spatial information of the small target image, and M is an integer greater than 1.

[0108] The second feature extraction module 203 is used to downsample the first feature map through the second network layer of the target model to obtain N first-class feature maps. The N first-class feature maps have different resolutions and include the semantic information of the small target image at different resolutions. N is an integer greater than 1.

[0109] The fusion module 204 is used to fuse the N first-class feature maps and the first feature map through the third network layer of the target model to obtain a second feature map;

[0110] The detection module 205 is used to detect the N first-class feature maps and the second feature map through the detection layer of the target model to obtain the small target image.

[0111] In this application embodiment, a detection device is provided. Using the above device, when extracting features from the image to be detected, spatial and semantic information of the small target image is extracted, and the spatial and semantic information are fused, so that the spatial and semantic information of the image to be detected can be richly represented, thereby improving the detection accuracy of the small target image.

[0112] Optionally, in the above Figure 10 Based on the corresponding embodiments, in another embodiment of the detection device 20 provided in this application,

[0113] The first network layer is an adaptive spatial parallel convolutional layer. The first feature extraction module 202 is used to segment the original feature map into M sub-maps through the M channels of the adaptive spatial parallel convolutional layer, and to perform parallel convolution on the M sub-maps to obtain M sub-feature maps. The convolutional layers corresponding to the M sub-maps are different, and M is an integer greater than 1. The M sub-feature maps are concatenated and feature batch normalization is performed to obtain a third feature map. The third feature map is concatenated and fused with the original feature map to obtain a fourth feature map. The fourth feature map is downsampled and feature extracted to obtain the first feature map.

[0114] In this embodiment, a detection device is provided. Using this device, the original feature map is segmented into multiple local sub-maps, and cascaded multi-convolution feature extraction is performed on the local sub-maps. This utilizes the cascaded relationship of the multi-convolutions to effectively extract the context of different local sub-maps, thereby obtaining spatial information for the small target image. This helps improve the detection performance of small target images.

[0115] Optionally, in the above Figure 10 Based on the corresponding embodiments, in another embodiment of the detection device 20 provided in this application, the first feature extraction module 202 is specifically used to divide the original feature map into two sub-maps through the two channels of the adaptive spatial parallel convolutional layer. One of the two sub-maps corresponds to the standard convolutional layer Conv2d, and the convolutional layer corresponding to the other sub-map includes a single-layer convolutional layer Conv and a standard convolutional layer Conv2d.

[0116] This application provides a detection device. Using this device, a design scheme for refining the convolutional layers in parallel convolution is provided when the original feature map is bisected. In the aforementioned application scenario, each sub-map uses a different convolutional layer for feature extraction, thereby achieving effective extraction of the context of different local sub-maps and obtaining spatial information for the small target image. This helps improve the detection performance of small target images.

[0117] Optionally, in the above Figure 10 Based on the corresponding embodiments, in another embodiment of the detection device 20 provided in this application, the convolution kernels of the standard convolutional layer Conv2d and the single-layer convolutional layer Conv are 3×3 or 1×1, and the stride is 1.

[0118] In this embodiment, a detection device is provided. Using this device, the convolution kernels of each convolutional layer are designed to be 3×3 or 1×1. This effectively extracts contextual features from small target images in different local sub-images, thereby obtaining rich spatial information for the small target image, which helps improve the detection performance of small target images.

[0119] Optionally, in the above Figure 10Based on the corresponding embodiments, in another embodiment of the detection device 20 provided in this application, the second network layer includes a sub-backbone network and a feature map pyramid network. The second feature extraction module 203 is specifically used to obtain Q second-class feature maps from the first feature map through X cross-stage double convolution transformation operators and 1 cross-stage double convolution transformation combination operator in the sub-backbone network, wherein X and Q are positive integers, the cross-stage double convolution transformation operator includes a convolution unit and a feature extraction unit, the cross-stage double convolution transformation combination operator includes a convolution unit and a spatial pyramid pooling unit, and the Q second-class feature maps include the first feature map; the Q second-class feature maps are then connected by a matrix of the feature map pyramid network to obtain the N first-class feature maps.

[0120] In this embodiment, a detection device is provided. Using this device, within the sub-backbone module and feature pyramid network structure, the spatial pyramid pooling unit can pool and cascade multi-scale local region features. Therefore, it can utilize global and local multi-scale features to obtain rich semantic information for small target images, thereby improving the detection accuracy of small target images.

[0121] Optionally, in the above Figure 10 Based on the corresponding embodiments, in another embodiment of the detection device 20 provided in this application, the second feature extraction module 203 is specifically used to obtain the first first-class feature map in the N first-class feature maps by passing the last output Q-th second-class feature map in the Q second-class feature maps through a feature map pyramid network according to the output time sequence.

[0122] The first first-type feature map is connected to the (Q-1)th second-type feature map among the Q second-type feature maps, and the second first-type feature map among the N first-type feature maps is obtained through the cross-stage double convolution transformation and combination operator in the feature map pyramid network; and so on, until the second second-type feature map among the Q second-type feature maps is connected to the (N-1)th first-type feature map, and the Nth first-type feature map among the N first-type feature maps is obtained through the cross-stage double convolution transformation and combination operator in the feature map pyramid network. The N first-type feature maps include the first first-type feature map, the second first-type feature map, and the Nth first-type feature map.

[0123] In this embodiment of the application, a detection device is provided. Using the above device, during the generation of the first type of feature map in the feature pyramid network, the information of each feature map in the sub-backbone network is fused, further increasing the semantic information of the first type of feature map. Therefore, rich semantic information for small target images can be obtained by utilizing global and local multi-scale features, thereby improving the detection accuracy of small target images.

[0124] Optionally, in the above Figure 10 Based on the corresponding embodiments, in another embodiment of the detection device 20 provided in this application, the third network layer is a multi-scale fusion network, and the fusion module 204 is specifically used to map the N first-class feature maps through an ascending filter array of sub-pixel convolutional layers to generate a fifth feature map, the resolution of the fifth feature map being greater than the resolution of the N first-class feature maps; and to connect and fuse the fifth feature map with the first feature map to obtain the second feature map.

[0125] In this embodiment, a detection device is provided. Using this device, during the fusion process, an ascending filter array of sub-pixel convolutional layers is used to map N first-class feature maps to obtain a high-resolution intermediate feature map. Simultaneously, the intermediate feature map, the remaining first-class feature maps, and the first feature map are fused to generate a second feature map that includes semantic and spatial information. In this process, the second feature map includes rich global and local spatial and semantic information, thereby improving the detection accuracy of small target images.

[0126] Optionally, in the above Figure 10 Based on the corresponding embodiments, in another embodiment of the detection device 20 provided in this application, the fusion module 204 is specifically used to map the (N-1)th first-class feature map of the N first-class feature maps through the ascending filter array of the sub-pixel convolution layer to generate a sixth feature map according to the output timing; connect and fuse the sixth feature map with the Nth first-class feature map of the N first-class feature maps, and upsample to generate the fifth feature map.

[0127] In this embodiment, a detection device is provided. Using this device, an ascending filter array of sub-pixel convolutional layers maps N first-class feature maps with resolutions close to the first feature map and containing more semantic information to obtain a high-resolution intermediate feature map. Simultaneously, this intermediate feature map, another first-class feature map of a different resolution obtained from the first-class feature map, and the first feature map are fused to generate a second feature map including semantic and spatial information. In this process, the first-class feature map undergoing high-resolution mapping has a larger resolution and higher semantic information; therefore, the second feature map will include richer global and local spatial and semantic information, thereby improving the detection accuracy of small target images.

[0128] Optionally, in the above Figure 10 Based on the corresponding embodiments, in another embodiment of the detection device 20 provided in this application, the fusion module 204 is specifically used to extract features from the first feature map through the convolutional layer of the third network layer to obtain a seventh feature map; and to connect and fuse the seventh feature map with the fifth feature map to obtain a second feature map.

[0129] In this embodiment, a detection device is provided. Using this device, the first feature map is processed through a convolutional layer for feature extraction, and then fused with the fifth feature map to obtain the second feature map. This allows the second feature map to include richer spatial and semantic information, both global and local, thereby improving the detection accuracy of small target images.

[0130] Optionally, in the above Figure 10 Based on the corresponding embodiments, in another embodiment of the detection device 20 provided in this application, the detection module 205 is specifically used to obtain a classification and localization result set by passing the N first-class feature maps and the second feature maps through the detection layer respectively; and to obtain the small target image by filtering the classification and localization results in the classification and localization result set through confidence threshold and non-maximum suppression processing.

[0131] In this embodiment, a detection device is provided. Using this device, the classification and localization results of multiple feature maps can be predicted, and the small target image can be obtained by performing confidence threshold filtering and non-maximum suppression processing based on the multiple classification and localization results. This allows the prediction result to integrate global and local features, thereby improving the detection accuracy of the small target image.

[0132] Optionally, in the above Figure 10Based on the corresponding embodiments, in another embodiment of the detection device 20 provided in this application, the acquisition module 201 is further used to acquire a sample set and an initial model. The initial model includes an initial network layer and an initial detection layer. The sample set includes a sample image of a sample object, the true position of the sample object in the sample image, and the true classification of the sample object in the sample image.

[0133] like Figure 11 As shown, the detection device 20 further includes a processing module 206, used to input the sample set into the initial model to obtain sample detection location and sample detection classification; based on the sample detection location, the sample detection classification, the true location, and the true classification, determine the joint difference between the sample detection location and the true location, as well as the sample detection classification and the true classification output by the initial model, through a joint loss function; based on the joint difference, adjust the model parameters of the initial model until the target conditions are met, and stop adjusting to obtain the target model.

[0134] In this embodiment, a detection device is provided. Using this device, during the training process of the target model, the sample set is segmented into multiple local sub-graphs, and cascaded multi-convolution feature extraction is performed on these sub-graphs. This utilizes the cascaded relationship of the multi-convolutions to effectively extract the context of different local sub-graphs, thereby obtaining spatial information for the small target image. This helps improve the detection performance of small target images. Simultaneously, by adjusting the model parameters of the initial model based on multiple joint dissimilarity measures, the predictive ability of the target model in detecting small target images can be improved.

[0135] Optionally, in the above Figure 10 Based on the corresponding embodiments, in another embodiment of the detection device 20 provided in this application, the joint loss function includes position loss, confidence loss and classification loss, wherein the position loss is the error between the sample detection position and the true position, the confidence loss is used to indicate the relative error of the predicted confidence score of the sample object, and the classification loss is used to indicate the error between the sample detection classification and the true classification.

[0136] In this application embodiment, a detection device is provided. Using the above device, the joint loss function can include multiple categories, thereby adjusting the model parameters of the initial model based on multiple joint differences, which can improve the predictive ability of the target model when detecting small target images.

[0137] The detection device provided in this application can be used on a server; please refer to [link / reference]. Figure 12 , Figure 12This is a schematic diagram of a server structure provided in an embodiment of this application. The server 300 can vary significantly due to different configurations or performance. It may include one or more central processing units (CPUs) 322 (e.g., one or more processors) and memory 332, and one or more storage media 330 (e.g., one or more mass storage devices) for storing application programs 342 or data 344. The memory 332 and storage media 330 can be temporary or persistent storage. The program stored in the storage media 330 may include one or more modules (not shown in the diagram), each module may include a series of instruction operations on the server. Furthermore, the CPU 322 may be configured to communicate with the storage media 330 and execute the series of instruction operations stored in the storage media 330 on the server 300.

[0138] Server 300 may also include one or more power supplies 326, one or more wired or wireless network interfaces 350, one or more input / output interfaces 358, and / or one or more operating systems 341, such as Windows Server. TM Mac OS X TM Unix TM Linux TM FreeBSD TM etc.

[0139] In the above embodiments, when the detection device is a server, the steps performed by the server can be based on... Figure 12 The server structure shown.

[0140] The detection device provided in this application can be used in terminal equipment; please refer to [link / reference]. Figure 13 For ease of explanation, only the parts relevant to the embodiments of this application are shown. For specific technical details not disclosed, please refer to the method section of the embodiments of this application. In the embodiments of this application, a smartphone is used as an example for illustration:

[0141] Figure 13 This is a block diagram illustrating a portion of the structure of a smartphone related to the terminal device provided in the embodiments of this application. (Reference) Figure 13 The smartphone includes components such as a radio frequency (RF) circuit 410, a memory 420, an input unit 430, a display unit 440, a sensor 450, an audio circuit 460, a wireless fidelity (WiFi) module 470, a processor 480, and a power supply 490. Those skilled in the art will understand that... Figure 13The smartphone structure shown does not constitute a limitation on smartphones and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0142] The following is combined with Figure 13 A detailed introduction to the various components of a smartphone:

[0143] RF circuit 410 can be used for receiving and transmitting signals during information transmission or calls. Specifically, it receives downlink information from the base station and processes it with processor 480; additionally, it transmits uplink data to the base station. Typically, RF circuit 410 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low-noise amplifier (LNA), a duplexer, etc. Furthermore, RF circuit 410 can also communicate wirelessly with networks and other devices. The aforementioned wireless communication can use any communication standard or protocol, including but not limited to Global System for Mobile Communication (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), email, Short Message Service (SMS), etc.

[0144] The memory 420 can be used to store software programs and modules. The processor 480 executes various functions and data processing of the smartphone by running the software programs and modules stored in the memory 420. The memory 420 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, applications required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created based on the use of the smartphone (such as audio data, phonebook, etc.). In addition, the memory 420 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0145] The input unit 430 can be used to receive input numerical or character information, and to generate key signal inputs related to user settings and function control of the smartphone. Specifically, the input unit 430 may include a touch panel 431 and other input devices 432. The touch panel 431, also known as a touch screen, can collect touch operations performed by the user on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near the touch panel 431), and drive the corresponding connected devices according to a pre-set program. Optionally, the touch panel 431 may include two parts: a touch detection device and a touch controller. The touch detection device detects the user's touch position and the signal generated by the touch operation, and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection device, converts it into touch point coordinates, sends it to the processor 480, and can receive and execute commands sent by the processor 480. In addition, the touch panel 431 can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch panel 431, the input unit 430 may also include other input devices 432. Specifically, other input devices 432 may include, but are not limited to, one or more of the following: physical keyboard, function keys (such as volume control buttons, power buttons, etc.), trackball, mouse, joystick, etc.

[0146] Display unit 440 can be used to display information input by the user or information provided to the user, as well as various menus of the smartphone. Display unit 440 may include a display panel 441, optionally configured as a liquid crystal display (LCD), organic light-emitting diode (OLED), or similar form. Further, touch panel 431 may cover display panel 441. When touch panel 431 detects a touch operation on or near it, it transmits the information to processor 480 to determine the type of touch event. Subsequently, processor 480 provides corresponding visual output on display panel 441 based on the type of touch event. Although in Figure 13 In this embodiment, the touch panel 431 and the display panel 441 are two separate components to realize the input and output functions of the smartphone. However, in some embodiments, the touch panel 431 and the display panel 441 can be integrated to realize the input and output functions of the smartphone.

[0147] Smartphones may also include at least one sensor 450, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor. The ambient light sensor can adjust the brightness of the display panel 441 according to the ambient light level, and the proximity sensor can turn off the display panel 441 and / or the backlight when the smartphone is moved to the ear. As a type of motion sensor, an accelerometer sensor can detect the magnitude of acceleration in various directions (generally three axes) and can detect the magnitude and direction of gravity when stationary. It can be used for applications that recognize the smartphone's posture (such as landscape / portrait switching, related games, magnetometer posture calibration), vibration recognition-related functions (such as pedometers, taps), etc. Other sensors that smartphones may also be equipped with, such as gyroscopes, barometers, hygrometers, thermometers, and infrared sensors, will not be described in detail here.

[0148] Audio circuit 460, speaker 461, and microphone 462 provide an audio interface between the user and the smartphone. Audio circuit 460 converts received audio data into electrical signals and transmits them to speaker 461, where speaker 461 converts them into sound signals for output. On the other hand, microphone 462 converts collected sound signals into electrical signals, which are received by audio circuit 460, converted into audio data, and then processed by processor 480 before being transmitted via RF circuit 410 to, for example, another smartphone, or the audio data can be output to memory 420 for further processing.

[0149] WiFi is a short-range wireless transmission technology. Smartphones, through their WiFi modules (470), can help users send and receive emails, browse web pages, and access streaming media, providing wireless broadband internet access. Although Figure 13 WiFi module 470 is shown, but it is understood that it is not an essential component of a smartphone and can be omitted as needed without changing the nature of the invention.

[0150] The processor 480 is the control center of the smartphone, connecting various parts of the smartphone through various interfaces and lines. It performs various functions and processes data by running or executing software programs and / or modules stored in the memory 420, and by calling data stored in the memory 420, thereby providing overall monitoring of the smartphone. Optionally, the processor 480 may include one or more processing units; optionally, the processor 480 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the aforementioned modem processor may also not be integrated into the processor 480.

[0151] The smartphone also includes a power supply 490 (such as a battery) that powers various components. Optionally, the power supply can be logically connected to the processor 480 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system.

[0152] Although not shown, smartphones may also include a camera, Bluetooth module, etc., which will not be described in detail here.

[0153] The steps performed by the terminal device in the above embodiments can be based on this Figure 13 The terminal device structure is shown.

[0154] This application also provides a computer-readable storage medium storing a computer program that, when run on a computer, causes the computer to perform the methods described in the foregoing embodiments.

[0155] This application also provides a computer program product including a program, which, when run on a computer, causes the computer to perform the methods described in the foregoing embodiments.

[0156] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0157] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.

[0158] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0159] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0160] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0161] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for detecting small targets in images, characterized in that, include: Obtain the original feature map of the image to be detected, including small target images; The original feature map is segmented into M sub-maps through the first network layer of the target model, and the context features of the M sub-maps are extracted and fused to obtain a first feature map. The first feature map includes the spatial information of the small target image, and M is an integer greater than 1. The first feature map is downsampled through the second network layer of the target model to obtain N first-class feature maps. The N first-class feature maps have different resolutions and include the semantic information of the small target image at different resolutions. N is an integer greater than 1. The second feature map is obtained by fusing the N first-class feature maps and the first feature map through the third network layer of the target model. The small target image is obtained by detecting the N first-class feature maps and the second-class feature maps through the detection layer of the target model; Wherein, the first network layer is an adaptive spatial parallel convolutional layer, and the process of segmenting the original feature map into M sub-maps through the first network layer of the target model, and extracting and fusing contextual features from the M sub-maps to obtain the first feature map includes: The original feature map is segmented into M sub-maps through the M channels of the adaptive spatial parallel convolutional layer, and the M sub-maps are then subjected to parallel convolution to obtain M sub-feature maps. The convolutional layers corresponding to the M sub-maps are different, and M is an integer greater than 1. The M sub-feature maps are connected through channels and subjected to feature batch normalization to obtain the third feature map; The third feature map is fused with the original feature map to obtain the fourth feature map; The first feature map is obtained by downsampling and feature extraction of the fourth feature map.

2. The method according to claim 1, characterized in that, The step of segmenting the original feature map into M sub-maps through the M channels of the adaptive spatial parallel convolutional layer includes: The original feature map is divided into two sub-maps by the two channels of the adaptive spatial parallel convolutional layer. One of the sub-maps corresponds to the standard convolutional layer Conv2d, and the other sub-map corresponds to a convolutional layer including a single convolutional layer Conv and a standard convolutional layer Conv2d.

3. The method according to claim 2, characterized in that, The standard convolutional layer Conv2d and the single-layer convolutional layer Conv have a 3×3 or 1×1 kernel and a stride of 1.

4. The method according to claim 1, characterized in that, The second network layer includes a sub-backbone network and a feature map pyramid network. The process of downsampling the first feature map through the second network layer of the target model to obtain N first-class feature maps includes: The first feature map is transformed into Q second-class feature maps by X cross-stage double convolution transformation operators and 1 cross-stage double convolution transformation combination operator in the sub-backbone network, wherein X and Q are positive integers, the cross-stage double convolution transformation operator includes a convolution unit and a feature extraction unit, the cross-stage double convolution transformation combination operator includes a convolution unit and a spatial pyramid pooling unit, and the Q second-class feature maps include the first feature map; The Q second-class feature maps are connected by a matrix of the feature map pyramid network to obtain the N first-class feature maps.

5. The method according to claim 4, characterized in that, The step of connecting the Q second-class feature maps through the feature map pyramid network to obtain the N first-class feature maps includes: According to the output sequence, the Qth second-class feature map, which is the last output of the Q second-class feature maps, is passed through the feature map pyramid network to obtain the first first-class feature map among the N first-class feature maps. The first first-type feature map is connected to the (Q-1)th second-type feature map among the Q second-type feature maps, and the second first-type feature map among the N first-type feature maps is obtained through the cross-stage double convolution transformation and combination operator in the feature map pyramid network. And so on, until the second second-class feature map in the Q second-class feature maps is connected to the (N-1)th first-class feature map in the N first-class feature maps, and the Nth first-class feature map in the N first-class feature maps is obtained by cross-stage double convolution transformation and combination operator in the feature map pyramid network. The N first-class feature maps include the first first-class feature map, the second first-class feature map, and the Nth first-class feature map.

6. The method according to claim 1, characterized in that, The third network layer is a multi-scale fusion network. The process of fusing the N first-class feature maps and the first feature map through the third network layer of the target model to obtain the second feature map includes: The N first-class feature maps are mapped to an ascending filter array of sub-pixel convolutional layers to generate a fifth feature map, the resolution of which is greater than the resolution of the N first-class feature maps; The second feature map is obtained by connecting and fusing the fifth feature map with the first feature map through a channel.

7. The method according to claim 6, characterized in that, The step of mapping the N first-class feature maps to an ascending filter array of sub-pixel convolutional layers to generate a fifth feature map includes: According to the output timing, the (N-1)th first-class feature map of the N first-class feature maps is mapped to the sixth feature map by the ascending filter array of the sub-pixel convolutional layer. The sixth feature map is fused with the Nth first-class feature map among the N first-class feature maps through channel connection, and then upsampled to generate the fifth feature map.

8. The method according to claim 7, characterized in that, The step of connecting and fusing the fifth feature map with the first feature map to obtain the second feature map includes: The first feature map is processed through the convolutional layer of the third network layer to extract features and obtain the seventh feature map. The second feature map is obtained by connecting and fusing the seventh feature map with the fifth feature map.

9. The method according to claim 1, characterized in that, The step of detecting the N first-class feature maps and the second-class feature map through the detection layer of the target model to obtain the small target image includes: The N first-class feature maps and the second-class feature maps are respectively passed through the detection layer to obtain a set of classification and localization results; The small target image is obtained by filtering the classification and localization results in the classification and localization result set through confidence thresholding and non-maximum suppression.

10. The method according to any one of claims 1 to 9, characterized in that, The method further includes: Obtain a sample set and an initial model. The initial model includes an initial network layer and an initial detection layer. The sample set includes sample images of sample objects, the true location of the sample objects in the sample images, and the true classification of the sample objects in the sample images. The sample set is input into the initial model to obtain the sample detection location and sample detection classification; Based on the sample detection location, the sample detection classification, the true location, and the true classification, the joint difference between the sample detection location and the true location output by the initial model, as well as between the sample detection classification and the true classification, is determined through a joint loss function. Based on the joint difference degree, the model parameters of the initial model are adjusted until the target conditions are met, and the adjustment is stopped to obtain the target model.

11. The method according to claim 10, characterized in that, The joint loss function includes location loss, confidence loss, and classification loss. The location loss is the error between the detected location of the sample and the true location. The confidence loss is used to indicate the relative error of the predicted confidence score of the sample object. The classification loss is used to indicate the error between the detected classification of the sample and the true classification.

12. A detection device, characterized in that, include: The acquisition module is used to acquire the original feature map of the image to be detected, including small target images; The first feature extraction module is used to segment the original feature map into M sub-maps through the first network layer of the target model, and to extract and fuse the context features of the M sub-maps to obtain a first feature map. The first feature map has the spatial information of the small target image, and M is an integer greater than 1. The second feature extraction module is used to downsample the first feature map through the second network layer of the target model to obtain N first-class feature maps. The N first-class feature maps have different resolutions and include the semantic information of the small target image at different resolutions. N is an integer greater than 1. The fusion module is used to fuse the N first-class feature maps and the first feature map through the third network layer of the target model to obtain a second feature map; The detection module is used to detect the N first-class feature maps and the second feature map through the detection layer of the target model to obtain the small target image; Wherein, the first network layer is an adaptive spatial parallel convolutional layer, and the first feature extraction module is specifically used for: The original feature map is segmented into M sub-maps through the M channels of the adaptive spatial parallel convolutional layer, and the M sub-maps are then subjected to parallel convolution to obtain M sub-feature maps. The convolutional layers corresponding to the M sub-maps are different, and M is an integer greater than 1. The M sub-feature maps are connected through channels and subjected to feature batch normalization to obtain the third feature map; The third feature map is fused with the original feature map to obtain the fourth feature map; The first feature map is obtained by downsampling and feature extraction of the fourth feature map.

13. The apparatus according to claim 12, characterized in that, The step of segmenting the original feature map into M sub-maps through the M channels of the adaptive spatial parallel convolutional layer includes: The original feature map is divided into two sub-maps by the two channels of the adaptive spatial parallel convolutional layer. One of the sub-maps corresponds to the standard convolutional layer Conv2d, and the other sub-map corresponds to a convolutional layer including a single convolutional layer Conv and a standard convolutional layer Conv2d.

14. The apparatus according to claim 13, characterized in that, The standard convolutional layer Conv2d and the single-layer convolutional layer Conv have a 3×3 or 1×1 kernel and a stride of 1.

15. The apparatus according to claim 12, characterized in that, The second network layer includes a sub-backbone network and a feature map pyramid network. The process of downsampling the first feature map through the second network layer of the target model to obtain N first-class feature maps includes: The first feature map is transformed into Q second-class feature maps by X cross-stage double convolution transformation operators and 1 cross-stage double convolution transformation combination operator in the sub-backbone network, wherein X and Q are positive integers, the cross-stage double convolution transformation operator includes a convolution unit and a feature extraction unit, the cross-stage double convolution transformation combination operator includes a convolution unit and a spatial pyramid pooling unit, and the Q second-class feature maps include the first feature map; The Q second-class feature maps are connected by a matrix of the feature map pyramid network to obtain the N first-class feature maps.

16. A computer device, characterized in that, include: Memory, processor, and bus system; The memory is used to store programs; The processor is configured to execute a program in the memory, and the processor is configured to execute the method of any one of claims 1 to 11 according to instructions in the program code; The bus system is used to connect the memory and the processor to enable communication between the memory and the processor.

17. A computer-readable storage medium comprising instructions that, when executed on a computer, cause the computer to perform the method as claimed in any one of claims 1 to 11.

18. A computer program product, characterized in that, The method includes computer instructions stored in a computer-readable storage medium; a processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions to cause the computer device to perform the method as described in any one of claims 1 to 11.

Images