A target detection method, apparatus, storage medium, and electronic device

Abstract

This invention discloses a target detection method, apparatus, storage medium, and electronic device. The method includes: acquiring an image to be detected; inputting the image to be detected into a pre-trained target detection model to obtain a target detection image output by the target detection model. The target detection model includes a feature extraction module, a spatial attention module, and a channel attention module. The extraction module extracts feature information at different levels. The spatial attention module performs spatial feature weighting on the first feature information extracted by the feature extraction module, and the channel attention module performs channel weighting on the second feature information extracted by the feature extraction module. The spatially weighted first feature information and the channel-weighted second feature information are fused to obtain the target detection image. By using an attention mechanism to focus on important features in both spatial and channel features and suppress unnecessary features, the accuracy of target detection can be improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a target detection method, apparatus, storage medium and electronic device. Background Technology

[0002] Saliency object detection, as a preprocessing step for other visual tasks, has always been a research hotspot in the field of computer vision. It can be widely applied to tasks such as automatic image cropping, image recognition and detection, video summarization, and object tracking.

[0003] Traditional saliency detection methods based on handcrafted features rely mainly on handcrafted features, which may fail to describe complex image scenes and structures, cannot adapt to new scenes and objects, and have poor generalization ability. Summary of the Invention

[0004] This invention provides a target detection method, apparatus, storage medium, and electronic device to improve the detection accuracy of targets.

[0005] In a first aspect, embodiments of the present invention provide a target detection method, comprising:

[0006] Acquire the image to be detected;

[0007] The image to be detected is input into a pre-trained target detection model to obtain the target detection image output by the target detection model. The target detection model includes a feature extraction module, a spatial attention module, and a channel attention module. The feature extraction module is used to extract feature information at different levels. The spatial attention module is used to perform spatial feature weighting on the first feature information extracted by the feature extraction module. The channel attention module is used to perform channel weighting on the second feature information extracted by the feature extraction module. The first feature information after spatial feature weighting and the second feature information after channel weighting are fused to obtain the target detection image.

[0008] Secondly, embodiments of the present invention also provide a target detection device, comprising:

[0009] The image acquisition module is used to acquire the image to be detected;

[0010] The target detection module is used to input the image to be detected into a pre-trained target detection model to obtain the target detection image output by the target detection model. The target detection model includes a feature extraction module, a spatial attention module, and a channel attention module. The feature extraction module is used to extract feature information at different levels. The spatial attention module is used to perform spatial feature weighting on the first feature information extracted by the feature extraction module. The channel attention module is used to perform channel weighting on the second feature information extracted by the feature extraction module. The first feature information after spatial feature weighting and the second feature information after channel weighting are fused to obtain the target detection image.

[0011] Thirdly, embodiments of this disclosure also provide an electronic device, the electronic device comprising:

[0012] One or more processors;

[0013] Storage device for storing one or more programs.

[0014] When the one or more programs are executed by the one or more processors, the one or more processors implement the target detection method as described in any of the embodiments of this disclosure.

[0015] Fourthly, embodiments of this disclosure also provide a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the target detection method as described in any of the embodiments of this disclosure.

[0016] The technical solution of this embodiment pre-sets a target detection model including a spatial attention module and a channel attention module. The target detection model performs target detection on the image to be detected. During the target detection process, the attention mechanism focuses on important features in the spatial and channel features and suppresses unnecessary features, which helps to improve the accuracy of target detection. Attached Figure Description

[0017] Figure 1 This is a schematic flowchart of a target detection method provided in an embodiment of the present invention;

[0018] Figure 2 This is a schematic diagram of the structure of a target detection model provided in an embodiment of the present invention;

[0019] Figure 3 This is a structural example diagram of a target detection model provided in an embodiment of the present invention;

[0020] Figure 4 This is a schematic diagram of the spatial attention module in an embodiment of the present invention;

[0021] Figure 5 This is a schematic diagram of the structure of a channel attention module provided in an embodiment of the present invention;

[0022] Figure 6 This is a flowchart of an image target detection method provided by an embodiment of the present invention;

[0023] Figure 7 This is a schematic diagram of the structure of a target detection device provided in an embodiment of the present invention;

[0024] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0025] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.

[0026] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0027] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0028] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0029] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0030] Figure 1This is a flowchart illustrating a target detection method provided in an embodiment of the present invention. This embodiment is applicable to the identification of regions of interest in an image. The method can be executed by the target detection device provided in this embodiment of the present invention. The target detection device can be implemented by software and / or hardware. The target detection device can be configured on electronic computing devices such as mobile phones, PCs, computers, and servers. Specifically, it includes the following steps:

[0031] S110. Obtain the image to be detected.

[0032] S120. The image to be detected is input into a pre-trained target detection model to obtain the target detection image output by the target detection model. The target detection model includes a feature extraction module, a spatial attention module, and a channel attention module. The feature extraction module is used to extract feature information at different levels. The spatial attention module is used to perform spatial feature weighting on the first feature information extracted by the feature extraction module. The channel attention module is used to perform channel weighting on the second feature information extracted by the feature extraction module. The first feature information after spatial feature weighting and the second feature information after channel weighting are fused to obtain the target detection image.

[0033] In some embodiments, the image to be detected can be a video frame image from a video, such as surveillance video or medical imaging video. By using each video frame image as the image to be detected and performing target detection, the region of interest (ROI), i.e., the target object, is determined within the video frame image. The original video is then processed based on the target detection images of each video frame image. The processing method can be based on generating a video summary or generating the movement trajectory of the target object, etc., and is not limited thereto; the processing method can be determined according to business requirements.

[0034] In some embodiments, the image to be detected may be an externally imported image, including but not limited to portrait images, landscape images, food images, medical images, etc. The externally imported image is used as the image to be detected for target detection. Subsequent processing can be performed based on the obtained target detection image. For example, the processing methods may include, but are not limited to, segmenting the target object in the image to be detected based on the target detection image, setting display effects on the target object based on the target detection image, etc. There are no limitations on this, and the processing method can be determined according to business needs.

[0035] In this embodiment, when the triggering conditions are met, the target detection method is executed on the image to be detected. The triggering conditions include, but are not limited to, the selection of a preset control, the import or upload of the image to be detected, and the entry of the detection interface.

[0036] A pre-defined object detection model is used. This model can be a neural network model, specifically a pre-trained network model with object detection capabilities. For example, see [link to example]. Figure 2 , Figure 2 This is a schematic diagram of the structure of a target detection model provided in an embodiment of the present invention. The target detection model includes a feature extraction module, a spatial attention module, and a channel attention module. Optionally, the feature extraction module includes multiple sub-modules or multiple convolutional blocks for sequentially extracting feature information at different levels. As the depth of the sub-modules or convolutional blocks increases, the depth of the extracted feature information also increases. By extracting feature information at different levels, the comprehensiveness of feature information can be improved, and the loss of feature information can be avoided.

[0037] In some optional embodiments, the feature extraction module includes a first feature extraction submodule for extracting shallow feature information and a second feature extraction submodule for extracting deep feature information. The first feature extraction submodule is connected to the spatial attention module, and the second feature extraction submodule is connected to the channel attention module. In this embodiment, the extracted feature information can be divided into first feature information and second feature information according to the number of layers in the feature extraction module. For example, the first feature information can be shallow feature information, and the second feature information can be deep feature information. The first feature information and the second feature information can be in the form of feature maps, data matrices, etc., and there can be at least one of the first feature information and the second feature information, which is not limited thereto.

[0038] Optionally, the number of the first feature extraction submodule and the second feature extraction submodule is at least one. Optionally, the first feature extraction submodule and the second feature extraction submodule may each include at least one convolutional block or convolutional layer capable of outputting feature information. For example, the first feature extraction submodule includes at least one convolutional block, each convolutional block outputting first feature information; the second feature extraction submodule includes at least one convolutional block, each convolutional block outputting second feature information. The data of the convolutional blocks in the first and second feature extraction submodules is not limited. For example, the first feature extraction submodule may include two convolutional blocks, and the second feature extraction submodule may include three convolutional blocks, correspondingly, two first feature information pieces and three second feature information pieces. The structure of each convolutional block may be the same or different; for example, a convolutional block may include two convolutional layers, or a convolutional block may include three convolutional layers. An activation function layer and / or a pooling layer may be set after the convolutional layers; this is not limited and can be set according to requirements. Optionally, the feature extraction module can be improved based on the VGG-16 network model. Specifically, it can be obtained by replacing the last fully connected layer in the VGG-16 network model with a convolutional layer. Correspondingly, the feature extraction module includes five convolutional blocks: VGG1-2, VGG2-2, VGG3-3, VGG4-3, and VGG5-3. Convolutional blocks VGG1-2 and VGG2-2 can belong to the first feature extraction submodule, used to output the first feature information, i.e., shallow feature information. Convolutional blocks VGG3-3, VGG4-3, and VGG5-3 can belong to the second feature extraction submodule, used to output the second feature information, i.e., deep feature information. See the example below. Figure 3 , Figure 3 This is a structural example diagram of a target detection model provided in an embodiment of the present invention.

[0039] The spatial attention module is connected to the first feature extraction submodule, specifically to each convolutional block within it, and processes the first feature information output by the first feature extraction submodule. The spatial attention module extracts spatial feature information from the first feature information and weights this extracted spatial feature information into the first feature information. Optionally, the spatial attention module may include pooling layers, convolutional layers, and activating function layers. The pooling layers may include parallel average pooling layers and max pooling layers. Average pooling layers average the feature points in the neighborhood, resulting in better background preservation, while max pooling layers maximize the feature points in the neighborhood, resulting in better texture extraction. By using parallel average pooling and max pooling layers, the first feature information is processed from different dimensions to preserve both background and texture features, improving the comprehensiveness of feature preservation and avoiding feature loss. See also... Figure 4 , Figure 4This is a schematic diagram of the spatial attention module in an embodiment of the present invention. Figure 4 In this context, R represents the output information of the spatial attention module, i.e., the first feature information, and R' represents the feature information output after processing by the spatial attention module.

[0040] For any input information, the spatial attention module performs the following processing steps: average pooling and max pooling are performed on the first feature information (i.e., the input information), the average pooling result and the max pooling result are concatenated and then convolved, the convolution result is nonlinearly processed to obtain spatial feature information, and the spatial feature information is weighted onto the first feature information to obtain the feature information output by the spatial attention module.

[0041] The channel attention module is connected to the second feature extraction submodule, specifically to each convolutional block within the second feature extraction submodule, and processes the second feature information output by the second feature extraction submodule. The channel attention module determines the dependencies between channels, obtains the weights of each channel, and performs weighted processing on the feature information of each channel using these weights.

[0042] Optionally, the channel attention module includes a pooling layer and two fully connected layers, wherein the pooling layer can be an average pooling layer, and each fully connected layer can be followed by a post-activation function layer. For example, see [link to example]. Figure 5 , Figure 5 This is a schematic diagram of the structure of a channel attention module provided in an embodiment of the present invention.

[0043] For any input information, the channel attention module performs the following processing: it performs average pooling on the second feature information (i.e., the input information), and then inputs the average pooling result into two fully connected layers in sequence. After normalization, the channel weights are obtained, and the channel features in the second feature information are weighted based on the channel weights.

[0044] A target detection image is obtained by fusing the feature information processed by the spatial attention module and the feature information processed by the channel attention module. The feature information processed by the spatial attention module and the feature information processed by the channel attention module can each be in the form of feature maps. The fusion of multiple feature maps can be achieved by averaging the pixel values ​​of corresponding pixels in each color channel, resulting in the fused target detection image. This target detection image includes both shallow spatial feature information and deep channel feature information, improving the comprehensiveness of the feature information and further enhancing the accuracy of target detection.

[0045] Based on the above embodiments, when there are two or more first feature information and second feature information respectively, see [reference needed]. Figure 3Different first feature information pieces have inconsistent data sizes, and different second feature information pieces also have inconsistent data sizes. In order to perform feature fusion, the sizes of the first and second feature information pieces need to be adjusted. Optionally, an upsampling module can be set in the target detection model to ensure that the data sizes of multiple first feature information pieces are consistent, the data sizes of multiple second feature information pieces are consistent, and the data sizes of the feature information to be fused are consistent.

[0046] Accordingly, the target detection model further includes: a first upsampling module, disposed between the first feature extraction submodule and the spatial attention module, for upsampling the first feature information to a first preset size; and a second upsampling module, disposed between the second feature extraction submodule and the channel attention module, for upsampling the second feature information to a second preset size.

[0047] Optionally, the first preset size can be the largest data size of the first feature information, and the second preset size can be the largest data size of the second feature information. It should be noted that the data size of the second feature information is smaller than the data size of the second feature information. A third upsampling module is also set after the channel attention module to upsample the processed second feature information to the first preset size.

[0048] Optionally, the first preset size and the second preset size can be the same, that is, the data size of the output image. The data size of the output image can be the same as the data size of the input image, or it can be the data size of the largest first feature information, or it can be a preset output data size. There is no limitation on this, and it can be determined according to the output requirements.

[0049] The technical solution of this embodiment pre-sets a target detection model including a spatial attention module and a channel attention module. The target detection model performs target detection on the image to be detected. During the target detection process, the attention mechanism focuses on important features in the spatial and channel features and suppresses unnecessary features, which helps to improve the accuracy of target detection.

[0050] Based on the above technical solution, after acquiring the image to be detected, the method further includes: determining the prior detection image of the image to be detected; correspondingly, the step of inputting the image to be detected into a pre-trained target detection model to obtain the detection result output by the target detection model includes: inputting the image to be detected and the prior detection image into the pre-trained target detection model to obtain the detection result output by the target detection model.

[0051] In this embodiment, prior object detection can be performed based on a pre-set prior detection model to obtain a prior detection image, which is the initial image for object detection. By jointly inputting the prior detection image and the image to be detected into the object detection model, prior guidance is provided for the object detection process based on the prior detection image, facilitating the improvement of the accuracy of object detection.

[0052] In some embodiments, the prior detection model may have the same structure as the feature extraction module in the object detection model. Exemplarily, it is obtained by improving VGG-16. Further, during the training process of the object detection model, the network parameters of the prior detection model are used as the initial network parameters of the feature extraction module in the object detection model, facilitating the acceleration of the training efficiency of the object detection model.

[0053] Exemplarily, refer to Figure 6 , Figure 6 which is a flowchart for object detection of an image provided by an embodiment of the present invention. Figure 6 In [the figure], the image to be detected is input into the prior detection model to obtain a prior detection image. The image to be detected and the prior detection image are input into a pre-trained object detection model to obtain the detection result output by the object detection model, that is, the object detection image.

[0054] Among them, the training process of the object detection model is as follows: Prepare the image training set and test set required for training. The training data set is enhanced through data enhancement methods such as cropping and horizontal flipping, so that the enhanced data set is 4 times the initial data set.

[0055] Based on the VGG-16 model, the last fully connected layer is changed to a convolutional layer to obtain a prior detection model. An RGB image is input for training to obtain an initial saliency detection map, that is, the prior detection image. Through the improvement of the VGG-16 model, an RGB image (i.e., the image to be detected) is used as the input of the network model, and the output is a saliency map. Based on the improved VGG-16 network model, the training data is resized to a unified size as the input, and the last two fully connected layers of the network are changed to fully convolutional layers. The entire network structure has 15 convolutional layer sequences for feature extraction from low dimension to high dimension. After the convolutional layers from VGG1-2 to VGG5-3, downsampling is performed, making the size of the original image reduced. The output after the VGG5-3 layer is 1 / 32 of the size of the original image. Then, a series of upsampling layers are performed until the output feature map is restored to the size of the input image. Thus, the network directly outputs a saliency map (i.e., the prior detection image).

[0056] The obtained saliency detection map, along with the input RGB image (i.e., the image to be detected), is used as input to an attention-based convolutional neural network (i.e., the object detection module). This convolutional neural network includes a feature extraction module, also based on an improved VGG-16 model. The low-level features output from VGG1-2 and VGG2-2 in the feature extraction module are upsampled to the size of conv1-2 and added to the spatial attention module to better utilize low-level features (i.e., shallow features). The high-level features output from VGG3-3, VGG4-3, and VGG5-3 in the feature extraction module are added to the channel attention module and upsampled to the size of conv1-2 to better utilize high-level features (i.e., deep features). The upsampled low-level and high-level features are then fused together to train the final saliency detection result.

[0057] Based on the above embodiments, this invention also provides a target detection device, see [link to previous embodiment]. Figure 7 , Figure 7 This is a schematic diagram of a target detection device provided in an embodiment of the present invention. The device includes:

[0058] Image acquisition module 210 is used to acquire the image to be detected;

[0059] The target detection module 220 is used to input the image to be detected into a pre-trained target detection model to obtain the target detection image output by the target detection model. The target detection model includes a feature extraction module, a spatial attention module, and a channel attention module. The feature extraction module is used to extract feature information at different levels. The spatial attention module is used to perform spatial feature weighting on the first feature information extracted by the feature extraction module. The channel attention module is used to perform channel weighting on the second feature information extracted by the feature extraction module. The first feature information after spatial feature weighting and the second feature information after channel weighting are fused to obtain the target detection image.

[0060] Optionally, the device may also include:

[0061] The prior detection image determination module is used to determine the prior detection image of the image to be detected after acquiring the image to be detected;

[0062] Accordingly, the target detection module 220 is used for:

[0063] The image to be detected and the prior detection image are input into a pre-trained target detection model to obtain the detection result output by the target detection model.

[0064] Optionally, the spatial attention module is used to perform average pooling and max pooling on the first feature information, concatenate the average pooling result and max pooling result and then perform convolution, perform nonlinear processing on the convolution result to obtain spatial feature information, and then weight the spatial feature information onto the first feature information.

[0065] Optionally, the channel attention module is used to perform average pooling on the second feature information, and input the average pooling result into two fully connected layers in sequence. After normalization, the channel weights are obtained, and the channel features in the second feature information are weighted based on the channel weights.

[0066] Optionally, the feature extraction module includes a first feature extraction submodule for extracting shallow feature information and a second feature extraction submodule for extracting deep feature information. The first feature extraction submodule is connected to the spatial attention module, and the second feature extraction submodule is connected to the channel attention module.

[0067] Optionally, the first feature extraction submodule includes at least one convolutional block, each convolutional block outputting first feature information; the second feature extraction submodule includes at least one convolutional block, each convolutional block outputting second feature information.

[0068] Optionally, the target detection model further includes:

[0069] A first upsampling module is disposed between the first feature extraction submodule and the spatial attention module, and is used to upsample the first feature information to a first preset size;

[0070] The second upsampling module is located between the second feature extraction submodule and the channel attention module, and is used to upsample the second feature information to a second preset size.

[0071] The target detection device provided in the embodiments of the present invention can execute the target detection method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the method execution.

[0072] The following is for reference. Figure 8 It illustrates an electronic device suitable for implementing embodiments of the present disclosure (e.g., Figure 8 The diagram below shows the structure of the terminal device or server 400. The terminal device in this embodiment may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and vehicle terminals (e.g., vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 8The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0073] like Figure 8 As shown, electronic device 400 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 401, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 402 or a program loaded from storage device 408 into random access memory (RAM) 403. RAM 403 also stores various programs and data required for the operation of electronic device 400. The processing device 401, ROM 402, and RAM 403 are interconnected via bus 404. Input / output (I / O) interface 405 is also connected to bus 404.

[0074] Typically, the following devices can be connected to I / O interface 405: input devices 406 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 407 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 408 including, for example, magnetic tapes, hard disks, etc.; and communication devices 409. Communication device 409 allows electronic device 400 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 8 An electronic device 400 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0075] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 409, or installed from a storage device 408, or installed from a ROM 402. When the computer program is executed by the processing device 401, it performs the functions defined in the methods of embodiments of this disclosure.

[0076] The electronic device provided in this embodiment belongs to the same inventive concept as the target detection method provided in the above embodiments. Technical details not described in detail in this embodiment can be found in the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.

[0077] This disclosure provides a computer storage medium storing a computer program that, when executed by a processor, implements the target detection method provided in the above embodiments.

[0078] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0079] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0080] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0081] The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to:

[0082] Acquire the image to be detected;

[0083] The image to be detected is input into a pre-trained target detection model to obtain the target detection image output by the target detection model. The target detection model includes a feature extraction module, a spatial attention module, and a channel attention module. The feature extraction module is used to extract feature information at different levels. The spatial attention module is used to perform spatial feature weighting on the first feature information extracted by the feature extraction module. The channel attention module is used to perform channel weighting on the second feature information extracted by the feature extraction module. The first feature information after spatial feature weighting and the second feature information after channel weighting are fused to obtain the target detection image.

[0084] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0085] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0086] The units described in the embodiments of this disclosure can be implemented in software or hardware. The names of the units / modules do not necessarily limit the specific unit itself.

[0087] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0088] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0089] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.

Claims

1. A target detection method, characterized in that, include: Acquire the image to be detected; The image to be detected is input into a pre-trained target detection model to obtain the target detection image output by the target detection model. The target detection model includes a feature extraction module, a spatial attention module, and a channel attention module. The feature extraction module includes a first feature extraction submodule for extracting first feature information and a second feature extraction submodule for extracting second feature information. The first feature information is shallow feature information, and the second feature information is deep feature information. The first feature extraction submodule is connected to the spatial attention module, and the second feature extraction submodule is connected to the channel attention module. The spatial attention module performs average pooling and max pooling on the first feature information, respectively, and combines the results of the average pooling and max pooling processes. After the images are stitched together, convolution processing is performed. The convolution result is then subjected to nonlinear processing to obtain spatial feature information. This spatial feature information is weighted onto the first feature information to obtain the first feature information after spatial feature weighting. The channel attention module is used to perform average pooling processing on the second feature information. The average pooling result is sequentially input into two fully connected layers and normalized to obtain channel weights. Based on the channel weights, each channel feature in the second feature information is weighted to obtain the second feature information after channel weighting. The first feature information after spatial feature weighting and the second feature information after channel weighting are fused to obtain the target detection image. The first feature information and the second feature information are obtained by upsampling and have the same size. After acquiring the image to be detected, the method further includes: The image to be detected is subjected to prior target detection using a pre-set prior detection model to obtain a prior detection image; Accordingly, the step of inputting the image to be detected into a pre-trained target detection model to obtain the detection result output by the target detection model includes: The image to be detected and the prior detection image are input into a pre-trained target detection model to obtain the detection result output by the target detection model. The prior detection model has the same structure as the feature extraction module in the target detection model.

2. The method according to claim 1, characterized in that, The first feature extraction submodule includes at least one convolutional block, each of which outputs first feature information; the second feature extraction submodule includes at least one convolutional block, each of which outputs second feature information.

3. The method according to claim 2, characterized in that, The target detection model also includes: A first upsampling module is disposed between the first feature extraction submodule and the spatial attention module, and is used to upsample the first feature information to a first preset size; The second upsampling module is located between the second feature extraction submodule and the channel attention module, and is used to upsample the second feature information to a second preset size.

4. A target detection device, characterized in that, include: The image acquisition module is used to acquire the image to be detected; The object detection module is used to input the image to be detected into a pre-trained object detection model to obtain the object detection image output by the object detection model. The object detection model includes a feature extraction module, a spatial attention module, and a channel attention module. The feature extraction module includes a first feature extraction submodule for extracting first feature information and a second feature extraction submodule for extracting second feature information. The first feature information is shallow feature information, and the second feature information is deep feature information. The first feature extraction submodule is connected to the spatial attention module, and the second feature extraction submodule is connected to the channel attention module. The spatial attention module is used to perform average pooling and max pooling processing on the first feature information, and then converts the average pooling result and the max pooling result into a single output. After the processing results are concatenated, convolution processing is performed. The convolution processing result is then subjected to nonlinear processing to obtain spatial feature information. The spatial feature information is weighted onto the first feature information to obtain the first feature information after spatial feature weighting. The channel attention module is used to perform average pooling processing on the second feature information. The average pooling processing result is sequentially input into two fully connected layers and normalized to obtain channel weights. Based on the channel weights, the channel features in the second feature information are weighted to obtain the second feature information after channel weighting. The first feature information after spatial feature weighting and the second feature information after channel weighting are fused to obtain the target detection image. The first feature information and the second feature information are obtained by upsampling and have the same size. The prior detection image determination module is used to perform prior target detection on the image to be detected using a pre-set prior detection model to obtain a prior detection image; Correspondingly, the target detection module is also used for: The image to be detected and the prior detection image are input into a pre-trained target detection model to obtain the detection result output by the target detection model. The prior detection model has the same structure as the feature extraction module in the target detection model.

5. An electronic device, characterized in that, The electronic device includes: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the target detection method as described in any one of claims 1-3.

6. A storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the target detection method as described in any one of claims 1-3.

Images