An image detection method and device, electronic equipment and storage medium

Abstract

The application relates to the technical field of image processing, and particularly provides an image detection method and device, aiming to solve the problems of poor detection effect and low detection precision of a detection target in an image. To this end, the image detection method comprises the following steps: inputting an image to be detected into an image detection model; extracting complementary features and redundant features of the image to be detected by using a feature extraction network; performing splicing processing on the complementary features and the redundant features to obtain first features; transmitting the first features to a feature fusion network; performing fusion processing on the first features by using the feature fusion network to obtain second features; transmitting the second features to the feature processing network; and sequentially performing positioning and classification processing on the second features by using the feature processing network to obtain a detection target. The complementary features and the redundant features of the image to be detected are spliced, fused, positioned and classified to obtain the detection target, so that the detection precision and speed of the image to be detected can be effectively improved, and the calculation amount of the algorithm can be greatly reduced.

Description

Technical Field

[0001] This application relates to the field of image processing technology, specifically to an image detection method, apparatus, electronic device, and computer-readable storage medium. Background Technology

[0002] Remote sensing images are aerial views taken from a bird's-eye perspective. These images contain varying target orientations and significant background areas, resulting in characteristics quite different from natural images taken vertically from the ground. Target features in remote sensing images are more blurred, and targets are smaller, such as airplanes, ships, cars, and windmills.

[0003] Data augmentation plays an important role in image processing. In the field of object detection, commonly used data augmentation methods include horizontal-vertical flipping, random cropping, scale transformation, elastic distortion, etc. However, they are all limited to specific datasets and are difficult to adapt to remote sensing image datasets.

[0004] In addition, if some images are taken at night or in poor lighting conditions, the low visibility will cause some targets to be lost, which will greatly affect the detection effect and reduce the detection accuracy of the image. Summary of the Invention

[0005] To address the above problems, this application provides an image detection method. This method inputs the image to be detected into an image detection model, uses a feature extraction network to extract complementary and redundant features from the image, concatenates these features to obtain a first feature, which is then transmitted to a feature fusion network. The feature fusion network fuses the first feature to obtain a second feature, which is then transmitted to a feature processing network. The feature processing network sequentially locates and classifies the second feature to obtain the detected target. This application, by concatenating, fusing, locating, and classifying the complementary and redundant features of the image to obtain the detected target, effectively improves the detection accuracy and speed of the image, while significantly reducing the computational load of the algorithm.

[0006] In a first aspect, embodiments of this application provide an image detection method, comprising: inputting an image to be detected into an image detection model, wherein the image detection model includes a feature extraction network, a feature fusion network, and a feature processing network; extracting complementary features and redundant features of the image to be detected using the feature extraction network, concatenating the complementary features and the redundant features to obtain a first feature, which is then transmitted to the feature fusion network; fusing the first feature using the feature fusion network to obtain a second feature, which is then transmitted to the feature processing network; and sequentially locating and classifying the second feature using the feature processing network to obtain a detection target.

[0007] Secondly, embodiments of this application provide an image detection device, comprising: an input module for inputting an image to be detected into an image detection model, wherein the image detection model includes a feature extraction network, a feature fusion network, and a feature processing network; an extraction module for extracting complementary and redundant features of the image to be detected using the feature extraction network, concatenating the complementary and redundant features to obtain a first feature, which is then transmitted to the feature fusion network; a feature fusion module for fusing the first feature using the feature fusion network to obtain a second feature, which is then transmitted to the feature processing network; and an acquisition module for sequentially locating and classifying the second feature using the feature processing network to obtain a detection target.

[0008] Thirdly, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements the method described in the first aspect above.

[0009] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in the first aspect above.

[0010] The technical solutions provided in this application embodiment have at least the following technical effects or advantages:

[0011] This application embodiment inputs the image to be detected into an image detection model, uses a feature extraction network to extract complementary and redundant features of the image to be detected, and concatenates the complementary and redundant features to obtain a first feature which is then transmitted to a feature fusion network. This improves the feature extraction capability for complex images to be detected, increases the detection accuracy of the image to be detected, reduces the computational load of the algorithm, and increases the detection speed.

[0012] Furthermore, the first feature is fused using a feature fusion network to obtain a second feature, which is then transmitted to the feature processing network. This allows for detailed fusion processing of the detailed information in the image to be detected, preventing the loss of the target image and improving detection accuracy.

[0013] Furthermore, by using a feature processing network to sequentially locate and classify the second feature to obtain the detection target, the consistency between image feature localization and classification of the image to be detected can be significantly improved, the detection error of the image to be detected can be reduced, and thus the detection accuracy of the image to be detected can be significantly improved.

[0014] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, in which specific embodiments of this application are described below. Attached Figure Description

[0015] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0016] Figure 1 A flowchart of an image detection method provided in an embodiment of this application is shown;

[0017] Figure 2 This paper shows a schematic diagram of the image detection model framework provided in an embodiment of this application;

[0018] Figure 3 It shows Figure 2 A schematic diagram of the feature extraction network structure;

[0019] Figure 4 This illustration shows a schematic diagram of the network training structure of the first feature extraction module and the second feature extraction module provided in an embodiment of this application;

[0020] Figure 5 It shows Figure 2 A schematic diagram of the structure of the feature processing network;

[0021] Figure 6 A schematic diagram of the structure of an image detection device provided in an embodiment of this application is shown.

[0022] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0023] Exemplary embodiments of this application will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of this application are shown in the drawings, it should be understood that this application may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of this application and to fully convey the scope of this application to those skilled in the art.

[0024] Remote sensing images are aerial views taken from a bird's-eye perspective. The orientation of targets in these images is arbitrary, and they also contain a large amount of background. Their characteristics differ significantly from natural images taken vertically from the ground. Target features in remote sensing images are more blurred, and targets are smaller, such as airplanes, ships, cars, and windmills.

[0025] Data augmentation plays an important role in image processing. In the field of object detection, commonly used data augmentation methods include horizontal-vertical flipping, random cropping, scale transformation, elastic distortion, etc. However, they are all limited to specific datasets and are difficult to adapt to remote sensing image datasets.

[0026] In addition, if some images are taken at night or in poor lighting conditions, the low visibility will cause some targets to be lost, which will greatly affect the detection effect and reduce the detection accuracy of the image.

[0027] Based on this, embodiments of this application provide an image detection method. The specific solution of these embodiments will be described below with reference to the accompanying drawings.

[0028] See Figure 1 The flowchart shown illustrates an image detection method, which specifically includes the following steps:

[0029] Step 101: Input the image to be detected into the image detection model.

[0030] In this embodiment, the image detection model includes a feature extraction network, a feature fusion network, and a feature processing network.

[0031] The image to be detected includes spectral images with different frequency distributions. A spectral image is an image that represents the frequency distribution of an input image. It is obtained by superimposing the one-dimensional Fourier transforms of all scan lines in the horizontal and vertical directions of the input image, and is used to represent the frequency distribution of the input image. In the embodiments of this application, the image to be detected may include a first spectral image and a second spectral image, and the frequency distributions of the first spectral image and the second spectral image are different.

[0032] See Figure 1 As shown, the first and second spectral images are input into the feature extraction network, the feature extraction network outputs the first feature, the first feature is transmitted to the feature fusion network to obtain the second feature, and the second feature is transmitted to the feature processing network to obtain the detection target.

[0033] Step 102: Use a feature extraction network to extract complementary and redundant features from the image to be detected, and concatenate the complementary and redundant features to obtain the first feature.

[0034] Based on the above Figure 1 As described in the image detection model, the first spectral image and the second spectral image are input into the feature extraction network, and the feature extraction network outputs the first feature. Specifically, the feature extraction network can be used to extract complementary features and redundant features of the image to be detected, and the complementary features and the redundant features are concatenated to obtain the first feature.

[0035] In this embodiment, the feature extraction network may include a first feature extraction module, a second feature extraction module, a first feature fusion module, and a feature extraction module.

[0036] Complementary features refer to the fact that the first spectral image contains feature information of the target being detected, and the second spectral image contains feature information of the target being detected.

[0037] Redundant features refer to the fact that the first spectral image includes feature information of other targets besides the detected target, and the second spectral image contains feature information of other targets besides the detected target.

[0038] See Figure 3 As shown, in one embodiment, a first feature extraction module can be used to extract a first complementary feature and a first redundant feature from a first spectral image, and transmit the first complementary feature and the first redundant feature to a first feature fusion module. A second feature extraction module can be used to extract a second complementary feature and a second redundant feature from a second spectral image, and transmit the second complementary feature and the second redundant feature to the first feature fusion module. Then, the first feature fusion module can be used to fuse the first complementary feature and the second complementary feature to obtain a third complementary feature. The first redundant feature and the second redundant feature can be fused to obtain a third redundant feature. The third complementary feature and the third redundant feature can be spliced ​​together to obtain the merged feature of the image to be detected, which is then transmitted to a feature extraction module. Finally, the feature extraction module can be used to extract features from the merged feature to obtain the first feature.

[0039] Furthermore, the feature extraction network has five layers, and each layer outputs a feature map information that has been downsampled. The downsampling factor can be preset by those skilled in the art based on experimental data, or it can be a factor obtained by those skilled in the art after adjusting the preset factor according to actual needs. This application embodiment does not make specific limitations. Preferably, the downsampling factor used in this application embodiment is 2 times.

[0040] Furthermore, the first feature extraction module includes a first layer and a second layer, with a max pooling layer between the first and second layers. The first layer includes a 1×1 convolutional layer 1, a normalized layer 1, and a ReLU activation function connected in sequence. The second layer includes a 1×1 convolutional layer 2, a normalized layer 2, a ReLU activation function, a 1×1 convolutional layer 3, a normalized layer 3, a ReLU activation function, a 1×1 convolutional layer 4, a normalized layer 4, and a ReLU activation function connected in sequence. The structure of the second feature extraction module is the same as that of the first feature extraction module, and will not be repeated here.

[0041] By setting up two layers for the first feature extraction module and the second feature extraction module, useful image features can be effectively extracted from the image, ensuring the alignment of pixels in the image features and improving the feature extraction capability of the image. At the same time, using two layers can also reduce the amount of computation and improve the running speed.

[0042] Furthermore, the feature extraction module includes a third level, a fourth level, and a fifth level. Moreover, the structure of the third level, the fourth level, and the fifth level is the same as that of the second level, and will not be repeated here.

[0043] Based on the above implementation methods, in some modified implementation methods, such as... Figure 4 As shown, before extracting the first complementary features and the first redundant features of the first spectral image using the first feature extraction module, the third spectral image and the fourth spectral image of each of the multiple training images can be obtained respectively. The features of the third spectral image of each training image are extracted using the first feature extraction module to obtain the first training feature. The features of the fourth spectral image of each training image are extracted using the second feature extraction module to obtain the second training feature. The first training feature and the second training feature are divided into two parts on an average basis in the channel dimension to obtain the first sub-feature and the second sub-feature corresponding to the first training feature, as well as the third sub-feature and the fourth sub-feature corresponding to the second training feature. Based on the first sub-feature and the third sub-feature, the first feature extraction module to be trained is trained using a spectral image classification network to obtain the first classification loss value. Based on the first classification loss value satisfying the first preset convergence condition, the first feature extraction module is obtained. Based on the second sub-feature and the fourth sub-feature, the second feature extraction module to be trained is trained using a domain adversarial spectral image classification network to obtain the second classification loss value. Based on the second classification loss value satisfying the second preset convergence condition, the second feature extraction module is obtained.

[0044] Furthermore, the spectrum image classification network includes two 1×1 convolutional layers, one ReLU activation function layer, and one sigmoid activation function layer, with Focalloss used as the loss function. The spectrum image classification network guides the network to learn features that allow the first and third sub-features to be extracted from each other in the feature space with a sufficiently large distance between them, until the first classification loss value is less than or equal to a preset first threshold. At this point, the network is considered to have met the first preset convergence condition, and the first feature extraction module is obtained.

[0045] Furthermore, the domain adversarial spectral image classification network includes one gradient inversion layer, two 1×1 convolutional layers, one ReLU activation function layer, and one sigmoid activation function layer. The Focal loss function is used. The domain adversarial spectral image classification network guides the network to learn features that allow the extraction of second and fourth sub-features that are sufficiently close in the feature space. This continues until the second classification loss value is less than or equal to a preset second threshold, at which point the second preset convergence condition is met, and the second feature extraction module is obtained.

[0046] It should be noted that the first classification loss value refers to the number or percentage of errors made by the spectral image classification network during training; the second classification loss value refers to the number or percentage of errors made by the domain adversarial spectral image classification network during training.

[0047] The first threshold and the second threshold can be values ​​preset by those skilled in the art based on experimental data, or values ​​obtained by those skilled in the art after adjusting the preset values ​​according to actual needs. This application does not impose specific limitations on the embodiments.

[0048] By training the first and second feature extraction modules, the feature extraction network is able to extract complementary and redundant features from the first and second spectral images. This eliminates the need to add an additional spectral image classification network and a domain adversarial spectral image classification network to the network, simplifying the structure of the feature extraction network, reducing the algorithm and detection time for image detection, effectively improving detection efficiency, and reducing computational load.

[0049] Based on the above embodiments, in some modified embodiments, the first feature fusion module is used to fuse the first complementary feature and the second complementary feature to obtain the third complementary feature. Specifically, the first feature fusion module can be used to add the first complementary feature and the second complementary feature point by point to obtain the third complementary feature; or, the first feature fusion module can be used to perform channel fusion processing on the first complementary feature and the second complementary feature to obtain the third complementary feature. Those skilled in the art can flexibly set it according to the actual situation, and the embodiments of this application do not make specific limitations.

[0050] Furthermore, after channel fusion processing of the first complementary feature and the second complementary feature using the first feature fusion module, an x3 convolution can be used to reduce the number of channels back to the original number to obtain the third complementary feature.

[0051] Based on the above implementation method, in some modified implementation methods, the first feature fusion module is used to fuse the first redundant feature and the second redundant feature to obtain the third redundant feature. Specifically, the first feature fusion module can be used to set a first preset weight for the first redundant feature and a second preset weight for the second redundant feature. The first redundant feature with the first preset weight is then fused with the second redundant feature with the second preset weight to obtain the third redundant feature.

[0052] It should be noted that both the first preset weight and the second preset weight can be weights set by those skilled in the art according to actual needs, or weights obtained by those skilled in the art after adjusting the set weights according to actual needs. This application embodiment does not impose specific limitations. Preferably, in this application embodiment, both the first preset weight and the second preset weight are 50%.

[0053] See Table (1) for the experimental results of the feature extraction network extracting the first feature of the image to be detected:

[0054]

[0055] Table (1)

[0056] Table (1) above shows the detection accuracy and detection speed obtained by using the feature extraction network of the prior art to extract features from everyday images, and the detection progress and detection data obtained by using the feature extraction network of this application to extract features from the first and second spectral images through each layer. It can be seen that by using two layers to extract features from the first and second spectral images and then fusing them, the detection accuracy obtained is 1.4% higher than that of the existing detection method. The method of extracting features first and then fusing them is more robust to inaccurate registration due to the network downsampling process, which significantly improves the feature extraction performance.

[0057] Furthermore, although the detection accuracy of feature extraction at the third level also improved by 1.4% AP50, the earlier the fusion layer is after feature extraction from the first and second spectral images, the more susceptible it is to mismatches. Conversely, the later the fusion layer is, the more likely the features will be submerged during the fusion process, and the greater the computational cost will be. Therefore, when the improvement in detection progress is the same, using the first two levels to extract features from the first and second spectral images can effectively reduce the computational cost and improve detection accuracy and speed.

[0058] Step 103: Use a feature fusion network to fuse the first feature to obtain the second feature.

[0059] In this embodiment, the feature fusion network uses an FPN+PAN network structure to fuse shallow and deep features obtained from the feature extraction network. The FPN network structure is a feature pyramid network capable of generating features of different sizes from the first feature; the PAN network structure is a method to enhance the representation of features generated by the FPN network.

[0060] Since the technique of fusing features through the network structure of FPN+PAN is a common technique used by those skilled in the art, it will not be described in detail in the embodiments of this application.

[0061] Step 104: Use the feature processing network to locate and classify the second feature sequentially to obtain the detection target.

[0062] In this embodiment, the feature processing network includes a localization module and a classification module connected in series.

[0063] The detection targets include the target location and the target category.

[0064] After obtaining the second feature through step 103 above, the feature processing network can be used to locate and classify the second feature in sequence to obtain the detection target.

[0065] See Figure 5 As shown, in one specific embodiment, the localization module can predict the bounding box of the target image based on the second feature, transmit the second feature and the bounding box to the classification module, and then use the classification module to extract the rotation-invariant features from the second feature based on the bounding box. The rotation-invariant features are then convolved according to a preset convolution size, and the convolved rotation-invariant features are classified to obtain the target location and target category.

[0066] By connecting the localization module and the classification module in series, the consistency of image features after the localization module and the classification module have performed localization and classification processing on the image can be effectively improved, as well as the consistency between localization accuracy and classification score can be effectively improved.

[0067] Based on the above implementation methods, in some modified implementation methods, a positioning quality module can be set in the positioning module. The second feature includes multiple feature points. The positioning module predicts the bounding box of the detected target based on the second feature. Specifically, the positioning module can predict the bounding box of each feature point based on the second feature, and then the positioning quality module can calculate the positioning quality of each bounding box. The bounding box with the highest positioning quality is taken as the bounding box of the target image.

[0068] By setting a localization quality module in the localization module to determine the bounding box of the target image, the computational load of the classification module for extracting rotation-invariant features of the bounding box can be reduced, effectively reducing the computational load and improving image detection efficiency.

[0069] Further, the bounding box includes the rotation angle of the bounding box formed by it and the horizontal or vertical line, the scale of the bounding box, and the aspect ratio of the bounding box. The classification module extracts rotation-invariant features from the second feature based on the bounding box. Specifically, an offset term can be determined based on the rotation angle, scale, and aspect ratio of the bounding box, and the second feature is rotated according to the offset term to obtain the rotation-invariant features. See Table (2) below for experimental results of image detection using the feature processing network of this application and existing image detection networks:

[0070] Detection Network IOU threshold Accuracy (AP50%) Speed ​​(FPS) Existing image detection networks 0.5 79.6 28.3 This application features a feature processing network. 0.5 83.0 27.0 This application features a feature processing network. 0.1 83.3 27.0

[0071] Table (2)

[0072] As shown in Table (2) above, the detection accuracy of the image is improved by 3.7% AP50 and the speed is increased by 1.3 FPS by using the feature processing network of this application, which effectively improves the detection progress and detection speed of the image.

[0073] The detection results obtained by the image detection method of this application and other existing image detection methods are shown in Table (3) below:

[0074]

[0075] Table (3)

[0076] As can be seen from the above table (3), the existing image detection methods can only detect one type of image, and the detection accuracy is not as good as the image detection method of this application. The image detection method of this application can significantly improve the detection accuracy of images and speed up the detection process, thereby improving the efficiency of image detection.

[0077] This application embodiment inputs the image to be detected into an image detection model, uses a feature extraction network to extract complementary and redundant features of the image to be detected, and concatenates the complementary and redundant features to obtain a first feature which is then transmitted to a feature fusion network. This improves the ability to extract features from complex images to be detected, increases the detection accuracy of the image to be detected, reduces the computational load of the algorithm, and increases the detection speed.

[0078] Furthermore, the first feature is fused using a feature fusion network to obtain a second feature, which is then transmitted to the feature processing network. This allows for detailed fusion processing of the detailed information in the image to be detected, preventing the loss of the target image and improving detection accuracy.

[0079] Furthermore, by using a feature processing network to sequentially locate and classify the second feature to obtain the detection target, the consistency between image feature localization and classification of the image to be detected can be significantly improved, the detection error of the image to be detected can be reduced, and thus the detection accuracy of the image to be detected can be significantly improved.

[0080] See Figure 6 This application also provides an image detection apparatus for performing the image detection method described in the above embodiments. The apparatus includes:

[0081] The input module 201 is used to input the image to be detected into the image detection model, wherein the image detection model includes a feature extraction network, a feature fusion network, and a feature processing network;

[0082] Extraction module 202 is used to extract complementary features and redundant features of the image to be detected using the feature extraction network, and to concatenate the complementary features and redundant features to obtain a first feature which is then transmitted to the feature fusion network.

[0083] Feature fusion module 203 is used to fuse the first feature using the feature fusion network to obtain a second feature, which is then transmitted to the feature processing network.

[0084] The acquisition module 204 is used to sequentially locate and classify the second feature using the feature processing network to obtain the detection target.

[0085] The image detection device provided in this application embodiment is based on the same inventive concept as the image detection method provided in the above embodiments, and has the same beneficial effects as the methods used, operated or implemented therein.

[0086] This application also provides an electronic device corresponding to the image detection method provided in the foregoing embodiments. Please refer to... Figure 7 This illustrates a schematic diagram of an electronic device provided by some embodiments of this application. For example... Figure 7 As shown, the electronic device 30 may include: a processor 300, a memory 301, a bus 302, and a communication interface 303. The processor 300, the communication interface 303, and the memory 301 are connected via the bus 302. The memory 301 stores a computer program that can run on the processor 300. When the processor 300 runs the computer program, it executes the image detection method provided in any of the foregoing embodiments of this application.

[0087] The memory 301 may include high-speed random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Communication between this system network element and at least one other network element is achieved through at least one physical port 303 (which can be wired or wireless), such as the Internet, wide area network, local area network, or metropolitan area network.

[0088] Bus 302 can be an ISA bus, PCI bus, or EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. The memory 301 is used to store programs. After receiving an execution instruction, the processor 300 executes the program. The image detection method disclosed in any of the foregoing embodiments of this application can be applied to the processor 300, or implemented by the processor 300.

[0089] The processor 300 may be an integrated circuit with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the processor 300 or by instructions in software form. The processor 300 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules may reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 301. The processor 300 reads the information in memory 301 and, in conjunction with its hardware, completes the steps of the above method.

[0090] The electronic device provided in this application embodiment and the image detection method provided in this application embodiment are based on the same inventive concept and have the same beneficial effects as the methods they adopt, operate or implement.

[0091] This application also provides a computer-readable storage medium corresponding to the image detection method provided in the foregoing embodiments, which stores a computer program (i.e., a program product) thereon. When the computer program is run by a processor, it executes the image detection method provided in any of the foregoing embodiments.

[0092] It should be noted that examples of the computer-readable storage medium may also include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical and magnetic storage media, which will not be elaborated here.

[0093] This application also provides a computer program product corresponding to the image detection method provided in the foregoing embodiments, including a computer program that is executed by a processor to implement the image detection method provided in the above embodiments.

[0094] The computer-readable storage medium and computer program product provided in the above embodiments of this application are based on the same inventive concept as the image detection method provided in the embodiments of this application, and have the same beneficial effects as the methods adopted, run or implemented by the application programs stored therein.

[0095] It should be noted that:

[0096] The algorithms and displays provided herein are not inherently related to any particular computer, virtual device, or other equipment. Various general-purpose devices can also be used in conjunction with the teachings herein. The required structure for constructing such devices is apparent from the above description. Furthermore, this application is not directed to any particular programming language. It should be understood that the content of this application described herein can be implemented using various programming languages, and the above description of specific languages ​​is for the purpose of disclosing the best mode of implementation of this application.

[0097] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of this application may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0098] Similarly, it should be understood that, in order to simplify this application and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of this application, various features of this application are sometimes grouped together into a single embodiment, figure, or description thereof. However, this method of disclosure should not be construed as reflecting an intention that the claimed application requires more features than are expressly recited in each claim. Rather, as reflected in the following claims, inventive aspects lie in fewer than all features of a single foregoing disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of this application.

[0099] Those skilled in the art will understand that modules in the device of the embodiments can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiments can be combined into a single module, unit, or component, and further, they can be divided into multiple sub-modules, sub-units, or sub-components. Except where at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or device so disclosed. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0100] Furthermore, those skilled in the art will understand that although some embodiments described herein include certain features but not others included in other embodiments, combinations of features from different embodiments are intended to be within the scope of this application and form different embodiments. For example, in the following claims, any of the claimed embodiments can be used in any combination.

[0101] The various component embodiments of this application can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some or all of the components in the virtual machine creation apparatus according to embodiments of this application. This application can also be implemented as a device or apparatus program (e.g., a computer program and computer program product) for performing part or all of the methods described herein. Such an implementation of this application can be stored on a computer-readable medium, or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0102] It should be noted that the above embodiments are illustrative of this application and not restrictive, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. This application can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

[0103] The above description is merely a preferred embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An image detection method, characterized in that, include: The image to be detected is input into the image detection model, wherein the image detection model includes a feature extraction network, a feature fusion network, and a feature processing network; The complementary and redundant features of the image to be detected are extracted using the feature extraction network. The complementary and redundant features are then concatenated to obtain a first feature, which is then transmitted to the feature fusion network. The first feature is fused using the feature fusion network to obtain a second feature, which is then transmitted to the feature processing network. The feature processing network is used to locate and classify the second feature sequentially to obtain the detection target; The image to be detected includes a first spectral image and a second spectral image. The feature extraction network includes a first feature extraction module, a second feature extraction module, a first feature fusion module, and a feature extraction module. The feature extraction network is used to extract complementary and redundant features from the image to be detected. The complementary and redundant features are then concatenated to obtain a first feature, including: The first complementary feature and the first redundant feature of the first spectral image are extracted using the first feature extraction module, and the first complementary feature and the first redundant feature are transmitted to the first feature fusion module. The second feature extraction module is used to extract the second complementary feature and the second redundant feature of the second spectral image, and the second complementary feature and the second redundant feature are transmitted to the first feature fusion module. The first feature fusion module fuses the first complementary feature and the second complementary feature to obtain a third complementary feature. The first redundant feature and the second redundant feature are fused to obtain a third redundant feature. The third complementary feature and the third redundant feature are then spliced ​​together to obtain the spliced ​​features of the image to be detected, which are then transmitted to the feature extraction module. The feature extraction module is used to extract features from the spliced ​​features to obtain the first feature.

2. The image detection method according to claim 1, characterized in that, Before extracting the first complementary feature and the first redundant feature of the first spectral image using the first feature extraction module, the method further includes: Obtain the third and fourth spectral images of each of the multiple training images; The first feature extraction module to be trained is used to extract the features of the third spectral image of each image to be trained, so as to obtain the first feature to be trained. The second feature extraction module to be trained is used to extract the features of the fourth spectral image of each image to be trained, so as to obtain the second feature to be trained; The first feature to be trained and the second feature to be trained are each divided into two equal parts to obtain the first sub-feature and the second sub-feature corresponding to the first feature to be trained, and the third sub-feature and the fourth sub-feature corresponding to the second feature to be trained. Based on the first sub-feature and the third sub-feature, the first feature extraction module to be trained is trained using a spectrum image classification network to obtain a first classification loss value. Based on the first classification loss value satisfying a first preset convergence condition, the first feature extraction module is obtained. Based on the second sub-feature and the fourth sub-feature, the second feature extraction module to be trained is trained using a domain adversarial spectrum image classification network to obtain a second classification loss value. Based on the second classification loss value satisfying the second preset convergence condition, the second feature extraction module is obtained.

3. The image detection method according to claim 1, characterized in that, The step of fusing the first complementary feature and the second complementary feature using the first feature fusion module to obtain the third complementary feature includes: using the first feature fusion module to perform point-by-point addition and fusion processing on the first complementary feature and the second complementary feature to obtain the third complementary feature.

4. The image detection method according to claim 1, characterized in that, The step of fusing the first complementary feature and the second complementary feature using the first feature fusion module to obtain the third complementary feature includes: The first feature fusion module is used to perform channel fusion processing on the first complementary feature and the second complementary feature to obtain the third complementary feature.

5. The image detection method according to claim 1, characterized in that, The step of fusing the first redundant feature and the second redundant feature using the first feature fusion module to obtain the third redundant feature includes: The first feature fusion module is used to set a first preset weight for the first redundant feature and a second preset weight for the second redundant feature. The first redundant feature with the first preset weight and the second redundant feature with the second preset weight are then fused to obtain a third redundant feature.

6. The image detection method according to claim 1, characterized in that, The feature processing network includes a localization module and a classification module connected in series. The detected target includes the target location and the target category. The step of using the feature processing network to sequentially localize and classify the second feature to obtain the detected target includes: The localization module uses the second feature to predict the bounding box of the detected target, and then transmits the second feature and the bounding box to the classification module. The classification module extracts rotation-invariant features from the second feature based on the bounding box, performs convolution processing on the rotation-invariant features according to a preset convolution size, and performs classification processing on the convolution-processed rotation-invariant features to obtain the target position and target category.

7. An image detection device, characterized in that, The device includes: An input module is used to input the image to be detected into an image detection model, wherein the image detection model includes a feature extraction network, a feature fusion network, and a feature processing network; The extraction module is used to extract complementary and redundant features of the image to be detected using the feature extraction network, and to concatenate the complementary and redundant features to obtain a first feature which is then transmitted to the feature fusion network. A feature fusion module is used to fuse the first feature using the feature fusion network to obtain a second feature, which is then transmitted to the feature processing network. The acquisition module is used to sequentially locate and classify the second feature using the feature processing network to obtain the detection target; The image to be detected includes a first spectral image and a second spectral image. The feature extraction network includes a first feature extraction module, a second feature extraction module, a first feature fusion module, and a feature extraction module. The extraction module is further used for: The first complementary feature and the first redundant feature of the first spectral image are extracted using the first feature extraction module, and the first complementary feature and the first redundant feature are transmitted to the first feature fusion module. The second feature extraction module is used to extract the second complementary feature and the second redundant feature of the second spectral image, and the second complementary feature and the second redundant feature are transmitted to the first feature fusion module. The first feature fusion module fuses the first complementary feature and the second complementary feature to obtain a third complementary feature. The first redundant feature and the second redundant feature are fused to obtain a third redundant feature. The third complementary feature and the third redundant feature are then spliced ​​together to obtain the spliced ​​features of the image to be detected, which are then transmitted to the feature extraction module. The feature extraction module is used to extract features from the spliced ​​features to obtain the first feature.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method according to any one of claims 1-6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1-6.

Images