Image detection model, image detection method and object detection network training method

Abstract

The invention provides an image detection model, an image detection method and an object detection network training method. According to the image detection method provided by the embodiment of the invention, the original image is processed through the feature extraction model to obtain the first image feature; processing the original image through a first potential encoder to obtain a second image feature; the first feature image is processed through the feature extractor to obtain a third image feature, and the first feature image can be an original image, a depth image or a heat image, that is, the first feature image is a multi-modal image. Therefore, the salient object detection of the multi-modal image can be realized, and the accuracy of the image detection result is improved.

Description

Technical Field

[0001] The present application relates to the field of image detection technology, and in particular to an image detection model, an image detection method, and an object detection network training method. Background Art

[0002] In the field of computer vision, salient object detection (SOD) is an important research direction, which aims to identify and segment the most important or most prominent objects in an image. In the prior art, salient object detection is generally performed on an image in a single-modal manner, that is, object detection is performed based on the original image, or depth image, or thermal image, and then the object in the image is determined based on the detection result.

[0003] However, the above-mentioned image detection method can only predict single-modal images, and the definition of object edges is not clear enough, which reduces the accuracy of image detection results. Summary of the invention

[0004] The main purpose of this application is to provide an image detection model, an image detection method and an object detection network training method, aiming to solve the technical problem of low accuracy of image detection results.

[0005] To achieve the above-mentioned object, the present application provides an image detection model, including a feature extraction model, a first latent encoder, a second latent encoder, a feature extractor, an object detection network and a codec module; The output end of the feature extraction model is connected to the first input end of the encoding and decoding module; The output end of the first latent encoder is connected to the object detection network and the second input end of the encoding and decoding module respectively; The output end of the second latent encoder is connected to the object detection network and the second input end of the encoding and decoding module respectively; An output end of the feature extractor is connected to the object detection network; The object detection network is also connected to the third input terminal of the encoding and decoding module.

[0006] Optionally, the image detection model further includes a multi-scale feature fusion convolution layer; The input end of the multi-scale feature fusion convolution layer is connected to the output end of the feature extraction model, and the output end of the multi-scale feature fusion convolution layer is connected to the first input end of the encoding and decoding module.

[0007] Optionally, the image detection model further includes a first convolutional layer; An input end of the first convolutional layer is connected to an output end of the first latent encoder, and an output end of the first convolutional layer is connected to the object detection network and a second input end of the encoding and decoding module, respectively.

[0008] Optionally, the image detection model further includes a second convolutional layer and a diffusion model; The input end of the second convolutional layer is connected to the output end of the second latent encoder, the output end of the second convolutional layer is connected to the input end of the diffusion model, and the output end of the diffusion model is connected to the object detection network and the second input end of the encoding and decoding module respectively.

[0009] Optionally, the object detection network includes a first zero convolutional layer and a feature adaptation network; The output end of the feature extractor is connected to the input end of the first zero convolution layer, the output end of the first zero convolution layer is connected to the first input end of the feature adaptation network, and the second input end of the feature adaptation network is connected to the output end of the first latent encoder and the output end of the second latent encoder.

[0010] Optionally, the object detection network further comprises a first encoding module, a second zero convolutional layer and a first intermediate module; The first input end of the first encoding module is connected to the output end of the feature adaptive network, the second input end of the first encoding module is connected to the output end of the first latent encoder and the output end of the second latent encoder, the first output end of the first encoding module is connected to the input end of the second zero convolution layer, and the second output end of the first encoding module is also connected to the first intermediate module.

[0011] Optionally, the encoding and decoding module includes a second encoding module, a second intermediate module, a decoding module and a decoder; An input end of the second encoding module is connected to an output end of the first latent encoder and an output end of the second latent encoder, a first output end of the second encoding module is connected to the object detection network, and a second output end of the second encoding module is connected to a first input end of the second intermediate module; The second input terminal of the second intermediate module is connected to the output terminal of the first intermediate module, the third input terminal of the second intermediate module is connected to the output terminal of the feature extraction model, and the output terminal of the second intermediate module is connected to the first input terminal of the decoding module; The second input end of the decoding module is connected to the object detection network, and the output end of the decoding module is connected to the decoder.

[0012] Optionally, the first encoding module and the second encoding module each include one 64*64 convolution layer, one 32*32 convolution layer, one 16*16 convolution layer, and one 8*8 convolution layer; The first intermediate module and the second intermediate module each include a 4*4 convolutional layer; The decoding module includes one 64*64 convolution layer, one 32*32 convolution layer, one 16*16 convolution layer, and one 8*8 convolution layer.

[0013] To achieve the above object, the present application also provides an image detection method, which is applied to the image detection model as described above, wherein the image detection model includes a feature extraction model, a first latent encoder, a second latent encoder, a feature extractor, an object detection network, and a codec module; The method comprises: Processing the original image through a feature extraction model to obtain a first image feature; Processing the original image through a first latent encoder to obtain a second image feature; Processing the first feature image by a feature extractor to obtain a third image feature; the first feature image includes at least one of an original image, a first depth image, and a first thermal image, the first depth image is determined based on the depth information of the original image, and the first thermal image is determined based on the thermal information of the original image; The first image feature, the second image feature and the third image feature are input into a trained object detection network to generate a target image; the target image is used to characterize a salient object in the original image.

[0014] To achieve the above object, the present application also provides a training method for an object detection network, which is applied to the image detection model as described above, wherein the image detection model includes a feature extraction model, a first latent encoder, a second latent encoder, a feature extractor, an object detection network, and a codec module; The training method of the object detection network comprises: Acquire a training set; the training set includes a training image, an annotated image, and a second feature image, the second feature image includes at least one of a training image, a second depth image, and a second thermal image, the second depth image is determined based on depth information of the training image, the second thermal image is determined based on thermal information of the training image, and the annotated image is used to characterize a salient object in the training image; Processing the training image through the feature extraction model and the first latent encoder to obtain a fourth image feature and a fifth image feature respectively; Processing the second feature image by a feature extractor to obtain a sixth image feature; Processing the annotated image through a second latent encoder to obtain a seventh image feature; Inputting the fourth image feature, the fifth image feature, the sixth image feature, and the seventh image feature into a preset object detection network to generate a predicted image; Iteratively training the object detection network based on a loss value between the predicted image and the labeled image; When the loss value between the predicted image and the labeled image is less than or equal to a preset value, a trained object detection network is obtained.

[0015] Compared with the prior art, the embodiments of the present application have the following beneficial effects: The present application provides an image detection model, an image detection method and an object detection network training method, wherein the image detection method comprises: processing the original image through a feature extraction model to obtain a first image feature; processing the original image through a first latent encoder to obtain a second image feature; processing the first feature image through a feature extractor to obtain a third image feature; the first feature image comprises at least one of the original image, the first depth image and the first heat image, the first depth image is determined based on the depth information of the original image, and the first heat image is determined based on the heat information of the original image; the first image feature, the second image feature and the third image feature are input into the trained object detection network to generate a target image; the target image is used to characterize a significant object in the original image. In an embodiment of the present application, the original image is processed through a feature extraction model to obtain a first image feature; the original image is processed through a first latent encoder to obtain a second image feature; the first feature image is processed through a feature extractor to obtain a third image feature, wherein the first feature image can be an original image, a depth image or a heat image, that is, the first feature image is a multimodal image. In addition, significant object detection of multimodal images can be achieved, thereby improving the accuracy of image detection results. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0017] Figure 1 It is one of the structural schematic diagrams of the image detection model provided in the embodiment of the present application; Figure 2This is the second structural diagram of the image detection model provided in the embodiment of the present application; Figure 3 is a flow chart of the image detection method provided in an embodiment of the present application; Figure 4 It is a flowchart of the training method of the object detection network provided in the embodiment of the present application.

[0018] Description of reference numerals: 100, feature extraction model; 200, first latent encoder; 300, second latent encoder; 400, feature extractor; 500, object detection network; 600, encoding and decoding module; 700, multi-scale feature fusion convolution layer; 800, first convolution layer; 900, second convolution layer; 1000, diffusion model; 510, first zero convolution layer; 520, feature adaptation network; 530, first encoding module; 540, second zero convolution layer; 550, first intermediate module; 610, second encoding module; 620, second intermediate module; 630, decoding module; 640, decoder. DETAILED DESCRIPTION

[0019] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0020] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0021] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0022] See also Figure 1 , Figure 1 This is one of the structural diagrams of the image detection model provided in the embodiment of the present application. Figure 1As shown, the image detection model provided in the embodiment of the present application includes a feature extraction model 100, a first latent encoder 200, a second latent encoder 300, a feature extractor 400, an object detection network 500 and a codec module 600; The output end of the feature extraction model 100 is connected to the first input end of the encoding and decoding module 600; The output end of the first latent encoder 200 is connected to the object detection network 500 and the second input end of the encoding and decoding module 600 respectively; The output end of the second latent encoder 300 is connected to the object detection network 500 and the second input end of the encoding and decoding module 600 respectively; The output end of the feature extractor 400 is connected to the object detection network 500; The object detection network 500 is also connected to the third input terminal of the encoding and decoding module 600 .

[0023] The working principle of the image detection model provided in the embodiment of the present application during the model reasoning process is as follows: The original image is processed by the feature extraction model 100 to obtain a first image feature; the original image is processed by the first latent encoder 200 to obtain a second image feature; the first feature image is processed by the feature extractor 400 to obtain a third image feature; wherein the first feature image includes at least one of the original image, the first depth image and the first thermal image, the first depth image is determined based on the depth information of the original image, and the first thermal image is determined based on the thermal information of the original image; the first image feature, the second image feature and the third image feature are input into the trained object detection network 500 to generate a target image.

[0024] The working principle of the image detection model provided in the embodiment of the present application during the model training process is as follows: Acquire a training set; wherein the training set includes a training image, an annotated image, and a second feature image, the second feature image includes at least one of a training image, a second depth image, and a second thermal image, the second depth image is determined based on depth information of the training image, the second thermal image is determined based on thermal information of the training image, and the annotated image is used to characterize a salient object in the training image.

[0025] The training image is processed by the feature extraction model 100 and the first latent encoder 200 to obtain a fourth image feature and a fifth image feature, respectively.

[0026] The second feature image is processed by the feature extractor 400 to obtain a sixth image feature.

[0027] The annotated image is processed by the second latent encoder 300 to obtain a seventh image feature.

[0028] The fourth image feature, the fifth image feature, the sixth image feature, and the seventh image feature are input into a preset object detection network 500 to generate a predicted image.

[0029] The object detection network 500 is iteratively trained based on the loss value between the predicted image and the labeled image.

[0030] When the loss value between the predicted image and the labeled image is less than or equal to a preset value, a trained object detection network 500 is obtained.

[0031] During the application process of the image detection model provided in the embodiment of the present application, the first feature image is processed by the feature extractor 400, wherein the first feature image is a multimodal image; during the training process, the second feature image is processed by the feature extractor 400, wherein the first feature image is a multimodal image; and thus, salient object detection of the multimodal image can be achieved, thereby improving the accuracy of the image detection results.

[0032] Optionally, the image detection model further includes a multi-scale feature fusion convolution layer 700; The input end of the multi-scale feature fusion convolution layer 700 is connected to the output end of the feature extraction model 100 , and the output end of the multi-scale feature fusion convolution layer 700 is connected to the first input end of the encoding and decoding module 600 .

[0033] See also Figure 2 , the image detection model also includes a multi-scale feature fusion convolution layer 700.

[0034] During the inference process and training process of the image detection model, the feature extraction model 100 extracts the image features of the training image, and processes the image features of the training image through the multi-scale feature fusion convolution layer 700. In this way, the features of the training image are retained during the training process, and the accuracy of the predicted image is improved.

[0035] Optionally, the image detection model further includes a first convolutional layer 800; The input end of the first convolutional layer 800 is connected to the output end of the first latent encoder 200, and the output end of the first convolutional layer 800 is connected to the object detection network 500 and the second input end of the encoding and decoding module 600 respectively.

[0036] Optionally, the image detection model further includes a second convolutional layer 900 and a diffusion model 1000; The input end of the second convolutional layer 900 is connected to the output end of the second latent encoder 300, the output end of the second convolutional layer 900 is connected to the input end of the diffusion model 1000, and the output end of the diffusion model 1000 is connected to the object detection network 500 and the second input end of the encoding and decoding module 600 respectively.

[0037] See also Figure 2 , see Figure 2 , the image detection model also includes a first convolutional layer 800, a second convolutional layer 900 and a diffusion model 1000.

[0038] During the inference and training process of the image detection model, the training image is processed by the first latent encoder 200 to obtain a coding feature map; and then the coding feature map is processed by the first convolutional layer 800.

[0039] During the training process of the image detection model, the annotated image is processed by the second latent encoder 300 to obtain a coding feature map, and then the coding feature map is processed by the second convolutional layer 900 and the diffusion model 1000. In this embodiment, the second coding feature map is processed by the diffusion model 1000 to gradually convert the data into noise, and then a reverse process is learned to gradually restore the original data from the noise, and finally generate high-quality image features.

[0040] Optionally, the object detection network 500 includes a first zero convolution layer 510 and a feature adaptation network 520; The output end of the feature extractor 400 is connected to the input end of the first zero convolution layer 510, the output end of the first zero convolution layer 510 is connected to the first input end of the feature adaptation network 520, and the second input end of the feature adaptation network 520 is connected to the output end of the first latent encoder 200 and the output end of the second latent encoder 300.

[0041] See also Figure 2 , the object detection network 500 also includes a first zero convolution layer 510 and a feature adaptation network 520.

[0042] During the inference process and training process of the image detection model, the image features are convolved through the first zero convolution layer 510 in the object detection network 500, and the convolved image features are input into the feature adaptation network 520 in the object detection network 500.

[0043] In this embodiment, a feature adaptation network 520 is set in the object detection network 500, and convolution processing is performed on the image features through the feature adaptation network 520, so that the output of the feature adaptation network 520 meets the requirements of the object detection network 500, thereby avoiding the loss of image information and improving the accuracy of the predicted image.

[0044] Optionally, the object detection network 500 further includes a first encoding module 530, a second zero convolution layer 540 and a first intermediate module 550; The first input end of the first encoding module 530 is connected to the output end of the feature adaptation network 520, the second input end of the first encoding module 530 is connected to the output end of the first latent encoder 200 and the output end of the second latent encoder 300, the first output end of the first encoding module 530 is connected to the input end of the second zero convolution layer 540, and the second output end of the first encoding module 530 is also connected to the first intermediate module 550.

[0045] Optionally, the encoding and decoding module 600 includes a second encoding module 610, a second intermediate module 620, a decoding module 630 and a decoder 640; An input end of the second encoding module 610 is connected to an output end of the first latent encoder 200 and an output end of the second latent encoder 300, a first output end of the second encoding module 610 is connected to the object detection network 500, and a second output end of the second encoding module 610 is connected to a first input end of the second intermediate module 620; The second input terminal of the second intermediate module 620 is connected to the output terminal of the first intermediate module 550, the third input terminal of the second intermediate module 620 is connected to the output terminal of the feature extraction model 100, and the output terminal of the second intermediate module 620 is connected to the first input terminal of the decoding module 630; A second input terminal of the decoding module 630 is connected to the object detection network 500 , and an output terminal of the decoding module 630 is connected to the decoder 640 .

[0046] See also Figure 2 , the object detection network 500 further includes a first encoding module 530, a second zero convolution layer 540 and a first intermediate module 550. Optionally, the first encoding module 530 includes one 64*64 convolution layer, one 32*32 convolution layer, one 16*16 convolution layer, and one 8*8 convolution layer; the first intermediate module 550 includes one 4*4 convolution layer.

[0047] See also Figure 2, the encoding and decoding module 600 includes a second encoding module 610, a second intermediate module 620, a decoding module 630 and a decoder 640. Optionally, the second encoding module 610 includes one 64*64 convolution layer, one 32*32 convolution layer, one 16*16 convolution layer, and one 8*8 convolution layer; the second intermediate module 620 includes one 4*4 convolution layer; the decoding module 630 includes one 64*64 convolution layer, one 32*32 convolution layer, one 16*16 convolution layer, and one 8*8 convolution layer.

[0048] During the training process of the image detection model, the image features output by the first convolutional layer 800 are fused with the image features output by the diffusion model 1000 to obtain fused image features, and the fused image features are respectively input into the feature adaptation network 520, the first encoding module 530 and the second encoding module 610; the image features output by the multi-scale feature fusion convolutional layer 700 are input into the second intermediate module 620.

[0049] The image features output by the feature adaptation network 520 are input to the first encoding module 530, and then the image features output by the first encoding module 530 are input to the second zero convolution layer 540. The image features output by the second zero convolution layer 540 and the image features output by the second encoding module 610 are merged and concatenated to obtain the output of the object detection network 500.

[0050] The decoding module 630 receives the output of the object detection network 500 , the output of the second intermediate module 620 , and the output of the second encoding module 610 , and obtains a predicted image through a decoder 640 .

[0051] During the inference process of the image detection model, the image features output by the first convolutional layer 800 are respectively input into the feature adaptation network 520, the first encoding module 530 and the second encoding module 610; the image features output by the multi-scale feature fusion convolutional layer 700 are input into the second intermediate module 620.

[0052] The image features output by the feature adaptation network 520 are input to the first encoding module 530, and then the image features output by the first encoding module 530 are input to the second zero convolution layer 540. The image features output by the second zero convolution layer 540 and the image features output by the second encoding module 610 are merged and concatenated to obtain the output of the object detection network 500.

[0053] The decoding module 630 receives the output of the object detection network 500 , the output of the second intermediate module 620 , and the output of the second encoding module 610 , and obtains a predicted image through a decoder 640 .

[0054] Please refer to Figure 3, shows a flow chart of an embodiment of the image detection method proposed in the present application. The embodiment of the present application can acquire and process relevant data based on artificial intelligence technology.

[0055] The image detection method proposed in the present application is applied to the image detection model as described above, wherein the image detection model includes a feature extraction model, a first latent encoder, a second latent encoder, a feature extractor, an object detection network, and an encoding and decoding module; The image detection method comprises: S310, processing the original image through a feature extraction model to obtain a first image feature.

[0056] Optionally, the feature extraction model is a ViT (Vision Transformer) model. It should be understood that the original image may be an RGB image.

[0057] It should be understood that the above-mentioned image detection model includes a feature extraction model. In this step, the image detection model obtains the original image and processes the original image through the feature extraction model to obtain the first image feature.

[0058] S320: Process the original image through a first latent encoder to obtain a second image feature.

[0059] It should be understood that the above-mentioned image detection model includes a first latent encoder.

[0060] In this step, the image detection model obtains the original image and processes the original image through the first latent encoder to obtain the second image feature.

[0061] S330, processing the first feature image by a feature extractor to obtain a third image feature.

[0062] It should be understood that the above-mentioned image detection model includes a feature extractor. The above-mentioned first feature image includes at least one of the original image, the first depth image and the first heat image, the first depth image is determined based on the depth information of the original image, and the first heat image is determined based on the heat information of the original image; the depth image can be understood as the depth map corresponding to the original image, and the heat map can be understood as the heat map corresponding to the original image.

[0063] In this step, the feature extractor may process the first feature image to obtain a third image feature.

[0064] S340: Input the first image feature, the second image feature, and the third image feature into a trained object detection network to generate a target image.

[0065] It should be understood that the above-mentioned image detection model includes an object detection network. Optionally, the above-mentioned object detection network is a SOD control network (ControlNet).

[0066] In this step, after obtaining the first image feature, the second image feature and the third image feature, the first image feature, the second image feature and the third image feature are input into the trained object detection network to generate a target image, wherein the target image is used to characterize the salient object in the original image.

[0067] The image detection method provided in the embodiment of the present application processes the original image through a feature extraction model to obtain a first image feature; processes the original image through a first latent encoder to obtain a second image feature; and processes the first feature image through a feature extractor to obtain a third image feature, wherein the first feature image can be an original image, a depth image, or a thermal image, that is, the first feature image is a multimodal image. In addition, salient object detection of multimodal images can be achieved, thereby improving the accuracy of image detection results.

[0068] Please refer to Figure 4 , shows a flow chart of an embodiment of the object detection network training method proposed in the present application. The embodiment of the present application can acquire and process relevant data based on artificial intelligence technology.

[0069] The object detection network training method proposed in the present application is applied to the image detection model as described above, wherein the image detection model includes a feature extraction model, a first latent encoder, a second latent encoder, a feature extractor, an object detection network, and an encoding and decoding module; The object detection network training method comprises: S410, obtaining a training set.

[0070] It should be noted that the above-mentioned training set includes training images, annotated images and second feature images. The second feature images include at least one of the training images, second depth images and second thermal images. The second depth image is determined based on the depth information of the training images, and the second thermal image is determined based on the thermal information of the training images. The annotated images are the original images after manual annotation, which are used to characterize the salient objects in the training images.

[0071] S420, processing the training image through the feature extraction model and the first latent encoder to obtain a fourth image feature and a fifth image feature respectively.

[0072] As described above, the image detection model includes a feature extraction model and a first latent encoder.

[0073] In this step, the training image is processed by the feature extraction model to obtain the fourth image feature; and the training image is processed by the first latent encoder to obtain the fifth image feature.

[0074] S430: Process the second feature image through a feature extractor to obtain a sixth image feature.

[0075] In this step, the second feature image may be processed by a feature extractor to obtain a sixth image feature, that is, the feature extractor processes at least one of the second feature image, the second thermal image, and the second depth image to obtain the sixth image feature.

[0076] S440: Process the labeled image through a second latent encoder to obtain a seventh image feature.

[0077] It should be understood that the image detection model also includes a second latent encoder.

[0078] In this step, the annotated image is processed by the second latent encoder to obtain the seventh image feature.

[0079] S450, inputting the fourth image feature, the fifth image feature, the sixth image feature, and the seventh image feature into a preset object detection network to generate a predicted image.

[0080] In this step, after obtaining the fourth image feature, the fifth image feature, the sixth image feature, and the seventh image feature, the fourth image feature, the fifth image feature, the sixth image feature, and the seventh image feature are input into the object detection network to obtain a predicted image.

[0081] S460: Iteratively train the object detection network based on the loss value between the predicted image and the labeled image.

[0082] S470, when the loss value between the predicted image and the labeled image is less than or equal to a preset value, obtaining a trained object detection network.

[0083] Furthermore, a loss value between the predicted image and the annotated image is calculated. The loss value is used to characterize the difference between the predicted image and the annotated image. When the loss value is less than or equal to a preset value, it indicates that the object detection network training is completed, and a trained object detection network is obtained.

[0084] In the embodiment of the present application, in the training of the object detection network, the second feature image is processed by the feature extractor, wherein the second feature image can be a training image, a depth image or a thermal image, that is, the second feature image is a multimodal image. Then, the multimodal image is introduced as a training set in the training of the object detection network to improve the accuracy of the image detection result.

[0085] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware online platform, and of course by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, disk, CD), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0086] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0087] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. An image detection model, characterized in that: including a feature extraction model, a first latent encoder, a second latent encoder, a feature extractor, an object detection network, and an encoding and decoding module; The output end of the feature extraction model is connected to the first input end of the encoding and decoding module; The output end of the first latent encoder is connected to the object detection network and the second input end of the encoding and decoding module respectively; The output end of the second latent encoder is connected to the object detection network and the second input end of the encoding and decoding module respectively; An output end of the feature extractor is connected to the object detection network; The object detection network is also connected to the third input terminal of the encoding and decoding module.

2. The image detection model according to claim 1, characterized in that: The image detection model also includes a multi-scale feature fusion convolution layer; The input end of the multi-scale feature fusion convolution layer is connected to the output end of the feature extraction model, and the output end of the multi-scale feature fusion convolution layer is connected to the first input end of the encoding and decoding module.

3. The image detection model according to claim 1, characterized in that: The image detection model also includes a first convolutional layer; An input end of the first convolutional layer is connected to an output end of the first latent encoder, and an output end of the first convolutional layer is connected to the object detection network and a second input end of the encoding and decoding module, respectively.

4. The image detection model according to claim 1, characterized in that: The image detection model also includes a second convolutional layer and a diffusion model; The input end of the second convolutional layer is connected to the output end of the second latent encoder, the output end of the second convolutional layer is connected to the input end of the diffusion model, and the output end of the diffusion model is connected to the object detection network and the second input end of the encoding and decoding module respectively.

5. The image detection model according to claim 1, characterized in that: The object detection network includes a first zero convolutional layer and a feature adaptation network; The output end of the feature extractor is connected to the input end of the first zero convolution layer, the output end of the first zero convolution layer is connected to the first input end of the feature adaptation network, and the second input end of the feature adaptation network is connected to the output end of the first latent encoder and the output end of the second latent encoder.

6. The image detection model according to claim 5, characterized in that: The object detection network also includes a first encoding module, a second zero convolutional layer, and a first intermediate module; The first input end of the first encoding module is connected to the output end of the feature adaptive network, the second input end of the first encoding module is connected to the output end of the first latent encoder and the output end of the second latent encoder, the first output end of the first encoding module is connected to the input end of the second zero convolution layer, and the second output end of the first encoding module is also connected to the first intermediate module.

7. The image detection model according to claim 6, characterized in that: The encoding and decoding module includes a second encoding module, a second intermediate module, a decoding module and a decoder; An input end of the second encoding module is connected to an output end of the first latent encoder and an output end of the second latent encoder, a first output end of the second encoding module is connected to the object detection network, and a second output end of the second encoding module is connected to a first input end of the second intermediate module; The second input terminal of the second intermediate module is connected to the output terminal of the first intermediate module, the third input terminal of the second intermediate module is connected to the output terminal of the feature extraction model, and the output terminal of the second intermediate module is connected to the first input terminal of the decoding module; The second input end of the decoding module is connected to the object detection network, and the output end of the decoding module is connected to the decoder.

8. The image detection model according to claim 7, characterized in that: The first encoding module and the second encoding module each include one 64*64 convolution layer, one 32*32 convolution layer, one 16*16 convolution layer, and one 8*8 convolution layer; The first intermediate module and the second intermediate module each include a 4*4 convolutional layer; The decoding module includes one 64*64 convolution layer, one 32*32 convolution layer, one 16*16 convolution layer, and one 8*8 convolution layer.

9. An image detection method, characterized in that: An image detection model as described in any one of claims 1 to 8, wherein the image detection model comprises a feature extraction model, a first latent encoder, a second latent encoder, a feature extractor, an object detection network, and an encoding and decoding module; The method comprises: Processing the original image through a feature extraction model to obtain a first image feature; Processing the original image through a first latent encoder to obtain a second image feature; Processing the first feature image by a feature extractor to obtain a third image feature; the first feature image includes at least one of an original image, a first depth image, and a first thermal image, the first depth image is determined based on the depth information of the original image, and the first thermal image is determined based on the thermal information of the original image; The first image feature, the second image feature and the third image feature are input into a trained object detection network to generate a target image; the target image is used to characterize a salient object in the original image.

10. A method for training an object detection network, characterized in that: An image detection model as described in any one of claims 1 to 8, wherein the image detection model comprises a feature extraction model, a first latent encoder, a second latent encoder, a feature extractor, an object detection network, and an encoding and decoding module; The method comprises: Acquire a training set; the training set includes a training image, an annotated image, and a second feature image, the second feature image includes at least one of a training image, a second depth image, and a second thermal image, the second depth image is determined based on depth information of the training image, the second thermal image is determined based on thermal information of the training image, and the annotated image is used to characterize a salient object in the training image; Processing the training image through the feature extraction model and the first latent encoder to obtain a fourth image feature and a fifth image feature respectively; Processing the second feature image by a feature extractor to obtain a sixth image feature; Processing the annotated image through a second latent encoder to obtain a seventh image feature; Inputting the fourth image feature, the fifth image feature, the sixth image feature, and the seventh image feature into a preset object detection network to generate a predicted image; Iteratively training the object detection network based on a loss value between the predicted image and the labeled image; When the loss value between the predicted image and the labeled image is less than or equal to a preset value, a trained object detection network is obtained.

Images