Image fusion detection method and device based on original output of image detector

Abstract

The present disclosure relates to the field of visual detection and image processing, and provides an image fusion detection method and device based on the original output of an image detector. The image fusion detection method includes: obtaining detection results obtained by detecting images of multiple sources of the same scene by multiple image detectors respectively; pairing the detection results of different image detectors to obtain paired detection results; based on the original output, fusing the paired detection results to obtain fused detection results. The present disclosure can solve the problem that the detection results after fusion may not be accurate enough. By using the original output as the input of the fusion process, it is conducive to designing a better fusion method and helping to obtain more accurate fusion classification results.

Description

Technical Field

[0001] The present disclosure relates to the field of visual detection and image processing, and in particular to an image fusion detection method and device based on the original output of an image detector. Background Art

[0002] With the development of image processing technology and the widespread application of image recognition technology, the accuracy requirements for image recognition are becoming higher and higher. Using multi-source image fusion technology for target detection can make full use of the complementary and redundant information of multi-source images, which helps to improve the target detection performance.

[0003] In the image fusion methods of the related art, when determining the category of the final detection result after fusion, most of them directly use the detection result categories of different detectors and the final detection scores for fusion. This makes the fused detection result completely dependent on the detection result category and the final detection score, and it is impossible to obtain more information of each detector in the detection process, which may make it difficult to further improve the accuracy of the fused detection result. Summary of the invention

[0004] The present disclosure provides an image fusion detection method and device based on the original output of an image detector, so as to at least solve the problem in the related art that it is difficult to obtain more detection information when the image detection results are fused, which makes it difficult to improve the accuracy of the fused detection results. The technical solution of the present disclosure is as follows:

[0005] According to a first aspect of the present disclosure, there is provided an image fusion detection method based on the original output of an image detector, the image fusion detection method comprising: obtaining detection results obtained by respectively detecting images of multiple signal sources of the same scene using multiple image detectors, wherein the signal sources correspond one-to-one to the image detectors, each detection result represents a target object detected in the image of the corresponding signal source, and the detection result comprises the original output of the corresponding image detector; pairing the detection results of different image detectors to obtain paired detection results; and based on the original output, fusing the paired detection results to obtain a fused detection result.

[0006] Optionally, the detection results of each image detector form a detection result set, and for any two image detectors among the multiple image detectors, the detection results of different image detectors are paired in the following manner: for each first detection result in the first detection result set of the first image detector among the two image detectors, the following operations are performed in sequence: in response to the presence of a candidate second detection result whose intersection-and-union ratio with the current first detection result is greater than or equal to a preset intersection-and-union ratio threshold in the current second detection result set of the second image detector among the two image detectors, the current first detection result is paired with a target second detection result among the candidate second detection results whose intersection-and-union ratio with the current first detection result is the largest; in response to the absence of the candidate second detection result in the current second detection result set, the current first detection result is paired with a null detection result.

[0007] Optionally, the image fusion detection method further includes: in response to pairing the current first detection result with the target second detection result, deleting other candidate second detection results except the target second detection result from the current second detection result set.

[0008] Optionally, the original output includes a classification score vector, and the paired detection results include a first pairing result and a second pairing result, wherein the step of fusing the paired detection results to obtain a fused detection result includes: determining a fused classification score vector by comparing the maximum classification score in a first classification score vector of the first pairing result with the maximum classification score in a second classification score vector of the second pairing result; and determining the fused detection result based on the fused classification score vector.

[0009] Optionally, the original output also includes confidence, wherein the fused classification score vector is determined in the following manner: in response to the difference between the maximum classification score in the first classification score vector of the first pairing result and the maximum classification score in the second classification score vector of the second pairing result being greater than a preset score threshold, the classification score vector with the larger maximum classification score between the first classification score vector and the second classification score vector is used as the fused classification score vector; in response to the difference between the maximum classification score in the first classification score vector and the maximum classification score in the second classification score vector is less than or equal to the preset score threshold, the classification score vector of the pairing result with higher confidence between the first pairing result and the second pairing result is used as the fused classification score vector.

[0010] Optionally, the fused detection result is determined in the following manner: the confidence of the first pairing result and the confidence of the second pairing result are fused to obtain a fused confidence; and the fused detection result is determined based on the fused confidence and the fused classification score vector.

[0011] Optionally, the fused detection result is determined in the following manner: in response to the maximum classification score in the fused classification score vector being greater than a preset score threshold, the classification of the fused detection result is determined to be a target in the image; in response to the maximum classification score in the fused classification score vector being less than or equal to the preset score threshold, the classification of the fused detection result is determined to be the background in the image.

[0012] Optionally, the original output also includes a regression branch output vector, and the fused detection result is determined in the following manner: based on the first regression branch output vector of the first pairing result and the second regression branch output vector of the second pairing result, determining the two larger values ​​in the sub-distribution of each regression branch output vector of the first regression branch output vector and the second regression branch output vector; based on the distance between the two values ​​and the sum of the two values, determining a first statistic of the first regression branch output vector and a second statistic of the second regression branch output vector, respectively; in response to the first statistic and the second statistic being both less than a preset statistic threshold, determining that the fused detection result is classified as the background in the image; in response to at least one of the first statistic and the second statistic being greater than or equal to the preset statistic threshold, determining that the fused detection result is classified as a target in the image.

[0013] Optionally, the image fusion detection method also includes: in response to the classification of the fused detection result as a target in the image, taking the category corresponding to the maximum element in the fused classification score vector as the category of the target, and determining the target detection position of the fused detection result by fusing the target detection position in the first pairing result and the target detection position in the second pairing result.

[0014] According to a second aspect of the present disclosure, there is provided an image fusion detection device based on the original output of an image detector, the image fusion detection device comprising: an acquisition unit, configured to acquire detection results obtained by respectively detecting images of multiple signal sources of the same scene by multiple image detectors, wherein the signal sources correspond one-to-one to the image detectors, each detection result represents a target object detected in the image of the corresponding signal source, and the detection result comprises the original output of the corresponding image detector; a pairing unit, configured to pair the detection results of different image detectors to obtain paired detection results; and a fusion unit, configured to fuse the paired detection results based on the original output to obtain a fused detection result.

[0015] According to a third aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing instructions executable by the processor, wherein the instructions executable by the processor, when executed by the processor, prompt the processor to execute the image fusion detection method based on the original output of the image detector according to an embodiment of the present disclosure.

[0016] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided. When instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the image fusion detection method based on the original output of the image detector according to an embodiment of the present disclosure.

[0017] According to a fifth aspect of the present disclosure, a computer program product is provided, which includes computer executable instructions, and when the computer executable instructions are executed by at least one processor, the image fusion detection method based on the original output of the image detector according to the embodiment of the present disclosure is implemented.

[0018] The technical solution provided by the present disclosure brings at least the following beneficial effects:

[0019] By adopting the present disclosure, detection results obtained by respectively detecting images of multiple sources of the same scene by multiple image detectors can be obtained. The detection results include the original outputs of the detectors. The detection results of different image detectors can be paired, and based on the original outputs, the paired detection results can be fused to obtain fused detection results. In this way, since the original outputs of the detectors contain more information, using the original outputs as the input of the fusion process is conducive to designing better fusion methods and helping to obtain more accurate fusion classification results.

[0020] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0022] Figure 1 is a schematic flowchart of an image fusion detection method according to an exemplary embodiment of the present disclosure.

[0023] Figure 2 is a schematic flowchart of an image fusion detection method according to an exemplary embodiment of the present disclosure.

[0024] Figure 3 It is a schematic diagram of an example process of obtaining paired detection results in the image fusion detection method according to an exemplary embodiment of the present disclosure.

[0025] Figure 4 It is a schematic block diagram of an example of the step of obtaining a fused detection result in the image fusion detection method according to an exemplary embodiment of the present disclosure.

[0026] Figure 5 It is a schematic block diagram of another example of the step of obtaining a fused detection result in the image fusion detection method according to an exemplary embodiment of the present disclosure.

[0027] Figure 6 It is a schematic block diagram of another example of the step of obtaining a fused detection result in the image fusion detection method according to an exemplary embodiment of the present disclosure.

[0028] Figure 7 It is a schematic diagram of an example process of obtaining a fused detection result in an image fusion detection method according to an exemplary embodiment of the present disclosure.

[0029] Figure 8 It is a schematic diagram of another example process of obtaining a fused detection result in the image fusion detection method according to an exemplary embodiment of the present disclosure.

[0030] Fig. 9 is a schematic block diagram of an image fusion detection device according to an exemplary embodiment of the present disclosure.

[0031] Fig.10 is a block diagram of an electronic device according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0032] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the present application will clearly and completely describe the technical solutions in the embodiments of the present disclosure in conjunction with the accompanying drawings.

[0033] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0034] It should be noted that the phrase "at least one of the items" in the present disclosure includes three types of parallel situations: "any one of the items", "a combination of any number of the items", and "all of the items". For example, "including at least one of A and B" includes the following three types of parallel situations: (1) including A; (2) including B; (3) including A and B. Another example is "executing at least one of step 1 and step 2" which means the following three types of parallel situations: (1) executing step 1; (2) executing step 2; (3) executing step 1 and step 2.

[0035] As mentioned above, in the related art, the fused detection results are completely dependent on the detection result category and the final detection score, and it is impossible to obtain more detection information of each detector, making it difficult to further improve the accuracy of the fused detection results.

[0036] In addition, multi-source image fusion technology can generally be divided into pixel-level, feature-level and decision-level fusion methods. In some special application scenarios such as distributed earth observation systems, the edge computing capabilities of sensor platforms such as intelligent drones and intelligent satellites are relatively limited, and the real-time requirements for information processing are high. Therefore, the pixel-level and feature-level fusion methods that require multi-source images to be transmitted to the fusion node for processing cannot meet the requirements.

[0037] For such application scenarios with limited computing power, deploying lightweight target detectors on multiple sensor platforms, performing preliminary detection on each platform, and finally sending the detection results with very small data scale to the fusion node for fusion analysis using the decision-level fusion method is a feasible solution for performing multi-source image fusion detection tasks. Therefore, how to achieve better fusion of the detection results of lightweight detectors to obtain more accurate image detection results is also one of the important issues that need to be paid attention to.

[0038] In view of the above problems, exemplary embodiments of the present disclosure propose an image fusion detection method based on the original output of an image detector, an image fusion detection device, an electronic device, a computer-readable storage medium, and a computer program product, which can solve or at least alleviate the above problems.

[0039] In a first aspect of an exemplary embodiment of the present disclosure, an image fusion detection method based on raw output of an image detector is provided.

[0040] The image fusion detection method according to the exemplary embodiment of the present disclosure can be applied to scenarios where users interact with software. For example, the software can be loaded on the user terminal, and the user can input image detection instructions on the user terminal, such as instructions for fusing the detection results of multiple image detectors to obtain fused detection results. The user terminal can obtain the final fusion result by executing the image fusion detection method according to the exemplary embodiment of the present disclosure.

[0041] Specifically, the user terminal can obtain the detection results obtained by multiple image detectors respectively detecting images of multiple sources for the same scene, wherein the sources correspond to the image detectors one by one, each detection result represents a target object detected in the image of the corresponding source, and the detection result includes the original output of the corresponding image detector. The user terminal can also pair the detection results of different image detectors to obtain paired detection results, and based on the original output, the paired detection results can be fused to obtain a fused detection result.

[0042] The above-mentioned user terminal can be, for example, a tablet computer, a laptop computer, a digital assistant, a wearable device, etc. However, the implementation scenario of the above method is only an example scenario. The image fusion detection method according to the exemplary embodiment of the present disclosure can also be applied to other application scenarios. For example, the user can request image detection from the server through the network on the user terminal (for example, a mobile phone, a desktop computer, a tablet computer, etc.), for example, the detection results of multiple image detectors are fused. The server can complete the request by executing the image fusion detection method according to the exemplary embodiment of the present disclosure. Here, the server can be an independent server, a server cluster, a cloud computing platform or a virtualization center.

[0043] According to the image fusion detection method of the exemplary embodiment of the present disclosure, more information of each detector in the detection process can be obtained by using the original output of the detector, which is conducive to better implementation of fusion and obtaining more accurate fusion classification results.

[0044] The following will refer to Figure 1 and Figure 2 An example of an image fusion detection method according to an embodiment of the present disclosure is described. Figure 1 As shown, the image fusion detection method may include the following steps:

[0045] In step S110, detection results obtained by respectively detecting images of multiple sources of the same scene using multiple image detectors may be obtained.

[0046] Here, the information source may correspond to the image detector one by one, each detection result may represent an object detected in the image of the corresponding information source, and the detection result may include the original output of the corresponding image detector. Here, the object may generally include a foreground target, but is not limited to a foreground target, and may also include a background, for example.

[0047] The image fusion detection method according to the embodiment of the present disclosure can be used to detect targets in images. The image detector can recognize the target or background and other contents in the input image, so that the target in the image can be determined from the recognized contents.

[0048] Specifically, multiple pre-trained image detectors can be used to detect multi-source image data of the same scene to identify objects, backgrounds, and other contents in the image. These image detectors have the same original output composition. For example, when an image is input into these image detectors, corresponding detection results can be obtained respectively, and these detection results have the same original output composition.

[0049] Multi-source image data from different sources can be obtained, for example, through different image capture methods. For example, for the same object, images of the object can be obtained using methods such as infrared cameras, natural light cameras, 3D scanning, microwave detection, video capture, etc.

[0050] In practical applications, taking the field of geological exploration as an example, multi-source image data may include, for example, optical remote sensing images, thermal infrared remote sensing images, radar remote sensing images, etc. taken at the same geographical location; taking the medical field as an example, multi-source image data may include, for example, positron emission tomography images and computer tomography images taken at the same inspection object.

[0051] The pre-trained multiple image detectors described above can be trained based on image data of multiple sources of the same training image sample. For example, each image sample can include image data of different sources, and an image detector can be trained for each source, so that multiple image detectors can be obtained.

[0052] The raw output of the image detector may refer to the preliminary result generated after the image is processed by the detector in the detection task. As an example, the raw output may include one or more of confidence, classification score, and regression branch output.

[0053] Here, we take a lightweight detector such as the Yolo series detector as an example. The most commonly used lightweight target detector is the Yolo series detector, which is a single-stage detector consisting of a feature extraction network and a detection head. With the development of the Yolo series detector, the structure of the detection head has been continuously improved, and the composition of the original output has also changed. From the Yolov3 detector to the Yolov7 detector, the original output of the detection head includes confidence, classification score, and regression branch output. The detection frame coordinates can be obtained by using the regression branch output, and the detection result category and the final detection score can be obtained by using the confidence and classification score. The original output of the detection head of the Yolov8 detector and the Yolov9 detector includes the classification score and the regression branch output, but does not include confidence. The detection frame coordinates can be obtained by using the regression branch output, and the detection result category and the final detection score can be obtained by using the classification score.

[0054] The raw output of the Yolov3 detector can include confidence c , classification score and the regression branch output u Confidence c is a scalar value between 0 and 1. Classification score is a vector, assuming the detector can detect C Class target, then yes C dimensional vector, each dimension takes a value between 0 and 1. According to the confidence c and classification scores The types of test results that can be obtained y The final test score s ,in, y is an integer ranging from 0 to C- The value of 1, s A scalar with values ​​between 0 and 1. Regression branch output u Used to get the detection box coordinates of the detection results b Commonly used Yolo series detectors such as Yolov5 detector, Yolor detector and Yolov7 detector are all detectors composed of the original output of the detection head of Yolov3 detector.

[0055] The raw output of the Yolov8 detector can include classification scores and the regression branch output , where the classification score The features of are the same as the classification scores in the original output of the Yolov3 detector, which can be used to obtain the category of the detection result and the final detection score. can be decomposed into four vectors , , and , the four vectors are L dimension, L The value of can be set by yourself or take a common value. Based on these four vectors, the detection box coordinates of the detection result can be obtained b The raw output composition of the Yolov9 detector is the same as that of the Yolov8 detector.

[0056] In this step, the two trained image detectors can be used to detect the multi-source image pair consisting of two paired multi-source images to obtain two sets of detection results. Since an image can contain one or more objects, Figure 2 As shown, the paired images may include paired multi-source image 1 and paired multi-source image 2, which may be respectively input into two image detectors such as lightweight object detector 1 and lightweight object detector 2, thereby obtaining detection result set 1 and detection result set 2. Here, the detection result set may include one or more detection results. In the case of using the original output as the input of the decision-level fusion algorithm, the original output of each detection result may be retained.

[0057] As an example, the detection process of the detector may include a preprocessing process of converting an input image into a tensor, a forward calculation process of inputting the tensor into a neural network to obtain an original output, a decoding process of obtaining a detection result category, a final detection score and a detection frame coordinate according to the original output, and a postprocessing process of removing redundant detection frames using a non-maximum suppression algorithm. Here, the detection process can be set according to actual needs and can be any existing detection process, and the present disclosure does not impose any particular limitation on this.

[0058] Taking the above Yolov3 detector as an example, two Yolov3 detectors can be trained using the image data of two sources of the same training image sample. These two detectors can be used to detect multiple source image data of the same scene respectively to obtain two detection results. The two detection results include the original output of the Yolov3 detector, that is, the confidence c , classification score and the regression branch output Similarly, taking the above Yolov8 detector as an example, two Yolov8 detectors can be trained using the image data of two sources of the same training image sample, and the two detectors can be used to detect multiple source image data of the same scene respectively to obtain two detection results. The two detection results include the original output of the Yolov8 detector, that is, the classification score and the regression branch output .

[0059] Here, it should be noted that the detection results of each image detector for the corresponding source image can be obtained in the process of executing the image fusion detection method according to the embodiment of the present disclosure, or can be obtained in advance before executing the method. The embodiment of the present disclosure does not impose any special restrictions on this.

[0060] It should also be noted that although the Yolo series detectors are described as an example in this article, the present disclosure does not impose any particular limitation on the detectors, and the image detection scheme according to the present disclosure can also be applied to other detectors.

[0061] In step S120, the detection results of different image detectors may be paired to obtain paired detection results.

[0062] According to an embodiment of the present disclosure, the detection result of each image detector forms a detection result set, and different detection result sets can be input into a fusion framework to generate multiple sets of paired detection results. Since the detection result sets may include multiple detection results respectively, and when fusing the detection results, it is necessary to fuse two detection results from different detectors, it is necessary to first obtain multiple sets of paired detection results, wherein each set of paired detection results includes two detection results from different detectors.

[0063] As an example, in step S120, for any two image detectors among the multiple image detectors, the detection results of different image detectors may be paired in the following manner:

[0064] For each first detection result in the first detection result set of the first image detector of the two image detectors, the following operations may be performed in sequence:

[0065] In response to the presence of a candidate second detection result whose intersection-and-union ratio with the current first detection result is greater than or equal to a preset intersection-and-union ratio threshold in the current second detection result set of the second image detector among the two image detectors, the current first detection result can be paired with a target second detection result whose intersection-and-union ratio with the current first detection result is the largest among the candidate second detection results; in response to the absence of a candidate second detection result in the current second detection result set, the current first detection result can be paired with a null detection result.

[0066] As an example, Can represent a test result, Can represent a fused detection result, where c Indicates confidence, represents the classification score, represents the regression branch output, Represents the coordinates of the detection box, Indicates the test result category. Represents the final detection score. and They can be expressed as M The first test result set of test results and N A second detection result set of detection results. It can represent the fused detection result set.

[0067] You can traverse the first test result set Each first test result in , calculate the detection frame coordinates of the first detection result With the second test result set The intersection over Union (IoU) between the detection box coordinates of each second detection result in .

[0068] In response to the maximum IoU among the IoUs between the detection frame of the first detection result and the detection frames of all the second detection results being less than or equal to a preset intersection-over-union ratio threshold α , you can compare this test result with the empty test result Pairing, get a set of paired detection results. Preset intersection-over-union ratio threshold α It can be set according to actual needs, for example, preset the intersection-and-union ratio threshold α It can be taken as 0.5.

[0069] In response to the maximum IoU among the IoUs between the detection box of the first detection result and the detection boxes of all the second detection results being greater than a preset intersection-over-union threshold α , the detection result can be paired with the second detection result corresponding to the maximum IoU to obtain a set of paired detection results.

[0070] The paired test results can be fused and the fused test results Add to An example of the fusion process will be described in detail below. Through the above pairing method, the corresponding detection results in different detection result sets can be accurately found, so that the selection of the detection object is more accurate, so that the subsequent fusion is also more accurate.

[0071] In addition, according to an embodiment of the present disclosure, the image fusion detection method may further include: in response to pairing the current first detection result with the target second detection result, deleting other candidate second detection results except the target second detection result from the current second detection result set.

[0072] Specifically, after pairing each first detection result, the current second detection result set The IoU between the first detection result and theα The other second test results from the current Delete and get a new second detection result set , the new second detection result set It can be used to pair with the next first detection result.

[0073] Since multiple detection results may be generated for the same object, that is, multiple detection boxes of different sizes may select the object, the above method can be used to find the second detection result with the largest intersection-over-union ratio with the current first detection result, and the other detection results with a large intersection-over-union ratio with the first detection result are also large (for example, greater than the threshold α ) may not be the most preferred pairing, and such a second test result can be deleted to speed up the subsequent pairing process.

[0074] After the above traversal process is completed, the remaining second detection results in the second detection result set can be combined with the empty detection results. Pair and merge, and add the fused result to middle.

[0075] Figure 3 It is a schematic diagram of an example process of obtaining paired detection results in the image fusion detection method according to an exemplary embodiment of the present disclosure.

[0076] like Figure 3 As shown, in step S301, it can be determined whether the first detection result set "Set 1" is a non-empty set.

[0077] If set 1 is not empty, then in step S302, a first detection result may be taken from set 1, and in step S303, it may be determined whether the second detection result set "set 2" is a non-empty set.

[0078] If set 2 is not empty, then in step S304, all second detection results can be taken from set 2, and in step S305, the IoU between the first detection result and each second detection result can be calculated. If set 2 is empty, then in step S3010, the first detection result can be paired with an empty detection result, and step S309 is executed.

[0079] In step S306, it can be determined whether there is an IoU higher than the preset intersection-over-union ratio threshold in the calculation result of step S305. If so, in step S307, other second detection results in set 2 with an IoU higher than the preset intersection-over-union ratio threshold except the second detection result corresponding to the maximum IoU can be deleted. In step S308, the first detection result can be paired with the second detection result corresponding to the maximum IoU. In step S309, the paired detection result can be added to the paired detection result set, and the process returns to step S301.

[0080] In step S301, if set 1 is empty, then in step S3011, it can be determined whether set 2 is a non-empty set. If set 2 is non-empty, then in step S3012, a second detection result can be taken from set 2, and in step S3013, the second detection result can be paired with the empty detection result. In step S3014, the paired detection result can be added to the paired detection result set, and the process returns to step S3011. If set 2 is empty, the traversal can be terminated.

[0081] In addition, preferably, according to an embodiment of the present disclosure, before pairing, the test results in each test result set can be arranged in a predetermined order, and the predetermined order can be, for example, sorted from large to small according to the weight of the test results, and the test results with a higher ranking can participate in the matching first, while the test results with a lower ranking are more likely to only match with empty results. For example, the weight of the confidence level can be greater than the weight of the final test score. The higher the confidence level of the test result, the more accurate the classification may be, so the test results with a higher confidence level can be placed in front.

[0082] For example, for detectors whose original output composition is the same as that of Yolov3 detector, they can be arranged from high to low according to the confidence level, and the sorting by confidence level is more meaningful; for detectors whose original output composition is the same as that of Yolov8 detector, there is no confidence level in the original output, and they can be arranged from high to low according to the final detection score.

[0083] Although the detection results are described above to be sorted, according to the embodiments of the present disclosure, this sorting method may not be adopted. For example, other methods may be adopted or no sorting may be performed.

[0084] In step S130, the paired detection results may be fused based on the original output to obtain a fused detection result.

[0085] In this step, the original outputs may be fused to obtain a fused detection result, so that the information or related quantity carried by each original output is introduced into the fused detection result, making the fused detection result more accurate.

[0086] As an example, Figure 4 As shown, in the case where the original output includes a classification score vector, assuming that the paired detection results include a first paired result and a second paired result, the paired detection results can be fused in the following manner to obtain a fused detection result:

[0087] In step S410, a fused classification score vector may be determined by comparing a maximum classification score in a first classification score vector of a first pairing result with a maximum classification score in a second classification score vector of a second pairing result.

[0088] In one example, when the original output also includes confidence, such as for a Yolov3 detector or similar detector, the fused classification score vector can be determined as follows:

[0089] In response to the difference between the maximum classification score in the first classification score vector of the first pairing result and the maximum classification score in the second classification score vector of the second pairing result being greater than a preset score threshold, the classification score vector with the larger maximum classification score between the first classification score vector and the second classification score vector is used as a fused classification score vector.

[0090] In response to the difference between the maximum classification score in the first classification score vector and the maximum classification score in the second classification score vector being less than or equal to a preset score threshold, the classification score vector of the pairing result with higher confidence between the first pairing result and the second pairing result is used as a fused classification score vector.

[0091] For example, if the first classification score vector The maximum classification score in and the second classification score vector The maximum classification score in The difference is greater than the preset score threshold β , classification score vector after fusion is the classification score vector with the largest maximum classification score; otherwise, the classification score vector after fusion is the classification score vector of the detection result with higher confidence. Here, the preset score threshold can be set according to actual needs, for example, it can be 0.5.

[0092] Through the above method, when the difference between the maximum classification scores is large, the fused score vector can be determined based on the maximum classification score, and when the difference between the two is not large, the fused score vector can be determined based on the confidence level, so that the calculation of the fused score vector is more accurate and reasonable.

[0093] In another example, when the original output does not include confidence, such as for a Yolov8 detector or a similar detector, the classification score vector with a larger maximum classification score between the first classification score vector and the second classification score vector can be used as a fused classification score vector.

[0094] For example, and Can represent the first classification score vector and the second classification score vector respectively. If the first classification score vector The maximum classification score in Greater than the second classification score vector The maximum classification score in , then the maximum classification score can be as the ensemble’s classification score vector; otherwise, the maximum classification score can be The classification score vector as the ensemble.

[0095] In step S420, a fused detection result may be determined based on the fused classification score vector.

[0096] Here, by comparing the maximum classification scores in the two classification score vectors, determining the fused classification score vector to determine the fused detection result, the target object can be classified more accurately, thereby improving the accuracy of the detection result.

[0097] In step S420 , the fused detection result may be determined based on the fused classification score vector according to whether the original output of the current image detector contains confidence.

[0098] In the example where the original output includes confidence, such as Figure 5 As shown, the fused detection results can be determined in the following ways:

[0099] In step S510, the confidence of the first pairing result and the confidence of the second pairing result may be fused to obtain a fused confidence.

[0100] As an example, the fused confidence can be calculated based on the product of two confidences. For example, the fused confidence can be obtained by the following formula (1):

[0101] (1)

[0102] in, represents the confidence of the fusion, represents the confidence of the first pairing result, Indicates the confidence of the second pairing result.

[0103] It should be noted that the above formula (1) is only an example of fusion method, and fusion can also be achieved through other calculation methods, for example, and The maximum value is .

[0104] In step S520, a fused detection result may be determined based on the fused confidence and the fused classification score vector.

[0105] In this step, the final detection score in the fused detection result can be determined based on the fused confidence and the fused classification score vector. For example, it can be determined by the following formula (2):

[0106] (2)

[0107] in, represents the final detection score in the fused detection results, represents the fused classification score vector, Represents the maximum classification score in the ensemble classification score vector.

[0108] It should be noted that the above formula (2) is only an example, and the final detection score may be determined by other calculation methods, for example, the formula may be weighted.

[0109] As an example, the fused detection result can be determined in the following manner: in response to the maximum classification score in the fused classification score vector being greater than a preset score threshold, the classification of the fused detection result is determined to be a target in the image; in response to the maximum classification score in the fused classification score vector being less than or equal to the preset score threshold, the classification of the fused detection result is determined to be the background in the image.

[0110] Specifically, if the final test score Above a pre-set threshold , the fused detection result is classified as the target; otherwise, the fused detection result is classified as the background. Here, the preset score threshold It can be set according to actual needs, for example, it can be 0.05.

[0111] Here, based on the maximum classification score of the fused detection results, the classification of the detected object can be judged more accurately, thereby improving the accuracy of image detection.

[0112] In addition, in response to the classification of the fused detection result as a target in the image, the category corresponding to the maximum element in the fused classification score vector is used as the category of the target, and the target detection position of the fused detection result is determined by fusing the target detection position in the first pairing result and the target detection position in the second pairing result.

[0113] Specifically, if the classification of the fused detection results is determined as the target, the category of the fused detection results is the fused classification score vector The index of the maximum element in , and the final target detection position can be determined based on the target detection positions of the two paired detection results. For example, the detection box of the fused detection result can be expressed by the following formula (3):

[0114] (3)

[0115] in, The detection box represents the fused detection result. represents the target detection position in the first pairing result, Indicates the target detection position in the second pairing result.

[0116] It should be noted that the above formula (3) is only an example, and the final target detection position may also be determined by other calculation methods, for example, the target detection positions of different pairing results may be weighted and then averaged.

[0117] According to an embodiment of the present disclosure, in an example where the original output includes confidence, when any detection result is paired with an empty detection result, the empty detection result may be given ,and , Each element in can be 0.5, The detection result may be the same as the paired one.

[0118] When the original output includes confidence, since confidence can better characterize the accuracy of the detection result, the confidence and classification score vectors can be fused separately to determine the fused detection result based on these two quantities to make it more accurate.

[0119] In examples where the raw output does not include confidence, other raw outputs of the image detector may be employed.

[0120] For example, taking the Yolov8 detector or similar detectors as an example, the original output also includes the regression branch output vector, such as Figure 6 As shown, the fused detection results can be determined in the following ways:

[0121] In step S610, two larger values ​​in the sub-distribution of each of the first regression branch output vector and the second regression branch output vector may be determined based on the first regression branch output vector of the first pairing result and the second regression branch output vector of the second pairing result.

[0122] In step S620, a first statistic of the first regression branch output vector and a second statistic of the second regression branch output vector may be determined based on the distance between the two values ​​of each regression branch output vector and the sum of the two values.

[0123] Here, the larger the sum between the two larger values ​​in the sub-distribution of each regression branch output vector, the larger the maximum value of the sub-distribution is, and the more likely it is to present a sharp shape. The smaller the distance between the two values, the closer the distribution is to a single-peak sharp shape. Based on such sum and distance, the statistics of the regression branch output vector are calculated respectively. The larger the statistic, the sharper and more concentrated the shape of the sub-distribution is.

[0124] As an example, the vector may be output for the first regression branch and the second regression branch output vector Calculate the first statistic separately and the second statistic .by For example, for four sub-distributions , , and , the first statistic can be determined by the following formula (4) :

[0125] (4)

[0126] in, and represents the output vector of the first regression branch The larger of the two values ​​in the sub-distribution, where =1,2,3,4, and represents the index of these two values, e is a natural constant, γ is a hyperparameter that can be set according to actual needs, for example, it can be 0.5. Here, the statistic is between 0 and 1. The second regression branch outputs the vector The second statistic The calculation method of can be similar to the above formula (4), which will not be repeated here.

[0127] In the above formula (4), the statistic is calculated for the sub-distributions of the four sides of the detection box respectively, so it is formally expressed as the summation followed by the average. The logarithm is used in the denominator in the calculation, considering that the minimum absolute value term in the denominator is 1. In this case, the value of the denominator is ln(e)=1. The larger the absolute value term, the larger the value of the denominator, and the minimum value of the denominator is 1. The minimum value of the numerator approaches 0, and the maximum value approaches 1. Therefore, the maximum value of the statistic approaches 1. The larger the statistic, the closer the shape of the detection box distribution is to the ideal unimodal concentrated distribution shape.

[0128] It should be noted that the above formula (4) is only an example, and the statistic may be determined by other calculation methods, for example, weighting may be performed on the above formula.

[0129] In step S630, in response to the first statistic and the second statistic being both smaller than a preset statistic threshold, it may be determined that the fused detection result is classified as the background in the image.

[0130] In step S640, in response to at least one of the first statistic and the second statistic being greater than or equal to a preset statistic threshold, it may be determined that the fused detection result is classified as a target in the image.

[0131] Here, the preset statistical threshold can be set according to actual needs, for example, it can be 0.7.

[0132] In this example, as described in step S410 above, the fused classification score vector may be a classification score vector with a larger maximum classification score. The maximum classification score in is taken as the final detection score, and the category of the fused detection result is the fused classification score vector The index of the maximum element in , and the final target detection position can be determined based on the target detection positions of the two paired detection results, for example, as shown in the above formula (3).

[0133] According to an embodiment of the present disclosure, in an example where the original output includes confidence, when any detection result is paired with an empty detection result, the empty detection result may be given ,and =0, Each element in can be , b e The detection result may be the same as the paired one.

[0134] In the above manner, even when the original output does not include confidence, the accuracy of the fused detection result can be improved by fusing the classification score vectors.

[0135] The following will refer to Figure 7 and Figure 8 The complete process of obtaining the fused detection results in the above example is described exemplarily.

[0136] In the example where the original output includes a classification score vector and confidence, such as Figure 7 As shown, in step S701, the confidence level of the first pairing result among the two pairing results may be and the confidence of the second pairing result Calculate the confidence of the fusion detection results .

[0137] In step S702, the maximum classification score in the first classification score vector can be determined. The maximum classification score in the second classification score vector Is the absolute value of the difference between β .

[0138] If it is greater than the preset score threshold β , then in step S703, the maximum classification score in the first classification score vector can be determined Is it greater than the maximum classification score in the second classification score vector? .like Greater than , then in step S704, the first classification score vector As the ensemble classification score vector ;like No more than , then in step S705, the second classification score vector As the ensemble classification score vector .

[0139] If it is not greater than the preset score threshold β , then in step S706, the confidence level of the first pairing result can be determined Is it greater than the confidence of the second pairing result? .like Greater than , then in step S707, the first classification score vector As the fused classification score vector; if No more than , then step S705 can be executed.

[0140] In step S708, the fused classification score vector can be determined .

[0141] In step S709, the fused classification score vector can be determined Is it greater than the preset threshold? If it is not greater than the preset threshold , then the classification of the fused detection result is determined to be background, and the process can be ended; if it is greater than the preset threshold , then in step S710, the classification of the fused detection result can be determined as the target, and the category of the fused detection result is the fused classification score vector The index of the largest element in and the process can be ended.

[0142] In the example where the original output includes a classification score vector and a regression branch output vector, such as Figure 8 As shown, in step S801, the maximum classification score in the first classification score vector can be determined. Is it greater than the maximum classification score in the second classification score vector? .like Greater than , then in step S802, the first classification score vector As the ensemble classification score vector ;like No more than , then in step S803, the second classification score vector As the ensemble classification score vector .

[0143] In step S804, the fused classification score vector can be determined .

[0144] In step S805, the vector can be output according to the regression branch and , respectively calculate the first statistic and the second statistic .

[0145] In step S806, the first statistic can be determined and the second statistic Are they all less than the preset threshold? If the first statistic and the second statistic At least one of the values ​​is not less than the preset threshold , then the classification of the fused detection result is determined to be background, and the process can be ended; if both are less than the preset threshold , then in step S807, the classification of the fused detection result can be determined as the target, and the category of the fused detection result is the fused classification score vector The index of the largest element in and the process can be ended.

[0146] As mentioned above, Figure 7 and Figure 8 For example, according to an exemplary embodiment of the present disclosure, the detection result may be composed of multiple parts of original output, which depends on the detection head structure of the detector. The present disclosure may design fusion methods for target detectors with different detection head structures.

[0147] According to the exemplary embodiments of the present disclosure, a lightweight target detector with different raw output compositions for two types of detection results can make full use of the effective information contained in the raw output of the detector by using the raw output of the detector such as the lightweight detector as the input of the decision-level fusion algorithm, and design a decision-level fusion algorithm consisting of a decision-level fusion algorithm framework and a fusion method, which can be applied to scenarios with limited computing resources and high real-time requirements such as distributed earth observation systems, and can improve performance indicators such as the average accuracy and number of false alarms of multi-source image target detection. It has the effect of improving the average accuracy of all categories of the decision-level fusion of the target detector on the test data set or reducing the number of false alarms.

[0148] In a second aspect of the exemplary embodiments of the present disclosure, there is provided an image fusion detection device based on the original output of an image detector. Fig. 9 is a schematic block diagram of an image fusion detection device according to an exemplary embodiment of the present disclosure. Fig. 9 As shown, the image fusion detection device may include an acquisition unit 910 , a pairing unit 920 and a fusion unit 930 .

[0149] The acquisition unit 910 can be configured to acquire detection results obtained by respectively detecting images of multiple signal sources of the same scene by multiple image detectors, wherein the signal sources correspond one-to-one to the image detectors, and each detection result represents a target object detected in the image of the corresponding signal source, and the detection result includes the original output of the corresponding image detector.

[0150] The pairing unit 920 may be configured to pair detection results of different image detectors to obtain paired detection results.

[0151] The fusion unit 930 may be configured to fuse the paired detection results based on the original output to obtain a fused detection result.

[0152] As an example, the detection results of each image detector form a detection result set. For any two image detectors among the multiple image detectors, the pairing unit 920 can be configured to pair the detection results of different image detectors in the following manner: for each first detection result in the first detection result set of the first image detector of the two image detectors, the following operations are performed in sequence: in response to the presence of a candidate second detection result whose intersection-and-union ratio with the current first detection result is greater than or equal to a preset intersection-and-union ratio threshold in the current second detection result set of the second image detector of the two image detectors, the current first detection result is paired with a target second detection result with the largest intersection-and-union ratio with the current first detection result among the candidate second detection results; in response to the absence of a candidate second detection result in the current second detection result set, the current first detection result is paired with a null detection result.

[0153] As an example, the pairing unit 920 may also be configured to: in response to pairing the current first detection result with the target second detection result, delete other candidate second detection results except the target second detection result from the current second detection result set.

[0154] As an example, the original output includes a classification score vector, and the paired detection result includes a first pairing result and a second pairing result, wherein the fusion unit 930 can be configured to: determine a fused classification score vector by comparing the maximum classification score in a first classification score vector of the first pairing result with the maximum classification score in a second classification score vector of the second pairing result; and determine a fused detection result based on the fused classification score vector.

[0155] As an example, the original output also includes confidence, wherein the fusion unit 930 can be configured to determine the fused classification score vector in the following manner: in response to the difference between the maximum classification score in the first classification score vector of the first pairing result and the maximum classification score in the second classification score vector of the second pairing result being greater than a preset score threshold, the classification score vector with the larger maximum classification score in the first classification score vector and the second classification score vector is used as the fused classification score vector; in response to the difference between the maximum classification score in the first classification score vector and the maximum classification score in the second classification score vector is less than or equal to the preset score threshold, the classification score vector of the pairing result with higher confidence in the first pairing result and the second pairing result is used as the fused classification score vector.

[0156] As an example, the fusion unit 930 can be configured to determine the fused detection result in the following manner: fuse the confidence of the first pairing result and the confidence of the second pairing result to obtain a fused confidence; determine the fused detection result based on the fused confidence and the fused classification score vector.

[0157] As an example, the fusion unit 930 can be configured to determine the fused detection result in the following manner: in response to the maximum classification score in the fused classification score vector being greater than a preset score threshold, determining that the fused detection result is classified as a target in the image; in response to the maximum classification score in the fused classification score vector being less than or equal to a preset score threshold, determining that the fused detection result is classified as the background in the image.

[0158] As an example, the original output also includes a regression branch output vector, and the fusion unit 930 can be configured to determine the fused detection result in the following manner: based on the first regression branch output vector of the first pairing result and the second regression branch output vector of the second pairing result, determine the two larger values ​​in the sub-distribution of each regression branch output vector in the first regression branch output vector and the second regression branch output vector; based on the distance between the two values ​​and the sum of the two values, determine the first statistic of the first regression branch output vector and the second statistic of the second regression branch output vector respectively; in response to the first statistic and the second statistic being both less than a preset statistic threshold, determine that the fused detection result is classified as the background in the image; in response to at least one of the first statistic and the second statistic being greater than or equal to the preset statistic threshold, determine that the fused detection result is classified as a target in the image.

[0159] As an example, the fusion unit 930 can also be configured to: in response to the classification of the fused detection result as a target in the image, take the category corresponding to the maximum element in the fused classification score vector as the category of the target, and determine the target detection position of the fused detection result by fusing the target detection position in the first pairing result and the target detection position in the second pairing result.

[0160] Regarding the device in the above-mentioned embodiment, the specific manner in which each unit performs the operation has been described in detail in the embodiment of the method. Each unit in the image fusion detection device can perform the corresponding steps in the method according to the image fusion detection method in the method embodiment of the first aspect above, and will not be elaborated here.

[0161] Fig.10 is a block diagram of an electronic device according to an exemplary embodiment of the present disclosure. Fig.10 As shown, the electronic device 10 includes a processor 101 and a memory 102 for storing processor executable instructions. Here, when the processor executable instructions are executed by the processor, the processor is prompted to execute the image fusion detection method based on the original output of the image detector as described in the above exemplary embodiment.

[0162] As an example, the electronic device 10 is not necessarily a single device, but may be any collection of devices or circuits that can execute the above instructions (or instruction sets) individually or in combination. The electronic device 10 may also be part of an integrated control system or system manager, or may be configured as a server that is interconnected with a local or remote (e.g., via wireless transmission) interface.

[0163] In the electronic device 10, the processor 101 may include a central processing unit (CPU), a graphics processing unit (GPU), a programmable logic device, a dedicated processor system, a microcontroller or a microprocessor. As an example and not a limitation, the processor 101 may also include an analog processor, a digital processor, a microprocessor, a multi-core processor, a processor array, a network processor, etc.

[0164] The processor 101 may execute instructions or codes stored in the memory 102, wherein the memory 102 may also store data. Instructions and data may also be sent and received over a network via a network interface device, wherein the network interface device may employ any known transmission protocol.

[0165] The memory 102 may be integrated with the processor 101, for example, by placing RAM or flash memory within an integrated circuit microprocessor or the like. In addition, the memory 102 may include a separate device, such as an external disk drive, a storage array, or any other storage device that can be used by a database system. The memory 102 and the processor 101 may be operatively coupled, or may communicate with each other, such as through an I / O port, a network connection, etc., so that the processor 101 can read files stored in the memory 102.

[0166] In addition, the electronic device 10 may also include a video display (such as a liquid crystal display) and a user interaction interface (such as a keyboard, a mouse, a touch input device, etc.) All components of the electronic device 10 may be connected to each other via a bus and / or a network.

[0167] In an exemplary embodiment, a computer-readable storage medium may also be provided. When the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device can perform the image fusion detection method based on the original output of the image detector as described in the above exemplary embodiment. The computer-readable storage medium may be, for example, a memory including instructions. Optionally, the computer-readable storage medium may be: read-only memory (ROM), random access memory (RAM), random access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disk storage, hard disk drive (HDD), solid state drive (SSD), card storage (such as, multimedia card, secure digital (SD) card or extreme digital (XD) card), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid state disk and any other device, any other device is configured to store computer programs and any associated data, data files and data structures in a non-transitory manner and provide the computer programs and any associated data, data files and data structures to a processor or computer so that the processor or computer can execute the computer program. The computer program in the above-mentioned computer-readable storage medium can be run in an environment deployed in a computer device such as a client, a host, an agent device, a server, etc. In addition, in one example, the computer program and any associated data, data files and data structures are distributed on a networked computer system, so that the computer program and any associated data, data files and data structures are stored, accessed and executed in a distributed manner by one or more processors or computers.

[0168] According to an exemplary embodiment of the present disclosure, a computer program product may also be provided, the computer program product comprising computer executable instructions, which, when executed by at least one processor, implement the image fusion detection method based on the original output of the image detector according to the exemplary embodiment of the present disclosure.

[0169] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

[0170] In addition, it should be noted that although several examples of each step are described above with reference to specific drawings, it should be understood that the embodiments of the present disclosure are not limited to the combinations given in the examples, and the steps appearing in different drawings may be combined, which are not exhaustively listed here.

[0171] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. An image fusion detection method based on the original output of an image detector, characterized in that: The image fusion detection method comprises: Acquire detection results obtained by detecting images of multiple sources of the same scene using multiple image detectors, wherein the sources correspond to the image detectors one by one, each detection result represents an object detected in the image of the corresponding source, and the detection result includes an original output of the corresponding image detector; Pairing the detection results of different image detectors to obtain paired detection results; Based on the original output, the paired detection results are fused to obtain a fused detection result, wherein the original output includes a classification score vector and a confidence level, and the paired detection results include a first pairing result and a second pairing result, The step of fusing the paired detection results to obtain a fused detection result includes: determining a fused classification score vector by comparing the maximum classification score in the first classification score vector of the first pairing result with the maximum classification score in the second classification score vector of the second pairing result; determining the fused detection result based on the fused classification score vector, The fused classification score vector is determined in the following manner: In response to the difference between the maximum classification score in the first classification score vector of the first pairing result and the maximum classification score in the second classification score vector of the second pairing result being greater than a preset score threshold, taking the classification score vector with the larger maximum classification score between the first classification score vector and the second classification score vector as the fused classification score vector; In response to the difference between the maximum classification score in the first classification score vector and the maximum classification score in the second classification score vector being less than or equal to the preset score threshold, the classification score vector of the pairing result with higher confidence between the first pairing result and the second pairing result is used as the fused classification score vector.

2. The image fusion detection method according to claim 1, characterized in that: The detection result of each image detector forms a detection result set. For any two image detectors among the multiple image detectors, the detection results of different image detectors are paired in the following manner: For each first detection result in the first detection result set of the first image detector of the two image detectors, the following operations are performed in sequence: In response to the presence of a candidate second detection result whose intersection-and-union ratio with the current first detection result is greater than or equal to a preset intersection-and-union ratio threshold in the current second detection result set of the second image detector of the two image detectors, pairing the current first detection result with a target second detection result whose intersection-and-union ratio with the current first detection result is the largest among the candidate second detection results; In response to the candidate second detection result not existing in the current second detection result set, the current first detection result is paired with a null detection result.

3. The image fusion detection method according to claim 2, characterized in that: The image fusion detection method also includes: In response to pairing the current first detection result with the target second detection result, other candidate second detection results except the target second detection result are deleted from the current second detection result set.

4. The image fusion detection method according to claim 1, characterized in that: The detection result of the fusion is determined by: fusing the confidence of the first pairing result and the confidence of the second pairing result to obtain a fused confidence; Based on the fused confidence and the fused classification score vector, a fused detection result is determined.

5. The image fusion detection method according to claim 1, characterized in that: The detection result of the fusion is also determined by: In response to a final detection score in the fused detection result being greater than a preset score threshold, determining that the fused detection result is classified as an object in the image; In response to a final detection score in the fused detection result being less than or equal to the preset score threshold, determining that the fused detection result is classified as background in the image.

6. The image fusion detection method according to claim 5, characterized in that: The image fusion detection method also includes: In response to the classification of the fused detection result as a target in an image, the category corresponding to the maximum element in the fused classification score vector is used as the category of the target, and the target detection position of the fused detection result is determined by fusing the target detection position in the first pairing result and the target detection position in the second pairing result.

7. An image fusion detection method based on the original output of an image detector, characterized in that: The image fusion detection method comprises: Acquire detection results obtained by detecting images of multiple sources of the same scene using multiple image detectors, wherein the sources correspond to the image detectors one by one, each detection result represents an object detected in the image of the corresponding source, and the detection result includes an original output of the corresponding image detector; Pairing the detection results of different image detectors to obtain paired detection results; Based on the original output, the paired detection results are fused to obtain a fused detection result, wherein the paired detection result includes a first pairing result and a second pairing result, The original output includes a regression branch output vector, and the fused detection result is determined by: Based on a first regression branch output vector of the first pairing result and a second regression branch output vector of the second pairing result, determining two larger values ​​in a sub-distribution of each of the first regression branch output vector and the second regression branch output vector; Determine, based on the distance between the two values ​​and the sum of the two values, a first statistic of the first regression branch output vector and a second statistic of the second regression branch output vector; In response to the first statistic and the second statistic being both less than a preset statistic threshold, determining that the fused detection result is classified as background in the image; In response to at least one of the first statistic and the second statistic being greater than or equal to the preset statistic threshold, determining that the fused detection result is classified as a target in the image.

8. The image fusion detection method according to claim 7, characterized in that: The original output also includes a classification score vector, and the image fusion detection method also includes: In response to the classification of the fused detection result as a target in the image, taking the category corresponding to the maximum element in the fused classification score vector as the category of the target, and determining the target detection position of the fused detection result by fusing the target detection position in the first pairing result and the target detection position in the second pairing result, The fused classification score vector is determined by comparing the maximum classification score in the first classification score vector of the first pairing result with the maximum classification score in the second classification score vector of the second pairing result.

9. An image fusion detection device based on the original output of an image detector, characterized in that: The image fusion detection device comprises: an acquisition unit, configured to acquire detection results obtained by detecting images of multiple sources of the same scene by multiple image detectors, respectively, wherein the sources correspond to the image detectors one by one, each detection result represents a target object detected in the image of the corresponding source, and the detection result includes an original output of the corresponding image detector; a pairing unit, configured to pair detection results of different image detectors to obtain paired detection results; a fusion unit configured to fuse the paired detection results based on the original output to obtain a fused detection result, wherein the original output includes a classification score vector and a confidence level, and the paired detection result includes a first paired result and a second paired result, The fusion unit is configured to: determine a fused classification score vector by comparing the maximum classification score in the first classification score vector of the first pairing result with the maximum classification score in the second classification score vector of the second pairing result; determine the fused detection result based on the fused classification score vector, The fusion unit is configured to determine the fused classification score vector in the following manner: In response to the difference between the maximum classification score in the first classification score vector of the first pairing result and the maximum classification score in the second classification score vector of the second pairing result being greater than a preset score threshold, taking the classification score vector with the larger maximum classification score between the first classification score vector and the second classification score vector as the fused classification score vector; In response to the difference between the maximum classification score in the first classification score vector and the maximum classification score in the second classification score vector being less than or equal to the preset score threshold, the classification score vector of the pairing result with higher confidence between the first pairing result and the second pairing result is used as the fused classification score vector.

10. An image fusion detection device based on the original output of an image detector, characterized in that: The image fusion detection device comprises: an acquisition unit, configured to acquire detection results obtained by detecting images of multiple sources of the same scene by multiple image detectors, respectively, wherein the sources correspond to the image detectors one by one, each detection result represents a target object detected in the image of the corresponding source, and the detection result includes an original output of the corresponding image detector; a pairing unit, configured to pair detection results of different image detectors to obtain paired detection results; a fusion unit configured to fuse the paired detection results based on the original output to obtain a fused detection result, wherein the paired detection result includes a first pairing result and a second pairing result, The original output includes a regression branch output vector, and the fusion unit is further configured to determine the fused detection result in the following manner: Based on a first regression branch output vector of the first pairing result and a second regression branch output vector of the second pairing result, determining two larger values ​​in a sub-distribution of each of the first regression branch output vector and the second regression branch output vector; Determine, based on the distance between the two values ​​and the sum of the two values, a first statistic of the first regression branch output vector and a second statistic of the second regression branch output vector; In response to the first statistic and the second statistic being both less than a preset statistic threshold, determining that the fused detection result is classified as background in the image; In response to at least one of the first statistic and the second statistic being greater than or equal to the preset statistic threshold, determining that the fused detection result is classified as a target in the image.

11. An electronic device, characterized in that: The electronic device comprises: processor; a memory for storing instructions executable by the processor, When the processor executable instructions are executed by the processor, the processor is prompted to execute the image fusion detection method based on the original output of the image detector according to any one of claims 1 to 8.

12. A computer-readable storage medium, characterized in that: When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the image fusion detection method based on the original output of an image detector according to any one of claims 1 to 8.

13. A computer program product comprising computer executable instructions, characterized in that: When the computer executable instructions are executed by at least one processor, the image fusion detection method based on the original output of the image detector according to any one of claims 1 to 8 is implemented.

Images