Multi-target detection model integration method, device, program product and electronic device
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- DIGITAL CHONGQING BIG DATA APPL DEV CO LTD
- Filing Date
- 2025-05-23
- Publication Date
- 2026-06-05
Smart Images
Figure CN120673028B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image detection technology, and in particular to multi-target detection model integration methods, apparatuses, program products, and electronic devices. Background Technology
[0002] In the field of video behavior detection, the detection and behavior recognition of various targets are required. For example, in security monitoring scenarios, it is necessary to detect targets such as objects thrown from heights and fires, and to recognize behaviors such as pedestrians loitering in restricted areas and vehicles illegally parked. In intelligent video monitoring scenarios using cameras, it is necessary to detect targets such as vehicles, pedestrians, and traffic signs on roads, and to recognize dynamic behaviors such as vehicles suddenly changing lanes and pedestrians crossing the road. To meet the needs of complex image detection, it is usually necessary to call two or more target detection models to handle different tasks. These two or more target detection models can include R-CNN series, YOLO series (such as YOLOv4, YOLOv8, and YOLOv10), Transformer detectors, etc.
[0003] When two or more object detection models are invoked to handle different tasks, each model outputs detection results for multiple targets, including the target category, bounding box location, and confidence score. Due to differences in detection logic and threshold settings, the confidence scores of different models may be inconsistent, leading to discrepancies in detection results for the same target. Using simple fusion strategies such as voting or averaging often fails to fully leverage the strengths of each model, potentially suppressing high-quality detection results. Summary of the Invention
[0004] This application aims to at least address the technical problems existing in the prior art and provide a method, apparatus, program product and electronic device for multi-target detection model integration.
[0005] In a first aspect, this application provides a method for integrating multi-object detection models, comprising: obtaining information on all detection boxes output by two or more object detection models after processing an image to be detected, wherein the information on each detection box includes an initial confidence level; constructing a first list, which is initialized to include all detection boxes; constructing a second list, which is initialized to be empty; determining the global importance weight of each object category; determining the dynamic weight of each object detection model for each object category; using the global importance weight of the object category to which detection box i belongs and the dynamic weight of the target detection model outputting detection box i for the object category to which detection box i belongs, weighting the initial confidence level of detection box i to obtain the weighted confidence level of detection box i, where i is the index of the detection box; repeatedly executing the following steps until the first list is empty, and when the first list is empty, outputting the detection boxes included in the second list: removing the detection box with the highest weighted confidence level from the first list and adding the removed detection box to the second list; fusing one or more detection boxes in the first list whose intersection-union ratio (IoU) with the removed detection box satisfies a preset overlap condition to obtain a new detection box, and adding the new detection box to the first list.
[0006] Secondly, this application provides a multi-object detection model integration apparatus for implementing the multi-object detection model integration method described in the first aspect of this application, comprising: a first list construction module for acquiring information of all detection boxes output by two or more object detection models after processing an image to be detected, wherein the information of each detection box includes an initial confidence level; constructing a first list, wherein the first list is initialized to include all detection boxes; a second list construction module for constructing a second list, wherein the second list is initialized to be empty; a weight determination module for determining the global importance weight of each object category; determining the dynamic weight of each object detection model for each object category; and a confidence weighting module for using the confidence level of the detection box i to perform a weighting operation. The global importance weight of the target category and the dynamic weight of the target detection model for the target category to which the output detection box i belongs are used to weight the initial confidence of the detection box i and obtain the weighted confidence of the detection box i, where i is the index of the detection box; the detection box fusion output module performs the following steps in a loop until the first list is empty, and when the first list is empty, it outputs the detection boxes included in the second list: remove the detection box with the highest weighted confidence from the first list and add the removed detection box to the second list; fuse more than one detection box in the first list that meets the preset overlap condition with the removed detection box to obtain a new detection box and add the new detection box to the first list.
[0007] Thirdly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the multi-target detection model integration method described in this application.
[0008] Fourthly, this application provides an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to execute the multi-target detection model integration method described in this application.
[0009] The beneficial technical effects of this application are as follows: Global importance weights are assigned to each target category according to actual task requirements, and the dynamic weights of each target detection model for each target category are obtained. The initial confidence of each detection box is double-weighted using the two weights mentioned above to obtain a weighted confidence, fully leveraging the advantages of each target detection model. The weighted confidence of the output detection boxes from different target detection models has better standard consistency compared to the initial confidence, thereby reducing the risk of suppressing high-quality detection results during the detection result fusion process. In the fusion process of all detection boxes, this application improves upon the existing Non-Maximum Suppression (NMS) algorithm by fusing detection boxes in the first list whose intersection-union ratio (IU) with the removed detection boxes meets a preset overlap condition into a new detection box, and adding the new detection box to the first list. This avoids the loss of detection boxes after fusion and improves the overall detection performance of the target detection system. Attached Figure Description
[0010] Figure 1 This is a flowchart illustrating a preferred embodiment of the multi-target detection model integration method of the present invention;
[0011] Figure 2 This is a schematic diagram of the structure of a multi-target detection model integration device in a preferred embodiment of the present invention;
[0012] Figure 3 This is a schematic diagram of the structure of an electronic device in a preferred embodiment of the present invention. Detailed Implementation
[0013] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.
[0014] In the description of this invention, it should be understood that the terms "longitudinal", "lateral", "up", "down", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.
[0015] In the description of this invention, unless otherwise specified and limited, it should be noted that the terms "installation", "connection" and "linking" should be interpreted broadly. For example, they can refer to mechanical or electrical connections, or internal connections between two components. They can be direct connections or indirect connections through an intermediate medium. Those skilled in the art can understand the specific meaning of the above terms according to the specific circumstances.
[0016] The execution entity of the multi-target detection model integration method provided by this invention includes, but is not limited to, at least one of the following electronic devices that can be configured to execute the method provided in the embodiments of this application: a server, a terminal, etc. In other words, the multi-target detection model integration method can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cloud server cluster. The server can be an independent server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.
[0017] The multi-target detection model integration method provided by this invention, in a preferred embodiment, is as follows: Figure 1 As shown, it includes:
[0018] Step S1: Obtain information on all detection boxes output by two or more object detection models after processing the image to be detected. The information of each detection box includes the initial confidence level. Construct a first list, which is initialized to include all detection boxes.
[0019] In this embodiment, each object detection model is used to process the image to be detected, obtaining the detection result of each model. The detection results of all object detection models are then integrated to obtain information for all bounding boxes. The information for each bounding box output by each object detection model includes the object category, initial confidence score, and bounding box position (e.g., the coordinates of the top-left and bottom-right corners). After integration, the information for each bounding box also includes the model identifier of the object detection model that output the bounding box. For example, the information for bounding box i is represented as follows: in, This represents the coordinates of the top-left corner of the detection box i. s represents the coordinates of the bottom right corner of the detection box i. i c represents the initial confidence level of detection box i. i Let m represent the target category of detection box i. i This represents the model identifier of the target detection model that generated the detection box i. The integrated detection box information is added to the first list to complete the initialization of the first list.
[0020] In this embodiment, depending on the application field, the image to be detected can be a traffic image, and the object detection model is used to detect target categories such as pedestrians, vehicles, and traffic signs in the traffic image; the image to be detected can also be a field image taken by a field camera, and the object detection model is used to detect various wild animals in the field image; the image to be detected can also be a production image from industrial production, and the object detection model is used to detect target categories such as equipment, parts, and workers in the production image. The object detection model is not limited to R-CNN series, YOLO series (such as YOLOv4, YOLOv8, and YOLOv10), Transformer, etc.
[0021] Step S2: Construct a second list, which is initialized to empty.
[0022] Step S3: Determine the global importance weight of each target category; determine the dynamic weight of each target detection model for each target category.
[0023] In this embodiment, the global importance weight of each target type can be determined according to the core requirements of the target recognition task, so as to adjust the overall priority of different target category detection boxes in the ranking. The value range of the global importance weight of each target type is [0,1]. For example, for the pedestrian behavior recognition task, the global importance weight of pedestrians can be set to 0.7, the global importance weight of vehicles can be set to 0.5, and the global importance weight of trees can be set to 0.3.
[0024] In this embodiment, in step S3, for example, the dynamic weights of each target detection model for each target category can be pre-stored. The value range of the dynamic weights is (0,1], and the value can be determined according to the detection accuracy of each target detection model for each target category. When executing the multi-target detection model integration method, the dynamic weights of each target detection model for each target category are read from the memory.
[0025] Step S4: Using the global importance weight of the target category to which the detection box i belongs and the dynamic weight of the target detection model of the output detection box i to the target category to which the detection box i belongs, the initial confidence of the detection box i is weighted to obtain the weighted confidence of the detection box i, where i is the detection box index and i is a positive integer.
[0026] For example, the weighted confidence score of detection box i for:
[0027]
[0028] in, The target detection model m represents the output detection box i. i For the target category c to which the detection box i belongs i Dynamic weights; α(c i ) indicates that the target category c to which the detection box i belongs. i The global importance weight.
[0029] Step S5: Repeat steps S51 and S52 until the first list is empty. When the first list is empty, output the information of the detection boxes included in the second list.
[0030] Step S51: Remove the detection box with the highest weighted confidence from the first list and add the removed detection box to the second list.
[0031] In this embodiment, the detection boxes in the first list are sorted in descending order of weighted confidence. After sorting, the detection box with the highest weighted confidence is removed from the first list, and the removed detection box with the highest weighted confidence is added to the second list.
[0032] Step S52: Merge one or more detection boxes in the first list whose intersection-union ratio with the removed detection boxes meets the preset overlap condition to obtain a new detection box, and add the new detection box to the first list.
[0033] For example, the preset overlap condition is that the intersection-union ratio (IU) of the detected boxes in the first list with the removed detected boxes is greater than the IU threshold, which can be preset. After removing the detected box with the highest weighted confidence, the IU of each detected box in the first list with the removed detected boxes is calculated one by one. Detected boxes in the first list with an IU ratio greater than the IU threshold are identified, and all identified detected boxes are merged to obtain a new detected box. The fusion method is not limited to selecting the detected box with the highest weighted confidence as the new detected box. Alternatively, the average position and average weighted confidence of all identified detected boxes can be calculated and used as the position and weighted confidence of the new detected box, respectively. The target category of the detected box with the highest initial confidence among all identified detected boxes is used as the target category of the new detected box.
[0034] In this embodiment, by obtaining weighted confidence scores and fusing all detection boxes based on these scores, the problem of suppressing high-quality detection results caused by inconsistent confidence standards among different target detection models can be reduced.
[0035] In a preferred embodiment, to balance the differences in detection capabilities of different target detection models for specific target types, step S3, which determines the dynamic weights of each target detection model for each target category, includes:
[0036] Step S3a1: Obtain the average precision (AP) of each target detection model for each target category, and normalize the obtained raw average precision.
[0037] In this embodiment, specifically, the average accuracy of each object detection model for each object category on the test sample set during the training process of each object detection model is obtained. For example, the i'th object detection model m... i' For the j'th target category c j' The average precision is expressed as
[0038] Step S3a2: Select the maximum value among the average accuracies of two or more object detection models for object category j', and use the selected maximum value as the maximum average accuracies of object category j', where j' is the object category index and is a positive integer.
[0039] For example, the maximum average precision of target class j' is expressed as: M represents the number of object detection models, m k Let represent the k-th object detection model.
[0040] Step S3a3: Divide the average accuracy of each object detection model for object category j' by the maximum average accuracy of object category j' to obtain the dynamic weight of each object detection model for object category j'. For example, the i'th object detection model m... i' For the j'th target category c j' The dynamic weights are represented as follows:
[0041]
[0042] In a preferred embodiment, dynamic weights are updated in multiple rounds. After each test using a test sample set on two or more target detection models, the dynamic weights are updated based on the obtained test data. This ensures that the dynamic weights are set according to the detection capabilities of the target detection models, resulting in better model balancing. Therefore, step S3, determining the dynamic weights of each target detection model for each target category, includes:
[0043] Step S3b1: Obtain the test data for the current round. The test data for the current round is obtained by testing two or more object detection models using the test sample set for the current round. The test data for the current round includes the average accuracy of each object detection model for each object category on the test sample set. Each test of two or more object detection models using the test sample set constitutes one round. For example, when the current round is t, the i'th object detection model m... i' For the j'th target category c j' The average precision of the test is expressed as
[0044] Step S3b2: Obtain the dynamic weight increment of each target detection model for each target category in the current round based on the test data of the current round.
[0045] In this embodiment, specifically, step S3b2 includes:
[0046] Step 1: Obtain the dynamic weights of each object detection model for each object category based on the test data of the current round. For example, let the current round be t, and the i'-th object detection model be m... i' For the j'th target category c j' The dynamic weight of the current round of testing is represented as follows:
[0047] Step 2: Subtract the dynamic weight of the target detection model for each target category in the previous round from the current round's dynamic weight for that target category. Multiply the difference by the learning rate δ to obtain the dynamic weight of each target detection model for each target category.
[0048] The dynamic weight increment for the current round. For example, the i'th object detection model m... i' For the j'th
[0049] Target category c j' The current round dynamic weight increment is: δ·Δω (t) (m i' ,c j' )=δ·[testω (t) (m i' ,c j' )-ω (t-1) (m i' ,c j' )]. ω (t-1) (m i' ,c j' ) represents the i'th target detection model m in the previous round (t-1). i' For the j'th target category c j' The dynamic weights. δ represents the learning rate, which typically ranges from 0.01 to 0.05.
[0050] Step S3b3: Calculate the sum of the current round dynamic weight increment of each target detection model for each target category and the dynamic weight of each target detection model for each target category in the previous round, and use this sum as the dynamic weight of each target detection model for each target category in the current round.
[0051] For example, the i'th target detection model m in the current round i' For the j'th target ω (t) (m i' ,c j' )=ω (t-1) (m i' ,c j' )+γ·Δω (t) (m i' ,c j' The value of γ typically ranges from 0.01 to 0.05.
[0052] In a preferred embodiment, step S52 involves fusing one or more detection boxes from the first list whose intersection-union ratio with the removed detection boxes satisfies a preset overlap condition to obtain a new detection box, including:
[0053] Step S521: Construct an overlapping detection box set, which includes one or more detection boxes in the first list whose intersection-union ratio with the removed detection boxes satisfies a preset overlap condition.
[0054] For example, the set of overlapping detection boxes B overlap Includes K detection boxes: B overlap ={b1,b2,…,b K}
[0055] Step S522: Determine the position weight of each detection box in the overlapping detection box set, where the position weight of detection box i is the dynamic weight of the target detection model of output detection box i for the target category to which detection box i belongs. The initial confidence s of the detection box i i The product of these factors, the position weight of detection box i, can be expressed as:
[0056] Step S523: Summate the position weights of the detection boxes in the overlapping detection box set to obtain the cumulative sum of position weights:
[0057] Step S524: Based on the position weights of the detection boxes, the position information of the detection boxes in the overlapping detection box set is weighted and summed to obtain the weighted sum of the detection box positions. Specifically, the x-axis coordinates of the top-left corner point, the y-axis coordinates of the top-left corner point, the x-axis coordinates of the bottom-right corner point, and the y-axis coordinates of the bottom-right corner point are weighted and summed respectively, as shown below: and
[0058] Step S525: Obtain the new detection box position information, specifically the coordinates of the top-left corner, by dividing the weighted sum of the detection box positions by the cumulative sum of the position weights. and the coordinates of the bottom right corner
[0059]
[0060] Step S526: Merge the initial confidence scores of the detection boxes in the overlapping detection box set to obtain the weighted confidence scores of the new detection boxes. The fusion method is not limited to taking the average or selecting the maximum value. Using the initial confidence scores here can avoid the detection boxes of high-priority target categories from excessively influencing the fusion result, thereby improving the accuracy of the fusion result.
[0061] Step S527: Take the target category of the detection box with the highest initial confidence in the overlapping detection box set as the target category of the new detection box.
[0062] In this embodiment, the position of the new detection box is obtained by weighted averaging of the positions of the detection boxes in the overlapping detection box set, which can well represent the overlapping detection box set and avoid the loss of high-quality detection results.
[0063] In this embodiment, more preferably, step S521, assembling the overlapping detection box set, includes:
[0064] Step S521a: Obtain the intersection-union (IU) threshold between each unremoved detection box in the first list and the removed detection boxes. Setting an IU threshold between any two detection boxes can more accurately identify detection boxes that effectively overlap with the removed detection boxes.
[0065] More preferably, the intersection-union (IU) threshold of the two detection boxes is adaptively obtained based on the dynamic weights of the target categories to which the detection boxes belong in the target detection models that output the two detection boxes, which can balance the performance differences between the two target detection models. Therefore, step S521a includes:
[0066] Step S521a1: Obtain the dynamic weights of the target detection model for each detection box that has not been removed from the first output list, denoted as the first dynamic weight. Let detection box j be a detection box that has not been removed from the first list; then the first dynamic weight is the dynamic weight of the target detection model of the output detection box j for the target type of detection box j.
[0067] Step S521a2: Obtain the dynamic weights of the target detection model for the removed detection boxes in relation to the target type of the removed detection boxes, denoted as the second dynamic weights. Let detection box i be the removed detection box, then the second dynamic weights are the dynamic weights of the target detection model for the removed detection box i in relation to the target type of the detection box i.
[0068] Step S521a3: Obtain the absolute value of the difference between the first dynamic weight and the second dynamic weight, denoted as the absolute value of the weight difference, such as...
[0069] Step S521a1: Multiply the absolute value of the weight difference by the weight difference sensitivity coefficient β to obtain the cross-union ratio threshold adjustment amount, such as...
[0070] Step S521a1, set the basic crossover ratio threshold τ base Subtract the intersection-union (IU) threshold adjustment to obtain the IU threshold between each unremoved detection box in the first list and the removed detection box. That is, the IU threshold τ between the unremoved detection box j and the removed detection box i in the first list. i,j for:
[0071] In this embodiment, β is the weight difference sensitivity coefficient, representing the adjustment amount of the cross-union ratio threshold per unit model weight difference, and its value ranges from [0.1, 0.3]; τ base The cross-union ratio threshold is based on the value of [0,1], and can generally be initialized to 0.5.
[0072] Step S521b: Calculate the intersection-union ratio (IUR) between each unremoved detection box in the first list and the removed detection boxes. If the IUR is greater than the IUR threshold between the detection box and the removed detection boxes, then add the detection box to the overlapping detection box set. Specifically, this can be represented as: Overlapping Detection Box Set B overlap ={b j |IoU(b i ,b j )>τ i,j}。 IoU(b i ,b j ) represents the intersection-union ratio of the detection boxes j that were not removed in the first list with the removed detection boxes i.
[0073] In this embodiment, IoU(b i ,b j The calculation formula for ) is as follows:
[0074] For the detection boxes removed from the first list and the detection boxes that were not removed The coordinates of the top left corner of the overlapping region are:
[0075]
[0076] The coordinates of the lower right corner of the overlapping region are:
[0077]
[0078] The area of the overlapping region is:
[0079] overlap_area =
[0080] max(0,overlap_x2-overlap_x1)×max(0,overlap_y2-overlap_y1)
[0081] The area of the union region is:
[0082] union_area = area_b i +area_b j -overlap_area;
[0083]
[0084] In a preferred embodiment, to obtain a more accurate and reasonable weighted confidence score for the new detection boxes and avoid the loss of high-quality detection results, step S526 involves fusing the initial confidence scores of the detection boxes in the overlapping detection box set to obtain the weighted confidence score of the new detection boxes, including:
[0085] Step S5261, from the set of overlapping detection boxes Boverlap Extract the maximum initial confidence from the initial confidence scores of the K bounding boxes (assuming there are K bounding boxes in total).
[0086] Step S5262: Based on the comparison between the initial confidence score of each detection box in the overlapping detection box set and the preset confidence filtering threshold θ, obtain the indicator function value of each detection box in the overlapping detection box set. Specifically, the indicator function f(s) is preset. i >θ), representing condition s i When θ holds, f(s) i >θ) takes 1, condition s i When θ does not hold, f(s) i >θ) is set to 0. θ is generally set to 0.3-0.5.
[0087] Step S5263: Summing the indicator function values of the detection boxes in the overlapping detection box set to obtain the accumulated indicator function value, which can be expressed as:
[0088] Step S5264: Multiply the accumulated value of the indicator function by the multi-model collaborative reward coefficient γ to obtain the accumulated correction value of the indicator function, which can be expressed as: The multi-model collaborative reward coefficient γ is generally taken in the range of 0.05-0.1.
[0089] Step S5265: The sum of the maximum initial confidence and the cumulative correction value of the indicator function is used as the weighted confidence of the new detection box.
[0090] This invention also discloses a multi-target detection model integration device, the structural schematic of which is shown in the figure below. Figure 2 As shown, in a preferred embodiment of the method for integrating multi-target detection models of cedar trees, the apparatus includes:
[0091] The first list construction module obtains information on all detection boxes output by two or more object detection models after processing the image to be detected. The information of each detection box includes the initial confidence level. The first list is constructed and initialized to include all detection boxes.
[0092] The second list construction module constructs a second list, which is initialized to empty.
[0093] The weight determination module determines the global importance weight of each target category and the dynamic weight of each target detection model for each target category.
[0094] The confidence weighting module uses the global importance weight of the target category to which the detection box i belongs and the dynamic weight of the target detection model of the output detection box i to the target category to which the detection box i belongs to the initial confidence of the detection box i to obtain the weighted confidence of the detection box i, where i is the index of the detection box.
[0095] The detection box fusion output module repeatedly executes the following steps until the first list is empty. When the first list is empty, it outputs the detection boxes included in the second list: remove the detection box with the highest weighted confidence from the first list and add the removed detection box to the second list; fuse one or more detection boxes in the first list that have an intersection-union ratio that meets the preset overlap condition with the removed detection box to obtain a new detection box and add the new detection box to the first list.
[0096] The present invention also discloses a computer program product, including a computer program that, when executed by a processor, implements the steps of the multi-target detection model integration method provided by the present invention. The computer program product should be understood as a software product that mainly implements its solution through a computer program, such as a program product integrated in the cloud or a software library.
[0097] The present invention also discloses an electronic device, in one embodiment of which the electronic device includes at least one processor; and a memory communicatively connected to the at least one processor; wherein,
[0098] The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to execute the multi-target detection model integration method provided by the present invention.
[0099] like Figure 3 The diagram shown is a schematic representation of an electronic device for a multi-target detection model integration method according to an embodiment of the present invention. The electronic device may include a processor 10, a memory 11, a communication bus 12, and a communication interface 13. It may also include a computer program, such as a multi-target detection model integration method program, stored in the memory 11 and executable on the processor 10.
[0100] In some embodiments, the processor 10 may be composed of integrated circuits, such as a single packaged integrated circuit or multiple integrated circuits with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The processor 10 is the control unit of the electronic device, connecting various components of the entire electronic device through various interfaces and lines. It executes programs or modules stored in the memory 11 (e.g., executing multi-target detection model integration methods) and calls data stored in the memory 11 to perform various functions of the electronic device and process data.
[0101] The memory 11 includes at least one type of readable storage medium, including flash memory, portable hard drive, multimedia card, card-type memory (e.g., SD or DX memory), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of an electronic device, such as a portable hard drive. In other embodiments, the memory 11 can be an external storage device of the electronic device, such as a plug-in portable hard drive, SmartMediaCard (SMC), SecureDigital (SD) card, FlashCard, etc. Furthermore, the memory 11 can include both internal and external storage units of the electronic device. The memory 11 can be used not only to store application software and various types of data installed on the electronic device, such as the code of a multi-target detection model integration method program, but also to temporarily store data that has been output or will be output.
[0102] The communication bus 12 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into an address bus, a data bus, a control bus, etc. The bus is configured to enable communication between the memory 11 and at least one processor 10, etc.
[0103] Communication interface 13 is used for communication between the aforementioned electronic device and other devices, including a network interface and a user interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a Wi-Fi interface, Bluetooth interface, etc.), typically used to establish communication connections between the electronic device and other electronic devices. The user interface may be a display, an input unit (such as a keyboard), or, optionally, a standard wired or wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen, etc. The display may also be appropriately referred to as a screen or display unit, used to display information processed in the electronic device and to display a visual user interface.
[0104] Figure 3 Only electronic devices with components are shown; it will be understood by those skilled in the art that... Figure 3 The structure shown does not constitute a limitation on the electronic device and may include fewer or more components than shown, or combine certain components, or have different component arrangements.
[0105] For example, although not shown, the electronic device may also include a power supply (such as a battery) to power various components. Preferably, the power supply can be logically connected to at least one processor 10 via a power management device, thereby enabling functions such as charging management, discharging management, and power consumption management. The power supply may also include one or more DC or AC power supplies, recharging devices, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components. The electronic device may also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described in detail here.
[0106] It should be understood that the embodiments are for illustrative purposes only and are not limited to this structure in the scope of the patent application.
[0107] Furthermore, if the modules / units integrated into the electronic device are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile. For example, a computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, or a read-only memory (ROM).
[0108] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.
Claims
1. A multi-target detection model ensemble method, characterized in that, include: Obtain information on all bounding boxes output by two or more object detection models after processing the image to be detected. The information for each bounding box includes the initial confidence score. Construct a first list, which is initialized to include all detection boxes; Construct a second list, which is initialized to empty; Determine the global importance weight for each target category; Determine the dynamic weights of each object detection model for each object category; Using detection box Global importance weights of the target category and output detection boxes The target detection model detects the bounding box. Dynamic weights of the target category, weighted detection boxes The initial confidence level is used to obtain the detection box. The weighted confidence level, For the detection box index; Repeat the following steps until the first list is empty, and when the first list is empty, output the detection boxes included in the second list: Remove the detection box with the highest weighted confidence from the first list and add the removed detection box to the second list; Merge one or more detection boxes from the first list whose intersection-union ratio with the removed detection boxes meets the preset overlap condition to obtain a new detection box and add the new detection box to the first list; The process of fusing one or more detection boxes from the first list whose intersection-union ratio with the removed detection boxes satisfies a preset overlap condition, to obtain a new detection box, includes: Construct an overlapping detection box set, the overlapping detection box set including one or more detection boxes in the first list whose intersection-union ratio with the removed detection boxes satisfies a preset overlap condition, the overlapping detection box set includes K detection boxes; Determine the position weight of each detection box in the set of overlapping detection boxes, wherein the detection box The position weight is the output detection box The target detection model detects the bounding box. Dynamic weight of the target category With detection box initial confidence level The product of the detection box The position weight can be expressed as: ; Represents the detection box Target category; Summing the positional weights of the detection boxes in the overlapping detection box set yields the cumulative sum of positional weights. Based on the position weights of the detection boxes, the position information of the detection boxes in the overlapping detection box set is weighted and summed to obtain the weighted sum of the detection box positions. The position information of the new detection box is obtained by dividing the weighted sum of the detection box positions by the sum of the position weights. The coordinates of the top left corner are used. and the coordinates of the bottom right corner : ； The initial confidence scores of the detection boxes in the overlapping detection box set are merged to obtain the weighted confidence scores of the new detection boxes; The target category of the detection box with the highest initial confidence in the overlapping detection box set is used as the target category of the new detection box.
2. The multi-target detection model integration method as described in claim 1, characterized in that, The determination of the dynamic weights of each target detection model for each target category includes: Obtain the average accuracy of each object detection model for each object category; Two or more object detection models for object categories The maximum value among the average precisions is selected, and this maximum value is taken as the target category. The maximum average accuracy, Index for target category; Each object detection model is assigned to the object category. Average accuracy divided by target category The maximum average accuracy is obtained for each object detection model for the target category. Dynamic weights.
3. The multi-target detection model integration method as described in claim 1, characterized in that, The determination of the dynamic weights of each target detection model for each target category includes: The test data for the current round is obtained by testing two or more target detection models using the test sample set for the current round. Based on the test data of the current round, obtain the dynamic weight increment of each target detection model for each target category in the current round; Calculate the sum of the dynamic weight increment of each target detection model for each target category in the current round and the dynamic weight of each target detection model for each target category in the previous round, and use the sum as the dynamic weight of each target detection model for each target category in the current round.
4. The multi-target detection model integration method as described in claim 1, characterized in that, The set of overlapping detection boxes includes: Obtain the intersection-union (IU) threshold of each unremoved bounding box in the first list with the removed bounding boxes; Calculate the intersection-union ratio (IUR) of each unremoved detection box in the first list with the removed detection boxes. If the IUR is greater than the IUR threshold between the detection box and the removed detection boxes, then add the detection box to the set of overlapping detection boxes.
5. The multi-target detection model integration method as described in claim 4, characterized in that, Obtain the intersection-union (IU) threshold between each unremoved bounding box in the first list and the removed bounding boxes, including: Get the dynamic weight of the target detection model for each detection box that has not been removed in the first output list, and denote it as the first dynamic weight. The dynamic weights of the target detection model for the removed detection boxes to the target type of the removed detection boxes are obtained and denoted as the second dynamic weights. Obtain the absolute value of the difference between the first dynamic weight and the second dynamic weight, and denot it as the absolute value of the weight difference; Multiply the absolute value of the weight difference by the weight difference sensitivity coefficient to obtain the crossover ratio threshold adjustment amount; The cross-union threshold (CUP) between each unremoved detection box in the first list and the removed detection boxes is obtained by subtracting the CUP threshold adjustment from the base CUP threshold.
6. The multi-target detection model integration method as described in claim 1, characterized in that, The initial confidence scores of the detection boxes in the fused overlapping detection box set are used to obtain the weighted confidence scores of the new detection boxes, including: Extract the maximum initial confidence score from the initial confidence scores corresponding to the detection boxes in the overlapping detection box set; Based on the comparison between the initial confidence score of each detection box in the overlapping detection box set and the preset confidence filtering threshold, the indicator function value of each detection box in the overlapping detection box set is obtained; Summing the indicator function values of the overlapping detection boxes in the set yields the accumulated indicator function value; The accumulated value of the indicator function is multiplied by the multi-model collaborative reward coefficient to obtain the accumulated correction value of the indicator function; The sum of the maximum initial confidence and the cumulative correction value of the indicator function is used as the weighted confidence of the new detection box.
7. A multi-target detection model integration device, characterized in that, The method for implementing the multi-target detection model ensemble method according to any one of claims 1-6 includes: The first list construction module obtains information on all detection boxes output by two or more object detection models after processing the image to be detected. The information of each detection box includes the initial confidence score. The first list is constructed and initialized to include all detection boxes. The second list construction module constructs a second list, which is initialized to empty. The weight determination module determines the global importance weight for each target category and the dynamic weight of each target detection model for each target category. The confidence-weighted module utilizes the detection box. Global importance weights of the target category and output detection boxes The target detection model detects the bounding box. Dynamic weights of the target category, weighted detection boxes The initial confidence level is used to obtain the detection box. The weighted confidence level, For the detection box index; The detection box fusion output module repeatedly executes the following steps until the first list is empty. When the first list is empty, it outputs the detection boxes included in the second list: remove the detection box with the highest weighted confidence from the first list and add the removed detection box to the second list; fuse one or more detection boxes in the first list that have an intersection-union ratio that meets the preset overlap condition with the removed detection box to obtain a new detection box and add the new detection box to the first list. The process of fusing one or more detection boxes from the first list whose intersection-union ratio with the removed detection boxes satisfies a preset overlap condition, to obtain a new detection box, includes: Construct an overlapping detection box set, the overlapping detection box set including one or more detection boxes in the first list whose intersection-union ratio with the removed detection boxes satisfies a preset overlap condition, the overlapping detection box set includes K detection boxes; Determine the position weight of each detection box in the set of overlapping detection boxes, wherein the detection box The position weight is the output detection box The target detection model detects the bounding box. Dynamic weight of the target category With detection box initial confidence level The product of the detection box The position weight can be expressed as: ; Represents the detection box Target category; Summing the positional weights of the detection boxes in the overlapping detection box set yields the cumulative sum of positional weights. Based on the position weights of the detection boxes, the position information of the detection boxes in the overlapping detection box set is weighted and summed to obtain the weighted sum of the detection box positions. The position information of the new detection box is obtained by dividing the weighted sum of the detection box positions by the sum of the position weights. The coordinates of the top left corner are used. and the coordinates of the bottom right corner : ； The initial confidence scores of the detection boxes in the overlapping detection box set are merged to obtain the weighted confidence scores of the new detection boxes; The target category of the detection box with the highest initial confidence in the overlapping detection box set is used as the target category of the new detection box.
8. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the steps of the multi-target detection model integration method according to any one of claims 1-6.
9. An electronic device, characterized in that, The electronic device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the multi-target detection model integration method as described in any one of claims 1 to 6.