Missed detection processing method for object detection

Abstract

This invention provides a method for handling missed detections in target detection, comprising: in response to determining the presence of a new target or a lost target based on historical targets in historical frames and the current target in the current frame, adding or removing template information from a template information set; preliminarily updating the target information of historical targets based on the target information of the current target; for each template information, in response to the existence of a corresponding historical target, performing template matching in the current frame based on the template information and the target information of the historical target to obtain the position of the lost target bounding box for supplementing and updating the target information of the historical target; the target information of the current target includes a target identifier and a target bounding box position; the target information of the historical target includes a target identifier, the number of times the target appears, the number of times the template is matched, and the target bounding box position; the template information includes a target identifier and an image.

Description

Technical Field

[0001] This invention relates to the field of target detection technology, and more specifically to a method for handling missed detections in target detection. Background Technology

[0002] Deep learning-based target tracking algorithms can guarantee the continuity and detection rate of targets to a certain extent, but the costs of data annotation and training are very high. Using multi-sensor fusion to complete missed targets increases hardware costs. Therefore, how to achieve automatic target completion after missed detection at a low cost is a pressing problem in target detection. Summary of the Invention

[0003] In view of the above problems, the present invention provides a target detection missed detection processing method that achieves automatic completion function after target detection missed detection at a low cost.

[0004] This invention provides a method for handling missed detections in target detection, comprising: in response to determining the presence of a new target or a lost target based on historical targets in historical frames and the current target in the current frame, adding or removing template information from a template information set; preliminarily updating the target information of historical targets based on the target information of the current target; for each template information, in response to the existence of a corresponding historical target, performing template matching in the current frame based on the template information and the target information of the historical target to obtain the position of the lost target bounding box for supplementing and updating the target information of the historical target; wherein, the target information of the current target includes a target identifier and a target bounding box position; the target information of the historical target includes a target identifier, the number of times the target appears, the number of times the template is matched, and the target bounding box position; and the template information includes a target identifier and an image.

[0005] According to an embodiment of the present invention, adding or removing template information in the template information set includes: in response to a target identifier in the target information of the current frame that is not present in the target information of a historical frame, determining that a new target has appeared; updating the template information set according to the first intersection-union ratio of the new target and historical targets whose template matching count is greater than zero; in response to a target identifier in the target information of a historical frame that is not present in the target information of the current frame, determining that a lost target has appeared; updating the template information set according to the template matching count of the lost target.

[0006] According to an embodiment of the present invention, updating the template information set based on a first intersection-union ratio (IUU) of the new target and historical targets whose template matching count is greater than zero includes: calculating a first IUU based on the target information of the new target and each historical target whose template matching count is greater than zero; determining whether there exists a first IUU greater than a first preset threshold; if yes, deleting the template information corresponding to the historical target in the template information set; if no, not changing the template information set; deleting the historical target whose template matching count is greater than zero; and updating the target information of the new target to the target information of the historical target.

[0007] According to an embodiment of the present invention, updating the template information set based on the number of template matches for the lost target includes: in response to the number of times the lost target appears being greater than a third preset threshold and the number of template matches being zero, storing the target identifier and image of the lost target as template information in the template information set; in response to the number of template matches for the lost target not being zero, determining whether the number of template matches is less than a fourth preset threshold; if yes, not changing the template information set; if no, deleting the template information corresponding to the lost target from the template information set.

[0008] According to an embodiment of the present invention, the target information of historical targets is initially updated based on the target information of the current target, including: in response to the target identifiers of the current target and the historical targets being consistent, the target occurrence count and target box position of the historical targets are updated based on the target information of the current target.

[0009] According to an embodiment of the present invention, template matching is performed in the current frame to obtain the position of a lost target box based on template information and target information of historical targets, in order to complete and update the target information of historical targets. This includes: determining a search region for the position of a lost target box in the current frame based on the position of the target box of historical targets; performing template matching within the search region using a normalized correlation coefficient to obtain the position of the lost target box; calculating a second intersection-union ratio (IUU) between the position of the lost target box and the position of the target box of historical targets; determining whether the second IUU is greater than a second preset threshold; if so, replacing the position of the target box of historical targets with the position of the lost target box; if not, deleting the template information.

[0010] According to an embodiment of the present invention, template matching is performed within a search area using a normalized correlation coefficient to obtain the location of the lost target box, including: using a first matrix within the search area to obtain a first normalized matrix where each element is subtracted from the mean of the first matrix; the first matrix is ​​a matrix selected by traversing within the search area according to the size of the second matrix; using a second matrix of historical targets to obtain a second normalized matrix where each element is subtracted from the mean of the second matrix; calculating the correlation coefficient using the inner product of the first and second normalized matrices and the sum of squares of the elements; and taking the location corresponding to the first normalized matrix with the largest correlation coefficient as the location of the lost target box.

[0011] A second aspect of the present invention provides a target detection missed detection processing apparatus. The apparatus can be used to implement a target detection missed detection processing method. The apparatus includes: a template information module, used to add or remove template information in a template information set in response to determining the presence of a new target or a lost target based on historical targets in historical frames and the current target in the current frame; a preliminary update module, used to preliminarily update the target information of historical targets based on the target information of the current target; and a completion update module, used for each template information, in response to the existence of a corresponding historical target for the template information, to perform template matching in the current frame based on the template information and the target information of the historical target to obtain the position of the lost target box for completing and updating the target information of the historical target; wherein, the target information of the current target includes a target identifier and a target box position; the target information of the historical target includes a target identifier, the number of times the target appears, the number of times the template is matched, and the target box position; and the template information includes a target identifier and an image.

[0012] A third aspect of the present invention provides an electronic device, comprising: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors perform the above-described method for handling missed detections in target detection.

[0013] A fourth aspect of the present invention also provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, cause the processor to perform the aforementioned method for handling missed detections in target detection.

[0014] The target detection missed detection processing method provided by this invention completes the missed detection bounding box by combining existing image information and target location information with a fast matching algorithm. Since no additional deep learning models or sensor hardware are required, it at least partially solves the technical problem of difficulty in completing missed target detection, achieving the technical effect of automatic completion of target detection missed detections at a low cost. Attached Figure Description

[0015] Figure 1 A flowchart illustrating a method for handling missed detections in target detection according to an embodiment of the present invention is shown schematically.

[0016] Figure 2 A flowchart illustrating the block completion logic according to an embodiment of the present invention is shown schematically;

[0017] Figure 3 A schematic diagram illustrating a search area image and a template image according to an embodiment of the present invention is shown.

[0018] Figure 4 This diagram illustrates template matching according to an embodiment of the present invention.

[0019] Figure 5 The illustration shows a schematic diagram of the normalized correlation coefficient calculation according to an embodiment of the present invention;

[0020] Figure 6 This schematic diagram illustrates a structural block diagram of a target detection missed detection processing device according to an embodiment of the present invention;

[0021] Figure 7 A block diagram of an electronic device suitable for implementing a missed detection processing method for target detection according to an embodiment of the present invention is shown schematically. Detailed Implementation

[0022] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the invention. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the invention for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concept of the invention.

[0023] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0024] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0025] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0026] Figure 1 A flowchart illustrating a method for handling missed detections in target detection according to an embodiment of the present invention is shown, as follows: Figure 1As shown, an embodiment of the present invention provides a method for handling missed detections in target detection, comprising: in response to determining the presence of a new target or a lost target based on historical targets in historical frames and the current target in the current frame, adding or removing template information in a template information set; initially updating the target information of historical targets based on the target information of the current target; for each template information, in response to the existence of a corresponding historical target, performing template matching in the current frame based on the template information and the target information of the historical target to obtain the position of the lost target bounding box for supplementing and updating the target information of the historical target; wherein, the target information of the current target includes a target identifier and a target bounding box position; the target information of the historical target includes a target identifier, the number of times the target appears, the number of times the template is matched, and the target bounding box position; the template information includes a target identifier and an image.

[0027] In this embodiment, three containers are defined: `history_objects[[track_id, occurrence_num, match_num, [x1, y1, x2, y2]]...]`, `current_objects[[track_id, [x1, y1, x2, y2]]...]`, and `template_lists[[track_id, template_img]...]`. `history_objects` represents information about all targets in the previous frame (including tracking ID (target identifier), the number of times the target appeared, the number of times the target template matched, and the coordinates of the template). `current_objects` represents all target information in the current frame. `template_lists` represents all template information (template information includes the target's `track_id` and the template image information). Furthermore, `history_objects` contains both detected targets and targets created from templates, while `current_objects` contains only detected targets in the current frame.

[0028] Through the embodiments of the present invention, by using the image information to adjust the template information that may need to be completed, a certain supplementary detection effect is achieved for the missed detections that occur when the target detection algorithm detects the target, making the target box more continuous or extending the number of times the target is detected.

[0029] Based on the above embodiments, adding or removing template information in the template information set includes: in response to a target identifier in the target information of the current frame that is not present in the target information of the historical frame, determining that a new target has appeared; updating the template information set according to the first intersection-union ratio of the new target and the historical target whose template matching count is greater than zero; in response to a target identifier in the target information of the historical frame that is not present in the target information of the current frame, determining that a lost target has appeared; updating the template information set according to the template matching count of the lost target.

[0030] Figure 2 The flowchart illustrating the box-filling logic according to an embodiment of the present invention is shown. `history_objects` is compared with `current_objects` to determine if a new target has appeared; `history_objects` is also compared with `current_objects` to determine if any old targets have been lost. Separate template information update processes are designed for new and old targets to facilitate template matching.

[0031] Through embodiments of the present invention, target changes are determined using target identifiers, and the template information set is adjusted according to the target changes.

[0032] Based on the above embodiments, the template information set is updated according to the first intersection-union ratio (IUU) of the new target and the historical targets whose template matching count is greater than zero. This includes: calculating the first IUU based on the target information of the new target and each historical target whose template matching count is greater than zero; determining whether there is a first IUU greater than a first preset threshold; if so, deleting the template information corresponding to the historical target in the template information set; if not, not changing the template information set; deleting the historical target whose template matching count is greater than zero; and updating the target information of the new target to the target information of the historical target.

[0033] In this embodiment, if a new target appears, the new target is compared with targets in history_objects where match_num>0 (match_num>0 means the target was generated by template matching, not detected by target detection) to calculate IoU (Intersection over Union). If IoU>0.8, the template corresponding to the target is deleted (i.e., if the track_id of the target in history_objects is known, the corresponding template can be found in template_lists and deleted), and the target in history_objects is also deleted, and the new target is added.

[0034] It should be noted that the Intersection over Union (IoU) is the ratio of the intersection of two regions to the union of the two regions. For example, target box A: top left corner coordinates (0, 0), bottom right corner coordinates (6, 8); target box B: top left corner coordinates (3, 2), bottom right corner coordinates (9, 10). The calculation process is as follows: S1, calculate the intersection. The top left corner coordinate of the intersection is the larger value of the top left corner coordinates of the two boxes, (max(0, 3), max(0, 2)) = (3, 2), and the bottom right corner coordinate is the smaller value of the bottom right corner coordinates of the two boxes (min(6, 9), min(8, 10)) = (6, 8). Therefore, the intersection area is (6-3) × (8-2) = 3 × 6 = 18. S2, calculate the union: the area of ​​target box A is (6-0) × (8-0) = 48, and the area of ​​target box B is (9-3) × (10-2) = 48. The union area is 48 + 48 - 18 = 78. S3, calculate IoU: IoU=18 / 78=0.2308.

[0035] According to the embodiments of the present invention, if the intersection-union ratio of the new target and the historical target is greater than a preset threshold, the new target detected has a higher priority than the historical target obtained by template matching because the original historical target was completed by template matching. Therefore, the corresponding template is deleted to prevent the template from being used to generate template targets in this round.

[0036] Based on the above embodiments, the template information set is updated according to the number of template matches for the lost target, including: in response to the number of times the lost target appears being greater than a third preset threshold and the number of template matches being zero, the target identifier and image of the lost target are stored as template information in the template information set; in response to the number of template matches for the lost target not being zero, it is determined whether the number of template matches is less than a fourth preset threshold; if yes, the template information set is not changed; if no, the template information corresponding to the lost target in the template information set is deleted.

[0037] In this embodiment, if an old target is lost, if the occurrence_num of the old target in history_objects is greater than 5 (meaning it is not a false positive) and match_num is equal to 0 (meaning it is not a target that matches the template, i.e., there is no template), then the track_id of the lost target is added to template_lists, and the area of ​​the target's coordinates [x1, y1, x2, y2] in history_objects is cropped from the previous frame and stored as the template_img. If match_num is greater than 0 and match_num is less than 15, it means that the target has a template, and the number of times the template has been created is less than 15, then the next stage of template matching is prepared. If match_num is greater than 15, then the corresponding template needs to be deleted to prevent it from performing template matching again, so as to avoid excessive completion and distortion of the target bounding box.

[0038] In embodiments of the present invention, the number of times a target appears is used to determine whether it is a false positive target; the number of template matches is used to determine whether it is a template-matched target with too many completion attempts. Based on the number of target appearances and the number of template matches, it is determined whether it is a missing target requiring further completion.

[0039] Based on the above embodiments, the target information of historical targets is initially updated according to the target information of the current target, including: in response to the target identifiers of the current target and the historical targets being consistent, the target occurrence count and target box position of the historical targets are updated according to the target information of the current target.

[0040] In this embodiment, history_objects is compared with current_objects in track_id, the coordinates of the corresponding target in current_objects are replaced with those in history_objects, and the occurrence_num of the corresponding target in history_objects is incremented by 1, and match_num is set to 0.

[0041] Through the embodiments of the present invention, the detected target information is updated to the historical targets, thereby completing the continuous updating of the actually detected targets.

[0042] Based on the above embodiments, according to the template information and the target information of historical targets, template matching is performed in the current frame to obtain the position of the lost target box for supplementing and updating the target information of historical targets. This includes: determining the search area for the position of the lost target box in the current frame based on the target box position of historical targets; performing template matching within the search area using a normalized correlation coefficient to obtain the position of the lost target box; calculating a second intersection-union ratio (IUU) between the position of the lost target box and the target box position of historical targets; determining whether the second IUU is greater than a second preset threshold; if so, replacing the target box position of historical targets with the position of the lost target box; if not, deleting the template information.

[0043] In this embodiment, the track_id of all templates in template_lists is traversed to match the target in history_objects. Based on the region of the target in history_objects, the region ((x1-5, y1-5), (x2+5, y2+5)) is extracted on the current image and matched with the template that matches the track_id (using the normalized correlation coefficient matching method). The new coordinates [x11, y11, x22, y22] are obtained by matching and the old [x1, y1, x2, y2] are calculated. If IoU > 0.6, it means that a match has been made. The new region of the target is replaced with the old region, and the occurrence_num and match_num of the target are increased by 1. If IoU < 0.6, it means that a match has not been made, and the template corresponding to the target is deleted.

[0044] Through the embodiments of the present invention, since the historical target box position of the target to be completed can be directly matched according to the target identifier, the search range for completion can be determined and the position of the lost target box can be determined within it.

[0045] Based on the above embodiments, template matching is performed within the search area using a normalized correlation coefficient to obtain the location of the lost target box. This includes: using a first matrix within the search area to obtain a first normalized matrix where each element is subtracted from the mean of the first matrix; the first matrix is ​​a matrix selected by traversing within the search area according to the size of the second matrix; using the second matrix of historical targets to obtain a second normalized matrix where each element is subtracted from the mean of the second matrix; calculating the correlation coefficient using the inner product of the first and second normalized matrices and the sum of squares of the elements; and taking the location corresponding to the first normalized matrix with the largest correlation coefficient as the location of the lost target box.

[0046] In this embodiment, with Figure 3Taking the matrix as an example, the image to be matched is I.Size(5,5), and the template image is T.Size(3,3). When traversing image I with the search box of Size(3,3), it needs to be traversed 3 times in the x-direction and 3 times in the y-direction. Then, let I represent the image to be matched. x and y represent the coordinates of the top-left corner element of the current search box in the matrix I. x' and y' represent the element coordinates of the matrix of T and the matrix of I bounded by the search box.

[0047] like Figure 4 As shown, the normalized correlation coefficient matching method is used as an example. At this time, the coordinates of the top-left corner of the search box are (x, y) = (0, 0). For example... Figure 5 The search box matrix and the template matrix are compared. First, the mean is subtracted. Second, the inner product of the two matrices and the product of the squares of the sums of the squares of their elements are calculated. Finally, the formula for the normalized correlation coefficient matching method is summarized as follows:

[0048] .

[0049] Through embodiments of the present invention, the normalized correlation coefficient matching method yields: a perfect match results in a value of 1, a perfectly negative correlation match results in a value of -1, and a perfect mismatch results in a value of 0. The correlation coefficient is used to match the most suitable location for the lost bounding box.

[0050] Based on the above-described method for handling missed detections in target detection, this invention also provides a device for handling missed detections in target detection. The following will be combined with... Figure 6 The device is described in detail.

[0051] Figure 6 The schematic diagram illustrates a structural block diagram of a target detection missed detection processing device according to an embodiment of the present invention.

[0052] like Figure 6 As shown, the target detection missed detection processing device of this embodiment can be used to implement the target detection missed detection processing method. The device includes: a template information module, used to add or remove template information in the template information set in response to determining the presence of a new target or a lost target based on historical targets in historical frames and the current target in the current frame; a preliminary update module, used to preliminarily update the target information of historical targets based on the target information of the current target; and a completion update module, used to perform template matching in the current frame to obtain the position of the lost target box for completing and updating the target information of historical targets for each template information in response to the existence of a corresponding historical target for the template information; wherein, the target information of the current target includes a target identifier and a target box position; the target information of the historical target includes a target identifier, the number of times the target appears, the number of times the template is matched, and the target box position; and the template information includes a target identifier and an image.

[0053] Figure 7A block diagram of an electronic device suitable for implementing a missed detection processing method for target detection according to an embodiment of the present invention is shown schematically.

[0054] like Figure 7 As shown, an electronic device 700 according to an embodiment of the present invention includes a processor 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage portion 708 into a random access memory (RAM) 703. The processor 701 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 701 may also include onboard memory for caching purposes. The processor 701 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0055] RAM 703 stores various programs and data required for the operation of electronic device 700. Processor 701, ROM 702, and RAM 703 are interconnected via bus 704. Processor 701 executes various operations of the method flow according to embodiments of the present invention by executing programs in ROM 702 and / or RAM 703. It should be noted that the programs may also be stored in one or more memories other than ROM 702 and RAM 703. Processor 701 may also execute various operations of the method flow according to embodiments of the present invention by executing programs stored in said one or more memories.

[0056] According to an embodiment of the present invention, the electronic device 700 may further include an input / output (I / O) interface 705, which is also connected to a bus 704. The electronic device 700 may also include one or more of the following components connected to the I / O interface 705: an input section 706 including a keyboard, mouse, etc.; an output section 707 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a LAN card, modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I / O interface 705 as needed. A removable medium 711, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 710 as needed so that computer programs read from it can be installed into the storage section 708 as needed.

[0057] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiments of the present invention.

[0058] According to embodiments of the present invention, a computer-readable storage medium may be a non-volatile computer-readable storage medium, such as including, but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of the present invention, a computer-readable storage medium may include ROM 702 and / or RAM 703 and / or one or more memories other than ROM 702 and RAM 703 described above.

[0059] Embodiments of the present invention also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code is used to cause the computer system to implement the methods provided in the embodiments of the present invention.

[0060] When the computer program is executed by the processor 701, it performs the functions defined in the system / apparatus of this invention. According to embodiments of the invention, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0061] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and may be downloaded and installed via the communication section 709, and / or installed from a removable medium 711. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0062] In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 709, and / or installed from the removable medium 711. When the computer program is executed by the processor 701, it performs the functions defined in the system of this embodiment of the invention. According to embodiments of the invention, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0063] According to embodiments of the present invention, program code for executing the computer programs provided in the embodiments of the present invention can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can be executed entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0064] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0065] Those skilled in the art will understand that the features described in the various embodiments and / or claims of the present invention can be combined or combined in various ways, even if such combinations or combinations are not explicitly described in the present invention. In particular, the features described in the various embodiments and / or claims of the present invention can be combined or combined in various ways without departing from the spirit and teachings of the present invention. All such combinations and / or combinations fall within the scope of the present invention.

[0066] The embodiments of the present invention have been described above. However, these embodiments are merely illustrative and not intended to limit the scope of the invention. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. The scope of the invention is defined by the appended claims and their equivalents. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of the invention, and all such substitutions and modifications should fall within the scope of the invention.

Claims

1. A method for handling missed detections in target detection, characterized in that, include: In response to determining whether a new target has appeared or a target has been lost based on the historical targets in historical frames and the current target in the current frame, the template information in the template information set is added or removed. The template information in the template information set for adding or removing template information includes: in response to a target identifier in the target information of the current frame that is not present in the target information of a historical frame, it is determined that a new target has appeared; the template information set is updated according to the first intersection-union ratio of the new target and historical targets whose template matching count is greater than zero; in response to a target identifier in the target information of a historical frame that is not present in the target information of the current frame, it is determined that a lost target has appeared; the template information set is updated according to the template matching count of the lost target. Based on the target information of the current target, the target information of the historical target is initially updated; For each template information, in response to the existence of a corresponding historical target, template matching is performed in the current frame based on the template information and the target information of the historical target to obtain the position of the missing target box for filling and updating the target information of the historical target; The target information of the current target includes the target identifier and the target bounding box position; the target information of the historical target includes the target identifier, the number of times the target appears, the number of times the template is matched, and the target bounding box position; the template information includes the target identifier and the image.

2. The method according to claim 1, wherein, The step of updating the template information set based on the first intersection-union ratio of the new target and historical targets with a template matching count greater than zero includes: Calculate the first intersection-union ratio based on the target information of the new target and each historical target whose template matching count is greater than zero; Determine whether the first intersection-union ratio is greater than the first preset threshold; If so, delete the template information corresponding to the historical target in the template information set; If not, the template information set is not changed; historical targets with a template matching count greater than zero are deleted; the target information of the new target is updated into the target information of the historical targets.

3. The method according to claim 1, wherein, The step of updating the template information set based on the number of template matches for the lost target includes: If the number of times the target of the lost target appears is greater than the third preset threshold and the number of template matching times is zero, then the target identifier and image of the lost target are stored as template information in the template information set. If the number of template matches for a lost target is not zero, it is determined whether the number of template matches is less than a fourth preset threshold. If yes, the template information set is not changed; if no, the template information corresponding to the lost target in the template information set is deleted.

4. The method according to claim 1, wherein, The step of initially updating the target information of the historical target based on the target information of the current target includes: If the target identifier of the current target is consistent with that of the historical target, then the target occurrence count and target bounding box position of the historical target are updated according to the target information of the current target.

5. The method according to claim 1, wherein, The step of performing template matching in the current frame to obtain the position of the missing target box based on the template information and the target information of the historical target, in order to complete and update the target information of the historical target, includes: Based on the target bounding box positions of historical targets, determine the search area at the location of the lost target bounding box in the current frame; Template matching is performed within the search area using normalized correlation coefficients to obtain the location of the missing target box; Calculate the second intersection-union ratio (IoU) between the location of the lost target box and the location of the historical target's target box; Determine whether the second crossover ratio is greater than the second preset threshold; If so, the target box position of the historical target is replaced with the lost target box position; If not, delete the template information.

6. The method according to claim 5, wherein, The step of performing template matching within the search area using a normalized correlation coefficient to obtain the location of the lost target box includes: Using the first matrix within the search area, a first normalized matrix is ​​obtained by subtracting the mean of the first matrix from each element; the first matrix is ​​a matrix selected by traversing the search area according to the size of the second matrix. Using the second matrix of historical targets, we obtain a second normalized matrix by subtracting the mean of the second matrix from each element; The correlation coefficient is calculated using the inner product of the first and second normalized matrices and the sum of squares of their elements. The position corresponding to the first normalized matrix with the largest correlation coefficient is taken as the position of the lost target box.

7. A target detection missed detection processing device, characterized in that, The apparatus is capable of implementing the method as described in claim 1, and the apparatus comprises: The template information module is used to add or remove template information from the template information set in response to the determination of the presence or loss of a new target based on historical targets in historical frames and the current target in the current frame. The addition or removal of template information from the template information set includes: determining that a new target has appeared if the target information in the current frame contains a target identifier that is not present in the target information of historical frames; updating the template information set based on the first intersection-union ratio (IUU) of the new target with historical targets whose template matching count is greater than zero; and determining that a lost target has appeared if the target information in historical frames contains a target identifier that is not present in the target information of the current frame; updating the template information set based on the template matching count of the lost target. The initial update module is used to initially update the target information of the historical target based on the target information of the current target; The completion and update module is used to perform template matching in the current frame to obtain the position of the missing target box for each template information in response to the existence of a corresponding historical target for the template information and the target information of the historical target, so as to complete and update the target information of the historical target. The target information of the current target includes the target identifier and the target bounding box position; the target information of the historical target includes the target identifier, the number of times the target appears, the number of times the template is matched, and the target bounding box position; the template information includes the target identifier and the image.

8. An electronic device, comprising: One or more processors; Storage device for storing one or more programs. Wherein, when the one or more programs are executed by the one or more processors, the one or more processors perform the method according to claim 1.

9. A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, cause the processor to perform the method according to claim 1.

Images