Depth pseudo label enhanced end-to-end dense scene pedestrian detection method

Abstract

The invention discloses a depth pseudo label enhanced end-to-end dense scene pedestrian detection method, and belongs to the technical field of pattern recognition and computer vision, and the method comprises the steps: adding a depth pseudo label in pedestrian image data, and constructing a pedestrian image data set in a dense scene; the method comprises the following steps: adding an auxiliary detection head of a multi-scale detection network based on region suggestion in a detection network based on query, and constructing an end-to-end detection model for forming more positive examples in a dense scene; training the end-to-end detection model by using the pedestrian image data set; and realizing pedestrian detection in a dense scene by using the trained end-to-end detection model. By adopting the pedestrian detection method provided by the invention, higher pedestrian detection precision can be realized in a densely shielded scene.

Description

Technical Field

[0001] The present invention relates to the field of pattern recognition and computer vision technology, and in particular to an end-to-end dense scene pedestrian detection method enhanced by deep pseudo-labels. Background Art

[0002] With the rapid development of intelligent technology, all walks of life have an increasingly strong demand for image-related intelligent technology. As a key technology for understanding images, computer vision has developed multiple research directions such as image classification, target detection, and target tracking. Among them, pedestrian detection, as an important subtask of target detection, aims to accurately give a detection box containing the position and size of a complete pedestrian in a two-dimensional image through classification and regression. Pedestrian detection is not only a basic task for many practical applications, but also plays a key role in scenarios such as autonomous driving and video surveillance.

[0003] In autonomous driving scenarios, pedestrian detection frames are the starting point for tracking pedestrians and predicting their trajectories, which in turn affects the vehicle's driving strategy planning. In the field of video surveillance, accurate pedestrian detection frames are the key elements for achieving tasks such as cross-camera feature matching and person re-identification.

[0004] Although pedestrian detection technology has been widely studied, it still faces many challenges due to the changes in background, lighting and appearance features in visual images. Especially in occluded scenes, the features of complete pedestrians are destroyed, making pedestrian detection more difficult. Occluded pedestrian detection mainly includes pedestrian detection under object occlusion and mutual occlusion between pedestrians (i.e. detection in dense scenes). In the case of object occlusion, the visible body parts of pedestrians are different and the features are not obvious, resulting in large errors in the model's estimation of pedestrian position and size. Mutual occlusion between pedestrians makes the position and size of each pedestrian blurred, making it particularly difficult to distinguish and retain the correct detection frame in the post-processing stage.

[0005] At present, the mainstream pedestrian detection solutions in dense scenes mainly include region proposal-based detection methods and query-based end-to-end detection methods.

[0006] Detection methods based on region proposals, such as Faster R-CNN, divide the detection task into two stages. The first stage generates several candidate boxes according to certain rules, and the second stage processes the candidate boxes in the previous stage and performs classification and regression on the boundaries of the detection boxes. This type of method performs well in relatively sparse scenes, but is prone to missed detection in dense pedestrian scenes.

[0007] Query-based detection methods use query mechanisms to accurately locate targets by directly treating target detection as a sequence-to-sequence task. DETR introduces learnable queries to represent objects and directly performs a single prediction for each instance. Although query-based detection methods have improved accuracy, they usually adopt a one-to-one detection strategy, and the number of detection boxes generated in dense scenes is relatively small, which cannot effectively cover all targets.

[0008] In summary, pedestrian detection faces many challenges in complex environments. Especially in two-dimensional images, since the camera can only capture scene information from a single angle, the overlap or occlusion between pedestrians significantly increases the difficulty of detection. Relying only on two-dimensional image information, it is difficult for the model to accurately distinguish partially occluded pedestrians, resulting in an increase in the missed detection rate, which seriously affects the detection performance. At the same time, the existing query-based network performs well in pedestrian detection, and can better capture target features through the query mechanism to achieve accurate positioning of pedestrians. However, the current query-based network usually adopts a one-to-one detection method, resulting in a small number of generated detection frames. Although this method works well in scenes with a small number of targets, in environments with densely distributed pedestrians, due to the large number of targets that need to be detected, the number of detection frames is not enough to cover all targets, thereby increasing the risk of missed detection. Therefore, the detection effect of existing technologies in dense scenes is not ideal. Summary of the invention

[0009] The present invention provides a deep pseudo-label enhanced end-to-end dense scene pedestrian detection method to solve the technical problem that the detection effect of the prior art is not ideal in an environment with densely distributed pedestrians.

[0010] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0011] On the one hand, the present invention provides a deep pseudo-label enhanced end-to-end dense scene pedestrian detection method, the deep pseudo-label enhanced end-to-end dense scene pedestrian detection method comprising:

[0012] Add deep pseudo labels to pedestrian image data to build a pedestrian image dataset in dense scenes;

[0013] Add a multi-scale region proposal-based detection network auxiliary detection head to the query-based detection network to build an end-to-end detection model that generates more positive samples in dense scenes.

[0014] Using the pedestrian image dataset to train the end-to-end detection model;

[0015] Leverage the trained end-to-end detection model to achieve pedestrian detection in dense scenes.

[0016] Furthermore, adding a deep pseudo label to the pedestrian image data includes:

[0017] The data augmentation method is adopted to crop pedestrians through instance segmentation technology and paste them into the original pedestrian image data to form pedestrian images in dense scenes. According to the order in which the pedestrians are pasted, the occlusion depth information in this dense scene is inferred and used as depth pseudo-labels.

[0018] Furthermore, the calculation formula of the occlusion depth information is as follows:

[0019]

[0020] Among them, i Occlusion depth information, indicating the degree to which an object is covered by other objects; ovl(b i ,b j ) is object b i Object b j The area covered; S(·) represents the area of ​​the region.

[0021] Furthermore, an auxiliary detection head of a multi-scale region proposal-based detection network is added to the query-based detection network to build an end-to-end detection model that generates more positive samples in dense scenes, including:

[0022] A region proposal-based branch is introduced into a query-based detection network. At the same time, a deep pseudo-labeling technology is introduced into the region proposal-based branch to implement a one-to-many label assignment strategy and train multiple parallel auxiliary detection heads; wherein the auxiliary detection heads are discarded after the training is completed.

[0023] Furthermore, the end-to-end detection model uses Deformable DETR as a benchmark network.

[0024] Furthermore, the end-to-end detection model uses Faster R-CNN as a one-to-many auxiliary detection head.

[0025] Furthermore, the loss function of the end-to-end detection model is designed to be a weighted sum of loss functions of multiple subtasks.

[0026] Furthermore, the calculation formula of the loss function of the end-to-end detection model is as follows:

[0027] L bbox =L query +L proposal

[0028]

[0029]

[0030] Among them, L bbox represents the loss function of the end-to-end detection model; L query represents the loss function of the query-based detection network, including classification loss, bounding box regression loss, and deep pseudo-label loss, and They represent the classification loss, bounding box regression loss, and deep pseudo-label loss corresponding to the query-based detection network respectively; L proposal represents the loss function of the branch based on region proposal, including classification loss, bounding box regression loss, and deep pseudo label loss, and They represent the classification loss, bounding box regression loss, and deep pseudo-label loss corresponding to the branch based on region proposal, respectively.

[0031] On the other hand, the present invention further provides an electronic device, comprising a processor and a memory; wherein the memory stores at least one instruction, and the instruction is loaded and executed by the processor to implement the above method.

[0032] In yet another aspect, the present invention further provides a computer-readable storage medium, wherein at least one instruction is stored in the storage medium, and the instruction is loaded and executed by a processor to implement the above method.

[0033] The beneficial effects brought about by the technical solution provided by the present invention include at least:

[0034] The present invention introduces a branch based on region proposals in the query-based detection network to enhance the detection capability of the model in dense scenes, making the model more feasible and flexible in practical applications. At the same time, deep pseudo-labeling technology is introduced in this branch to further increase the richness and accuracy of information, thereby improving the accuracy and robustness of pedestrian detection in dense scenes and effectively reducing missed detection in dense scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0036] Figure 1 It is a schematic diagram of the execution flow of the deep pseudo-label enhanced end-to-end dense scene pedestrian detection method provided by an embodiment of the present invention;

[0037] Figure 2 is a framework diagram of an end-to-end model provided by an embodiment of the present invention;

[0038] Figure 3 It is a system block diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0039] To make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0040] First of all, it should be noted that in the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplary" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "exemplarily" is intended to present the concept in a concrete way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either of the two.

[0041] First embodiment

[0042] This embodiment provides a deep pseudo-label enhanced end-to-end dense scene pedestrian detection method, which aims to combine the query-based detection network with the region proposal-based branch and enhance the occluded pedestrian information of the image to improve the accuracy of dense scene pedestrian detection. The method can be implemented by an electronic device, which can be a terminal or a server. The execution process of the method is as follows Figure 1 As shown, the following steps are included:

[0043] S1, add deep pseudo labels to pedestrian image data to build a pedestrian image dataset in dense scenes;

[0044] S2, adds a multi-scale auxiliary detection head based on region proposal detection network to the query-based detection network, and builds an end-to-end detection model that generates more positive samples in dense scenes;

[0045] S3, training the end-to-end detection model using the pedestrian image dataset;

[0046] S4, uses the trained end-to-end detection model to achieve pedestrian detection in dense scenes.

[0047] Specifically, this embodiment introduces a region proposal-based branch into the query-based detection network to enhance the detection capability of the model in dense scenes. At the same time, deep pseudo-labeling technology is introduced into this branch to further increase the richness and accuracy of information, thereby effectively reducing missed detections in dense scenes.

[0048] To implement the above technical solution, this embodiment adopts a data augmentation method to crop pedestrians through instance segmentation technology and paste them into the training set to enhance the robustness of the model and its adaptability to dense scenes. During the pasting process, due to the order in which the pedestrians are pasted, we can infer the depth information in this occluded dense scene and generate an occluded depth pseudo-label based on it. This pseudo-label not only provides additional supervision signals for the model, but also effectively helps it learn the characteristic performance of pedestrians under different occlusion conditions. Specifically, the calculation formula for occlusion depth is as follows:

[0049]

[0050] Among them, od (Overlay Depth) represents the occlusion depth pseudo-label of the augmented pedestrian, which indicates the degree to which an object is covered by other objects; ovl (b i ,b j ) is object b i Object b j The area covered; S(·) represents the area of ​​the region. For any object b in the image i , there exists a set of objects O i , these objects are i When the pedestrian is not blocked, the initial depth value is 1.0. When the newly pasted pedestrian overlaps with the pedestrian, the depth of the pedestrian will be changed dynamically.

[0051] Afterwards, this embodiment introduces the one-to-many label assignment strategy of Faster R-CNN in the query-based network, and trains multiple parallel auxiliary heads to enhance the learning ability of the encoder in the end-to-end detector. The design of these auxiliary heads enables the model to process complex feature representations more effectively during training, thereby improving the accuracy and robustness of detection. Specifically, these parallel auxiliary heads not only provide diverse supervision signals, but also perform additional customized positive queries by extracting positive sample coordinates. This approach aims to improve the training efficiency of positive samples in the decoder, enable the model to converge faster, and optimize the positioning ability of the detection box. In this way, the model is able to learn more detailed target features, thereby achieving higher detection accuracy in dense scenes. Specifically, the one-to-one detection network in the end-to-end detection model constructed in this embodiment selects Deformable DETR as the baseline network, and selects Faster R-CNN as the one-to-many auxiliary detection head. The structure of the end-to-end detection model is as follows Figure 2 shown.

[0052] It should be noted that in the method of this embodiment, the auxiliary head is discarded after the training is completed. This means that the solution of this embodiment does not introduce additional parameters or computational costs to the original detector. This not only ensures the efficiency of the model, but also simplifies the reasoning process. In addition, the method of this embodiment does not need to re-execute the manually designed non-maximum suppression (NMS) process, thereby avoiding the computational bottlenecks and delays that may occur in traditional methods. This optimization enables the detector of this embodiment to have faster reasoning speed and lower resource consumption while maintaining high performance.

[0053] In summary, through the introduction of parallel auxiliary heads and the extraction of customized positive queries, the method of this embodiment effectively improves the overall performance of the end-to-end detector while maintaining its computational efficiency, making the model more feasible and flexible in practical applications.

[0054] Furthermore, the loss calculation formula of the network after combining the query-based network and the region-based proposal can be designed as the weighted sum of the loss functions of multiple subtasks. Specifically, the loss calculation formula is as follows:

[0055] L bbox =L query +L proposal

[0056]

[0057]

[0058] Among them, L bbox represents the loss function of the end-to-end detection model; L query represents the loss function of the query-based detection network, including classification loss, bounding box regression loss, and deep pseudo-label loss, and They represent the classification loss, bounding box regression loss, and deep pseudo-label loss corresponding to the query-based detection network respectively; L proposal represents the loss function of the branch based on region proposal, including classification loss, bounding box regression loss, and deep pseudo label loss, and They represent the classification loss, bounding box regression loss, and deep pseudo-label loss corresponding to the branch based on region proposal, respectively.

[0059] The model of this embodiment is applied to the Crowdhuman dataset, and the final detection effect is shown in Table 1. Among them, the Crowdhuman dataset is often used for pedestrian detection in dense scenes. It contains 15,000 images, of which there are 470K instances in the training set and the validation set. The human density is significantly higher than Cityperson, reaching 22.6 people per image, and there are 2.4 paired overlapping instances (IoU greater than 0.5). It can be seen from Table 1 that the model of this embodiment produces better results in dense scene pedestrian detection on the Crowdhuman dataset.

[0060] In the test phase, we remove the one-to-many branch based on region proposals and only use the one-to-one detection branch based on queries, so that the entire network still has end-to-end superiority. For dense scenes, we increase the maximum number of instances generated for each image in the decoder to 200. As can be seen from Table 1, the model of this embodiment produces better results in dense scene pedestrian detection on the Crowdhuman dataset.

[0061] Table 1 Detection effect of the model in this embodiment on the Crowdhuman dataset

[0062]

[0063] Next, the superiority of the present invention over the prior art is verified by comparative experiments; this experiment consists of two parts, the first part is to generate deep pseudo labels by data augmentation, and the second part is to add one-to-many branches to the one-to-one query-based network structure, and to increase the depth in the loss function calculation to provide more information for the detection model. The impact of each part on the results is shown in Table 2:

[0064] Table 2 Detection effects of different models

[0065]

[0066]

[0067] Among them, for the one-to-one detection network, this embodiment selects Deformable DETR as the baseline network, and this embodiment selects Faster R-CNN as the one-to-many auxiliary detection head. Since this embodiment uses data augmentation in generating pseudo labels, this embodiment also compares the comparative experiments of adding only data augmentation on the basis of the joint one-to-one and one-to-many detection networks and using data augmentation plus deep pseudo labels.

[0068] In summary, this embodiment provides an end-to-end dense scene pedestrian detection method enhanced by deep pseudo-labeling, which introduces a branch based on region proposals in the query-based detection network to enhance the detection ability of the model in dense scenes, making the model more feasible and flexible in practical applications. At the same time, deep pseudo-labeling technology is introduced in this branch to further increase the richness and accuracy of information, thereby improving the accuracy and robustness of pedestrian detection in dense scenes and effectively reducing missed detections in dense scenes.

[0069] Second embodiment

[0070] This embodiment provides an electronic device, such as Figure 3 As shown, the electronic device includes: a processor and a memory; wherein the processor and the memory can be connected via a communication bus; the memory stores at least one instruction, and the instruction is loaded and executed by the processor to implement the method of the first embodiment. In addition, the electronic device may also include a transceiver, the processor and the transceiver can be connected via a communication bus, and the transceiver is used to communicate with other devices.

[0071] Next, combine Figure 3 The following is a detailed introduction to the various components of the electronic device:

[0072] Among them, the processor is the control center of the electronic device, and the electronic device may include multiple processors, each of which may be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). The processor here may be a processor or a general term for multiple processing elements. For example, the processor is one or more central processing units (CPUs), or other general-purpose processors, application specific integrated circuits (ASICs), or one or more integrated circuits configured to implement an embodiment of the present invention, such as one or more microprocessors (digital signal processors, DSPs), or one or more field programmable gate arrays (field programmable gate arrays, FPGAs), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor may execute various functions of the electronic device by running or executing software programs stored in the memory and calling data stored in the memory.

[0073] In a specific implementation, as an embodiment, the processor may include one or more CPUs, such as Figure 3 The CPU0 and CPU1 shown in the figure are, of course, only exemplary.

[0074] The memory is used to store the software program for executing the solution of the present invention, and the execution is controlled by the processor. The specific implementation method can refer to the above method embodiment and will not be repeated here.

[0075] Optionally, the memory may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory may be integrated with the processor or exist independently and accessed through the interface circuit ( Figure 3 (not shown) is coupled to the processor, which is not specifically limited in this embodiment of the present invention.

[0076] The transceiver may include a receiver and a transmitter ( Figure 3 The receiver is used to implement the receiving function, and the transmitter is used to implement the sending function. The transceiver can be integrated with the processor or exist independently and communicate with the electronic device through the interface circuit ( Figure 3 (not shown) is coupled to the processor, which is not specifically limited in this embodiment of the present invention.

[0077] In addition, it should be noted that Figure 3 The structure of the electronic device shown in the figure does not constitute a limitation on the device, and the actual device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. In addition, the technical effects achieved by the electronic device when executing the method of the first embodiment above can refer to the technical effects described in the first embodiment above, so they are not repeated here.

[0078] Third embodiment

[0079] This embodiment provides a computer-readable storage medium, which stores at least one instruction, and the instruction is loaded and executed by a processor to implement the method of the first embodiment. The computer-readable storage medium may be a ROM, a random access memory, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc. The instructions stored therein may be loaded by a processor in a terminal to execute the method.

[0080] In addition, it should be noted that the present invention can be provided as a method, an apparatus or a computer program product. Therefore, the embodiment of the present invention can be in the form of a full or partial hardware embodiment, a full or partial software embodiment or an embodiment combining software and hardware. Moreover, when implemented using software, the embodiment of the present invention can be in the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program codes. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center containing one or more available media sets. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a DVD), or a semiconductor medium. The semiconductor medium may be a solid state hard disk.

[0081] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0082] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device, so that a series of operation steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable terminal device provide for implementing the process in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0083] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. The terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of more restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements. In addition, the term "and / or" is only an association relationship describing the associated objects, indicating that there can be three relationships, for example, A and / or B, which can represent: A exists alone, A and B exist at the same time, and B exists alone, wherein A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding. "At least one" means one or more, and "plurality" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can be represented by: a, b, c, ab, ac, bc or abc, where a, b, c can be single or plural.

[0084] In addition, it can be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0085] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0086] In several embodiments provided by the present invention, it should be understood that the disclosed equipment, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of functional modules / units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point, the coupling or direct coupling or communication connection between each other shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms. The unit described as a separate component may or may not be physically separated, and the component displayed as a unit may or may not be a physical unit, that is, it may be located in one place, or it may be distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, each functional unit in each embodiment of the present invention can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit.

[0087] If the method is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0088] Finally, it should be noted that the above is only a preferred embodiment of the present invention. It should be pointed out that although the preferred embodiment of the present invention has been described, for ordinary technicians in this technical field, once the basic creative concept of the present invention is known, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the protection scope of the present invention. Therefore, the attached claims are intended to be interpreted as including the preferred embodiment and all changes and modifications that fall within the scope of the embodiments of the present invention.

Claims

1. A deep pseudo-label enhanced end-to-end dense scene pedestrian detection method, characterized in that: The deep pseudo-label enhanced end-to-end dense scene pedestrian detection method comprises: Add deep pseudo labels to pedestrian image data to build a pedestrian image dataset in dense scenes; Add a multi-scale region proposal-based detection network auxiliary detection head to the query-based detection network to build an end-to-end detection model that generates more positive samples in dense scenes. Using the pedestrian image dataset to train the end-to-end detection model; Leverage the trained end-to-end detection model to achieve pedestrian detection in dense scenes.

2. The deep pseudo-label enhanced end-to-end dense scene pedestrian detection method according to claim 1, characterized in that: The adding of deep pseudo labels to pedestrian image data comprises: The data augmentation method is adopted to crop pedestrians through instance segmentation technology and paste them into the original pedestrian image data to form pedestrian images in dense scenes. According to the order in which the pedestrians are pasted, the occlusion depth information in this dense scene is inferred and used as the depth pseudo-label.

3. The deep pseudo-label enhanced end-to-end dense scene pedestrian detection method according to claim 2, characterized in that: The calculation formula of the occlusion depth information is as follows: Among them, i Occlusion depth information, indicating the degree to which an object is covered by other objects; ovl(b i ,b j ) is object b i Object b j The area covered; S(·) represents the area of ​​the region.

4. The deep pseudo-label enhanced end-to-end dense scene pedestrian detection method according to claim 1, characterized in that: The method adds a multi-scale auxiliary detection head of a region proposal-based detection network to the query-based detection network to construct an end-to-end detection model that generates more positive samples in dense scenes, including: A region proposal-based branch is introduced into a query-based detection network. At the same time, a deep pseudo-labeling technology is introduced into the region proposal-based branch to implement a one-to-many label assignment strategy and train multiple parallel auxiliary detection heads; wherein the auxiliary detection heads are discarded after the training is completed.

5. The deep pseudo-label enhanced end-to-end dense scene pedestrian detection method according to claim 4, characterized in that: The end-to-end detection model uses Deformable DETR as the baseline network.

6. The deep pseudo-label enhanced end-to-end dense scene pedestrian detection method according to claim 5, characterized in that: The end-to-end detection model uses Faster R-CNN as a one-to-many auxiliary detection head.

7. The deep pseudo-label enhanced end-to-end dense scene pedestrian detection method according to claim 6, characterized in that: The loss function of the end-to-end detection model is designed to be a weighted sum of loss functions of multiple subtasks.

8. The deep pseudo-label enhanced end-to-end dense scene pedestrian detection method according to claim 7, characterized in that: The calculation formula of the loss function of the end-to-end detection model is as follows: L bbox =L query +L proposal Among them, L bbox represents the loss function of the end-to-end detection model; L query represents the loss function of the query-based detection network, including classification loss, bounding box regression loss, and deep pseudo-label loss, and They represent the classification loss, bounding box regression loss, and deep pseudo-label loss corresponding to the query-based detection network respectively; L proposal represents the loss function of the branch based on region proposal, including classification loss, bounding box regression loss, and deep pseudo label loss, and They represent the classification loss, bounding box regression loss, and deep pseudo-label loss corresponding to the branch based on region proposal, respectively.

Images