Deep pseudo-label augmented end-to-end dense scene pedestrian detection method

Abstract

The application discloses a kind of end-to-end dense scene pedestrian detection methods of depth pseudo label enhancement, belong to pattern recognition and computer vision technical field, the method includes: adding depth pseudo label in pedestrian image data, constructs the pedestrian image dataset under dense scene;Increase the auxiliary detection head of the detection network based on region proposal of multi-scale in the detection network based on query, construct the end-to-end detection model that forms more positive examples in dense scene;The end-to-end detection model is trained using the pedestrian image dataset;Pedestrian detection in dense scene is realized using the end-to-end detection model trained.The pedestrian detection method of the application can realize higher pedestrian detection precision in occlusion dense scene.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of pattern recognition and computer vision, in particular to a deep pseudo-label enhanced end-to-end dense scene pedestrian detection method. BACKGROUND

[0002] With the rapid development of intelligent technology, the demand for image-related intelligent technology in various industries is becoming increasingly strong. As a key technology for understanding images, computer vision has developed multiple research directions such as image classification, object detection, and object tracking. Among them, pedestrian detection, as an important sub-task of object detection, aims to accurately give the position and size of the detection box containing the complete pedestrian in the two-dimensional image through classification and regression. Pedestrian detection is not only a basic task for many practical applications, but also plays a key role in autonomous driving, video surveillance and other scenarios.

[0003] In the autonomous driving scenario, the pedestrian detection box is the starting basis for pedestrian tracking and trajectory prediction, which in turn affects the driving strategy planning of the vehicle. In the field of video surveillance, accurate pedestrian detection boxes are key elements for cross-camera feature matching and person re-identification tasks.

[0004] Although pedestrian detection technology has been extensively studied, it still faces many challenges due to changes in background, lighting and appearance features in visual images. In particular, in occlusion scenarios, the features of complete pedestrians are destroyed, making pedestrian detection more difficult. Occluded pedestrian detection mainly includes pedestrian detection under object occlusion and mutual occlusion between pedestrians (i.e. detection in dense scenes). In the case of object occlusion, the visible body parts of pedestrians vary and their features are not obvious, resulting in large errors in the model's estimation of pedestrian position and size. Mutual occlusion between pedestrians makes the position and size of each pedestrian ambiguous, and it is particularly difficult to distinguish and retain the correct detection box in the post-processing stage.

[0005] Currently, the main dense scene pedestrian detection schemes mainly include region proposal-based detection methods and query-based end-to-end detection methods.

[0006] Region proposal-based detection methods, such as Faster R-CNN, divide the detection task into two stages: the first stage generates a number of candidate boxes according to certain rules, and the second stage processes the candidate boxes from the first stage and performs classification and regression of the detection box boundaries. This type of method performs well in relatively sparse scenes, but in the case of dense pedestrians, it is prone to missed detection.

[0007] The query-based detection method directly regards target detection as a sequence-to-sequence task, and uses a query mechanism to achieve accurate positioning of the target. DETR introduces a learnable query to represent an object, and directly performs a single prediction for each instance. Although the query-based detection method improves the accuracy, the number of detection boxes generated in a dense scene is relatively small due to the one-to-one detection strategy, which cannot effectively cover all targets.

[0008] In summary, pedestrian detection faces many challenges in complex environments. Especially in two-dimensional images, the overlap or occlusion between pedestrians significantly increases the difficulty of detection due to the single angle of view of the camera. Relying solely on two-dimensional image information, the model has difficulty accurately distinguishing partially occluded pedestrians, leading to an increase in missed detection rates and severely affecting detection performance. At the same time, existing query-based networks perform well in pedestrian detection, better capturing target features through the query mechanism to achieve accurate positioning of pedestrians. However, current query-based networks typically use a one-to-one detection approach, resulting in a small number of generated detection boxes. Although this method works well in scenarios with a small number of targets, in environments with dense pedestrian distribution, the number of detection boxes is insufficient to cover all targets, increasing the risk of missed detection. Therefore, the existing technology does not perform well in dense scenes. SUMMARY

[0009] The present application provides a deep pseudo-label enhanced end-to-end dense scene pedestrian detection method to solve the technical problem of the existing technology not being ideal in dense scene detection.

[0010] To solve the above technical problems, the present application provides the following technical solutions:

[0011] On the one hand, the present application provides a deep pseudo-label enhanced end-to-end dense scene pedestrian detection method, which comprises:

[0012] Adding deep pseudo-labels to pedestrian image data to construct a pedestrian image dataset in a dense scene;

[0013] Adding a multi-scale region proposal-based detection network auxiliary detection head to the query-based detection network to construct an end-to-end detection model that generates more positive examples in a dense scene;

[0014] Training the end-to-end detection model using the pedestrian image dataset;

[0015] Using the trained end-to-end detection model to detect pedestrians in a dense scene.

[0016] Further, the adding of the depth pseudo label in the pedestrian image data comprises:

[0017] By using a data augmentation method, pedestrians are cropped by instance segmentation technology and pasted into original pedestrian image data to form pedestrian images in dense scenes, and according to the order of pasting of the pedestrians, occlusion depth information in the dense scenes is inferred and used as the depth pseudo label.

[0018] Further, the calculation formula of the occlusion depth information is as follows:

[0019]

[0020] Wherein, od i is the occlusion depth information, indicating the degree of coverage of an object by other objects; ovl(b i ,b j is the area covered by object b i by object b j ; S(·) represents the area of the region.

[0021] Further, an auxiliary detection head of a multi-scale region proposal-based detection network is added in the query-based detection network to construct an end-to-end detection model that forms more positive examples in dense scenes, comprising:

[0022] A region proposal-based branch is introduced in the query-based detection network, and a depth pseudo label technology is introduced in the region proposal-based branch to realize a one-to-many label assignment strategy and train multiple parallel auxiliary detection heads; wherein the auxiliary detection heads are discarded after training.

[0023] Further, the end-to-end detection model takes Deformable DETR as the benchmark network.

[0024] Further, the end-to-end detection model takes Faster R-CNN as a one-to-many auxiliary detection head.

[0025] Further, the loss function of the end-to-end detection model is designed as the weighted sum of the loss functions of multiple subtasks.

[0026] Further, the calculation formula of the loss function of the end-to-end detection model is as follows:

[0027] L bbox = L query + L proposal

[0028]

[0029]

[0030] wherein, L bbox represents the loss function of the end-to-end detection model; L query represents the loss function of the query-based detection network, including classification loss, bounding box regression loss and deep pseudo label loss, and respectively represent the classification loss, bounding box regression loss and deep pseudo label loss corresponding to the query-based detection network; L proposal represents the loss function of the region proposal-based branch, including classification loss, bounding box regression loss and deep pseudo label loss, and respectively represent the classification loss, bounding box regression loss and deep pseudo label loss corresponding to the region proposal-based branch.

[0031] In still another aspect, the present application also provides an electronic device comprising a processor and a memory; wherein the memory has at least one instruction stored therein, which is loaded and executed by the processor to implement the above method.

[0032] In still another aspect, the present application also provides a computer-readable storage medium, wherein the storage medium has at least one instruction stored therein, which is loaded and executed by the processor to implement the above method.

[0033] The technical solution provided by the present application has at least the following beneficial effects:

[0034] The present application introduces a region proposal-based branch in the query-based detection network to enhance the detection capability of the model in dense scenes, so that the model has better feasibility and flexibility in actual application. At the same time, the deep pseudo label technology is introduced in the branch to further increase the richness and accuracy of information, thereby improving the precision and robustness of pedestrian detection in dense scenes and effectively reducing the missed detection in dense scenes. BRIEF DESCRIPTION OF DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0036] Figure 1 is an execution flow diagram of the deep pseudo label enhanced end-to-end dense scene pedestrian detection method provided by the embodiments of the present application;

[0037] Figure 2 is a framework diagram of the end-to-end model provided by the embodiments of the present application;

[0038] Figure 3 is a system block diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0039] To make the objectives, technical solutions, and advantages of the present application clearer, further detailed descriptions will be made to the embodiments of the present application with reference to the drawings.

[0040] First of all, it should be noted that in the embodiments of the present application, the words such as "exemplarily", "for example" are used to represent as an example, illustration or description. Any embodiment or design scheme described as "exemplary" in the present application should not be interpreted as more preferred or more advantageous than other embodiments or design schemes. Rather, the word "exemplarily" is intended to present the concept in a specific manner. In addition, in the embodiments of the present application, the meaning expressed by "and / or" can be both, or can be either one of the two.

[0041] First Embodiment

[0042] The present embodiment provides a deep pseudo-label enhanced end-to-end dense scene pedestrian detection method, which aims to combine the query-based detection network with the region proposal-based branch, and enhance the occluded pedestrian information of the image. To improve the accuracy of dense scene pedestrian detection. The method can be implemented by an electronic device, which can be a terminal or a server. The execution flow of the method is as shown in Figure 1 The method comprises the following steps:

[0043] S1, adding deep pseudo-labels in pedestrian image data, and constructing pedestrian image dataset in dense scene;

[0044] S2, adding a multi-scale region proposal-based detection network auxiliary detection head in the query-based detection network, and constructing an end-to-end detection model that forms more positive examples in dense scene;

[0045] S3, training the end-to-end detection model using the pedestrian image dataset;

[0046] S4, realizing pedestrian detection in dense scene by using the trained end-to-end detection model.

[0047] Specifically, the present embodiment introduces a region proposal-based branch in the query-based detection network to enhance the detection capability of the model in dense scene. At the same time, the deep pseudo-label technology is introduced in the branch to further increase the richness and accuracy of the information, so as to effectively reduce the missed detection in dense scene.

[0048] To achieve the above technical solutions, the embodiment adopts a data augmentation method, crops pedestrians through instance segmentation technology, and pastes them into the training set to enhance the robustness and adaptability of the model to dense scenes. In the pasting process, due to the order of pasting pedestrians, we can infer the depth information in this occlusion dense scene, and generate occlusion depth pseudo labels accordingly. This pseudo label not only provides additional supervision signal for the model, but also effectively helps it learn the feature performance of pedestrians in different occlusion conditions. Specifically, the calculation formula of occlusion depth is as follows:

[0049]

[0050] Where od(Overlay Depth) represents the occlusion depth pseudo label of the augmented pedestrian, and the variable represents the degree to which an object is covered by other objects; ovl(b i ,b j ) is the area covered by object b i by object b j . S(·) represents the area of the region. For any object b i in the image, there is a set of objects O i that cause occlusion to b i . When the pedestrian is not occluded, the initial value of its depth is 1.0, and when the newly pasted pedestrian has an overlapping area with the pedestrian, the depth of the pedestrian will change dynamically.

[0051] After that, the embodiment introduces the Faster R-CNN one-to-many label assignment strategy in the query-based network, trains multiple parallel auxiliary heads, and enhances the learning ability of the encoder in the end-to-end detector. The design of these auxiliary heads enables the model to more effectively process complex feature representations during training, thereby improving the accuracy and robustness of detection. Specifically, these parallel auxiliary heads not only provide diverse supervision signals, but also extract positive sample coordinates for additional customized positive queries. This approach aims to improve the training efficiency of positive samples in the decoder, enabling the model to converge faster and optimize the positioning ability of the detection box. In this way, the model can learn more detailed target features, thereby achieving higher detection accuracy in dense scenes. Specifically, the one-to-one detection network in the end-to-end detection model constructed by the embodiment selects Deformable DETR as the benchmark network and Faster R-CNN as the one-to-many auxiliary detection head. The structure of the end-to-end detection model is as shown in Figure 2 .

[0052] It should be noted that in the method of the present embodiment, the auxiliary head is discarded after training is completed. This means that the solution of the present embodiment does not introduce additional parameters or computational costs to the original detector. This not only ensures the efficiency of the model, but also simplifies the inference process. In addition, the method of the present embodiment does not need to re-execute the manually designed non-maximum suppression (NMS) process, thereby avoiding the computational bottleneck and delay that may occur in the traditional method. This optimization enables the detector of the present embodiment to maintain high performance while having faster inference speed and lower resource consumption.

[0053] In summary, by introducing parallel auxiliary heads and extracting customized positive queries, the method of the present embodiment effectively improves the overall performance of the end-to-end detector while maintaining its computational efficiency, making the model more feasible and flexible in practical applications.

[0054] Further, the network loss calculation formula based on the combination of the query-based network and the region proposal-based network can be designed as the weighted sum of the loss functions of multiple subtasks. Specifically, the loss calculation formula is as follows:

[0055] L bbox = L query + L proposal

[0056]

[0057]

[0058] wherein L bbox denotes the loss function of the end-to-end detection model; L query denotes the loss function of the query-based detection network, including the classification loss, the bounding box regression loss, and the deep pseudo-label loss, and denote the classification loss, the bounding box regression loss, and the deep pseudo-label loss of the query-based detection network, respectively; L proposal denotes the loss function of the region proposal-based branch, including the classification loss, the bounding box regression loss, and the deep pseudo-label loss, and denote the classification loss, the bounding box regression loss, and the deep pseudo-label loss of the region proposal-based branch, respectively.

[0059] The model of the embodiment is applied to the Crowdhuman dataset, and the final detection effect is shown in Table 1. Among them, the Crowdhuman dataset is commonly used for dense scene pedestrian detection. It contains 15,000 images, of which there are 470K instances in the training set and the validation set. The human density is obviously higher than that of Cityperson, reaching 22.6 people per image, and there are 2.4 paired overlapping instances (IoU greater than 0.5). As can be seen from Table 1, the model of the embodiment produces better results for dense scene pedestrian detection on the Crowdhuman dataset.

[0060] In the test phase, we remove the one-to-many branch based on region proposal, and only use the one-to-detection branch based on query, so that the entire network still has end-to-end superiority. For dense scenes, we expand the maximum number of instances generated by each image in the decoder to 200. As can be seen from Table 1, the model of the embodiment produces better results for dense scene pedestrian detection on the Crowdhuman dataset.

[0061] Table 1 Detection effect of the model of the embodiment on the Crowdhuman dataset

[0062]

[0063] Next, the superiority of the present application compared with the prior art is verified by means of comparative experiments; the experiment includes two parts, the first part is data augmentation to generate deep pseudo labels, and the second part is to add a one-to-many branch in the one-to-one query-based network structure, and to add a deep loss function calculation to provide more information for the detection model. The influence of each part on the result is shown in Table 2:

[0064] Table 2 Detection effect of different models

[0065]

[0066]

[0067] Among them, the one-to-one detection network, the embodiment selects Deformable DETR as the benchmark network, and the embodiment selects Faster R-CNN as the one-to-many auxiliary detection head. Since data augmentation is used in generating pseudo labels in the embodiment, the embodiment also compares the addition of data augmentation and the use of data augmentation and deep pseudo labels based on the combination of one-to-one and one-to-many detection networks.

[0068] In summary, the embodiment provides a deep pseudo-label enhanced end-to-end dense scene pedestrian detection method. A region proposal based branch is introduced in the query based detection network to enhance the detection capability of the model in the dense scene, so that the model has better feasibility and flexibility in actual application. At the same time, the deep pseudo-label technology is introduced in the branch to further increase the richness and accuracy of the information, thereby improving the precision and robustness of pedestrian detection in the dense scene, and effectively reducing the missed detection in the dense scene.

[0069] Second embodiment

[0070] The embodiment provides an electronic device, such as Figure 3 As shown in the figure, the electronic device comprises a processor and a memory; wherein the processor and the memory can be connected through a communication bus; the memory stores at least one instruction, which is loaded and executed by the processor to implement the method of the first embodiment. In addition, the electronic device can also comprise a transceiver, and the processor and the transceiver can be connected through a communication bus, and the transceiver is used for communication with other devices.

[0071] Next, the specific implementation of the electronic device will be described in combination with Figure 3 The specific components of the electronic device will be described in detail as follows:

[0072] The processor is the control center of the electronic device, and the electronic device can comprise a plurality of processors, each of which can be a single-CPU or a multi-CPU. The processor herein can be one processor or a general term for a plurality of processing elements. For example, the processor is one or more central processing units (CPU), which can also be other general-purpose processors, application specific integrated circuits (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application, such as one or more microprocessors (digital signal processors, DSP), or one or more field programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor. The processor can perform various functions of the electronic device by running or executing software programs stored in the memory and calling data stored in the memory.

[0073] In a specific implementation, as an example, the processor can include one or more CPUs, such as CPU0 and CPU1 shown in FIG. 10, of course, this is only an exemplary description. Figure 3

[0074] The memory is used to store a software program for implementing the scheme of the present application, and is controlled by the processor to perform, and the specific implementation can refer to the method embodiments described above, which will not be repeated here.

[0075] Optionally, the memory can be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM) or other type of dynamic storage device that can store information and instructions, and can also be an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage medium or other magnetic storage device, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and capable of being accessed by a computer, but not limited to this. The memory can be integrated with the processor, or can exist independently, and is coupled with the processor through the interface circuit (not shown in FIG. 10) of the electronic device, and the embodiments of the present application do not make specific limitations here. Figure 3

[0076] The transceiver can include a receiver and a transmitter (not shown separately in FIG. 10). Among them, the receiver is used to realize the receiving function, and the transmitter is used to realize the transmitting function. The transceiver can be integrated with the processor, or can exist independently, and is coupled with the processor through the interface circuit (not shown in FIG. 10) of the electronic device, and the embodiments of the present application do not make specific limitations here. Figure 3 Figure 3

[0077] In addition, it needs to be explained that, Figure 3 The structure of the electronic device shown in FIG. 10 does not constitute a limitation on the device, and the actual device can include more or fewer components than the diagram, or combine certain components, or different component arrangements. In addition, the technical effects achieved by the electronic device when executing the method of the first embodiment described above can refer to the technical effects described in the first embodiment described above, and therefore, will not be repeated here.

[0078] Third embodiment​​​​

[0079] The embodiment provides a computer readable storage medium, at least one instruction is stored in the storage medium, the instruction is loaded and executed by a processor to realize the method of the first embodiment. Wherein, the computer readable storage medium can be a ROM, a random access memory, a CD-ROM, a magnetic tape, a floppy disk and an optical data storage device, etc. The instruction stored therein can be loaded and executed by the processor in the terminal to execute the above method.

[0080] In addition, it should be noted that the present application can be provided as a method, an apparatus or a computer program product. Therefore, the embodiments of the present application can be in the form of entirely or partially hardware embodiments, entirely or partially software embodiments or embodiments combining software and hardware aspects. Moreover, when implemented by software, the embodiments of the present application can be in the form of a computer program product implemented on one or more computer usable storage media including computer usable program code. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded and executed on a computer, the flow or function described in the embodiments of the present application is generated entirely or partially. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable devices. The computer instructions can be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (such as infrared, wireless, microwave, etc.) mode. The computer readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. containing one or more available medium sets. The available medium can be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium. The semiconductor medium can be a solid state disk.

[0081] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams according to the method, terminal device (system) and computer program product of the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be realized by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, an embedded processor or other programmable data processing terminal device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device produce a machine that implements the flow specified in the flowcharts and / or block diagrams. Figure 1 The flow or the combination of multiple flows and / or blocks Figure 1 The device that implements the function specified in the flow or the combination of multiple flows and / or blocks.

[0082] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable data processing terminal equipment to cause a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0083] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. The terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element. Furthermore, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone, where A and B can be singular or plural. Additionally, the character " / " in this text generally indicates an "or" relationship between the preceding and following objects, but it can also indicate an "AND / OR" relationship. Please refer to the context for specific interpretations. "At least one" refers to one or more items, while "more than" refers to two or more items. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or multiple items. For example, at least one of a, b, or c can be represented as: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0084] In addition, it should be understood that the sequence of the above processes does not mean the order of execution in various embodiments of the present application, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0085] Those skilled in the art can appreciate that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized in electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. A skilled person can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0086] In several embodiments provided by the present application, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative, for example, the division of functional modules / units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another device, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed units can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms. The units described as separate components can be or can not be physically separated, and the components displayed as units can be or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the present embodiment. In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit.

[0087] If the method is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or the part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in the embodiments of the present application. The aforementioned storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0088] Finally, it should be pointed out that the above description is only the preferred embodiments of the present application. It should be noted that although the preferred embodiments of the present application have been described, for those skilled in the art, once the basic creative concept of the present application is known, a number of improvements and refinements can be made without departing from the principles of the present application. These improvements and refinements should also be considered as the protection scope of the present application. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all modifications and variations falling within the scope of the embodiments of the present application.

Claims

1. An end-to-end pedestrian detection method for dense scenes with deep pseudo-label enhancement, characterized in that, The end-to-end dense scene pedestrian detection method enhanced by deep pseudo-labels includes: Add depth pseudo-labels to pedestrian image data to construct a pedestrian image dataset for dense scenes; Add a multi-scale region-propagation-based auxiliary detection head to the query-based detection network to build an end-to-end detection model that generates more positive samples in dense scenes; The end-to-end detection model is trained using the pedestrian image dataset; Pedestrian detection in dense scenes is achieved using a pre-trained end-to-end detection model; Adding depth pseudo-labels to pedestrian image data includes: The data augmentation method is used to crop pedestrians using instance segmentation technology and paste them into the original pedestrian image data to form pedestrian images in dense scenes. Based on the order in which the pedestrians are pasted, the occlusion depth information in such dense scenes is inferred and used as a depth pseudo-label. The method of adding a multi-scale region-propagation-based auxiliary detection head to the query-based detection network to construct an end-to-end detection model that generates more positive examples in dense scenes includes: A region-based proposal branch is introduced into the query-based detection network. At the same time, a deep pseudo-labeling technique is introduced into the region-based proposal branch to implement a one-to-many label allocation strategy and train multiple parallel auxiliary detection heads. The auxiliary detection heads are discarded after training.

2. The end-to-end pedestrian detection method for dense scenes with depth pseudo-label enhancement as described in claim 1, characterized in that, The formula for calculating the occlusion depth information is as follows: Among them, od i Occlusion depth information indicates the degree to which an object is covered by other objects; ovl(b i ,b j ) is object b i Object b j The covered area; S(·) represents the area of ​​the region.

3. The end-to-end pedestrian detection method for dense scenes with depth pseudo-label enhancement as described in claim 1, characterized in that, The end-to-end detection model uses Deformable DETR as the baseline network.

4. The end-to-end pedestrian detection method for dense scenes with depth pseudo-label enhancement as described in claim 3, characterized in that, The end-to-end detection model uses Faster R-CNN as a one-to-many auxiliary detection head.

5. The end-to-end pedestrian detection method for dense scenes with depth pseudo-label enhancement as described in claim 4, characterized in that, The loss function of the end-to-end detection model is designed as a weighted sum of the loss functions of multiple sub-tasks.

6. The end-to-end pedestrian detection method for dense scenes with depth pseudo-label enhancement as described in claim 5, characterized in that, The formula for calculating the loss function of the end-to-end detection model is as follows: L bbox =L quety +L proposal Among them, L bbox L represents the loss function of the end-to-end detection model; query This represents the loss function of the query-based detection network, including classification loss, bounding box regression loss, and deep pseudo-label loss. and L represents the classification loss, bounding box regression loss, and deep pseudo-label loss for the query-based detection network, respectively; proposal This represents the loss function for the region-based proposal branch, including classification loss, bounding box regression loss, and deep pseudo-label loss. and These represent the classification loss, bounding box regression loss, and deep pseudo-label loss corresponding to the branch based on region proposal, respectively.

Images