Anchors-free fast pedestrian detection method and system for road monitoring scene

Abstract

The application relates to an anchor-free frame fast pedestrian detection method and system for a road monitoring scene, which comprises the following steps: S1, extracting road monitoring image features x by using a feature extraction network; S2, inputting the road monitoring image features x into a self-defined instance integral module to obtain an instance integral value s; and S3, inputting the feature network output f and the instance integral value s into a detection head for pedestrian detection. The application provides an anchor-free frame fast pedestrian detection method and system for a road monitoring scene, which is designed without an anchor frame, does not need to calculate multi-scale features, connects a self-developed instance integral module after a feature extraction network, and then directly connects a single detection head, so that the model layer number and output dimension are greatly reduced, and the inference speed of the detection system and the pedestrian detection performance in the road monitoring scene are improved.

Description

Technical Field

[0001] The present application relates to the technical field of pedestrian detection, and in particular to a method and system for fast pedestrian detection without anchor frames for road monitoring scenarios. Background Art

[0002] Mainstream object detection methods can be categorized into two types: two-stage and one-stage. Two-stage methods, such as Faster R-CNN and Mask R-CNN, first identify candidate bounding boxes for the object in the image. These methods then detect and correct each candidate bounding box to produce a predicted bounding box. While these methods offer high detection accuracy, they are relatively slow. Currently, the vast majority of real-time pedestrian detection algorithms are based on frameworks based on one-stage methods, such as various versions of YOLO. For the most popular YOLO v5 and v7 models, the anchor box settings determine the detector's performance. While the preset anchor boxes can cover most detected objects when properly configured, they may still underperform for models of certain object sizes. Furthermore, the camera positions in road surveillance scenarios are complex and varied, and the preset anchor boxes often fail to cover all scenarios, potentially impacting the robustness of the detection system. Although YOLO v8 lacks an anchor box design, it essentially outputs a set of predicted bounding boxes within a given scale range. Consequently, the output dimensionality increases in various architectures, making model training more challenging. Summary of the Invention

[0003] In response to the technical problems in the above-mentioned background technology, this application proposes a fast pedestrian detection method and system without anchor frames for road monitoring scenarios.

[0004] In the first aspect, the present application proposes a fast pedestrian detection method without anchor boxes for road monitoring scenarios, comprising the following steps:

[0005] S1: Use feature extraction network to extract road monitoring image features x;

[0006] S2: Input the road monitoring image feature x into the customized instance integration module to obtain the instance integration value s;

[0007] S3: Input the feature network output f and instance integral value s into the detection head for pedestrian detection.

[0008] By adopting the above technical solution, the self-developed instance integration module is connected after the feature extraction network, and then directly connected to a single detection head, which greatly reduces the number of model layers and output dimensions, thereby improving the inference speed of the detection system and the pedestrian detection performance in road monitoring scenarios.

[0009] Preferably, the customized instance integration module in S2 is obtained by the following steps:

[0010] S201: For each pixel position of the feature map, the linear layer is input to calculate the query q, key k and value v;

[0011] S202: Calculate the transpose of the last two dimensions of the k matrix;

[0012] S203: Calculate the matrix product of q and the transposed matrix of k to obtain the correlation coefficient r of each pixel point to feature map;

[0013] S204: using a custom ScaledSiLU function to constrain the value of r to a distribution between 0 and 1;

[0014] S205: Input v into the softmax function and calculate the activation value;

[0015] S206: Calculate the matrix product of r and the activated v to obtain the instance integral value s.

[0016] Preferably, in S204, the customized ScaledSiLU function is:

[0017] ScaledSiLU(x)=SiLU(x) / max[1,d*Mean(SiLU(x))];

[0018] Where d is a constant coefficient.

[0019] Preferably, the S3 specifically includes: inputting the feature network output f and the instance integral value s into the detection head to obtain the predicted box coordinates, confidence and predicted category.

[0020] In a second aspect, the present application also proposes an anchor-free rapid pedestrian detection system for road monitoring scenarios, the system comprising:

[0021] A road monitoring image module configured to extract road monitoring image features x using a feature extraction network;

[0022] an instance integral value calculation module configured to input a road monitoring image feature x into a custom instance integral module to obtain an instance integral value s;

[0023] The pedestrian detection module is configured to input the feature network output f and the instance integral value s into the detection head for pedestrian detection.

[0024] Preferably, the instance integral value calculation module is specifically used to:

[0025] For each pixel position in the feature map, the linear layer is input to calculate the query q, key k and value v;

[0026] Calculate the transpose of the last two dimensions of the k matrix;

[0027] Calculate the matrix product of q and the transposed matrix of k to obtain the correlation coefficient r of each pixel point to the feature map;

[0028] Use the custom ScaledSiLU function to constrain the value of r to a distribution between 0 and 1;

[0029] Input v into the softmax function and calculate the activation value;

[0030] Calculate the matrix product of r and the activated v to obtain the instance integral value s.

[0031] Preferably, the customized ScaledSiLU function is:

[0032] ScaledSiLU(x)=SiLU(x) / max[1,d*Mean(SiLU(x))];

[0033] Where d is a constant coefficient.

[0034] Preferably, the pedestrian detection module is specifically used to: input the feature network output f and the instance integral value s into the detection head to obtain the predicted box coordinates, confidence and predicted category.

[0035] This application proposes an anchor-free fast pedestrian detection method for road monitoring scenarios. While designing without anchor frames, it does not require the calculation of multi-scale features. A self-developed instance integration module is connected after the feature extraction network, and then directly connected to a single detection head, which greatly reduces the number of model layers and output dimensions, thereby improving the inference speed of the detection system and the pedestrian detection performance in road monitoring scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The accompanying drawings are included to provide a further understanding of the embodiments and are incorporated into and constitute a part of this specification. The accompanying drawings illustrate the embodiments and, together with the description, serve to explain the principles of the present application. Other embodiments and many of the expected advantages of the embodiments will be readily apparent as they become better understood by reference to the following detailed description. The elements of the drawings are not necessarily to scale with respect to each other. Like reference numerals designate corresponding similar parts.

[0037] Figure 1 This is a flowchart of the present application's method for fast pedestrian detection without anchor frames for road monitoring scenarios.

[0038] Figure 2 This is a schematic diagram of a customized instance integration module in one embodiment of the present application.

[0039] Figure 3 This is a schematic diagram of the module structure of an anchor-free rapid pedestrian detection system for road monitoring scenarios in one embodiment of the present application.

[0040] Figure 4 It is a structural diagram of a computer system suitable for implementing the electronic device of the embodiment of the present application. DETAILED DESCRIPTION

[0041] The present application will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely for the purpose of explaining the relevant invention and are not intended to limit the invention. It should also be noted that, for ease of description, only portions relevant to the relevant invention are shown in the accompanying drawings.

[0042] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0043] Figure 1 The flowchart of the present invention is a method for fast pedestrian detection without anchor frames for road monitoring scenes. Figure 1 , the method specifically comprises the following steps:

[0044] S1: Use feature extraction network to extract road monitoring image features x;

[0045] S2: Input the road monitoring image feature x into the customized instance integration module to obtain the instance integration value s;

[0046] The customized instance integration module in S2 is obtained by the following steps:

[0047] S201: For each pixel position of the feature map, the linear layer is input to calculate the query q, key k and value v;

[0048] S202: Calculate the transpose of the last two dimensions of the k matrix;

[0049] S203: Calculate the matrix product of q and the transposed matrix of k to obtain the correlation coefficient r of each pixel point to feature map;

[0050] S204: using a custom ScaledSiLU function to constrain the value of r to a distribution between 0 and 1;

[0051] The custom ScaledSiLU function is:

[0052] ScaledSiLU(x)=SiLU(x) / max[1,d*Mean(SiLU(x))];

[0053] Where d is a constant coefficient

[0054] S205: Input v into the softmax function and calculate the activation value;

[0055] S206: Calculate the matrix product of r and the activated v to obtain the instance integral value s.

[0056] S3: Input the feature network output f and instance integral value s into the detection head for pedestrian detection.

[0057] The S3 specifically includes: inputting the feature network output f and the instance integral value s into the detection head to obtain the predicted box coordinates, confidence and predicted category.

[0058] In a specific embodiment, the following describes a method for fast pedestrian detection without anchor frames for road monitoring scenarios in the present application:

[0059] Mainstream object detection methods can be categorized into two types: two-stage and one-stage. Two-stage methods, such as Faster R-CNN and Mask R-CNN, first identify candidate bounding boxes for the object in the image. These methods then perform detection and correction on each candidate box to produce a predicted box. While these methods offer high accuracy, they are relatively slow. Currently, the majority of real-time pedestrian detection algorithms are based on frameworks based on one-stage methods, such as various versions of YOLO. For example, the YOLO v5 and YOLO v7 pipelines typically pre-set three anchor boxes at each scale. A deep learning network is then used to extract image features at each scale. Within the feature map at each scale, the algorithm globally learns to detect objects with similar scales to the anchor boxes. The latest YOLO v8 detection algorithm removes the need for anchor boxes at each scale and instead learns a probability distribution for the values ​​of the predicted box coordinates. This distribution is then discretely sampled to obtain a set of predicted boxes, thereby achieving object detection. Pedestrian recognition in road surveillance scenarios places high demands on the real-time performance of both the algorithm and the system, making two-stage object detection methods inapplicable in this context. For the most popular YOLO v5 and v7 models, the performance of the detector is determined by the anchor box settings. While the preset anchor boxes can cover most detection objects when properly set, there may still be cases where the detection effect on models of certain object scales is poor. Furthermore, the camera positions in road surveillance scenarios are complex and varied, and the preset anchor boxes often cannot cover all scenarios, which may affect the robustness of the detection system. Although YOLO v8 is anchor-free, it essentially outputs a set of prediction boxes within a given scale range. Therefore, the output dimension increases in multiple architectures, making model training more difficult. Therefore, we propose an anchor-free fast pedestrian detection method and system for road surveillance scenarios. While adopting an anchor-free design, it eliminates the need to calculate multi-scale features. Instead, a self-developed instance integration module is connected after the feature extraction network, and then directly connected to a single detection head. This significantly reduces the number of model layers and output dimensions, thereby improving the inference speed of the detection system and pedestrian detection performance in road surveillance scenarios.

[0060] In a specific embodiment, the anchor-free fast pedestrian detection method for a road monitoring scenario in this embodiment specifically includes the following steps:

[0061] Step 1. Use feature extraction network to extract road monitoring image features f;

[0062] Step 2. Define the self-developed instance integration module in steps 3 to 8. The module structure is shown in Figure 2 ;

[0063] Step 3. For each pixel position in the feature map, input the linear layer to calculate: query q, key k, value v;

[0064] Step 4. Calculate the transpose of the last two dimensions of the k matrix;

[0065] Step 5. Calculate the matrix product of q and the transposed matrix of k to obtain the correlation coefficient r of each pixel pair feature map;

[0066] Step 6. Use the self-built ScaledSiLU function to constrain the value of r to a distribution between 0 and 1, indicating the degree of correlation between two pixels: ScaledSiLU(x) = SiLU(x) / max[1,d*Mean(SiLU(x))], where d is a constant coefficient.

[0067] Step 7. Input v into the softmax function and calculate the activation value;

[0068] Step 8. Calculate the matrix product of r and v to obtain the weighted sum of each pixel based on r on the v function, which represents the area s of the detected object;

[0069] Step 9. Input the feature network output f and the instance integral value s into the detection head to obtain the predicted box coordinates, confidence and predicted category.

[0070] This application proposes an anchor-free fast pedestrian detection method for road monitoring scenarios. While designing without anchor frames, it does not require the calculation of multi-scale features. A self-developed instance integration module is connected after the feature extraction network, and then directly connected to a single detection head, which greatly reduces the number of model layers and output dimensions, thereby improving the inference speed of the detection system and the pedestrian detection performance in road monitoring scenarios.

[0071] Further references Figure 3 As an implementation of the above-mentioned method, the present application provides an embodiment of an anchor-free fast pedestrian detection system for road monitoring scenes. Figure 1 Corresponding to the method embodiment shown, the system can be specifically applied to various electronic devices.

[0072] refer to Figure 3, a fast pedestrian detection system without anchor boxes for road monitoring scenarios, including:

[0073] A road monitoring image module 101 is configured to extract road monitoring image features x using a feature extraction network;

[0074] The instance integral value calculation module 102 is configured to input the road monitoring image feature x into a custom instance integral module to obtain an instance integral value s;

[0075] The pedestrian detection module 103 is configured to input the feature network output f and the instance integral value s into the detection head for pedestrian detection.

[0076] Reference below Figure 4 , which shows a structural diagram of a computer system 200 suitable for implementing an electronic device of an embodiment of the present application. Figure 4 The electronic device shown is merely an example and should not limit the functions and scope of use of the embodiments of the present application.

[0077] like Figure 4 As shown, the computer system 200 includes a central processing unit (CPU) 201, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 202 or a program loaded from a storage unit 208 into a random access memory (RAM) 203. Various programs and data required for the operation of the system 200 are also stored in the RAM 203. The CPU 201, the ROM 202, and the RAM 203 are connected to each other via a bus 204. An input / output (I / O) interface 205 is also connected to the bus 204.

[0078] The following components are connected to the I / O interface 205: an input section 206 including a keyboard, a mouse, and the like; an output section 207 including a liquid crystal display (LCD) and speakers; a storage section 208 including a hard disk; and a communication section 209 including a network interface card such as a LAN card or a modem. The communication section 209 performs communication processing via a network such as the Internet. A drive 220 is also connected to the I / O interface 205 as needed. A removable medium 211, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 220 as needed, so that a computer program read therefrom can be installed in the storage section 208 as needed.

[0079] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 209, and / or installed from a removable medium 211. When the computer program is executed by the central processing unit (CPU) 201, the above-mentioned functions defined in the method of the present application are performed.

[0080] As another aspect, the present application also provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiment; or may exist independently without being assembled into the electronic device. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed by the electronic device, the electronic device implements the following when executing. Figure 1 The method shown in .

[0081] It should be noted that the computer-readable storage medium described in this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this application, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. This propagated data signal can take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable storage medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. Program code contained on a computer-readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wireline, optical cable, RF, or any suitable combination thereof.

[0082] Computer program code for performing the operations of the present application can be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0083] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, program segment or a part of code, and the module, program segment or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0084] The above describes the specific embodiments of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

[0085] In the description of this application, it should be understood that the terms "upper", "lower", "inside", "outside", etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings. They are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, they cannot be understood as limiting this application. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of multiple such elements. The simple fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used for improvement. Any reference signs in the claims should not be interpreted as limiting the scope.

Claims

1. A fast pedestrian detection method without anchor boxes for road monitoring scenarios, characterized by: The method comprises the following steps: S1: Use feature extraction network to extract road monitoring image features x; S2: Input the road monitoring image feature x into a custom instance integration module to obtain the instance integration value s. The custom instance integration module is obtained by the following steps: S201: For each pixel position in the feature map, input the linear layer to calculate the query q, key k, and value v; S202: Calculate the transpose of the last two dimensions of the k matrix; S203: Calculate the matrix product of q and the transpose of k to obtain the correlation coefficient r of each pixel point to the feature map; S204: Use the custom ScaledSiLU function to constrain the value of r to a distribution between 0 and 1. The custom ScaledSiLU function is: ScaledSiLU(x)=SiLU(x) / max[1,d*Mean(SiLU(x))];where d is a constant coefficient;S205:V is input into the softmax function and the activation value is calculated;S206:Compute the matrix product of r and the activated v to obtain the instance integral value s; S3: Input the feature network output f and instance integral value s into the detection head for pedestrian detection.

2. The anchor-free fast pedestrian detection method for road monitoring scenarios according to claim 1, characterized in that: The S3 specifically includes: inputting the feature network output f and the instance integral value s into the detection head to obtain the predicted box coordinates, confidence and predicted category.

3. An anchor-free rapid pedestrian detection system for road monitoring scenarios, characterized by: The system comprises: A road monitoring image module configured to extract road monitoring image features x using a feature extraction network; The instance integral value calculation module is configured to input the road monitoring image feature x into the customized instance integral module to obtain the instance integral value s; the instance integral value calculation module is specifically used to: for each pixel position of the feature map, input the linear layer to calculate the query q, key value k and value v; calculate the transpose of the last two dimensions of the k matrix; calculate the matrix product of q and the k transpose to obtain the correlation coefficient r of each pixel point to the feature map; use the customized ScaledSiLU function to constrain the value of r to a distribution between 0 and 1; the customized ScaledSiLU function is: ScaledSiLU(x)=SiLU(x) / max[1,d* Mean(SiLU(x))]; where d is a constant coefficient; input v into the softmax function and calculate the activation value; calculate the matrix product of r and the activated v to obtain the instance integral value s; The pedestrian detection module is configured to input the feature network output f and the instance integral value s into the detection head for pedestrian detection.

4. The anchor-free rapid pedestrian detection system for road monitoring scenarios according to claim 3 is characterized by: The pedestrian detection module is specifically used to: input the feature network output f and the instance integral value s into the detection head to obtain the predicted box coordinates, confidence and predicted category.

Images