A pedestrian speed measurement method, system, medium and computing device

Abstract

The present invention relates to a pedestrian speed measurement method, system, medium and computing device, which comprises: loading original image data; performing target detection on the original image data and outputting the detection result; performing associated tracking of targets in different frames on the target detection result and outputting the detection result that meets the tracking constraint condition; calculating the pixel difference of the vertex of the detection frame and the time difference between the front and back frames; performing cumulative time judgment on the time difference and the pixel difference to ensure that the stored data is within a preset time threshold; mapping the pixel difference to obtain the actual distance, and then accumulating according to the pixels to obtain the moving distance in the X direction and the Y direction around the time threshold; calculating the moving distance in the X and Y directions in the obtained real scene according to the Pythagorean theorem; and obtaining the target moving speed by using the speed calculation formula according to the moving distance and the accumulated time. The present invention can realize accurate real-time speed detection of pedestrians.

Description

Technical Field

[0001] The present invention relates to the technical field of speed measurement, and in particular to a pedestrian speed measurement method, system, medium and computing device based on a monocular camera. Background Art

[0002] Pedestrian speed measurement is an important supporting technology for the realization of intelligent safety warning, protection, monitoring and other functions of pedestrians in various scenarios. For example, the recognition of pedestrians' running, walking and other states, and the rapid movement of large areas caused by abnormalities in the crowd. The traditional way is to implement it through security personnel patrolling on site or observing the monitoring screen. Or use speed radar to achieve it. However, since there are a large number of monitoring equipment installed in existing subway stations, the use of monitoring equipment can save costs to a large extent. With the development of artificial intelligence technology this year, some manufacturers have also tried to use monitoring screens to achieve speed measurement. However, due to technical reasons, the speed measurement effect is not ideal.

[0003] The existing speed measurement radar is mainly a radar device whose transmitter emits electromagnetic wave energy in a certain direction in space through an antenna. Objects in this direction reflect the electromagnetic waves. The radar antenna receives the reflected waves and sends them to the receiving device for processing to extract certain information about the object (the distance from the target object to the radar, the rate of change of distance or radial velocity, azimuth, altitude, etc.). Its main disadvantages are high cost and difficulty in measuring the speed of pedestrians in highly complex scenes. Summary of the invention

[0004] In view of the above problems, the purpose of the present invention is to provide a pedestrian speed measurement method, system, medium and computing device based on a monocular camera, which can realize accurate real-time pedestrian speed detection.

[0005] To achieve the above-mentioned purpose, the present invention adopts the following technical scheme: a pedestrian speed measurement method, which comprises: step S1: loading original image data; step S2: performing target detection on the original image data and outputting the detection result; step S3: performing associated tracking of targets in different frames on the target detection result, and outputting the detection result that meets the tracking constraint condition; step S4: calculating the pixel difference of the vertex of the detection frame and the time difference between the previous and next frames; step S5: accumulating the time difference and the pixel difference for time judgment to ensure that the stored data is within a preset time threshold; step S6: mapping the pixel difference to obtain the actual distance, and then accumulating according to the pixels to obtain the X-direction and Y-direction moving distance of the time threshold; step S7: calculating the movement distance of the time threshold in the X and Y directions in the obtained real scene according to the Pythagorean theorem; step S8: according to the movement distance and the accumulated time, using the speed calculation formula to obtain the target movement speed.

[0006] Furthermore, in step S1, the original image data is loaded by a monocular camera or a video file.

[0007] Furthermore, in step S2, the YOLOv3 deep neural network model is used to complete multi-target detection, and output the detection box category and coordinate information.

[0008] Furthermore, in step S3, the DeepSORT algorithm is used to realize the associated tracking of targets in different frames, and the associated tracked target category and coordinate information are output.

[0009] Furthermore, in step S4, the specific calculation includes:

[0010] Step S41: Initialize a preset time memory and a pixel difference memory of the detection frame vertices, and decompose the original motion pixel difference of the detection frame vertices into the X direction and the Y direction;

[0011] Step S42: Calculate the pixel difference between the vertices of the detection frames of the two previous and next frames in the X direction and store it in a pixel difference memory;

[0012] Step S43: Calculate the time difference between the previous and next two frames and record it in the time memory;

[0013] Step S44: Calculate the pixel difference between the vertices of the detection frames of the two previous and next frames in the Y direction and store it in a pixel difference memory.

[0014] Further, in step S5, the cumulative time judgment method is: storing the pixel difference while recording the timestamp of the current frame, and then judging whether the time difference between the beginning and the end in the time memory is greater than the set time threshold. If it is greater than the time threshold, the pixel difference in the pixel difference memory is accumulated and summed, and the process goes to step S6 to perform the mapping calculation between the pixel and the actual distance, and delete the data of the starting position in the time memory and the pixel difference memory; if it is less than the time threshold, it returns to step S4 and recalculates until the result is satisfied.

[0015] Furthermore, in step S6, the pixel difference mapping formula is:

[0016]

[0017]

[0018]

[0019] Where: w1 is the distance between the position of the vertical point of the camera and the nearest end of the vertical field of view; w2 is the distance from the nearest end of the vertical field of view to the center line of the field of view; w3 is the distance from the farthest end of the vertical field of view to the center line of the field of view; w4 is the distance from the farthest end of the vertical field of view to the extension line of the center line of the field of view; h is the height of the camera; θ1 is the angle between the vertical line of the camera and the center line of the camera field of view; n is the number of pixels in the Y direction; m is a constant, and its value range is [1,n-2]; x0 is the actual distance corresponding to the first pixel vertically upward from the center of the camera imaging plane; x1 is the actual distance corresponding to the second pixel vertically upward from the center of the camera imaging plane; Y is the distance in the real scene corresponding to each pixel in the vertical direction of the camera imaging plane; H0 is the distance from each pixel to the ground in the vertical direction of the camera imaging plane; θ0 is half of the horizontal field of view angle of the camera; w5 is half of the horizontal field of view width; X is the distance in the real scene corresponding to each pixel in the horizontal direction of the camera imaging plane.

[0020] A pedestrian speed measurement system comprises: a loading module, a target detection module, a tracking module, a difference calculation module, a judgment module, a mapping module, a motion distance acquisition module and a target speed acquisition module; the loading module is used to load original image data; the target detection module performs target detection on the original image data and outputs the detection result; the tracking module performs associated tracking of targets in different frames on the target detection result and outputs the detection result that meets the tracking constraint condition; the difference calculation module is used to calculate the pixel difference of the vertex of the detection frame and the time difference between the previous and next frames; the judgment module performs cumulative time judgment on the time difference and the pixel difference to ensure that the stored data is within a preset time threshold; the mapping module maps the pixel difference to obtain the actual distance, and then accumulates according to the pixels to obtain the X-direction and Y-direction moving distance of the time threshold; the motion distance acquisition module calculates the moving distance of the time threshold in the X and Y directions in the real scene according to the Pythagorean theorem; the target speed acquisition module obtains the target motion speed according to the motion distance and the accumulated time by using the speed calculation formula.

[0021] A computer-readable storage medium storing one or more programs, wherein the one or more programs include instructions, which, when executed by a computing device, cause the computing device to perform any of the above methods.

[0022] A computing device comprises: one or more processors, a memory and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include instructions for executing any of the above methods.

[0023] Due to the adoption of the above technical solutions, the present invention has the following advantages: 1. Based on a monocular camera, the present invention realizes accurate real-time speed detection of pedestrians through deep learning and spatial mapping methods, and optimizes according to multi-coordinate information. 2. The present invention maps pixels to their actual distances one by one through spatial mapping, and uses multi-coordinate comprehensive threshold judgment and other methods to improve the accuracy of pedestrian speed detection in monocular camera scenes, and the speed detection accuracy is not affected in the case of severe occlusion, and the speed calculation process is refined, and finally the accuracy rate reaches more than 90%. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a schematic diagram of a method flow in an embodiment of the present invention.

[0025] Figure 2 It is a vertical cross-sectional view of camera imaging in an embodiment of the present invention.

[0026] Figure 3 It is a horizontal cross-sectional view of camera imaging in an embodiment of the present invention. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solution and advantages of the embodiment of the present invention clearer, the technical solution of the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings of the embodiment of the present invention. Obviously, the described embodiment is a part of the embodiment of the present invention, not all of the embodiments. Based on the described embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of the present invention.

[0028] In the description of the present invention, the corresponding abbreviations and key terms are defined as follows:

[0029] YOLOv3: A convolutional neural network framework used in industrial scenarios. Its special convolutional layer organization makes it significantly more accurate for small targets than other convolutional neural network models.

[0030] DeepSORT: A multi-target tracking algorithm that uses motion models and appearance information for data association. In the real-time target tracking process, it extracts the target's apparent features for nearest neighbor matching, which can improve the target tracking effect in the case of occlusion. At the same time, it also reduces the problem of target ID jumps. In addition, it uses more reliable metrics instead of association metrics, allowing convolutional neural networks to be trained on large-scale pedestrian datasets and extract features, which has increased the network's robustness to loss and obstacles.

[0031] The present invention maps the distance of the actual horizontal plane to the pixel points on the imaging plane of the monocular camera according to the set spatial mapping method. Since the mapping method performs one-to-one mapping according to pixels, it has higher accuracy than the traditional grid method. The vertices of the pedestrian detection frame are used as pedestrian positioning points to measure the speed of pedestrians. A high speed measurement accuracy can still be obtained when a large area of ​​the pedestrian's body is blocked. A comprehensive threshold judgment is performed based on the moving distance of the coordinates of the four vertices of the pedestrian detection frame per unit time, thereby realizing filtering of pedestrians in non-walking states. High-speed and high-accuracy pedestrian speed measurement is realized. The present invention is described in detail below in conjunction with the embodiments.

[0032] In a first embodiment of the present invention, Figure 1 As shown, a pedestrian speed measurement method based on a monocular camera is provided, which includes:

[0033] Step S1: Loading original image data;

[0034] In this embodiment, the original image data is loaded by a monocular camera or a video file.

[0035] Step S2: Perform target detection on the original image data and output the detection result;

[0036] In this embodiment, the YOLOv3 deep neural network model is used to complete multi-target detection and output the detection box category and coordinate information.

[0037] Step S3: performing associated tracking of targets in different frames on the target detection results, and outputting detection results that meet the tracking constraint conditions;

[0038] In this embodiment, the DeepSORT algorithm is used to realize the association tracking of targets in different frames, and the category and coordinate information of the associated tracked targets are output;

[0039] Step S4: Calculate the pixel difference of the detection frame vertices and the time difference between the previous and next frames;

[0040] The calculations include:

[0041] Step S41: Initialize the preset time memory and the pixel difference memory of the detection frame vertices, and decompose the original motion pixel difference of the detection frame vertices into the X direction and the Y direction, in preparation for iterative calculation of the pixel difference of the detection frame vertices.

[0042] Step S42: Calculate the pixel difference between the vertices of the detection frames of the two previous and next frames in the X direction and store it in a pixel difference memory.

[0043] Step S43: Calculate the time difference between the two frames and record it in the time memory.

[0044] Step S44: Calculate the pixel difference between the vertices of the detection frames of the two previous and next frames in the Y direction and store it in a pixel difference memory.

[0045] Step S5: The time difference and the pixel difference are accumulated to determine the time, so as to ensure that the stored data is within a preset time threshold.

[0046] Specifically, the cumulative time judgment method is: while storing the pixel difference, record the timestamp of the current frame, and then judge whether the time difference between the beginning and the end of the time memory is greater than the set time threshold. If it is greater than the time threshold, the pixel difference in the pixel difference memory is accumulated and summed, and the process goes to step S6 to perform the mapping calculation between the pixel and the actual distance, and deletes the data of the starting position in the time memory and the pixel difference memory; if it is less than the time threshold, it returns to step S4 to recalculate until the result is satisfied;

[0047] In this embodiment, the preset time threshold is 1 second.

[0048] Step S6: Map the pixel difference to obtain the actual distance, and then accumulate according to the pixels to obtain the moving distance in the X direction and the Y direction when the time threshold moves left and right (for example, moving left and right for 1 second).

[0049] In this embodiment, the pixel difference mapping formula is:

[0050]

[0051]

[0052]

[0053] like Figure 2 and Figure 3 As shown, in the above formula, w1 is the distance between the position of the vertical point of the camera and the nearest end of the vertical field of view; w2 is the distance from the nearest end of the vertical field of view to the center line of the field of view; w3 is the distance from the farthest end of the vertical field of view to the center line of the field of view; w4 is the distance from the farthest end of the vertical field of view to the extension line of the center line of the field of view; h is the height of the camera; θ1 is the angle between the vertical line of the camera and the center line of the camera field of view; n is the number of pixels in the Y direction; m is a constant, and its value range is [1,n-2]; x0 is the actual distance corresponding to the first pixel vertically upward from the center of the camera imaging plane; x1 is the actual distance corresponding to the second pixel vertically upward from the center of the camera imaging plane; Y is the distance in the real scene corresponding to each pixel in the vertical direction of the camera imaging plane; H0 is the distance from each pixel point to the ground in the vertical direction of the camera imaging plane; θ0 is half of the horizontal field of view angle of the camera; w5 is half of the horizontal field of view width; X is the distance in the real scene corresponding to each pixel in the horizontal direction of the camera imaging plane.

[0054] Step S7: Calculate the movement distance by moving the time thresholds in the X and Y directions in the real scene to the left and right according to the Pythagorean theorem.

[0055] Step S8: Based on the movement distance obtained in step S7 and the accumulated time obtained in step S5, the target movement speed is obtained using the speed calculation formula (v=s / t).

[0056] In a second embodiment of the present invention, a pedestrian speed measurement system is provided, which includes: a loading module, a target detection module, a tracking module, a difference calculation module, a judgment module, a mapping module, a movement distance acquisition module and a target speed acquisition module;

[0057] The loading module is used to load the original image data;

[0058] The target detection module performs target detection on the original image data and outputs the detection results;

[0059] The tracking module performs associated tracking of targets in different frames based on the target detection results, and outputs the detection results that meet the tracking constraints;

[0060] The difference calculation module is used to calculate the pixel difference of the detection box vertices and the time difference between the previous and next frames;

[0061] The judgment module accumulates the time difference and the pixel difference to make a time judgment, ensuring that the stored data is within the preset time threshold;

[0062] The mapping module maps the pixel difference to obtain the actual distance, and then accumulates the pixels to obtain the X-direction and Y-direction movement distance of the time threshold.

[0063] The movement distance acquisition module calculates the movement distance according to the Pythagorean theorem by moving the time thresholds in the X and Y directions in the real scene to the left and right.

[0064] The target speed acquisition module obtains the target movement speed using the speed calculation formula based on the movement distance and accumulated time.

[0065] In a third embodiment of the present invention, a computer-readable storage medium storing one or more programs is provided. The one or more programs include instructions. When the instructions are executed by a computing device, the computing device executes any method in the first embodiment described above.

[0066] In a fourth embodiment of the present invention, a computing device is provided, comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by one or more processors, and the one or more programs include instructions for executing any method as described in the first embodiment above.

[0067] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0068] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0069] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0070] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

Claims

1. A method for measuring pedestrian speed, characterized in that: include: Step S1: Loading original image data; Step S2: Perform target detection on the original image data and output the detection result; Step S3: performing associated tracking of targets in different frames on the target detection results, and outputting detection results that meet the tracking constraint conditions; Step S4: Calculate the pixel difference of the detection frame vertices and the time difference between the previous and next frames; Step S5: accumulating the time difference and the pixel difference to determine the time, and ensuring that the stored data is within a preset time threshold; Step S6: Map the pixel difference to obtain the actual distance, and then accumulate according to the pixels to obtain the X-direction and Y-direction movement distance of the time threshold; Step S7: Calculate the movement distance by moving the time thresholds in the X and Y directions in the real scene to the left and right according to the Pythagorean theorem; Step S8: According to the movement distance and the accumulated time, the target movement speed is obtained by using the speed calculation formula; In step S6, the pixel difference mapping formula is: Where: w1 is the distance between the vertical point of the camera and the nearest end of the vertical field of view; w2 is the distance from the nearest end of the vertical field of view to the center line of the field of view; w3 is the distance from the farthest end of the vertical field of view to the center line of the field of view; w4 is the distance from the farthest end of the vertical field of view to the extended line of the center line of the field of view; h is the height of the camera; θ1 is the angle between the vertical line of the camera and the center line of the camera field of view; n is the number of pixels in the Y direction; m is a constant with a value range of [1, n-2]; x0 is the actual distance corresponding to the first pixel vertically upward from the center of the camera imaging plane; x1 is the actual distance corresponding to the second pixel vertically upward from the center of the camera imaging plane; Y is the distance in the real scene corresponding to each pixel in the vertical direction of the camera imaging plane; H0 is the distance from each pixel to the ground in the vertical direction of the camera imaging plane; θ0 is half of the horizontal field of view angle of the camera; w5 is half of the horizontal field of view width; X is the distance in the real scene corresponding to each pixel in the horizontal direction of the camera imaging plane.

2. The measuring method according to claim 1, characterized in that: In step S1, the original image data is loaded by using a monocular camera or a video file.

3. The measuring method according to claim 1, characterized in that: In step S2, the YOLOv3 deep neural network model is used to complete multi-target detection and output the detection box category and coordinate information.

4. The measuring method according to claim 1, characterized in that: In step S3, the DeepSORT algorithm is used to realize the associated tracking of targets in different frames, and the associated tracked target category and coordinate information are output.

5. The measuring method according to claim 1, characterized in that: In step S4, the specific calculation includes: Step S41: Initialize a preset time memory and a pixel difference memory of the detection frame vertices, and decompose the original motion pixel difference of the detection frame vertices into the X direction and the Y direction; Step S42: Calculate the pixel difference between the vertices of the detection frames of the two frames in the X direction and store it in a pixel difference memory; Step S43: Calculate the time difference between the previous and next two frames and record it in the time memory; Step S44: Calculate the pixel difference between the vertices of the detection frames of the two previous and next frames in the Y direction and store it in a pixel difference memory.

6. The measuring method according to claim 1, characterized in that: In step S5, the cumulative time judgment method is: storing the pixel difference while recording the timestamp of the current frame, and then judging whether the time difference between the beginning and the end of the time memory is greater than the set time threshold. If it is greater than the time threshold, the pixel difference in the pixel difference memory is accumulated and summed, and the process goes to step S6 to perform the mapping calculation between the pixel and the actual distance, and delete the data of the starting position in the time memory and the pixel difference memory; if it is less than the time threshold, the process returns to step S4 and recalculates until the result is satisfied.

7. A pedestrian speed measurement system, characterized in that: include: Loading module, target detection module, tracking module, difference calculation module, judgment module, mapping module, motion distance acquisition module and target speed acquisition module; The loading module is used to load original image data; The target detection module performs target detection on the original image data and outputs the detection result; The tracking module performs associated tracking of targets in different frames on the target detection results, and outputs the detection results that meet the tracking constraint conditions; The difference calculation module is used to calculate the pixel difference of the detection frame vertices and the time difference between the previous and next frames; The judgment module performs cumulative time judgment on the time difference and the pixel difference to ensure that the stored data is within a preset time threshold; The mapping module maps the pixel difference to obtain the actual distance, and then accumulates the pixels to obtain the moving distance in the X direction and the Y direction of the time threshold. The movement distance acquisition module calculates the movement distance by moving the time thresholds in the X and Y directions in the real scene to the left and right according to the Pythagorean theorem; The target speed acquisition module obtains the target movement speed using a speed calculation formula according to the movement distance and the accumulated time; In the mapping module, the pixel difference mapping formula is: Where: w1 is the distance between the vertical point of the camera and the nearest end of the vertical field of view; w2 is the distance from the nearest end of the vertical field of view to the center line of the field of view; w3 is the distance from the farthest end of the vertical field of view to the center line of the field of view; w4 is the distance from the farthest end of the vertical field of view to the extended line of the center line of the field of view; h is the height of the camera; θ1 is the angle between the vertical line of the camera and the center line of the camera field of view; n is the number of pixels in the Y direction; m is a constant with a value range of [1, n-2]; x0 is the actual distance corresponding to the first pixel vertically upward from the center of the camera imaging plane; x1 is the actual distance corresponding to the second pixel vertically upward from the center of the camera imaging plane; Y is the distance in the real scene corresponding to each pixel in the vertical direction of the camera imaging plane; H0 is the distance from each pixel to the ground in the vertical direction of the camera imaging plane; θ0 is half of the horizontal field of view angle of the camera; w5 is half of the horizontal field of view width; X is the distance in the real scene corresponding to each pixel in the horizontal direction of the camera imaging plane.

8. A computer-readable storage medium storing one or more programs, characterized in that: The one or more programs include instructions, which, when executed by a computing device, cause the computing device to perform any one of the methods of claims 1 to 6.

9. A computing device, characterized in that include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include instructions for executing any one of the methods described in claims 1 to 6.

Images