Highly real-time pedestrian tracking method, device and readable medium

Abstract

The invention discloses a highly real-time pedestrian tracking method, device and readable medium, which relate to the field of pedestrian following, establish a corresponding relationship between the actual size of a pedestrian and the size of the pedestrian in an image taken by the pedestrian following a ground track; obtain a current frame image taken by the pedestrian following the ground track, input it into a trained target detection model, obtain a detection result, perform a similarity comparison between the detection result and the detection result of the previous moment in a time series data record, and judge whether it is the same person according to the comparison result; determine whether there is a historical tracking person in the time series data record in the detection result, update the tracking person, perform weighted interpolation on the displacement of the tracking person, and obtain the predicted displacement of the tracking person; obtain the movement direction and speed of the tracking person according to the corresponding relationship and the predicted displacement, determine the movement direction and movement speed of the pedestrian following the ground track according to the movement direction and speed, and control the pedestrian to move following the ground track, so as to solve the problems of low real-time performance and high cost.

Description

Technical Field

[0001] The present invention relates to the field of pedestrian following, and in particular to a high-real-time pedestrian tracking method, device and readable medium. Background Art

[0002] Pedestrian tracking has a wide range of application scenarios. Pedestrian tracking requires equipment to follow the scenes, such as the camera tracking performers on the stage, robots tracking and intercepting intruders outside the dangerous area, etc. Compared with monitoring and other record-keeping tracking that does not require changing its own position, it requires higher real-time performance and flexibility.

[0003] The current manual tracking method is costly and cannot maintain work efficiency for a long time. The invention patent with application number CN202111471521.1 and patent name "A Pedestrian Tracking and Monitoring Method and System Based on Deep Neural Network" uses machine vision algorithms and predicts pedestrian movements based on time series information, but it uses a large amount of computational SSD model and feature matching, which makes it difficult to achieve high real-time performance, and the motion prediction method is linear interpolation, which has poor adaptability to pedestrian movement changes during the following process and has low real-time performance. The invention patent with application number CN 201710620228.4 and patent name "Pedestrian Tracking Method and Device" requires the use of depth equipment to collect depth information, which has high hardware requirements for the equipment and cannot be achieved using only ordinary optical cameras. Summary of the invention

[0004] In view of the above-mentioned technical problems, the purpose of the embodiments of the present application is to propose a highly real-time pedestrian tracking method, device and readable medium to solve the technical problems mentioned in the above background technology section.

[0005] In a first aspect, an embodiment of the present application provides a highly real-time pedestrian tracking method, comprising the following steps:

[0006] S1, establishing a corresponding relationship between the actual size of the pedestrian and the size of the pedestrian in the image taken by the pedestrian following the ground track;

[0007] S2, obtaining the current frame image taken by the pedestrian following the ground track, and inputting it into the trained target detection model to obtain the detection result, and performing a similarity comparison between the detection result and the detection result of the previous moment in the time series data record, and judging whether it is the same person according to the comparison result;

[0008] S3, determining whether there is a historical tracked person in the time series data record in the detection result, updating the tracked person, performing weighted interpolation on the displacement of the tracked person, and obtaining a predicted displacement of the tracked person;

[0009] S4, obtaining the movement direction and speed of the tracked person according to the corresponding relationship and the predicted displacement, determining the movement direction and speed of the pedestrian following the ground track according to the movement direction and speed, and controlling the pedestrian to move following the ground track according to the movement direction and speed.

[0010] Preferably, step S1 specifically includes:

[0011] Obtain the actual average height and average shoulder width of pedestrians in the target scene;

[0012] Get the average height and average width of pedestrians in the images taken by pedestrians following the ground track;

[0013] A correspondence is established based on the actual average height and average shoulder width of pedestrians and the average height and average width of pedestrians in the image.

[0014] Preferably, the target detection model includes a YOLO model.

[0015] Preferably, the time series data record includes N detection results adjacent to the current moment.

[0016] Preferably, the similarity comparison in step S2 includes IoU calculation.

[0017] Preferably, step S3 specifically includes:

[0018] Obtain the historical tracking person from the time series data record, and check whether there is a historical tracking person in the detection result. If yes, the historical tracking person is used as the tracking person. Otherwise, the person closest to the center of the lens of the pedestrian following the ground track is selected as the tracking person and stored in the time series data record;

[0019] Perform weighted interpolation according to the following formula to obtain the predicted displacement P of the tracked person:

[0020] P = q*(O 1 -O 2 )+(1-q)*q* (O 2 -O 3 )+(1-q) 2 *q* (O 3 -O 4 )+……+ (1-q) (N-2) *q*(O (N-2) -O (N-1) )+(1-q) (N-1) * (O (N-1) -O N );

[0021] Among them, 0.5≤q<1, O 1 is the detection result of the tracked person in the current frame image, and so on.N is the detection result of the tracked person in the Nth detection.

[0022] Preferably, step S4 specifically includes:

[0023] If the x-direction value in the predicted displacement is greater than 0, the tracking person's movement direction is to the right; if the x-direction value in the predicted displacement is less than 0, the tracking person's movement direction is to the left; the movement direction of the pedestrian following the ground track is the same as the movement direction of the tracking person;

[0024] The speed V of the tracking person is calculated according to the following formula 1 :

[0025] V 1 =P / b*a;

[0026] Among them, a is the actual average shoulder width of pedestrians in the corresponding relationship, and b is the average width of pedestrians in the image taken when pedestrians follow the ground track;

[0027] The moving speed V of the pedestrian following the ground track is calculated according to the following formula: 2 :

[0028] V 2 =α*V 1 ;

[0029] Among them, α>1.

[0030] In a second aspect, an embodiment of the present application provides a highly real-time pedestrian tracking device, comprising:

[0031] a corresponding relationship establishing module, configured to establish a corresponding relationship between the actual size of the pedestrian and the size of the pedestrian in the image taken by the pedestrian following the ground track;

[0032] The detection comparison module is configured to obtain a current frame image taken by a pedestrian following the ground track, input the image into a trained target detection model, obtain a detection result, compare the detection result with the detection result of the previous moment in the time series data record for similarity, and determine whether it is the same person according to the comparison result;

[0033] A prediction module is configured to determine whether there is a historical tracked person in the time series data record in the detection result to update the tracked person, perform weighted interpolation on the displacement of the tracked person, and obtain a predicted displacement of the tracked person;

[0034] The speed calculation module is configured to obtain the movement direction and speed of the tracked person according to the corresponding relationship and the predicted displacement, determine the movement direction and movement speed of the pedestrian following the ground track according to the movement direction and movement speed, and control the pedestrian to move following the ground track according to the movement direction and movement speed.

[0035] In a third aspect, an embodiment of the present application provides an electronic device comprising one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by one or more processors, the one or more processors implement a method as described in any implementation manner in the first aspect.

[0036] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any implementation manner in the first aspect.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] (1) The real-time pedestrian tracking method of the present invention uses the YOLO model to quickly detect the position of the pedestrian, predicts the direction of the pedestrian's movement in the next frame based on the timing information and the previous detection results, and then guides the motion system to follow.

[0039] (2) The real-time pedestrian tracking method of the present invention can realize the tracking and identification of the same pedestrian and the estimation of the movement tendency. No special equipment is required and it can be realized on the basis of the existing pedestrian following ground rail system, which can save labor costs and equipment costs and improve the real-time tracking performance.

[0040] (3) The real-time pedestrian tracking method of the present invention can effectively solve the problem of low real-time performance in the prior art and has low computational complexity. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0042] Figure 1 is an exemplary device architecture diagram to which an embodiment of the present application may be applied;

[0043] Figure 2 A flowchart of a high-real-time pedestrian tracking method according to an embodiment of the present application;

[0044] Figure 3 A flowchart of a high-real-time pedestrian tracking method according to an embodiment of the present application;

[0045] Figure 4 A schematic diagram of a high real-time pedestrian tracking device according to an embodiment of the present application;

[0046] Figure 5It is a schematic diagram of the structure of a computer device suitable for implementing the electronic device of the embodiment of the present application. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] Figure 1 An exemplary device architecture 100 is shown to which a high-real-time pedestrian tracking method or a high-real-time pedestrian tracking device according to an embodiment of the present application can be applied.

[0049] like Figure 1 As shown, the device architecture 100 may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0050] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Various applications, such as data processing applications, file processing applications, etc., can be installed on terminal devices 101, 102, 103.

[0051] Terminal devices 101, 102, 103 may be hardware or software. When terminal devices 101, 102, 103 are hardware, they may be various electronic devices, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, etc. When terminal devices 101, 102, 103 are software, they may be installed in the electronic devices listed above. They may be implemented as multiple software or software modules (for example, software or software modules for providing distributed services), or as a single software or software module. No specific limitation is made here.

[0052] The server 105 may be a server that provides various services, such as a background data processing server that processes files or data uploaded by the terminal devices 101, 102, and 103. The background data processing server may process the acquired files or data and generate processing results.

[0053] It should be noted that the high real-time pedestrian tracking method provided in the embodiment of the present application can be executed by the server 105 or by the terminal devices 101, 102, 103. Accordingly, the high real-time pedestrian tracking device can be set in the server 105 or in the terminal devices 101, 102, 103.

[0054] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided as required. In the case where the processed data does not need to be obtained remotely, the above device architecture may not include a network, but only a server or a terminal device.

[0055] Figure 2 A highly real-time pedestrian tracking method provided by an embodiment of the present application is shown, comprising the following steps:

[0056] S1, establishing a corresponding relationship between the actual size of the pedestrian and the size of the pedestrian in the image taken when the pedestrian follows the ground track.

[0057] Specifically, refer to Figure 3 , collect pedestrian images in simulated or real target scenes, collect pedestrian image data Images1, use labelImg software to annotate Images1 with pedestrians, save them as yolo format txt files, and obtain annotated Labels1. Images1 and Labels1 together constitute the pedestrian dataset Dataset1.

[0058] In a specific embodiment, step S1 specifically includes:

[0059] Obtain the actual average height and average shoulder width of pedestrians in the target scene;

[0060] Get the average height and average width of pedestrians in the images taken by pedestrians following the ground track;

[0061] A correspondence is established based on the actual average height and average shoulder width of pedestrians and the average height and average width of pedestrians in the image.

[0062] Specifically, the actual average height and average shoulder width of pedestrians in the target scene are obtained, and the average height and average width of pedestrian annotations in Labels1 are calculated. By combining the actual data with the annotation data, the pedestrians themselves can be used as the measurement scale to derive the correspondence between the displacement of pixels in the images taken by pedestrians following the ground track and the actual displacement.

[0063] S2, obtain the current frame image taken by the pedestrian following the ground track, and input it into the trained target detection model to obtain the detection result, compare the detection result with the detection result of the previous moment in the time series data record for similarity, and determine whether it is the same person based on the comparison result.

[0064] Specifically, during actual detection, for videos of pedestrians following the ground track, frame images are acquired at fixed intervals. For high-speed cameras, only one frame from multiple adjacent frames may be taken for detection to reduce input repetitiveness.

[0065] In a specific embodiment, the target detection model includes a YOLO model. The training process of the target detection model is as follows:

[0066] Divide the pedestrian dataset DataSet1 into training set, validation set and test set according to a certain ratio; select different versions and structures of the YOLO model according to the actual detection requirements, nano / s / m / l structure, the calculation amount and accuracy are continuously improved from left to right, but the time consumption is also increased and the real-time performance is reduced. Load the official weights for transfer learning. Use the test set to test the trained YOLO model, judge the model according to the detection rate and positioning accuracy, if it does not meet the requirements, extend the training time, or adjust the hyperparameters and retrain the model.

[0067] In a specific embodiment, the time series data record includes N detection results adjacent to the current moment. To save memory, the time series data record Record1 only stores the results of the N adjacent detections, and stores the detection results of the current frame image. 1 After that, discard the test results before N times O （N+1） , so that the total number of detection results in the time series data record Record1 is maintained at N. Then combined with the detection result O of the current frame image 1 And O in the time series data record Record1 2 , O 3 , ... O N Identify the same person.

[0068] In a specific embodiment, the similarity comparison in step S2 includes IoU calculation.

[0069] Specifically, in the embodiment of the present application, the IoU series function is used to determine the same person, that is, the IoU series function is used to determine the same person. I The detection box in O (I+1) The detection box in the IoU calculation is performed, and for each O I The detection box selection in O (I+1)The detection frame that can minimize the IoU is identified as the same person. Among them, the IoU series functions include original IoU, DIoU, GIoU, CIoU, ALPHAIoU, etc. Except for the original IoU which is not recommended because the intersection of the detection frames may be 0 under high-speed motion, all others can be used. Using IoU to identify the same person is only a simplified simulation, which is not as accurate as feature recognition and other methods in theoretical terms. However, the following ability of the pedestrian following the track system is limited and can only follow one person. Even if the IoU judgment confuses the following object, in most cases it can still ensure that the pedestrian follows at least one person. In actual calculations, as long as it is determined that O 1 With O 2 Whether they belong to the same person, the same person identification in the other detection results can be cached and recorded in the time series data record Record1.

[0070] S3, determining whether there is a historical tracked person in the time series data record in the detection result, updating the tracked person, performing weighted interpolation on the displacement of the tracked person, and obtaining a predicted displacement of the tracked person.

[0071] In a specific embodiment, step S3 specifically includes:

[0072] Obtain the historical tracking person from the time series data record, and check whether there is a historical tracking person in the detection result. If yes, the historical tracking person is used as the tracking person. Otherwise, the person closest to the center of the lens of the pedestrian following the ground track is selected as the tracking person and stored in the time series data record;

[0073] Perform weighted interpolation according to the following formula to obtain the predicted displacement P of the tracked person:

[0074] P = q*(O 1 -O 2 )+(1-q)*q* (O 2 -O 3 )+(1-q) 2 *q* (O 3 -O 4 )+……+ (1-q) (N-2) *q*(O (N-2) -O (N-1) )+(1-q) (N-1) * (O (N-1) -O N );

[0075] Among them, 0.5≤q<1, O 1 is the detection result of the tracked person in the current frame image, and so on. N is the detection result of the person being tracked in the Nth detection, and requires that the sum of the coefficients before the difference of all adjacent detection results is 1.

[0076] Specifically, obtain the historical tracking person Person0 from the time series data record Record1, check whether Person0 still exists in Output1, if so, Person1 = Person0, otherwise select the person closest to the center of the lens as Person1, put the Person1 record back into the time series data record Record1, and update Person0.

[0077] Specifically, interpolation means that the predicted displacement is a combination of multiple displacements. The process of assigning weights to each displacement is called interpolation. Weighted interpolation emphasizes giving greater weights to adjacent output results to better simulate the real-time movement of pedestrians. If Person1 does not exist in a certain detection result, it is considered to have entered from both sides based on prior knowledge. Depending on whether Person1 is on the left or right side of the screen in the adjacent frame, the frame of Person1 in the adjacent frame is moved left / right out of the camera and a corresponding inferred position is added.

[0078] S4, obtaining the movement direction and speed of the tracked person according to the corresponding relationship and the predicted displacement, determining the movement direction and speed of the pedestrian following the ground track according to the movement direction and speed, and controlling the pedestrian to move following the ground track according to the movement direction and speed.

[0079] In a specific embodiment, step S4 specifically includes:

[0080] If the x-direction value in the predicted displacement is greater than 0, the tracking person's movement direction is to the right; if the x-direction value in the predicted displacement is less than 0, the tracking person's movement direction is to the left; the movement direction of the pedestrian following the ground track is the same as the movement direction of the tracking person;

[0081] The speed V of the tracking person is calculated according to the following formula 1 :

[0082] V 1 =P / b*a;

[0083] Among them, a is the actual average shoulder width of pedestrians in the corresponding relationship, and b is the average width of pedestrians in the image taken when pedestrians follow the ground track;

[0084] The moving speed V of the pedestrian following the ground track is calculated according to the following formula: 2 :

[0085] V 2 =α*V 1 ;

[0086] Among them, α>1.

[0087] Specifically, due to the particularity of the ground rail system movement (only one degree of freedom), the direction of movement can be simplified to left (front) or right (back), which can be judged according to whether the x direction in the predicted displacement P is greater than / less than 0. According to the coordinate system of the cv2 library, x>0 means rightward. The ground rail can be simplified as a straight track on the ground, and the camera's perspective is perpendicular to the track, so the left and right of the camera are the front and back on the track. There are only two front and back choices with a difference of 180 degrees in the direction, so the direction of movement can be determined according to the magnitude of the x direction value in the predicted displacement. The speed V1 of the tracked person is multiplied by a pre-set control factor α greater than one, and is passed to the motion system of the pedestrian following the ground rail to make it move. α>1 is to make up for the possible difference between the predicted actual speed and the actual speed, and to avoid the situation where the pedestrian speed exceeds the ground rail speed and the distance gradually increases. Since the ground rail movement speed is greater than the pedestrian speed, the ground rail may catch up with and exceed the pedestrian and move repeatedly. It is necessary to control the movement threshold of the ground rail, and do not move when the pedestrian deviates from the visual center by a small amount and the speed in the movement command is less than the threshold.

[0088] Further references Figure 4 As an implementation of the methods shown in the above figures, the present application provides an embodiment of a high real-time pedestrian tracking device. Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0089] The embodiment of the present application provides a highly real-time pedestrian tracking device, comprising:

[0090] A corresponding relationship establishing module 1 is configured to establish a corresponding relationship between the actual size of the pedestrian and the size of the pedestrian in the image taken when the pedestrian follows the ground track;

[0091] The detection and comparison module 2 is configured to obtain the current frame image taken by the pedestrian following the ground track, input the image into the trained target detection model, obtain the detection result, compare the detection result with the detection result of the previous moment in the time series data record for similarity, and determine whether it is the same person according to the comparison result;

[0092] Prediction module 3 is configured to determine whether there is a historical tracked person in the time series data record in the detection result to update the tracked person, perform weighted interpolation on the displacement of the tracked person, and obtain a predicted displacement of the tracked person;

[0093] The speed calculation module 4 is configured to obtain the movement direction and speed of the tracked person according to the corresponding relationship and the predicted displacement, determine the movement direction and movement speed of the pedestrian following the ground track according to the movement direction and movement speed, and control the pedestrian to move along the ground track according to the movement direction and movement speed.

[0094] Reference below Figure 5, which shows an electronic device (eg Figure 1 A schematic diagram of the structure of a computer device 500 (a server or terminal device as shown). Figure 5 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0095] like Figure 5 As shown, the computer device 500 includes a central processing unit (CPU) 501 and a graphics processing unit (GPU) 502, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 503 or a program loaded from a storage part 509 to a random access memory (RAM) 504. In the RAM 504, various programs and data required for the operation of the device 500 are also stored. The CPU 501, GPU 502, ROM 503 and RAM 504 are connected to each other through a bus 505. An input / output (I / O) interface 506 is also connected to the bus 505.

[0096] The following components are connected to the I / O interface 506: an input section 507 including a keyboard, a mouse, etc.; an output section 508 including a display such as a liquid crystal display (LCD), etc. and a speaker, etc.; a storage section 509 including a hard disk, etc.; and a communication section 510 including a network interface card such as a LAN card, a modem, etc. The communication section 510 performs communication processing via a network such as the Internet. A drive 511 can also be connected to the I / O interface 506 as needed. A removable medium 512, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 511 as needed so that a computer program read therefrom is installed into the storage section 509 as needed.

[0097] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication part 510, and / or installed from a removable medium 512. When the computer program is executed by a central processing unit (CPU) 501 and a graphics processing unit (GPU) 502, the above-mentioned functions defined in the method of the present application are executed.

[0098] It should be noted that the computer-readable medium described in the present application may be a computer-readable signal medium or a computer-readable medium or any combination of the above two. The computer-readable medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared or semiconductor device, device or component, or any combination of the above. More specific examples of computer-readable media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution device, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium may also be any computer-readable medium other than a computer-readable medium that can send, propagate or transmit a program for use by or in conjunction with an instruction execution device, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0099] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0100] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the devices, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based device that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0101] The modules involved in the embodiments described in this application may be implemented by software or hardware, and the modules described may also be set in a processor.

[0102] As another aspect, the present application also provides a computer-readable medium, which may be included in the electronic device described in the above embodiment; or it may exist independently and not be assembled into the electronic device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by the electronic device, the electronic device: establishes a corresponding relationship between the actual size of the pedestrian and the size of the pedestrian in the image taken by the pedestrian following the ground track; obtains the current frame image taken by the pedestrian following the ground track, and inputs the trained target detection model to obtain the detection result, and compares the detection result with the detection result of the previous moment in the time series data record for similarity, and determines whether it is the same person according to the comparison result; determines whether there is a historical tracking person in the time series data record in the detection result, updates the tracking person, and performs weighted interpolation on the displacement of the tracking person to obtain the predicted displacement of the tracking person; obtains the movement direction and speed of the tracking person according to the corresponding relationship and the predicted displacement, determines the movement direction and movement speed of the pedestrian following the ground track according to the movement direction and speed, and controls the movement of the pedestrian following the ground track according to the movement direction and movement speed.

[0103] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above invention concept. For example, the above features are replaced with the technical features with similar functions disclosed in this application (but not limited to) by each other to form a technical solution.

Claims

1. A highly real-time pedestrian tracking method, characterized in that: The following steps are involved: S1, establishing a corresponding relationship between the actual size of the pedestrian and the size of the pedestrian in the image taken by the pedestrian following the ground track; S2, obtaining a current frame image taken by a pedestrian following the ground track, and inputting the image into a trained target detection model to obtain a detection result, performing a similarity comparison between the detection result and the detection result at the previous moment in the time series data record, and determining whether the pedestrian is the same person according to the comparison result; S3, determining whether there is a historical tracked person in the time series data record in the detection result to update the tracked person, performing weighted interpolation on the displacement of the tracked person to obtain the predicted displacement of the tracked person, the step S3 specifically includes: Obtaining a historical tracking person from the time series data record, searching whether the historical tracking person exists in the detection result, and if so, taking the historical tracking person as the tracking person, otherwise selecting a person closest to the center of the lens of the pedestrian following the ground track as the tracking person, and storing the person in the time series data record; The predicted displacement P of the tracked person is obtained by performing weighted interpolation according to the following formula: P=q*(O1-O2)+(1-q)*q*(O2-O3)+(1-q) 2 *q*(O3-O4)+……+(1-q) (N-2) *q*(O (N-2) -O (N-1) )+(1-q) (N-1) *(O (N-1) -O N ); Among them, 0.5≤q<1, O1 is the detection result of the tracked person in the current frame image, and so on, O N is the detection result of the tracked person described in the Nth detection; S4, obtaining the movement direction and speed of the tracked person according to the corresponding relationship and the predicted displacement, determining the movement direction and speed of the pedestrian following the track according to the movement direction and speed, and controlling the pedestrian to move following the track according to the movement direction and speed.

2. The high real-time pedestrian tracking method according to claim 1, characterized in that: The step S1 specifically includes: Obtain the actual average height and average shoulder width of pedestrians in the target scene; Get the average height and average width of pedestrians in the images taken by pedestrians following the ground track; The corresponding relationship is established according to the actual average height and average shoulder width of the pedestrians and the average height and average width of the pedestrians in the image.

3. The high real-time pedestrian tracking method according to claim 1, characterized in that: The target detection model includes a YOLO model.

4. The high real-time pedestrian tracking method according to claim 1, characterized in that: The time series data record includes N detection results adjacent to the current moment.

5. The high real-time pedestrian tracking method according to claim 1, characterized in that: The similarity comparison in step S2 includes IoU calculation.

6. The high real-time pedestrian tracking method according to claim 1, characterized in that: The step S4 specifically includes: If the x-direction value in the predicted displacement is greater than 0, the movement direction of the tracked person is to the right; if the x-direction value in the predicted displacement is less than 0, the movement direction of the tracked person is to the left; the movement direction of the pedestrian following the ground track is the same as the movement direction of the tracked person; The speed V1 of the tracked person is calculated according to the following formula: V1=P / b*a; Wherein, a is the actual average shoulder width of the pedestrian in the corresponding relationship, and b is the average width of the pedestrian in the image taken when the pedestrian follows the ground track; The moving speed V2 of the pedestrian following the ground track is calculated according to the following formula: V2 = α * V1; Among them, α>1.

7. A highly real-time pedestrian tracking device, characterized in that: include: a corresponding relationship establishing module, configured to establish a corresponding relationship between the actual size of the pedestrian and the size of the pedestrian in the image taken by the pedestrian following the ground track; The detection comparison module is configured to obtain a current frame image taken by a pedestrian following the ground track, input the image into a trained target detection model, obtain a detection result, compare the detection result with the detection result at a previous moment in the time series data record for similarity, and determine whether the pedestrian is the same person based on the comparison result; The prediction module is configured to determine whether there is a historical tracking person in the time series data record in the detection result to update the tracking person, and perform weighted interpolation on the displacement of the tracking person to obtain the predicted displacement of the tracking person, specifically including: Obtaining a historical tracking person from the time series data record, searching whether the historical tracking person exists in the detection result, and if so, taking the historical tracking person as the tracking person, otherwise selecting a person closest to the center of the lens of the pedestrian following the ground track as the tracking person, and storing the person in the time series data record; The predicted displacement P of the tracked person is obtained by performing weighted interpolation according to the following formula: P=q*(O1-O2)+(1-q)*q*(O2-O3)+(1-q) 2 *q*(O3-O4)+……+(1-q) (N-2) *q*(O (N-2) -O (N-1) )+(1-q) (N-1) *(O (N-1) -O N ); Among them, 0.5≤q<1, O1 is the detection result of the tracked person in the current frame image, and so on, O N is the detection result of the tracked person described in the Nth detection; The speed calculation module is configured to obtain the movement direction and speed of the tracked person according to the corresponding relationship and the predicted displacement, determine the movement direction and movement speed of the pedestrian following the ground track according to the movement direction and movement speed, and control the pedestrian to move following the ground track according to the movement direction and movement speed.

8. An electronic device comprising: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images