Pedestrian tracking method, device, computer equipment and storage medium

Abstract

The embodiment of the present invention discloses a pedestrian tracking method, device, computer equipment and storage medium. The method includes: if it is determined that the indoor navigation robot establishes a follow-up navigation relationship with the target pedestrian, then each time the tracking time point is reached, the detection collection image is obtained through the indoor environment camera; at least one detection target is identified, and the tracking prediction target for the target pedestrian is marked; according to the pedestrian image collected by the indoor navigation robot for the target pedestrian and the set follow-up navigation route, the first fusion contrast feature corresponding to each detection target and the second fusion contrast feature corresponding to the tracking prediction target are calculated; according to the similarity between each first fusion contrast feature and the second fusion contrast feature, the target pedestrian is identified in the detection target to track the target pedestrian. The problem that the indoor navigation robot does not have the ability to actively follow in real time is solved, the function of real-time following of the customer is realized, and the intelligence level of the robot is improved.

Description

Technical Field

[0001] The embodiments of the present invention relate to computer vision technology, and more particularly to a pedestrian tracking method, apparatus, computer equipment and storage medium. Background Art

[0002] At present, in the lobby of a bank branch, the lobby manager often bears a very heavy workload. In addition to providing relevant business consultation, he also has to guide customers to the designated location to handle relevant business. Relatively speaking, the workload is quite heavy.

[0003] However, the indoor robots used in banks to guide customers in the bank lobby often do not provide real-time guidance services and are difficult to adjust the tracking direction and objects in real time. Especially when there are many customers in the branch lobby, the indoor robots may make misjudgments, resulting in a poor customer experience. Summary of the invention

[0004] The embodiments of the present invention provide a pedestrian tracking method, device, computer equipment and storage medium, which solve the problem that indoor robots do not have the ability to actively follow in real time, and realize the service function of real-time following of customers.

[0005] In a first aspect, an embodiment of the present invention provides a pedestrian tracking method, which includes:

[0006] If it is determined that the indoor navigation robot establishes a following navigation relationship with the target pedestrian, then each time the tracking time point is reached, a detection collection image is obtained through the indoor environment camera;

[0007] Identifying at least one detection target in the detection acquisition image, and identifying a tracking prediction target for the target pedestrian in the detection acquisition image;

[0008] According to the pedestrian images collected by the indoor navigation robot for the target pedestrian and the set follow-up navigation route, a first fused contrast feature corresponding to each detected target and a second fused contrast feature corresponding to the tracking prediction target are calculated;

[0009] According to the similarity between each first fused contrast feature and the second fused contrast feature, a target pedestrian is identified in the detection target to track the target pedestrian.

[0010] In a second aspect, an embodiment of the present invention further provides a pedestrian tracking device, the pedestrian tracking device comprising:

[0011] The detection and acquisition image acquisition module is used to acquire the detection and acquisition image through the indoor environment camera every time the tracking time point is reached if it is determined that the indoor navigation robot has established a following navigation relationship with the target pedestrian;

[0012] A tracking prediction target recognition module, used to recognize at least one detection target in the detection acquisition image, and identify the tracking prediction target for the target pedestrian in the detection acquisition image;

[0013] The first fusion contrast feature and the second fusion contrast feature calculation module are used to calculate the first fusion contrast feature corresponding to each detection target and the second fusion contrast feature corresponding to the tracking prediction target according to the pedestrian image collected by the indoor navigation robot for the target pedestrian and the set follow-up navigation route;

[0014] The target pedestrian tracking module is used to identify the target pedestrian in the detection target according to the similarity between each first fusion contrast feature and the second fusion contrast feature, so as to track the target pedestrian.

[0015] In a third aspect, an embodiment of the present invention further provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the pedestrian tracking method as described in any embodiment of the present invention is implemented.

[0016] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored, wherein when the computer program is executed by a processor, the pedestrian tracking method as described in any embodiment of the present invention is implemented.

[0017] The technical solution provided by the embodiment of the present invention is as follows: if it is determined that the indoor navigation robot has established a follow-up navigation relationship with the target pedestrian, then each time a tracking time point is reached, a detection acquisition image is obtained through the indoor environment camera; at least one detection target is identified in the detection acquisition image, and a tracking prediction target for the target pedestrian is identified in the detection acquisition image; based on the pedestrian image collected by the indoor navigation robot for the target pedestrian and the set follow-up navigation route, the first fusion comparison feature corresponding to each detection target and the second fusion comparison feature corresponding to the tracking prediction target are calculated; based on the similarity between each first fusion comparison feature and the second fusion comparison feature, the target pedestrian is identified in the detection target to track the target pedestrian. The problem that the indoor navigation robot does not have the ability to actively follow in real time is solved, the function of following the customer in real time is realized, and the intelligence level of the indoor navigation robot is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 A flowchart of a pedestrian tracking method provided in Embodiment 1 of the present invention;

[0019] Figure 2a A flowchart of another pedestrian tracking method provided in Embodiment 2 of the present invention;

[0020] Figure 2b A schematic diagram of the structure of the shortest distance calculation in another pedestrian tracking method provided in the second embodiment of the present invention;

[0021] Figure 3 A flowchart of another pedestrian tracking method provided in Embodiment 3 of the present invention;

[0022] Figure 4 is a schematic structural diagram of a pedestrian tracking device provided by a fourth embodiment of the present invention;

[0023] Figure 5 It is a structural diagram of a computer device provided in Embodiment 5 of the present invention. DETAILED DESCRIPTION

[0024] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0025] Embodiment 1

[0026] Figure 1 This is a flow chart of a pedestrian tracking method provided in Embodiment 1 of the present invention. This embodiment is applicable to the case where an indoor navigation robot actively follows in real time. The method of this embodiment can be executed by a pedestrian tracking device, which can be implemented by software and / or hardware, and can be configured in a server.

[0027] Accordingly, the method specifically comprises the following steps:

[0028] S110: If it is determined that the indoor navigation robot establishes a following navigation relationship with the target pedestrian, then each time a tracking time point is reached, a detection collection image is acquired through the indoor environment camera.

[0029] Among them, the following navigation relationship can be that the indoor navigation robot collects the initial image of the target pedestrian and establishes a task relationship with the target pedestrian, so as to realize the following function and real-time navigation function of the indoor navigation robot for the target pedestrian. Specifically, when the customer chooses to be navigated by the indoor navigation robot, the indoor navigation robot collects the initial image of the customer. At this time, it is determined that the indoor navigation robot and the target pedestrian establish a following navigation relationship, that is, a task relationship. The tracking time point can be the image collection cycle preset by the indoor camera monitoring platform. Specifically, at each cycle time point, the indoor environment camera realizes the collection of indoor environment images. Detecting the collected image can be that each time a collection time cycle is triggered, the corresponding image to be detected is collected by the indoor environment camera.

[0030] For example, suppose there are three indoor navigation robots in the bank hall, namely robot 1, robot 2 and robot 3. Robot 2 is in working state, and the indoor environment camera is set to collect images every 3 seconds. Assume that at 9:30:35 on January 28, 2022, two customers come to the bank hall to do business. Robot 1 first collects the initial image of customer 1, so robot 1 establishes a follow-up navigation relationship with customer 1. Since robot 2 is in working state, robot 3 establishes a follow-up navigation relationship with customer 2.

[0031] Specifically, at 9:30:35 on January 28, 2022, robot 1 first captured the initial image of customer 1. Since the indoor environment camera is set to capture images every 3 seconds, the indoor environment camera captured the detection image at 9:30:38 on January 28, 2022.

[0032] S120: Identify at least one detection target in the detection acquisition image, and identify a tracking prediction target for the target pedestrian in the detection acquisition image.

[0033] The detection target can be the target that needs to be tracked in real time, which is identified by the yolov3 model in the detection acquisition image. The tracking prediction target can be the state of the recognition target in the current cycle predicted according to the recognition target corresponding to the image collected in the previous cycle, which is the prediction target.

[0034] Optionally, at least one detection target is identified in the detection acquisition image, and a tracking prediction target for the target pedestrian is identified in the detection acquisition image, including: inputting the detection acquisition image into a pre-trained yolov3 model to obtain at least one detection target; obtaining a historical recognition area of ​​the target pedestrian in a historical acquisition image acquired at a previous tracking time point; and using a Kalman tracking algorithm to identify a tracking prediction target for the target pedestrian in the detection acquisition image based on the historical recognition area.

[0035] Among them, the yolov3 model can be a classic representative of the one-stage target detection algorithm. It only needs one complete process to complete the target recognition and location positioning, and can complete the target detection task in near real time. The historical acquisition image can be a detection acquisition image acquired by an indoor environment camera at the previous tracking time point. The historical recognition area can be the corresponding area to be recognized in the historical acquisition image.

[0036] Among them, the Kalman tracking algorithm can be an algorithm that uses the linear system state equation and the system input and output observation data to optimally estimate the system state. Since the observation data includes the influence of noise and interference in the system, the optimal estimation can also be regarded as a filtering process.

[0037] Specifically, according to the historical recognition area in the historical acquisition image, the corresponding state vector can be obtained as X = (x, y, w, h, dx, dy, dw, dh), where (x, y, w, h) is the representation value of the detected historical recognition area, and (dx, dy, dw, dh) is the speed of the detected target in the x-axis direction, the speed in the y-axis direction, the width change and the height change. Then the observation matrix can be determined as z t =(x,y,w,h), that is, only the pedestrian position size corresponding to the detection target is used as the observation vector value.

[0038] According to formula (1) and formula (2), the current state of the system can be calculated and the current covariance error The formula is as follows:

[0039]

[0040]

[0041] in, represents the state of the system at the previous moment, P t-1 Represents the covariance error of the system at the previous moment.

[0042] According to formula (3), formula (4) and formula (5), the system state based on the observation matrix can be calculated and system covariance error P t , the formula is as follows:

[0043]

[0044]

[0045]

[0046] Where H represents the measurement model matrix, R represents the measurement noise covariance, and K t Denotes covariance gain. By adopting the Kalman tracking algorithm, according to the historical identification area, a tracking prediction target for the target pedestrian is identified in the detection acquisition image.

[0047] The advantage of this setting is that by inputting the detection and acquisition images into the pre-trained yolov3 model, at least one detection target is obtained, and the Kalman tracking algorithm is used to identify the tracking prediction of the target pedestrian. In this way, the corresponding prediction target at the current moment can be obtained more accurately based on the detection target at the previous moment, so as to determine the location information of the target pedestrian at the current moment, so that the indoor navigation robot can follow the target pedestrian.

[0048] S130. Calculate first fused comparison features corresponding to each detection target and second fused comparison features corresponding to the tracking prediction target based on the pedestrian images collected by the indoor navigation robot for the target pedestrian and the set follow-up navigation route.

[0049] The first fused comparison feature may be a fused comparison feature obtained by calculating the features extracted from the image collected by the indoor navigation robot for the target pedestrian and the features extracted from each detection target. The second fused comparison feature may be a fused comparison feature obtained by calculating the features extracted from the image collected by the indoor navigation robot for the target pedestrian and the features extracted from each tracking prediction target.

[0050] Specifically, the first fusion comparison feature can be obtained by calculating the pedestrian image collected by the indoor navigation robot for the target pedestrian and the pedestrian images in each detection target, as well as the follow-up navigation route set by the indoor navigation robot for the target pedestrian and the three-dimensional indoor position corresponding to the detection center of the pedestrian in each detection target.

[0051] Furthermore, the second fusion comparison feature can be obtained by calculating the pedestrian image collected by the indoor navigation robot for the target pedestrian and the pedestrian images in each tracking prediction target, as well as the following navigation route set by the indoor navigation robot for the target pedestrian and the three-dimensional indoor position corresponding to the detection center of the pedestrian in each tracking prediction target.

[0052] S140: According to the similarity between each first fusion contrast feature and the second fusion contrast feature, identify a target pedestrian in the detection target to track the target pedestrian.

[0053] In this embodiment, by obtaining the first fusion contrast features and the second fusion contrast features, the similarity of the first fusion contrast features and the second fusion contrast features can be calculated respectively. The smaller the difference between the two, the greater the possibility that the detected target and the predicted target can be matched successfully, that is, the greater the possibility that they are the same person. In this way, the real-time following function of the indoor navigation robot can be realized.

[0054] The technical solution provided by the embodiment of the present invention is as follows: if it is determined that the indoor navigation robot has established a follow-up navigation relationship with the target pedestrian, then each time a tracking time point is reached, a detection acquisition image is obtained through the indoor environment camera; at least one detection target is identified in the detection acquisition image, and a tracking prediction target for the target pedestrian is identified in the detection acquisition image; based on the pedestrian image collected by the indoor navigation robot for the target pedestrian and the set follow-up navigation route, the first fusion comparison feature corresponding to each detection target and the second fusion comparison feature corresponding to the tracking prediction target are calculated; based on the similarity between each first fusion comparison feature and the second fusion comparison feature, the target pedestrian is identified in the detection target to track the target pedestrian. The problem that the indoor navigation robot does not have the ability to actively follow in real time is solved, the function of following the customer in real time is realized, and the intelligence level of the indoor navigation robot is improved.

[0055] Embodiment 2

[0056] Figure 2a This is a flowchart of another indoor robot navigation control method provided in the second embodiment of the present invention. This embodiment is refined on the basis of the above embodiments, and further refines the calculation of the first fusion comparison feature corresponding to each detection target and the second fusion comparison feature corresponding to the tracking prediction target based on the pedestrian image collected by the indoor navigation robot for the target pedestrian and the set follow-up navigation route, referring to Figure 2a , the method specifically comprises the following steps:

[0057] S210: If it is determined that the indoor navigation robot establishes a following navigation relationship with the target pedestrian, then each time a tracking time point is reached, a detection collection image is acquired through the indoor environment camera.

[0058] S220: Identify at least one detection target in the detection acquisition image, and identify a tracking prediction target for the target pedestrian in the detection acquisition image.

[0059] S230. Calculate the three-dimensional indoor position of each detection target and tracking prediction target according to the two-dimensional image position of each detection target and tracking prediction target in the detection acquisition image.

[0060] The two-dimensional image position may refer to the position of a plane image that does not contain depth information. The corresponding two-dimensional coordinate position of the target pedestrian under the target camera can be determined from the acquired captured image. The three-dimensional indoor position may be the three-dimensional coordinate position of the target pedestrian under the target camera, which can be calculated based on the two-dimensional image coordinates and the camera parameter matrix.

[0061] Specifically, assuming that the information of the detection frame of the captured image is (x, y, w, h), the center coordinates of the detection frame of the captured image can be further determined as Assuming that the target pedestrian is located at the center of the detection frame of the captured image, the two-dimensional image coordinates corresponding to the target pedestrian are: That is, you can make That is, the position of the two-dimensional image is O1(X A ,Y A ). According to the formula [u,v,1] T ~P est [X,Y,Z,1] T The 3D indoor position O(X) corresponding to the target pedestrian can be calculated based on the 2D image position corresponding to the target pedestrian. A1 ,Y A1 ,Z A1 ). Where [u,v] represents the two-dimensional image position, [X,Y,Z] represents the three-dimensional indoor position, and the camera parameter matrix P est .

[0062] S240: Calculate the distance between each of the three-dimensional indoor positions and the follow-up navigation route as the first sub-feature.

[0063] The first sub-feature may be a description feature of the distance between each three-dimensional indoor position and the following navigation route, and the first sub-feature may be a sub-feature in the fusion comparison feature.

[0064] Specifically, the first sub-feature may be represented by a parameter d, which represents a parameter of the distance between each three-dimensional indoor position and the follow-up navigation route.

[0065] Continuing with the previous example, Figure 2b As shown in the figure, O(X A1 ,Y A1 ,Z A1 ) is the calculated three-dimensional indoor position, L represents the straight line of the planned route, and the shortest distance d can be calculated between point O and the planned route L. d can be used as a separate feature to match pedestrians, that is, d is the first sub-feature.

[0066] S250, calculating the feature matching degree between each detection target and the pedestrian image, and the feature matching degree between the tracking prediction target and the pedestrian image as the second sub-feature.

[0067] The second sub-feature may be the feature matching degree between each detection target or each tracking prediction target and the pedestrian image, and the second sub-feature may be two sub-features in the fused comparison feature.

[0068] Specifically, the second sub-feature can be represented by a parameter f, which represents a parameter of the feature matching degree between each detection target and the pedestrian image.

[0069] S260 , calculating the image color feature of each detection target and tracking prediction target as the third sub-feature, and calculating the frame size feature of each detection target and tracking prediction target as the fourth sub-feature.

[0070] Among them, the third sub-feature can be the image color feature of each detection target and the image color feature of each tracking prediction target, and the sub-feature corresponding to the calculated fusion feature. Similarly, it can be known that the fourth sub-feature can be the frame size feature of each detection target and the frame size feature of each tracking prediction target, and the sub-feature corresponding to the calculated fusion feature.

[0071] Specifically, the third sub-feature can be represented by parameters (r, g, b), which represent the characteristic parameters of the image color. Furthermore, the fourth sub-feature can be represented by parameters (x, y, w, h), which represent the characteristic parameters of the frame size.

[0072] S270, fusing the first sub-feature, the second sub-feature, the third sub-feature and the fourth sub-feature corresponding to each detection target respectively to obtain a first fused comparison feature corresponding to each detection target respectively.

[0073] Continuing with the previous example, we can obtain the first fused comparison feature α1 = (x1, y1, w1, h1, r1, g1, b1, d1, f1).

[0074] S280: Fusing the first sub-feature, the second sub-feature, the third sub-feature, and the fourth sub-feature corresponding to the tracking prediction target to obtain a second fused comparison feature corresponding to the tracking prediction target.

[0075] By the same token, we can obtain the second fusion comparison feature α2 = (x2, y2, w2, h2, r2, g2, b2, d2, f2).

[0076] S290: According to the similarity between each first fusion contrast feature and the second fusion contrast feature, identify the target pedestrian in the detection target to track the target pedestrian.

[0077] The technical solution provided by the embodiment of the present invention is as follows: if it is determined that the indoor navigation robot establishes a following navigation relationship with the target pedestrian, then each time a tracking time point is reached, a detection acquisition image is obtained through an indoor environment camera; at least one detection target is identified in the detection acquisition image, and a tracking prediction target for the target pedestrian is identified in the detection acquisition image; the three-dimensional indoor position of each detection target and the tracking prediction target is calculated according to the two-dimensional image position of each detection target and the tracking prediction target in the detection acquisition image; the distance between each three-dimensional indoor position and the following navigation route is calculated as a first sub-feature; the feature matching degree between each detection target and the pedestrian image, and the feature matching degree between the tracking prediction target and the pedestrian image are calculated as a second sub-feature; according to the first sub-feature and the second sub-feature corresponding to each detection target and the tracking prediction target respectively, a first fusion comparison feature corresponding to each detection target and a second fusion comparison feature corresponding to the tracking prediction target are obtained; according to the similarity between each first fusion comparison feature and the second fusion comparison feature, the target pedestrian is identified in the detection target to track the target pedestrian. By obtaining the first sub-feature, the second sub-feature, the third sub-feature and the fourth sub-feature, the first fusion contrast feature and the second fusion contrast feature are calculated. In this way, the target pedestrian can be identified and followed more accurately, thereby realizing the function of the indoor navigation robot to follow the customer in real time.

[0078] Embodiment 3

[0079] Figure 3 This is a flow chart of another indoor robot navigation control method provided by Embodiment 3 of the present invention. This embodiment is refined on the basis of the above embodiments, and further refines the identification of target pedestrians in the detection target based on the similarity between each first fusion contrast feature and the second fusion contrast feature, refer to Figure 3 , the method specifically comprises the following steps:

[0080] S310: If it is determined that the indoor navigation robot establishes a following navigation relationship with the target pedestrian, then each time a tracking time point is reached, a detection collection image is acquired through the indoor environment camera.

[0081] S320: Identify at least one detection target in the detection acquisition image, and identify a tracking prediction target for the target pedestrian in the detection acquisition image.

[0082] S330. Calculate first fused comparison features corresponding to each detection target and second fused comparison features corresponding to the tracking prediction target based on the pedestrian images collected by the indoor navigation robot for the target pedestrian and the set follow-up navigation route.

[0083] S340: Calculate feature difference values ​​between each sub-feature included in each first fused contrast feature and each sub-feature matched in the second fused contrast feature.

[0084] The feature difference value can be collected by collecting images from two adjacent frames with a short time interval, and the difference of each sub-feature can be calculated respectively.

[0085] Exemplarily, assuming that the third sub-feature is taken as an example, the formula for calculating the feature difference value between the third sub-features is as follows:

[0086]

[0087]

[0088]

[0089] Among them, r 检,i , g 检,i and b 检,i They represent the red, green and blue values ​​of the target pedestrian i in the detection target, respectively, and r 预,j , g 预,j and b 预,j They represent the red, green and blue values ​​of the target pedestrian j in the tracking prediction target, Δr i,j , Δg i,j and Δb i,j is the feature difference value between the third sub-feature of the detected target and the target pedestrian of the tracked prediction target. i,j , Δg i,j and Δb i,j The smaller the value of is, the more likely it is that the target pedestrian i of the detection target and the target pedestrian j of the tracking prediction target are the same person. In other words, the smaller the feature difference value between the third sub-features is, the more likely it is that the target pedestrian i of the detection target and the target pedestrian j of the tracking prediction target are the same person.

[0090] S350, calculating the fusion feature difference value corresponding to each detection target according to each feature difference value and the feature weight allocation strategy corresponding to each detection target.

[0091] The feature weight allocation strategy may be to allocate corresponding weights to each feature difference value according to whether the current detection target has an occlusion relationship with other detection targets. The fused feature difference value may be calculated according to each feature difference value and the corresponding feature weight value.

[0092] Optionally, according to each feature difference value and the feature weight allocation strategy corresponding to each detection target, the fused feature difference value corresponding to each detection target is calculated, including: obtaining the current detection target, and the first feature difference value, the second feature difference value, the third feature difference value and the fourth feature difference value corresponding to the current detection target; judging whether there is an occlusion relationship between the current detection target and other detection targets according to the frame overlap relationship of each detection target in the detection acquisition image; if not, adopting an adaptive weight adjustment scheme to determine the weight values ​​corresponding to the first feature difference value, the second feature difference value, the third feature difference value and the fourth feature difference value respectively; if yes, according to a preset fixed weight allocation scheme, the weight values ​​corresponding to the first feature difference value, the second feature difference value, the third feature difference value and the fourth feature difference value respectively are allocated; according to the first feature difference value, the second feature difference value, the third feature difference value, the fourth feature difference value corresponding to the current detection target and the matching weight value, the fused feature difference value corresponding to the current detection target is calculated.

[0093] The first feature difference value may be a difference value calculated between the first sub-features, the second feature difference value may be a difference value calculated between the second sub-features, the third feature difference value may be a difference value calculated between the third sub-features, and the fourth feature difference value may be a difference value calculated between the fourth sub-features. The frame overlap relationship may be to determine whether there is an overlap relationship between the frame of the current detection target and the frames of other detection targets.

[0094] Among them, the adaptive weight adjustment scheme is to calculate the corresponding weight value according to the first feature difference value, the second feature difference value, the third feature difference value and the fourth feature difference value. The fixed weight allocation scheme can be that when there is an occlusion relationship between the current detection target and other detection targets, after multiple data calculations, the optimal weight values ​​corresponding to the first feature difference value, the second feature difference value, the third feature difference value and the fourth feature difference value are obtained.

[0095] Optionally, the adaptive weight adjustment scheme is used to determine the weight values ​​corresponding to the first feature difference value, the second feature difference value, the third feature difference value, and the fourth feature difference value, respectively, including: calculating the first adaptive weight value, the second adaptive weight value, the third adaptive weight value, and the fourth adaptive weight value according to the first feature difference value, the second feature difference value, the third feature difference value, and the fourth feature difference value, respectively; obtaining the first adaptive weight value, the second adaptive weight value, the third adaptive weight value, and the fourth adaptive weight value, performing normalization processing, and calculating the first feature weight value, the second feature weight value, the third feature weight value, and the fourth feature weight value, respectively.

[0096] The first adaptive weight value may be calculated based on the first feature difference value and is a weight assigned to the first feature difference value. The second adaptive weight value may be calculated based on the second feature difference value and is a weight assigned to the second feature difference value. The third adaptive weight value may be calculated based on the third feature difference value and is a weight assigned to the third feature difference value. The fourth adaptive weight value may be calculated based on the fourth feature difference value and is a weight assigned to the fourth feature difference value.

[0097] Among them, the first feature weight value can be normalized according to the adaptive weight value to obtain the feature weight value corresponding to the first feature difference value. The second feature weight value can be normalized according to the adaptive weight value to obtain the feature weight value corresponding to the second feature difference value. The third feature weight value can be normalized according to the adaptive weight value to obtain the feature weight value corresponding to the third feature difference value. The fourth feature weight value can be normalized according to the adaptive weight value to obtain the feature weight value corresponding to the fourth feature difference value.

[0098] Continuing from the previous example, we know that Δr i,j , Δg i,j and Δb i,j is the feature difference value between the third sub-features. Therefore, Δd can be set i,j is the feature difference value between the first sub-features; Δf can be set i,j is the feature difference value between the second sub-features; Δx can be set i,j , Δy i,j , Δw i,j and Δh i,j is the feature difference value between the fourth sub-features. Therefore, the first adaptive weight value, the second adaptive weight value, the third adaptive weight value and the fourth adaptive weight value can be calculated respectively from the first feature difference value, the second feature difference value, the third feature difference value and the fourth feature difference value. and The formula is as follows:

[0099]

[0100]

[0101]

[0102]

[0103] The calculated weight value is and The adaptive weight coefficients can be normalized to calculate the first feature weight value, the second feature weight value, the third feature weight value and the fourth feature weight value ω1, ω2, ω3 and ω4 respectively as follows:

[0104]

[0105]

[0106]

[0107]

[0108] The advantage of such a setting is that by judging whether the current detection target has an occlusion relationship with other detection targets according to the overlapping relationship of the frames of each detection target in the detection acquisition image, an adaptive weight adjustment scheme and a preset fixed weight allocation scheme are adopted, and further, the weight values ​​corresponding to the first feature difference value, the second feature difference value, the third feature difference value and the fourth feature difference value are obtained to calculate the fused feature difference value corresponding to the current detection target. In this way, the fused feature difference value can be calculated more accurately to judge the current detection target, so that the indoor navigation robot can accurately perform real-time following and navigation functions.

[0109] S360: If it is determined that the minimum fusion feature difference value is less than or equal to the preset matching threshold, the detection target corresponding to the minimum fusion feature difference value is identified as a target pedestrian.

[0110] Among them, the preset matching threshold can be a threshold value of a set fusion feature difference value, which can be obtained based on big data detection and pre-set.

[0111] Specifically, assuming that the preset matching threshold is K. If the minimum fusion feature difference value is less than or equal to the preset matching threshold K, the detection target corresponding to the minimum fusion feature difference value is identified as the target pedestrian; if the minimum fusion feature difference value is greater than the preset matching threshold K, the detection target corresponding to the minimum fusion feature difference value cannot be identified as the target pedestrian.

[0112] S370: Track the target pedestrian.

[0113] The technical solution provided by the embodiment of the present invention is as follows: if it is determined that the indoor navigation robot establishes a follow-up navigation relationship with the target pedestrian, then each time a tracking time point is reached, a detection acquisition image is obtained through an indoor environment camera; at least one detection target is identified in the detection acquisition image, and a tracking prediction target for the target pedestrian is identified in the detection acquisition image; according to the pedestrian image collected by the indoor navigation robot for the target pedestrian and the set follow-up navigation route, a first fusion comparison feature corresponding to each detection target and a second fusion comparison feature corresponding to the tracking prediction target are calculated; the feature difference values ​​between each sub-feature included in each first fusion comparison feature and each sub-feature matched in the second fusion comparison feature are calculated respectively; according to each feature difference value and a feature weight allocation strategy corresponding to each detection target, a fusion feature difference value corresponding to each detection target is calculated; if it is determined that the minimum fusion feature difference value is less than or equal to a preset matching threshold, the detection target corresponding to the minimum fusion feature difference value is identified as the target pedestrian, so as to track the target pedestrian. By calculating the fusion feature difference value and comparing it with the preset matching threshold, the target pedestrian can be identified more accurately, so that the real-time tracking function for customers can be performed more scientifically.

[0114] Embodiment 4

[0115] Figure 4 is a schematic diagram of the structure of a pedestrian tracking device provided in the fourth embodiment of the present invention. The pedestrian tracking device provided in this embodiment can be implemented by software and / or hardware, and can be configured in a server to implement a pedestrian tracking method in the embodiment of the present invention. Figure 4 As shown, the device may specifically include: a detection and acquisition image acquisition module 410, a tracking and prediction target recognition module 420, a first fusion contrast feature and a second fusion contrast feature calculation module 430 and a target pedestrian tracking module 440.

[0116] The detection and acquisition image acquisition module 410 is used to acquire the detection and acquisition image through the indoor environment camera every time the tracking time point is reached if it is determined that the indoor navigation robot has established a following navigation relationship with the target pedestrian;

[0117] A tracking prediction target identification module 420, configured to identify at least one detection target in the detection acquisition image, and identify a tracking prediction target for the target pedestrian in the detection acquisition image;

[0118] The first fusion contrast feature and the second fusion contrast feature calculation module 430 is used to calculate the first fusion contrast feature corresponding to each detection target and the second fusion contrast feature corresponding to the tracking prediction target according to the pedestrian image collected by the indoor navigation robot for the target pedestrian and the set follow-up navigation route;

[0119] The target pedestrian tracking module 440 is used to identify the target pedestrian in the detection target according to the similarity between each first fusion contrast feature and the second fusion contrast feature, so as to track the target pedestrian.

[0120] The technical solution provided by the embodiment of the present invention is as follows: if it is determined that the indoor navigation robot has established a follow-up navigation relationship with the target pedestrian, then each time a tracking time point is reached, a detection acquisition image is obtained through the indoor environment camera; at least one detection target is identified in the detection acquisition image, and a tracking prediction target for the target pedestrian is identified in the detection acquisition image; based on the pedestrian image collected by the indoor navigation robot for the target pedestrian and the set follow-up navigation route, the first fusion comparison feature corresponding to each detection target and the second fusion comparison feature corresponding to the tracking prediction target are calculated; based on the similarity between each first fusion comparison feature and the second fusion comparison feature, the target pedestrian is identified in the detection target to track the target pedestrian. The problem that the indoor navigation robot does not have the ability to actively follow in real time is solved, the function of following the customer in real time is realized, and the intelligence level of the indoor navigation robot is improved.

[0121] Based on the above embodiments, the tracking prediction target identification module 420 can be specifically used to: input the detection acquisition image into a pre-trained yolov3 model to obtain at least one detection target; obtain the historical identification area of ​​the target pedestrian in the historical acquisition image collected at the previous tracking time point; use the Kalman tracking algorithm to identify the tracking prediction target for the target pedestrian in the detection acquisition image based on the historical identification area.

[0122] On the basis of the above embodiments, the first fusion contrast feature and the second fusion contrast feature calculation module 430 may specifically include: a three-dimensional indoor position calculation unit, used to calculate the three-dimensional indoor position of each detection target and the tracking prediction target according to the two-dimensional image position of each detection target and the tracking prediction target in the detection acquisition image; a first sub-feature determination unit, used to calculate the distance between each of the three-dimensional indoor positions and the following navigation route as the first sub-feature; a second sub-feature calculation unit, used to calculate the feature matching degree between each detection target and the pedestrian image, and the feature matching degree between the tracking prediction target and the pedestrian image, as the second sub-feature; a first fusion contrast feature and a second fusion contrast feature acquisition unit, used to obtain a first fusion contrast feature corresponding to each detection target and a second fusion contrast feature corresponding to the tracking prediction target according to the first sub-feature and the second sub-feature corresponding to each detection target and the tracking prediction target respectively.

[0123] On the basis of the above embodiments, the third sub-feature and the fourth sub-feature calculation unit may be specifically used to: after calculating the feature matching degree between each detection target and the pedestrian image, and the feature matching degree between the tracking prediction target and the pedestrian image as the second sub-feature, calculate the image color feature of each detection target and the tracking prediction target as the third sub-feature, and calculate the frame size feature of each detection target and the tracking prediction target as the fourth sub-feature;

[0124] On the basis of the above embodiments, the first fused contrast feature and the second fused contrast feature obtaining unit can be specifically used to: fuse the first sub-features, second sub-features, third sub-features and fourth sub-features corresponding to each detection target, to obtain the first fused comparison features corresponding to each detection target; fuse the first sub-features, second sub-features, third sub-features and fourth sub-features corresponding to the tracking prediction target, to obtain the second fused comparison features corresponding to the tracking prediction target.

[0125] Based on the above embodiments, the target pedestrian tracking module 440 may specifically include: a feature difference value calculation unit, which is used to respectively calculate the feature difference values ​​between each sub-feature included in each first fusion contrast feature and each sub-feature matched in the second fusion contrast feature; a fusion feature difference value calculation unit, which is used to calculate the fusion feature difference value corresponding to each detection target according to each feature difference value and the feature weight allocation strategy corresponding to each detection target; a target pedestrian recognition unit, which is used to identify the detection target corresponding to the minimum fusion feature difference value as a target pedestrian if it is determined that the minimum fusion feature difference value is less than or equal to a preset matching threshold.

[0126] On the basis of the above embodiments, it also includes a fusion feature difference value calculation unit, which can be specifically used to: obtain a current detection target, and a first feature difference value, a second feature difference value, a third feature difference value, and a fourth feature difference value corresponding to the current detection target; determine whether the current detection target has an occlusion relationship with other detection targets according to the frame overlap relationship of each detection target in the detection acquisition image; if not, adopt an adaptive weight adjustment scheme to determine the weight values ​​corresponding to the first feature difference value, the second feature difference value, the third feature difference value, and the fourth feature difference value respectively; if so, allocate the weight values ​​corresponding to the first feature difference value, the second feature difference value, the third feature difference value, and the fourth feature difference value respectively according to a preset fixed weight allocation scheme; calculate the fusion feature difference value corresponding to the current detection target according to the first feature difference value, the second feature difference value, the third feature difference value, and the fourth feature difference value corresponding to the current detection target and the matching weight value.

[0127] On the basis of the above embodiments, the adaptive weight adjustment scheme is used to determine the weight values ​​corresponding to the first feature difference value, the second feature difference value, the third feature difference value and the fourth feature difference value, respectively. It can be specifically used to: calculate the first adaptive weight value, the second adaptive weight value, the third adaptive weight value and the fourth adaptive weight value according to the first feature difference value, the second feature difference value, the third feature difference value and the fourth feature difference value, respectively; obtain the first adaptive weight value, the second adaptive weight value, the third adaptive weight value and the fourth adaptive weight value, perform normalization processing, and calculate the first feature weight value, the second feature weight value, the third feature weight value and the fourth feature weight value, respectively.

[0128] The above-mentioned pedestrian tracking device can execute the pedestrian tracking method provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0129] Embodiment 5

[0130] Figure 5 Schematic diagram of the structure of a computer device provided by Embodiment 5 of the present invention. Figure 5 As shown, the device includes a processor 510, a memory 520, an input device 530, and an output device 540; the number of processors 510 in the device can be one or more. Figure 5 A processor 510 is taken as an example; the processor 510, the memory 520, the input device 530 and the output device 540 in the device can be connected via a bus or other means. Figure 5 The example of connecting through bus is taken in the following.

[0131] The memory 520, as a computer-readable storage medium, can be used to store software programs, computer executable programs and modules, such as program instructions / modules corresponding to the pedestrian tracking method in the embodiment of the present invention (for example, the detection and acquisition image acquisition module 410, the tracking and prediction target recognition module 420, the first fusion contrast feature and the second fusion contrast feature calculation module 430 and the target pedestrian tracking module 440). The processor 510 executes various functional applications and data processing of the device by running the software programs, instructions and modules stored in the memory 520, that is, implements the above-mentioned pedestrian tracking method, which includes: if it is determined that the indoor navigation robot establishes a following navigation relationship with the target pedestrian, then each time the tracking time point is reached, a detection acquisition image is obtained through the indoor environment camera; at least one detection target is identified in the detection acquisition image, and a tracking prediction target for the target pedestrian is identified in the detection acquisition image; based on the pedestrian image collected by the indoor navigation robot for the target pedestrian and the set following navigation route, a first fusion comparison feature corresponding to each detection target and a second fusion comparison feature corresponding to the tracking prediction target are calculated; based on the similarity between each first fusion comparison feature and the second fusion comparison feature, the target pedestrian is identified in the detection target to track the target pedestrian.

[0132] The memory 520 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required for a function; the data storage area may store data created according to the use of the terminal, etc. In addition, the memory 520 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 520 may further include a memory remotely arranged relative to the processor 510, and these remote memories may be connected to the device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0133] The input device 530 may be used to receive input digital or character information and generate key signal input related to user settings and function control of the device. The output device 540 may include a display device such as a display screen.

[0134] Embodiment 6

[0135] Embodiment 6 of the present invention also provides a computer-readable storage medium, wherein the computer-readable instructions are used to execute a pedestrian tracking method when executed by a computer processor, the method comprising: if it is determined that the indoor navigation robot establishes a following navigation relationship with the target pedestrian, then each time a tracking time point is reached, a detection acquisition image is acquired through an indoor environment camera; at least one detection target is identified in the detection acquisition image, and a tracking prediction target for the target pedestrian is identified in the detection acquisition image; based on the pedestrian image collected by the indoor navigation robot for the target pedestrian and the set following navigation route, a first fusion comparison feature corresponding to each detection target and a second fusion comparison feature corresponding to the tracking prediction target are calculated; based on the similarity between each first fusion comparison feature and the second fusion comparison feature, the target pedestrian is identified in the detection target to track the target pedestrian.

[0136] Of course, the computer-readable storage medium provided in the embodiment of the present invention has computer-readable instructions that are not limited to the method operations described above, and can also execute related operations in the pedestrian tracking method provided in any embodiment of the present invention.

[0137] Through the above description of the implementation methods, the technicians in the relevant field can clearly understand that the present invention can be implemented by means of software and necessary general hardware, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, a read-only memory (ROM), a random access memory (RAM), a flash memory (FLASH), a hard disk or an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0138] It is worth noting that in the embodiment of the above-mentioned pedestrian tracking device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0139] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A pedestrian tracking method, characterized in that: include: If it is determined that the indoor navigation robot establishes a following navigation relationship with the target pedestrian, then each time the tracking time point is reached, a detection collection image is obtained through the indoor environment camera; Identifying at least one detection target in the detection acquisition image, and identifying a tracking prediction target for the target pedestrian in the detection acquisition image; According to the pedestrian images collected by the indoor navigation robot for the target pedestrian and the set follow-up navigation route, a first fused contrast feature corresponding to each detected target and a second fused contrast feature corresponding to the tracking prediction target are calculated; According to the similarity between each first fused contrast feature and the second fused contrast feature, a target pedestrian is identified in the detection target to track the target pedestrian; The method of calculating the first fusion contrast feature corresponding to each detection target and the second fusion contrast feature corresponding to the tracking prediction target according to the pedestrian image collected by the indoor navigation robot for the target pedestrian and the set follow-up navigation route includes: Calculating the three-dimensional indoor position of each detection target and tracking prediction target according to the two-dimensional image position of each detection target and tracking prediction target in the detection acquisition image; Calculating the distance between each of the three-dimensional indoor positions and the following navigation route as a first sub-feature; Calculate the feature matching degree between each detected target and the pedestrian image, and the feature matching degree between the tracking prediction target and the pedestrian image as the second sub-feature; According to the first sub-features and the second sub-features respectively corresponding to each detection target and the tracking prediction target, a first fused contrast feature corresponding to each detection target and a second fused contrast feature corresponding to the tracking prediction target are obtained.

2. The method according to claim 1, characterized in that Identifying at least one detection target in the detection acquisition image, and identifying a tracking prediction target for the target pedestrian in the detection acquisition image, including: Input the detected image into the pre-trained yolov3 model to obtain at least one detection target; Obtain the historical recognition area of ​​the target pedestrian in the historical acquisition image acquired at the previous tracking time point; A Kalman tracking algorithm is used to identify a tracking prediction target for the target pedestrian in the detection acquisition image according to the historical recognition area.

3. The method according to claim 1, characterized in that After calculating the feature matching degree between each detected target and the pedestrian image, and the feature matching degree between the tracked predicted target and the pedestrian image as the second sub-feature, it also includes: Calculate the image color feature of each detected target and tracked predicted target as the third sub-feature, and calculate the frame size feature of each detected target and tracked predicted target as the fourth sub-feature; According to the first sub-features and the second sub-features corresponding to each detection target and the tracking prediction target, respectively, a first fusion contrast feature corresponding to each detection target and a second fusion contrast feature corresponding to the tracking prediction target are obtained, including: The first sub-feature, the second sub-feature, the third sub-feature and the fourth sub-feature corresponding to each detection target are fused to obtain a first fused contrast feature corresponding to each detection target; The first sub-feature, the second sub-feature, the third sub-feature and the fourth sub-feature corresponding to the tracking prediction target are fused to obtain a second fused contrast feature corresponding to the tracking prediction target.

4. The method according to any one of claims 1 to 3, characterized in that: According to the similarity between each first fusion contrast feature and the second fusion contrast feature, a target pedestrian is identified in the detection target, including: Respectively calculating feature difference values ​​between each sub-feature included in each first fusion contrast feature and each sub-feature matched in the second fusion contrast feature; According to each feature difference value and the feature weight allocation strategy corresponding to each detection target, the fusion feature difference value corresponding to each detection target is calculated; If it is determined that the minimum fusion feature difference value is less than or equal to the preset matching threshold, the detection target corresponding to the minimum fusion feature difference value is identified as a target pedestrian.

5. The method according to claim 4, characterized in that According to each feature difference value and the feature weight allocation strategy corresponding to each detection target, the fusion feature difference value corresponding to each detection target is calculated, including: Acquire a current detection target, and a first feature difference value, a second feature difference value, a third feature difference value, and a fourth feature difference value corresponding to the current detection target; According to the overlapping relationship of the frames of the detection targets in the detection acquisition image, it is determined whether there is an occlusion relationship between the current detection target and other detection targets; If not, an adaptive weight adjustment scheme is adopted to determine weight values ​​corresponding to the first feature difference value, the second feature difference value, the third feature difference value, and the fourth feature difference value, respectively; If yes, then according to a preset fixed weight allocation scheme, weight values ​​corresponding to the first feature difference value, the second feature difference value, the third feature difference value, and the fourth feature difference value are allocated respectively; A fused feature difference value corresponding to the current detection target is calculated based on the first feature difference value, the second feature difference value, the third feature difference value, the fourth feature difference value and the matched weight value corresponding to the current detection target.

6. The method according to claim 5, characterized in that The method of adopting an adaptive weight adjustment scheme to determine weight values ​​corresponding to the first feature difference value, the second feature difference value, the third feature difference value, and the fourth feature difference value, respectively, includes: According to the first feature difference value, the second feature difference value, the third feature difference value and the fourth feature difference value, a first adaptive weight value, a second adaptive weight value, a third adaptive weight value and a fourth adaptive weight value are calculated respectively; The first adaptive weight value, the second adaptive weight value, the third adaptive weight value and the fourth adaptive weight value are obtained, and normalized to obtain the first feature weight value, the second feature weight value, the third feature weight value and the fourth feature weight value respectively.

7. A pedestrian tracking device, characterized in that: include: The detection and acquisition image acquisition module is used to acquire the detection and acquisition image through the indoor environment camera every time the tracking time point is reached if it is determined that the indoor navigation robot has established a following navigation relationship with the target pedestrian; A tracking prediction target recognition module, used to recognize at least one detection target in the detection acquisition image, and identify the tracking prediction target for the target pedestrian in the detection acquisition image; The first fusion contrast feature and the second fusion contrast feature calculation module are used to calculate the first fusion contrast feature corresponding to each detection target and the second fusion contrast feature corresponding to the tracking prediction target according to the pedestrian image collected by the indoor navigation robot for the target pedestrian and the set follow-up navigation route; A target pedestrian tracking module, used for identifying a target pedestrian in the detection target according to the similarity between each first fusion contrast feature and the second fusion contrast feature, so as to track the target pedestrian; Wherein, the first fusion contrast feature and the second fusion contrast feature calculation module include: A three-dimensional indoor position calculation unit, used to calculate the three-dimensional indoor position of each detection target and tracking prediction target according to the two-dimensional image position of each detection target and tracking prediction target in the detection acquisition image; A first sub-feature determination unit, configured to calculate a distance between each of the three-dimensional indoor positions and the follow-up navigation route as a first sub-feature; A second sub-feature calculation unit, used to calculate the feature matching degree between each detection target and the pedestrian image, and the feature matching degree between the tracking prediction target and the pedestrian image, as the second sub-feature; The first fused contrast feature and the second fused contrast feature obtaining unit are used to obtain the first fused contrast feature corresponding to each detection target and the second fused contrast feature corresponding to the tracking prediction target according to the first sub-features and the second sub-features corresponding to each detection target and the tracking prediction target respectively.

8. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the pedestrian tracking method according to any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the pedestrian tracking method as described in any one of claims 1 to 6 is implemented.

Images