A human target recognition and tracking method suitable for quadruped robot inspection
Through the dual-mode motion and observation model under the particle filtering framework, combined with Gaussian motion and feature point tracking model, the jitter and occlusion problems of human target recognition tracking during the inspection of four-legged robots are solved, and stable and high-precision target tracking is achieved. Effect.
Patent Information
- Application Number
- CN202111192988.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-10-13
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2041-10-13
AI Technical Summary
During the inspection, the four-legged robot is prone to drift due to camera shake and fast target movement during the inspection, and it is difficult to achieve stable tracking.
Using the dual-mode motion model and observation model under the particle filtering framework, combining the Gaussian motion model and feature point tracking model, target state prediction and position estimation are performed through the classification detector and the target recognizer, reducing the impact of jitter and occlusion, and improving tracking accuracy .
During the movement of the four-legged robot, fast and stable identification and tracking of human targets is achieved, reducing the probability of tracking drift caused by camera bumps and target occlusion, and improving tracking accuracy and reliability.
Smart Images

Figure CN113989701B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the fields of robotics and computer vision, and in particular to a human target recognition and tracking method suitable for a quadruped robot inspection scenario. Background Art
[0002] The application field of mobile robots is rapidly expanding. Whether it is a service robot or a military combat robot, human target recognition and tracking is one of the important functions that intelligent service robots need to have. Human target recognition and tracking is a key technology to achieve the task of mobile robots following the target person. It is necessary to provide the relative position of the human target to the mobile robot. At present, human target recognition and tracking technology has been widely used in fields such as video surveillance and intelligent driving.
[0003] The application scenarios of existing human target recognition and tracking algorithms are mostly fixed camera monitoring scenarios or stable moving platforms. However, in the application scenarios of legged robots, the frequent camera shaking and the rapid movement of the target make the existing recognition and tracking algorithms easily drift and often fail to track. The target person tracking technology for quadruped robot following operations faces the following challenges: (1) changes in background and lighting; (2) the non-rigidity of the human body causes the target person's posture to change; (3) the target person is blocked by other objects or pedestrians; (4) image blur and picture jumps caused by frequent shaking. Summary of the invention
[0004] The purpose of the present invention is to overcome the defects of the above-mentioned technology and provide a human target recognition and tracking method suitable for quadruped robot inspection, which can quickly and stably realize the purpose of human target recognition and tracking when the quadruped robot moves and the camera is violently shaken.
[0005] A human target recognition and tracking method suitable for quadruped robot inspection is performed under the framework of particle filtering, comprising the following steps:
[0006] S1: Initialization: Get the initial state X of the human target from the video frame t-1 , the particle set at time t is By X t-1 It is generated in an iterative manner, where Ns represents the number of particles;
[0007] S2: Target state prediction: The motion model uses a combination of Gaussian motion model and feature point tracking model to complete target state prediction. The Gaussian motion model transfers particles to cover the tracking target to solve the problem of human targets being blocked; the feature point tracking model predicts the particle motion state to reduce the probability of tracking drift when the picture jumps due to camera shake;
[0008] S3: System observation: The observation model uses a combination of classification detector and target identifier to complete the target person position estimation; the classification detector of the observation model detects the pedestrian class, and then the target identifier recognizes the target person to complete the target person position estimation;
[0009] S4: Resampling: Obtain the particle weights through the observation model, resample after normalizing the weights, update the classifier template, and obtain the final particle set at the current moment;
[0010] S5: Target tracking result: Calculate each target state according to the final particle set, and obtain the target tracking result by weighted average; check whether the information of the next moment arrives. If so, return to step S2 and use the final particle set as the input of the next filtering process; otherwise, the target tracking process ends.
[0011] Preferably, the state vector of the particle in the motion model is X=[x, y, s, μ, ν] T , where (x, y) is the two-dimensional position coordinate in the image coordinate system; s is the pixel width of the particle; (μ, ν) are the velocity components on the x-axis and y-axis respectively; the particle set at time t is By X t-1 It is generated in an iterative manner, where Ns represents the number of particles;
[0012] Preferably, the Gaussian motion model performs the particle transfer process as follows:
[0013]
[0014] in, is the state vector of the ith particle at time t, and the process noise It obeys a normal distribution with zero mean and assumes that each dimensional state variable is independent of each other; the noise variance of the position (x, y) and pixel width s is proportional to the size of the target; the noise variance of the velocity component (μ, ν) is a constant.
[0015] Preferably, the feature point tracking model is:
[0016] Extract feature points in the rectangular frame of the tracking result at time t-1 and calculate the center of the feature points Then use the Lucas-Canade optical flow method to track the feature points of the current frame and calculate the center of the tracked feature points, which is recorded as The expected state of the particle is denoted by Given by formula (2):
[0017]
[0018] Among them, it is assumed that the center of the feature point of the previous frame The center of the feature point in the current frame And the corresponding tracking rectangle result The relative position relationship remains unchanged, and finally the feature points and the particles generated by the model are connected by It is obtained by sampling from a Gaussian distribution with a mean value;
[0019] The target position is given by tracking feature points, as shown in formula (3):
[0020]
[0021] in, is Gaussian noise, is the tracking result of the previous frame; Ω(·) is the tracking result of the previous frame Mapped to the particle set at time t Expected state
[0022] Preferably, the motion model adopts a particle transfer model combining a Gaussian motion model and a feature point tracking model:
[0023] X t =ηX t +(1-η)X t (4)
[0024] Among them, η represents the percentage of particles transmitted by the Gaussian motion model, 0≤η≤1; all particles are divided into two parts, one part is transmitted by the Gaussian motion model, there are ηNs particles; the other part is transmitted by the feature point tracking model, and there are (1-η)Ns particles left; the value of η is adaptively adjusted according to the instantaneous displacement detector. When instantaneous displacement is detected, the weight of the feature point tracking model is increased, that is, the value of η is reduced.
[0025] Preferably, the classification detector is used to detect the probability of pedestrians, and adopts a pedestrian detection algorithm based on HOG features and SVM classifier, unifies the image block corresponding to each particle into a 64×128 pixel block size, and then extracts the HOG features of the pixel block, which are input into a pre-trained SVM classifier to obtain the classifier score of each particle, and finally the score is subjected to non-maximum suppression processing to obtain the final result.
[0026] Preferably, the target identifier is used to accurately identify the target person, including:
[0027] 1) Template pool preparation: First, search the storage space for existing templates based on the target person's identity description. If so, read the template directly from the storage space. If not, prepare online and use the image of the target person as the first template in the template pool. Then, enrich the template pool by identifying the target person. The starting template Tinit Collect several frames of images from the target person’s video every s seconds, that is, Where N i Indicates the number of starting templates;
[0028] 2) Feature descriptor: By extracting channel histogram features as the feature descriptor of the particle, the LUV color space, gradient amplitude and gradient histogram in six directions (0°, 30°, 60°, 90°, 120°, 150°) of the image at time t are directly extracted on the three channels of the LUV color space to extract the histogram features of the pixel blocks corresponding to the particles. The six-directional gradient histogram channel sums the pixel values of the pixel blocks corresponding to each directional channel; if Represents particles The three-channel histogram feature of the LUV color space, the directional gradient histogram feature of the j-th pixel block is Then the characteristic description of the i-th particle at time t can be expressed as The first three parts describe the color information, and the last N parts describe the local contour information;
[0029] 3) Template matching: The Bhattacharyya distance is used to measure the matching similarity between the particle and the template. The distance is defined as Where C is the candidate particle, T is the template, denote the i-th characteristic component of the candidate particle and the template respectively, and the function b(·) denotes the Bhattacharyya distance between the two vectors; therefore, the candidate particle C and the starting template T init The distance between them is d(C,T init )=min{d(C,T i )}, take the minimum value of the distance between the candidate particle and the starting template, similarly, the candidate particle C and the current template T curr The distance between them is d(C,T curr )=min{d(C,T i )},T i ∈T curr ; The distance between candidate C and the entire template pool is defined as the weighted sum of its distances from the templates at two moments, as shown in formula (5):
[0030] d=γd(C,T init )+(1-γ)d(C,T curr ). (5)
[0031] Preferably, the observation model consists of a classification detector and a target identifier, as shown in formula (6):
[0032]
[0033] in, represents the probability that the particle given by the classification detector is a pedestrian, represents the probability that the particle given by the target identifier is the target person, Represents the confidence weight parameter.
[0034] Preferably, the step S4 comprises:
[0035] Particle weight Iterate and update through formula (7):
[0036]
[0037] in, is the observation model, and the particle weights are calculated by equation (7) and then normalized so that Then all particles are weighted averaged to obtain the tracking result.
[0038] The present application adopts a human target recognition and tracking method suitable for quadruped robot inspection. Through the above technical solution, the beneficial effects of the present application are:
[0039] In response to the need for human target recognition and tracking during the movement of a quadruped robot, this application provides a dual-modal combined motion model and observation model under the particle filter framework, which can to a certain extent solve the problems of camera up and down shaking, target being blocked, and field of view jumping (instantaneous displacement of the target) during the movement of the quadruped robot. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0041] Figure 1 A flowchart of a target recognition and tracking algorithm provided in an embodiment of the present application;
[0042] Figure 2 A flow chart of a particle filter observation model classification detector for detecting pedestrians provided in an embodiment of the present application;
[0043] Figure 3 A flowchart of the particle filter observation model target identifier provided in an embodiment of the present application. DETAILED DESCRIPTION
[0044] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.
[0045] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more related listed items.
[0046] This application proposes a dual-modal complementary observation and motion model based on a particle filter framework to meet the needs of human target recognition and tracking during the movement of a quadruped robot. On the one hand, the pedestrian category is detected through the classification detector of the observation model, and then the target person is identified through the target identifier; on the other hand, the Gaussian motion model of the motion model is used to transfer particles to cover the tracking target to solve the problem of jitter in the video field of view of the quadruped robot, and then the feature point tracking model is used to predict the particle motion state to reduce the probability of tracking drift when the picture jumps.
[0047] The present application provides a human target recognition and tracking method suitable for quadruped robot inspection, which performs human target recognition and tracking under the framework of particle filtering.
[0048] The idea of particle filtering (PF) is based on the Monte Carlo method, which uses a set of particles to represent probability and can be used in any form of state space model. Its core idea is to express its distribution by extracting random state particles from the posterior probability, which is a sequential importance sampling method. Assume that the state space model of the system is as follows:
[0049] X k =f k (X k-1 ,V k-1 ) (1)
[0050] Z k =h k (X k ,N k-1 ) (2)
[0051] Formula (1) represents the state transition model, where X k represents the state vector of the system at time k, V k The noise vector representing the state, f krepresents the state transfer equation; Formula (2) represents the observation model, where Z k represents the measured value of system k, N k represents the system measurement noise, h k Represents the observation equation. Particle filtering is divided into prediction and update based on Bayesian filtering. Prediction refers to calculating the prior probability of the state at time k based on the system's motion model and the posterior probability of the state at time k-1, which is given by formula (3):
[0052] p(X k |Z 1:k-1 )=∫p(X k |X k-1 )p(X k-1 |Z 1:k-1 )dX k-1 (3)
[0053] Among them, p(X k-1 |Z 1:k-1 ) represents the result of the previous iteration, p(X k |X k-1 ) is obtained from the state transition model.
[0054] Updating means updating the prior probability to the posterior probability through the observation model according to the Bayesian rule, which can be obtained from formula (4):
[0055] p(X k |Z 1:k-1 )∝p(Z k |X k )p(X|Z 1:k-1 ) (4)
[0056] Where p(Z k |X k ) is obtained from the observation model.
[0057] See also Figure 1 The human target recognition and tracking method of the present application comprises the following steps:
[0058] S1: Initialization: Get the initial state X of the human target from the video frame t-1 , the particle set at time t is By X t-1 It is generated in an iterative manner, where Ns represents the number of particles;
[0059] S2: Target state prediction: The motion model uses a combination of Gaussian motion model and feature point tracking model to complete target state prediction. The Gaussian motion model transfers particles to cover the tracking target to solve the problem of human targets being blocked; the feature point tracking model predicts the particle motion state to reduce the probability of tracking drift when the picture jumps due to camera shake;
[0060] The motion model of the particle filter in this application includes two models: Gaussian motion and point feature tracking. Through this dual-modal complementary motion model, the generated and transmitted particles can accurately track and cover the target even in the video sequence where the quadruped robot camera frequently shakes and the target in the field of view moves rapidly. The state vector of the particle in the motion model is X = [x, y, s, μ, ν] T The meanings of the components are as follows: (x, y), the two-dimensional position coordinates in the image coordinate system; s, the pixel width of the particle; (μ, ν), the velocity components on the x-axis and y-axis respectively. In order to reduce the dimension of the state space, the algorithm assumes that the pixel aspect ratio δ of the particle is fixed, that is, h = δs. The fixed value of δ is taken as 0.42 and remains unchanged during the entire algorithm experiment. The particle set at time t is By X t-1 Generate in an iterative manner, where Ns represents the number of particles.
[0061] Gaussian motion model: Pedestrians’ movements are random, especially when the camera is shaking. It is difficult to accurately predict the next move of the human target. The target may appear anywhere near the position of the previous frame. Therefore, the Gaussian motion model is used for transmission coverage. Assuming that the movement of the human target between two consecutive frames is uniform, the particles are given greater coverage by adding Gaussian noise. The i-th particle at time t is The process of random walk iterative particle transfer is as follows:
[0062]
[0063] Process noise It obeys the normal distribution with zero mean, and assumes that each dimensional state variable is independent of each other. Among them, the noise variance of the position (x, y) and the pixel width s is proportional to the size of the target. The larger the width of the target, the larger the variance; the noise variance of the velocity component is set to a constant. When the motion continuity assumption holds, the particles transmitted by the Gaussian motion model are very representative. As long as the human target appears near the position of the previous frame, these particles can accurately cover the target. However, the Gaussian motion model can only solve the problem of camera shake. When the picture jumps, that is, when the human target is instantly displaced, the human target deviates from the original position by a large margin. The particles generated by the Gaussian motion model cannot cover the target extensively, and the particles lose their representativeness. Of course, we can also use more particles to cover a larger area, but this will cause excessive computational load and will also include more background interference. Therefore, another feature point tracking model is used to complement this problem.
[0064] The tracking frame of the Gaussian motion model provided in the embodiment of the present application tracks a human target. The blue tracking frame represents the vicinity of the target position of the previous frame covered by the Gaussian motion model, and the red tracking frame represents the tracking result. However, this method will cause tracking drift when the target undergoes instantaneous displacement (screen transition).
[0065] Feature point tracking model: extract feature points (Harris corner points) in the tracking result rectangle at time t-1, and then calculate the center of these feature points. We use their position mean to represent the center, which is recorded as Then use the Lucas-Canade optical flow method to track these feature points in the current frame and calculate the center of the tracked feature points, which is recorded as The expected state of the particle is denoted by Given by formula (4):
[0066]
[0067] In the above formula, it is assumed that the center of the feature point of the previous frame The center of the feature point in the current frame And the corresponding tracking rectangle result The relative position relationship remains unchanged, and finally the feature points and the particles generated by the model are connected by The target position is obtained by sampling the Gaussian distribution with the mean value. The target position is relatively accurately obtained by tracking the feature points, as shown in formula (5):
[0068]
[0069] in, is Gaussian noise, is the tracking result of the previous frame. Ω(·) is the core process of feature point tracking, which converts the tracking result of the previous frame into Mapped to the particle set at time t Expected state Although feature point tracking can solve the problem of screen transition, that is, instantaneous displacement of the target, when the human target in the picture is completely occluded, the feature point tracking model will lose its effectiveness and cause tracking drift. Therefore, the advantages of the Gaussian motion model can be combined to deal with the target occlusion problem.
[0070] In the tracking process of the feature point tracking model provided in the embodiment of the present application, during the movement of the quadruped robot, the jitter of the camera can easily produce instantaneous displacement. We use feature point tracking to predict the position where the target will appear. The yellow tracking box indicates the particle Gaussian distribution sampling generated by feature point tracking. However, this method will lose its effect on the problem of target occlusion. When the target is completely occluded, the tracked and extracted feature points will eventually gather on the occluder. In this way, the extracted features of the occluder have lost their transmission value, which eventually leads to tracking failure.
[0071] Combination of two models: The workflow is to first calculate the feature points of the image frame of the target detection result and The Euclidean distance d between the two represents the pixel displacement of the target in the image coordinate system; the larger the value of d, the greater the possibility of instantaneous displacement of the target. We set it to When the pixel is equal to 0, it is determined that the detected target has instantaneous displacement. When instantaneous displacement occurs, the Gaussian motion model cannot successfully cover and capture the target. At this time, the feature point tracking model can be used for tracking. Therefore, the respective advantages of the Gaussian motion model and the feature point tracking model are combined to complement each other, as shown in formula (6):
[0072] X t =ηX t +(1-η)X t (6)
[0073] Where η represents the percentage of particles transmitted by the Gaussian motion model, 0≤η≤1; all particles are divided into two parts, one part is transmitted by the Gaussian motion model, with ηNs particles; the other part is transmitted by the feature point tracking model, with the remaining (1-η)Ns particles. The value of η will be adaptively adjusted according to the instantaneous displacement detector. When the instantaneous displacement is detected, the feature point tracking model is given a greater weight, that is, the value of η is reduced. For example, if the value of η is 0.2, it means that the role of the feature point tracking model is dominant; when no instantaneous displacement is detected, the value of η is increased, so that the role of the Gaussian motion model is dominant, avoiding the failure of the feature point tracking model when the human target is occluded.
[0074] The respective advantages of the Gaussian motion model and the feature point tracking model are combined to solve the problems of instantaneous displacement and target occlusion, producing a relatively good robust effect.
[0075] S3: System observation: The observation model uses a combination of classification detector and target identifier to complete the target person position estimation; the classification detector of the observation model detects the pedestrian class, and then the target identifier recognizes the target person to complete the target person position estimation;
[0076] Classification detector: Refer to the classic pedestrian detection algorithm based on HOG (Histogram of Oriented Gradients) features and SVM classifier. Unify the image block corresponding to each particle into a 64×128 pixel block size, then extract the HOG features of the pixel block and use it as input to the pre-trained SVM classifier to obtain the classifier score of each particle. Finally, the score is processed by non-maximum suppression to obtain the final result. Because the score of the SVM classifier may be negative, we map the score through an exponential function and normalize it to finally use it as the likelihood probability that the particle is a pedestrian. However, this classification detector can only classify pedestrians. Each pedestrian can get a high-score detection result, but it cannot focus on a specific pedestrian target and is easily disturbed by other pedestrian targets. Therefore, a target detector is also needed to accurately identify the target person.
[0077] like Figure 2 As shown, the pedestrian detection algorithm based on the histogram of oriented gradients (HOG) feature and the SVM classifier provided in the embodiment of the present application first needs to pre-train the SVM classifier on the pedestrian detection data set, and then the pixel block corresponding to each particle is unified into a size of 64×128, and then the HOG feature descriptor of each pixel block is extracted, and each extraction is input into the trained SVM classifier.
[0078] Target Identifier: The classification detector gives the probability of detecting pedestrians, while the target detector is used to accurately identify the target person. We use the usual feature + matching method, which mainly consists of three parts: template pool preparation, feature descriptor, and template matching.
[0079] 1) Template pool preparation: In order to accurately identify the target person, it is necessary to prepare the template image of the target person in advance, which provides an identification basis for the subsequent identification of the target person. Generally, multiple images of the target person need to be prepared in the template pool, covering various postures of the target person, in order to enhance the recognition accuracy. First, search whether the template already exists in the storage according to the provided identity description of the target person (such as name, number code, etc.). If it exists, read the template directly from the storage space to avoid reloading duplicate templates; if it does not exist, prepare online, and it will be continuously updated before the template pool reaches the storage boundary. The picture of the target person will be used as the first template in the template pool, and the target person identified by subsequent identification will continue to be used to enrich the template pool. The starting template T init We collect several frames of images every s seconds from the target person’s video, namely Where N i Indicates the number of templates at the start. If time t≤1+(N i -1)s, there is only the starting template in the template pool.
[0080] 2) Feature descriptor: By extracting channel histogram features as feature descriptors of image particles. The 10 channels of the image at time t, including LUV color space, gradient amplitude, and gradient histograms in six directions (0°, 30°, 60°, 90°, 120°, 150°), are directly extracted from the histogram features of the pixel blocks corresponding to the particles on the three channels of the LUV color space. For the directional gradient histogram channel, the pixel values of the pixel blocks corresponding to each directional channel are summed. These channel histogram features will contain the color information and contour (gradient) information of the human target. If Represents particles The three-channel histogram feature of the LUV color space, the directional gradient histogram feature of the j-th pixel block is Then the characteristic description of the i-th particle at time t can be expressed as It has N+3 in total, the first three parts describe the color information, and the last N parts describe the local contour information.
[0081] 3) Template matching: The Bhattacharyya distance is used to measure the matching similarity between the particle and the template. The distance is defined as Where C is the candidate particle, T is the template, denote the i-th characteristic component of the candidate particle and the template respectively, and the function b(·) denotes the Bhattacharyya distance between the two vectors. init The distance between them is d(C,T init )=min{d(C,T i )}, take the minimum value of the distance between the candidate particle and the starting template, similarly, the candidate particle C and the current template T curr The distance between them is d(C,T curr )=min{d(C,T i )},T i ∈T curr The distance between candidate C and the entire template pool is defined as the weighted sum of its distances from the templates at two moments, as shown in the following formula.
[0082] d=γd(C,T init )+(1-γ)d(C,T curr ) (7)
[0083] Observation model: It consists of a classification detector and a target identifier, as shown below:
[0084]
[0085] in, It represents the probability that the particle given by the classification detector is a pedestrian. It represents the probability that the particle given by the target identifier is the target person. represents the confidence weight parameter, The larger the value, the more likely the observation model is to trust the target identifier, and vice versa.
[0086] like Figure 3As shown, the target identifier provided in the embodiment of the present application is based on a template matching method to accurately identify the target person, and is mainly composed of three parts: a template pool, feature descriptor extraction, and template matching. The quality of template preparation in the template pool will directly affect the final matching probability result. Therefore, we have designed a dynamically updated template pool, which is composed of target person templates at different times.
[0087] S4: Resampling: Obtain the particle weights through the observation model, resample after normalizing the weights, update the classifier template, and obtain the final particle set at the current moment;
[0088] The importance distribution of particle filtering is q(·)=p(X t |X t-1 ), when the observed value Y at time t is obtained t (When the particle weight Iterate and update through formula (9):
[0089]
[0090] in, is the observation model.
[0091] S5: Target tracking result: Calculate each target state according to the final particle set, and obtain the target tracking result by weighted average; check whether the information of the next moment arrives. If so, return to step S2 and use the final particle set as the input of the next filtering process; otherwise, the target tracking process ends.
[0092] After the particle weight is calculated by formula (5), it needs to be normalized first, so that Then all particles are weighted averaged to obtain the tracking result.
[0093] In the description of the present invention, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. "Multiple" means two or more, unless otherwise clearly and specifically defined.
[0094] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.
[0095] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.
Claims
1. A human target recognition and tracking method suitable for quadruped robot inspection, which performs human target recognition and tracking under the framework of particle filtering. It is characterized in that The method comprises the following steps: S1: Initialization: Get the initial state X of the human target from the video frame t-1 , the particle set at time t is By X t-1 It is generated in an iterative manner, where Ns represents the number of particles; S2: Target state prediction: The motion model uses a combination of Gaussian motion model and feature point tracking model to complete target state prediction. The Gaussian motion model transfers particles to cover the tracking target to solve the problem of human targets being blocked; the feature point tracking model predicts the particle motion state to reduce the probability of tracking drift when the picture jumps due to camera shake; S3: System observation: The observation model uses a combination of classification detector and target identifier to complete the target person position estimation; the classification detector of the observation model detects the pedestrian class, and then the target identifier recognizes the target person to complete the target person position estimation; S4: Resampling: Obtain the particle weights through the observation model, resample after normalizing the weights, update the classifier template, and obtain the final particle set at the current moment; S5: Target tracking result: Calculate each target state according to the final particle set, and obtain the target tracking result by weighted average; Check whether the information of the next moment arrives, if so, return to step S2, and use the final particle set as the input of the next filtering process; otherwise, the target tracking process ends; The classification detector is used to detect the probability of pedestrians. The pedestrian detection algorithm based on HOG features and SVM classifier is used to unify the image block corresponding to each particle into a 64×128 pixel block size, and then the HOG features of the pixel block are extracted and input into the pre-trained SVM classifier to obtain the classifier score of each particle. Finally, the score is processed by non-maximum suppression to obtain the final result; The target identifier is used to accurately identify the target person, including: 1) Template pool preparation: First, search the storage space for existing templates based on the target person's identity description. If so, read the template directly from the storage space. If not, prepare online and use the image of the target person as the first template in the template pool. Then, enrich the template pool by identifying the target person. The starting template T init Collect several frames of images from the target person’s video every s seconds, that is, Where N i Indicates the number of starting templates; 2) Feature descriptor: By extracting channel histogram features as the feature descriptor of the particle, the LUV color space, gradient amplitude and gradient histogram in six directions (0°, 30°, 60°, 90°, 120°, 150°) of the image at time t are directly extracted on the three channels of the LUV color space to extract the histogram features of the pixel blocks corresponding to the particles. The six-directional gradient histogram channel sums the pixel values of the pixel blocks corresponding to each directional channel; if Represents particles The three-channel histogram feature of the LUV color space, the directional gradient histogram feature of the j-th pixel block is Then the characteristic description of the i-th particle at time t can be expressed as The first three parts describe the color information, and the last N parts describe the local contour information; 3) Template matching: The Bhattacharyya distance is used to measure the matching similarity between the particle and the template. The distance is defined as Where C is the candidate particle, T is the template, denote the i-th characteristic component of the candidate particle and the template respectively, and the function b(·) denotes the Bhattacharyya distance between the two vectors; therefore, the candidate particle C and the starting template T init The distance between them is d(C,T init )=min{d(C,T i )}, take the minimum value of the distance between the candidate particle and the starting template, similarly, the candidate particle C and the current template T curr The distance between them is d(C,T curr )=min{d(C,T i )},T i ∈T curr ; The distance between candidate C and the entire template pool is defined as the weighted sum of its distances from the templates at two moments, and γ indicates which moment the target detector is more inclined to trust, as shown in formula (5): d=γd(C,T init )+(1-γ)d(C,T curr ) (5)。 2. The human target recognition and tracking method according to claim 1, It is characterized in that The state vector of the particle in the motion model is X = [x, y, s, μ, ν] T , where (x, y) is the two-dimensional position coordinate in the image coordinate system; s is the pixel width of the particle; (μ, ν) are the velocity components on the x-axis and y-axis respectively; the particle set at time t is By X t-1 Generate in an iterative manner, where Ns represents the number of particles.
3. The human target recognition and tracking method according to claim 2, It is characterized in that The particle transfer process of the Gaussian motion model is: in, is the state vector of the ith particle at time t, and the process noise It obeys a normal distribution with zero mean and assumes that each dimensional state variable is independent of each other; the noise variance of the position (x, y) and pixel width s is proportional to the size of the target; the noise variance of the velocity component (μ, ν) is a constant.
4. The human target recognition and tracking method according to claim 3, It is characterized in that The feature point tracking model is: Extract feature points in the rectangular frame of the tracking result at time t-1 and calculate the center of the feature points Then use the Lucas-Canade optical flow method to track the feature points of the current frame and calculate the center of the tracked feature points, which is recorded as The expected state of the particle is denoted by Given by formula (2): Among them, it is assumed that the center of the feature point of the previous frame The center of the feature point in the current frame And the corresponding tracking rectangle result The relative position relationship remains unchanged, and finally the feature points and the particles generated by the model are connected by It is obtained by sampling from a Gaussian distribution with a mean value; The target position is given by feature point tracking, as shown in formula (3): in, is Gaussian noise, is the tracking result of the previous frame; Ω(·) is the tracking result of the previous frame Mapped to the particle set at time t Expected state 5. The human target recognition and tracking method according to claim 4, It is characterized in that The motion model adopts a particle transfer model combining a Gaussian motion model and a feature point tracking model: X t =ηX t +(1-n)X t (4) Among them, η represents the percentage of particles transmitted by the Gaussian motion model, 0≤η≤1; all particles are divided into two parts, one part is transmitted by the Gaussian motion model, there are ηNs particles; the other part is transmitted by the feature point tracking model, and there are (1-η)Ns particles left; the value of η is adaptively adjusted according to the instantaneous displacement detector. When instantaneous displacement is detected, the weight of the feature point tracking model is increased, that is, the value of η is reduced.
6. The human target recognition and tracking method according to claim 1, It is characterized in that The observation model consists of a classification detector and a target identifier, as shown in formula (6): in, represents the probability that the particle given by the classification detector is a pedestrian, represents the probability that the particle given by the target identifier is the target person, Represents the confidence weight parameter.
7. The human target recognition and tracking method according to claim 1, It is characterized in that The step S4 comprises: Particle weight Iterate and update through formula (7): in, is the observation model, and the particle weights are calculated by equation (7) and then normalized so that Then all particles are weighted averaged to obtain the tracking result.
