Pedestrian re-identification device and method

Abstract

To provide a pedestrian re-identification device, method, and computer storage medium, which improve accuracy of pedestrian re-identification in scenes where a pedestrian changed his / her posture significantly or is identified across different cameras.SOLUTION: A pedestrian re-identification device 100 is provided, comprising: a pedestrian detection unit configured to detect pedestrians from each video frame of a video sequence; a feature extraction unit configured to extract appearance features of each pedestrian detected in the video frames; and a pedestrian identification unit configured to match detected appearance features of each pedestrian with target appearance features of a target pedestrian, including at least first and second appearance features associated with an outline of a target pedestrian, to identify the target pedestrian in the video frames.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present disclosure relates to image processing, and in particular to pedestrian re-identification devices, methods and computer storage media. [Background technology]

[0002] Pedestrian re-identification is a technique that analyzes images and video sequences from multiple cameras with overlapping or non-overlapping fields of view to determine whether a specific target pedestrian is present. Unlike general pedestrian tracking, pedestrian re-identification can identify a specific target pedestrian in images and video sequences captured across different cameras to realize long-term tracking and monitoring of a specific target pedestrian, and therefore has great potential for application in surveillance fields, criminal investigations, etc. Summary of the Invention [Problem to be solved by the invention]

[0003] Currently, pedestrian re-identification generally determines whether a detected pedestrian corresponds to a known target pedestrian by matching the features of each pedestrian detected in an image or video sequence with the features of a known target pedestrian in a pre-constructed target pedestrian library. The effect is often not ideal. The main reasons are that the posture and appearance of the same pedestrian in images captured by different cameras may be significantly different due to differences in the shooting angles, field of view, lighting conditions, etc. between different cameras; pedestrians often have relatively strong mobility, so their posture and appearance also change to a certain extent over time; and the features of the pre-constructed target pedestrian are single, so misidentification is likely to occur when the posture and appearance of the same pedestrian changes. [Means for solving the problem]

[0004] According to one aspect of the present disclosure, there is provided a pedestrian re-identification device including: a pedestrian detection unit configured to detect pedestrians in each video frame of a video sequence; a feature extraction unit configured to extract appearance features of each pedestrian detected in the video frames; and a pedestrian identification unit configured to match the appearance features of each detected pedestrian with target appearance features of a target pedestrian including at least a first appearance feature and a second appearance feature associated with an outline of the target pedestrian, in order to identify a target pedestrian from the video frames.

[0005] According to another aspect of the present disclosure, there is provided a pedestrian re-identification device including a processor and a memory storing computer program instructions, which, when executed by the processor, causes the processor to perform the steps of: detecting pedestrians in each video frame of a video sequence; extracting appearance features for each pedestrian detected in the video frames; and matching the appearance features of each detected pedestrian with target appearance features of a target pedestrian including at least a first appearance feature and a second appearance feature associated with an outline of the target pedestrian, in order to identify a target pedestrian from the video frames.

[0006] According to another aspect of the present disclosure, there is provided a pedestrian re-identification method including the steps of detecting pedestrians in each video frame of a video sequence, extracting appearance features of each pedestrian detected in the video frames, and matching the appearance features of each detected pedestrian with target appearance features of a target pedestrian including at least a first appearance feature and a second appearance feature associated with an outline of the target pedestrian to identify a target pedestrian from the video frames.

[0007] According to another aspect of the present disclosure, there is provided a computer readable storage medium having stored thereon computer program instructions which, when executed by a processor, implements the pedestrian re-identification method described above. Effect of the Invention

[0008] The devices, methods, and computer storage media according to the above aspects of the present disclosure improve the accuracy of pedestrian re-identification in situations where the pedestrian's pose changes significantly or when identifying across cameras. [Brief description of the drawings]

[0009] These and / or other aspects and advantages of the present disclosure will become more apparent and become more understandable in the following detailed description of exemplary embodiments of the present disclosure, which refers to the drawings.

[0010] [Figure 1] 1(a) and 1(b) are diagrams illustrating exemplary misidentification cases during pedestrian re-identification. [Diagram 2] FIG. 2 is an exemplary configuration block diagram of a pedestrian re-identification device according to an embodiment of the present disclosure. [Diagram 3] 3(a) and 3(b) are schematic diagrams showing examples of a first appearance feature and a second appearance feature among target appearance features according to an embodiment of the present disclosure. [Figure 4] 1 is a schematic diagram of examples of multiple appearance features among target appearance features according to an embodiment of the present disclosure. [Diagram 5] FIG. 2 is a hardware block diagram of an exemplary computing device for implementing a pedestrian re-identification device according to an embodiment of the present disclosure. [Figure 6] 1 is a schematic flowchart of a pedestrian re-identification method according to an embodiment of the present disclosure. [Figure 7] 7(a) and 7(b) are schematic diagrams of the testing results of different pedestrian re-identification methods. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] In order to make those skilled in the art better understand the present disclosure, the following clearly and completely describes the technical solutions in the embodiments of the present disclosure with reference to the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only some embodiments of the present disclosure, and not all of the embodiments. Any other embodiments obtained by those skilled in the art based on the embodiments of the present disclosure without creative labor are within the scope of the present disclosure.

[0012] As described above, current pedestrian re-identification generally matches the features of each pedestrian detected in an image or video sequence with the features of a known target pedestrian in a pre-constructed target pedestrian library to determine whether the detected pedestrian corresponds to the known target pedestrian. However, due to various factors such as pedestrian posture changes, conventional pedestrian re-identification methods are prone to misidentifying pedestrians. In particular, when the target pedestrian library for feature matching has a single feature, if the same pedestrian's posture or appearance changes significantly, it may be misidentified as another pedestrian. With reference to FIG. 1(a) and FIG. 1(b), an exemplary case of pedestrian misidentification that may occur in pedestrian re-identification will be described.

[0013] FIG. 1(a) shows a case of pedestrian misidentification that occurs when the target feature of a target pedestrian is a feature associated with its half body, but the pedestrian appears in the screen with his whole body, and the feature associated with the whole body is extracted and matched. As shown in the left diagram of FIG. 1(a), when a certain pedestrian just enters the field of view of the camera, the half body appears on the screen, and at that time, a new ID is assigned to the pedestrian, and the feature associated with the half body of the pedestrian on the screen is set as the target appearance feature and is used for subsequent feature matching. After that, as shown in the right diagram of FIG. 1(a), when the pedestrian enters the field of view of the camera, the whole body appears on the screen, and since there is a large difference between the feature associated with the whole body extracted from this screen and the feature associated with the half body previously constructed as described above, matching errors are likely to occur, and the pedestrian is misidentified as another pedestrian.

[0014] FIG. 1(b) shows an exemplary case of pedestrian misidentification occurring when the target feature of a target pedestrian is a feature associated with the whole body, the pedestrian appears in the screen with only half of his / her body, and the feature associated with the half of his / her body is extracted and matched. As shown in the left diagram of FIG. 1(b), when a pedestrian appears in the screen with his / her whole body, the feature associated with the whole body of the pedestrian may be extracted from the screen as the target appearance feature and used for subsequent feature matching. Then, as shown in the right diagram of FIG. 1(b), for example, when the pedestrian is occluded or moves out of the camera's field of view, the pedestrian appears in the screen with only half of his / her body, and since there is a large difference between the feature associated with the half of his / her body extracted from the screen and the feature associated with the whole body previously constructed as described above, the pedestrian is easily misidentified as another pedestrian.

[0015] As can be seen from this, in such a situation, if the posture of the pedestrian changes significantly, the features extracted from the captured video frame and the features of the target pedestrian constructed in advance may differ in the external shape of the pedestrian corresponding to each other, and for example, when a feature corresponding to the half of the pedestrian is extracted and the features of the target pedestrian are corresponding to the whole body, or when a feature corresponding to the whole body of the pedestrian is extracted and the features of the target pedestrian are corresponding to the half body, the difference between the two matching features is large, and pedestrians are likely to be misidentified. In such a situation, in the present disclosure, in order to match the corresponding features of the detected pedestrian, the features of the target pedestrian are constructed by comprehensively considering multiple appearance features corresponding to different external shapes of the target pedestrian, so that it is possible to provide a sufficiently accurate identification result in a scene where the posture of the pedestrian changes significantly or identification is performed across cameras.

[0016] Next, a pedestrian re-identification device according to an embodiment of the present invention will be described with reference to Fig. 2. Fig. 2 is an exemplary configuration block diagram of a pedestrian re-identification device 100 according to an embodiment of the present disclosure. As shown in Fig. 2, the pedestrian re-identification device 100 may include a pedestrian detection unit 110, a feature extraction unit 120, and a pedestrian identification unit 130. The main functions of each unit of the pedestrian re-identification device 100 will be described below.

[0017] The pedestrian detection unit 110 can detect a pedestrian in each video frame of a video sequence. As described above, unlike general pedestrian tracking, the pedestrian re-identification technique according to the present disclosure can identify a specific target pedestrian in images and video sequences captured across different cameras, and can realize long-term tracking and monitoring of the specific target pedestrian. More specifically, this video sequence is an analysis target video sequence from which it is necessary to identify whether or not a target pedestrian is present, and this video sequence may be captured by a camera different from the video frame in which the target pedestrian was previously captured and its appearance features were extracted, or may be captured by the same camera at a different timing.

[0018] Specifically, the video sequence analyzed by the pedestrian detection unit 110 may include one or more video frames, each of which may be consecutive or non-consecutive video frames captured by a single camera or by multiple cameras with overlapped or non-overlapping fields of view. The pedestrian detection unit 110 may detect pedestrians from each video frame of the video sequence by any suitable image detection technique in the art, and the present disclosure is not limited thereto. For example, the pedestrian detection unit 110 may perform processes such as foreground segmentation, edge extraction, and motion detection for each video frame to identify each sub-image region corresponding to each pedestrian appearing in the video frame. For example, the sub-image region of the video frame may be represented by a rectangular frame circumscribing the body contour of the detected pedestrian. Also, for example, the pedestrian detection unit 110 may perform pedestrian detection for each video frame by a pedestrian detection classifier trained in advance using a machine learning method such as a neural network or a support vector machine, and determine the location where each pedestrian appearing in the video frame is located.

[0019] The feature extraction unit 120 may extract appearance features for each pedestrian detected from the video frames. In the present disclosure, the appearance features may include features reflecting the appearance morphology of the pedestrian, such as color features, texture features, shape features, and face features, but the present disclosure is not limited thereto. Methods for extracting each of the above features are well known in the art, and therefore will not be described here.

[0020] The pedestrian identification unit 130 may match the appearance features of each detected pedestrian with the target appearance features of the target pedestrian, and identify the target pedestrian from the video frame based on the matching result. Specifically, the pedestrian identification unit 130 may match the appearance features of each detected pedestrian with the appearance features of the target pedestrian, and perform pedestrian re-identification by identifying whether the pedestrian detected in the video frame is the target pedestrian based on the similarity between the two. Alternatively, the number of target pedestrians may be multiple, and accordingly, the pedestrian identification unit 130 may match the appearance features of each detected pedestrian with the appearance features of each constructed target pedestrian, and identify whether the pedestrian detected in the video frame corresponds to any of the target pedestrians based on the similarity between the two, thereby performing pedestrian re-identification. It will be understood that the target appearance features of the target pedestrian may be included in a pedestrian appearance feature library constructed in advance, and the pedestrian appearance feature library may be stored locally in the pedestrian re-identification device, or may be stored in a server accessible to the pedestrian re-identification device. Alternatively, the target appearance features of the target pedestrian may be generated from an image when necessary and input from outside, instead of being stored in advance in the pedestrian appearance feature library. The present disclosure does not limit the method of storing or acquiring the target appearance features of the target pedestrian.

[0021] In the embodiment of the present disclosure, a target pedestrian can be identified from a video frame using various feature matching methods, and for completeness of description, exemplary methods are described below. For example, the pedestrian identification unit 130 may calculate a feature distance between the appearance feature of each detected pedestrian and the target appearance feature of the target pedestrian, such as Manhattan distance, Euclidean distance, Pearl distance, etc., and compare it with a preset threshold to determine whether the detected pedestrian corresponds to the target pedestrian. Alternatively, the number of target pedestrians may be multiple, and in this case, the pedestrian identification unit 130 may calculate each feature distance from each target pedestrian for the detected pedestrian, identify the closest feature distance among them, and if the closest feature distance is smaller than the preset threshold, identify the detected pedestrian as the target pedestrian corresponding to the closest feature distance. The pedestrian identification unit 130 may output a pedestrian re-identification result based on the matching result. For example, if the detected pedestrian corresponds to a certain target pedestrian, the detected pedestrian is associated with the identity of the target pedestrian, and if the detected pedestrian cannot be matched with the target pedestrian, a new identity is assigned to the detected pedestrian.

[0022] As described above, when the posture of a pedestrian changes significantly, if the features extracted from the captured video frame and the pre-constructed features of the target pedestrian are not identical to each other, the difference between the two features is large, and the result of erroneous identification of the pedestrian is likely to occur. In the present disclosure, the target appearance features of the target pedestrian include at least a first appearance feature and a second appearance feature corresponding to the target pedestrian's appearance, thereby enabling the pre-constructed multiple appearance features to fully reflect different characteristics corresponding to different appearances of the target pedestrian, thereby improving the accuracy of pedestrian re-identification. Note that the appearance in the present disclosure may refer to the form in which the target pedestrian appears on the screen, for example, the whole body appearance, the half body appearance, the front appearance, the side appearance, the back appearance, etc., but it will be understood that the present disclosure is not limited thereto. Therefore, the first appearance feature and the second appearance feature are two appearance features corresponding to different appearances of the target pedestrian. Hereinafter, the pedestrian identification operation of the pedestrian re-identification device 100 will be specifically described for different examples of the first appearance feature and the second appearance feature.

[0023] According to an embodiment of the present disclosure, the first appearance feature of the target pedestrian may be a feature associated with the whole body, and the second appearance feature of the target pedestrian may be a feature associated with the half body. As shown in FIG. 3(a), the first appearance feature of the target pedestrian may be a feature associated with the whole body in front, and the second appearance feature may be a feature associated with the half body in front. It should be understood that the half body in the present disclosure is not strictly limited to half the size of the whole body, and may be any appropriate ratio of the whole body, such as 1 / 4, 1 / 3, etc., as necessary. Hereinafter, the pedestrian identification operation by the pedestrian re-identification device 100 in this case will be specifically described.

[0024] As described above, the pedestrian detection unit 110 may detect a pedestrian in each video frame of a video sequence and identify each sub-image region corresponding to each pedestrian appearing in the video frame, and the pedestrian detection unit 110 may identify whether the pedestrian appears in the whole body or half body in the video frame based on the aspect ratio of the detected pedestrian in the video frame (e.g., the aspect ratio of the pedestrian's circumscribing rectangular frame), a full-body image / half-body image classifier trained based on a neural network, or other methods. Therefore, when the pedestrian detection unit 110 identifies the detected pedestrian as appearing in the video frame with its whole body, the feature extraction unit 120 may extract features associated with the whole body, and may extract appearance features such as color features, texture features, shape features, and face features from the sub-image region in which the pedestrian is located. Additionally, the feature extraction unit 120 may cut out a half-body image of the detected pedestrian from the video frame and extract features associated with that half-body. For example, the feature extraction unit 120 may cut out the upper half of the sub-image area in which the pedestrian is located and extract appearance features such as color features, texture features, shape features, and facial features associated with that half-body from the cut-out half of the sub-image area.

[0025] Next, the pedestrian identification unit 130 may match the features associated with the entire body of the detected pedestrian with the features associated with the entire body of the target pedestrian, match the features associated with the half body with the features associated with the half body of the target pedestrian, and if at least one of the features associated with the entire body of the detected pedestrian and the features associated with the half body can be matched with the target pedestrian, the pedestrian detected in the video frame may be matched with the identity of the target pedestrian. That is, if the pedestrian identification unit 130 can match one of the features associated with the entire body of the detected pedestrian and the features associated with the cut-out half body with the corresponding features of the target pedestrian, the pedestrian can be accurately identified without being identified as another pedestrian. Alternatively, the pedestrian identification unit 130 may first match either one of the features associated with the entire body of the detected pedestrian and the features associated with the half body with the corresponding features of the target pedestrian, and if the matching is successful, the matching operation for the remaining one feature is omitted, and only if the matching is unsuccessful, the matching for the remaining one feature is continued. This can reduce the amount of calculation in the matching process. In addition, if the pedestrian identification unit 130 determines that a pedestrian detected in a video frame cannot be matched with a target pedestrian, i.e., features associated with the entire body of the detected pedestrian cannot be matched with features associated with the entire body of the target pedestrian, and features associated with one half of the detected pedestrian cannot be matched with features associated with one half of the target pedestrian, the pedestrian identification unit 130 may assign a new identity to the detected pedestrian.

[0026] Alternatively, the pedestrian re-identification device 100 may further include a feature update unit (not shown). The feature update unit may update the target appearance feature of the target pedestrian based on the identification result of the pedestrian identification unit 130. Specifically, when a match is made between the pedestrian detected in the video frame and the target pedestrian, the feature update unit may update the target appearance feature of the target pedestrian using the feature associated with the whole body of the detected pedestrian and the feature associated with the half body. For example, when the pedestrian detected in the video frame is identified as corresponding to a certain target pedestrian and the detected pedestrian is associated with the identity of the target pedestrian, if the target pedestrian lacks the feature associated with the half body or the feature associated with the whole body, the feature update unit directly adds the feature associated with the whole body or the half body of the detected pedestrian in the video frame to the missing part of the target pedestrian, and for the feature associated with the existing whole body of the pedestrian, the feature update unit replaces the existing feature associated with the whole body of the target pedestrian with the feature associated with the whole body of the detected pedestrian in the video frame. Alternatively, the feature update unit may replace the features associated with the entire body of the detected pedestrian in the video frame with the features associated with the existing entire body of the target pedestrian, and may integrate the features associated with the existing half body of the target pedestrian as features associated with the entire body of the target pedestrian after updating, and for the features associated with the existing half body of the target pedestrian, the feature update unit may replace the features associated with the existing half body of the target pedestrian with the features associated with the half body extracted after cropping the video frame, or may integrate the features associated with the extracted half body and the features associated with the existing half body of the target pedestrian as features associated with the half body of the target pedestrian after updating. Alternatively, the feature update unit may transmit the features associated with the entire body of the detected pedestrian and the features associated with the half body to a server that maintains target appearance features of a target pedestrian, and cause the server to update the target appearance features of the target pedestrian based on the received features, so that the updated features can be tracked over time to characteristics corresponding to different external shapes of the target pedestrian.Additionally, if the pedestrian identification unit 130 determines that a detected pedestrian in a video frame cannot be matched with a target pedestrian and assigns a new identity to the detected pedestrian, the feature update unit may construct target appearance features, i.e., the features associated with the whole body of the detected pedestrian in the video frame may be the features associated with the whole body of a target pedestrian, and the features associated with the half body extracted after cropping the video frame may be the features associated with the half body of the target pedestrian, for subsequent feature matching.

[0027] The above describes the corresponding operations of the feature extraction unit 120 and the pedestrian identification unit 130 when the pedestrian detection unit 110 determines that the detected pedestrian appears in the video frame with the entire body. Next, the corresponding operations of the feature extraction unit 120 and the pedestrian identification unit 130 when the pedestrian detection unit 110 determines that the detected pedestrian appears in the video frame with only half of the body will be described.

[0028] Specifically, when the pedestrian detection unit 110 determines that the detected pedestrian appears in the video frame with only half of its body, the feature extraction unit 120 may extract features associated with the half of its body, for example, from the sub-image region in which the pedestrian exists, and may extract appearance features, such as color features, texture features, shape features, and facial features, associated with the half of its body. Thus, the pedestrian identification unit 130 may match features associated with the half of the detected pedestrian with features associated with the half of a target pedestrian, and if the features associated with the half of the pedestrian can be matched with the target pedestrian, the pedestrian detected in the video frame may be matched with the identity of the target pedestrian. In addition, when the pedestrian detection unit 130 determines that the pedestrian detected in the video frame cannot be matched with the target pedestrian, i.e., if the features associated with the half of the detected pedestrian cannot be matched with the features associated with the half of the target pedestrian, the pedestrian identification unit 130 may assign a new identity to the detected pedestrian.

[0029] Alternatively, the feature update unit may update the target appearance feature of the target pedestrian based on the identification result of the pedestrian identification unit 130. Specifically, when the feature associated with the half body of the detected pedestrian and the feature associated with the half body of the target pedestrian are matched in the video frame, the feature update unit may update the target appearance feature of the target pedestrian using the feature associated with the half body of the detected pedestrian. For example, the feature update unit may replace the feature associated with the existing half body of the target pedestrian with the feature associated with the half body of the detected pedestrian extracted from the video frame, or may integrate the extracted feature associated with the half body and the feature associated with the existing half body of the target pedestrian as the feature associated with the half body of the target pedestrian after updating. Alternatively, the feature update unit may transmit the feature associated with the half body of the detected pedestrian to a server that maintains the target appearance feature of the target pedestrian, and the server may update the target appearance feature of the target pedestrian based on the received feature, and allow the updated feature to track characteristics corresponding to different external shapes of the target pedestrian. In addition, if the pedestrian identification unit 130 determines that a detected pedestrian in a video frame cannot be matched with a target pedestrian and assigns a new identity to the detected pedestrian, the feature update unit may construct target appearance features, i.e., features associated with a half-body of the detected pedestrian in the video frame, to be provided as features associated with a half-body of the target pedestrian for subsequent feature matching.

[0030] In the above, for convenience of explanation, the first appearance feature of the target pedestrian is a feature associated with the whole front body, and the second appearance feature is a feature associated with the front half body. However, the features associated with the whole body and the half body in the present disclosure are not limited to the features extracted from the front target pedestrian. For example, the first appearance feature may be a feature associated with the whole side body of the target pedestrian, and the second appearance feature may be a feature associated with the half side body, or the first appearance feature may be a feature associated with the whole back body of the target pedestrian, and the second appearance feature may be a feature associated with the half back body. The pedestrian identification unit 130 may identify the pedestrian from the video frame by matching the features associated with the whole body / half body of the side / back of the detected pedestrian with the corresponding features of the target pedestrian, and the present disclosure is not limited thereto.

[0031] According to another embodiment of the present disclosure, the first and second appearance features of the target pedestrian may be any two of a feature associated with the front face, a feature associated with the side face, and a feature associated with the back face. As shown in FIG. 3(b), the first appearance feature of the target pedestrian may be a feature associated with its front face, and the second appearance feature may be a feature associated with its side face. It will be understood that the first appearance feature and the second appearance feature may be a feature associated with the front face and a feature associated with the back face, or a feature associated with the side face and a feature associated with the back face, respectively. Hereinafter, the pedestrian identification operation by the pedestrian re-identification device 100 in this case will be described in detail.

[0032] As described above, the pedestrian detection unit 110 may detect pedestrians in each video frame of the video sequence and identify each sub-image region corresponding to each pedestrian appearing in the video frame. Correspondingly, the feature extraction unit 120 may extract appearance features, such as color features, texture features, and shape features, associated with the front face of the sub-image region in which the pedestrian is located, such as color features, texture features, and shape features, appearance features, such as color features, texture features, and shape features, associated with the side face of the sub-image region, or appearance features, such as color features, texture features, and shape features, associated with the back face of the sub-image region, using feature extraction methods such as color features, texture features, and shape features. Next, the pedestrian identification unit 130 may match the features associated with the front face, the features associated with the side face, or the features associated with the back face of the detected pedestrian with the first and second appearance features (e.g., the features associated with the front face and the features associated with the side face) of a target pedestrian, and may associate the pedestrian detected in the video frame with the identity of the target pedestrian when the features associated with the front face, the features associated with the side face, or the features associated with the back face of the pedestrian can be matched to the target pedestrian. That is, since a pedestrian has a strong mobility and its posture changes from moment to moment, the target pedestrian has a plurality of appearance features associated with its plurality of external shapes. Therefore, if the feature actually extracted for the detected pedestrian and associated with a certain external shape can match either the first appearance feature or the second appearance feature of the target pedestrian, the pedestrian identification unit 130 can accurately identify the pedestrian without identifying it as another pedestrian. Alternatively, the feature extraction unit 120 may further determine whether the pedestrian appears in the video frame from the front, side, or back. For example, the feature extraction unit 120 may use a trained neural network as a classifier, or may identify whether the pedestrian appears in the video frame from the front, side, or back by analyzing the aspect ratio of the sub-image region in which the pedestrian is located, the proportion of the pedestrian occluded by hair, and the like.The pedestrian identification unit 130 may then determine whether the extracted features of the pedestrian are associated with the front, side, or back of the pedestrian, match the extracted features with the corresponding features of the target pedestrian, and omit the matching operation with the remaining features of the target pedestrian, thereby reducing the amount of computation in the matching process. Also, when the pedestrian identification unit 130 determines that the detected pedestrian in the video frame cannot be matched with the target pedestrian, i.e., when the pedestrian identification unit 130 determines that the features associated with the front, side, or back of the detected pedestrian cannot be matched with any of the features associated with the front and side of the target pedestrian, the pedestrian may be assigned a new identity.

[0033] Alternatively, the pedestrian re-identification device 100 may further include a feature update unit (not shown). The feature update unit may update the target appearance feature of the target pedestrian based on the identification result of the pedestrian identification unit 130. Specifically, when the pedestrian detected in the video frame and the target pedestrian can be matched, the feature update unit may update the target appearance feature of the target pedestrian using a feature associated with the front face, a feature associated with the side face, or a feature associated with the back face of the detected pedestrian. For example, when the pedestrian identification unit 130 identifies a pedestrian detected in a video frame as corresponding to a target pedestrian and associates the detected pedestrian with the identity of the target pedestrian, the feature update unit may replace the existing feature associated with the front of the target pedestrian with the feature associated with the front of the detected pedestrian in the video frame, or may integrate the feature associated with the front of the detected pedestrian in the video frame with the existing feature associated with the front of the target pedestrian as an updated feature associated with the front of the target pedestrian. Alternatively, the feature update unit may transmit the feature associated with the front of the pedestrian to a server that maintains target appearance features of the target pedestrian, and the server may update the target appearance features of the target pedestrian based on the received features. In addition, when the pedestrian identification unit 130 determines that a pedestrian detected in a video frame cannot be matched to a target pedestrian and assigns a new identity to the detected pedestrian, the feature update unit may construct target appearance features for subsequent feature matching.

[0034] In addition, in the present disclosure, the appearance features of the target pedestrian are not limited to two types, and may be more diverse. For example, as shown in FIG. 4, a plurality of appearance features associated with each target pedestrian may be set as the target appearance features of the target pedestrian. Here, the first appearance feature may be a feature associated with the entire front of the target pedestrian, the second appearance feature may be a feature associated with the entire side of the target pedestrian, the third appearance feature may be a feature associated with the entire back of the target pedestrian, the fourth appearance feature may be a feature associated with the front half of the target pedestrian, the fifth appearance feature may be a feature associated with the side half of the target pedestrian, the sixth appearance feature may be a feature associated with the back half of the target pedestrian, and so on. It will be understood that any of the above various appearance features may be set as the appearance features of the target pedestrian to perform subsequent feature matching. In this case, the feature extraction unit 120 may extract appearance features, such as color features, texture features, shape features, etc., associated with the detected pedestrian's front full body, side full body, back full body, front half body, side half body, back half body, or other external shapes. Thus, the pedestrian identification unit 130 may match the extracted features with a plurality of external features of a target pedestrian, and if the external features of the detected pedestrian can be matched with any of the external features of the target pedestrian, the pedestrian detected in the video frame is associated with the identity of the target pedestrian, thereby accurately identifying the pedestrian without identifying the pedestrian as another pedestrian. Alternatively, the feature extraction unit 120 may further determine whether the pedestrian appears in the video frame as the front full body, side full body, back full body, front half body, side half body, back half body, or other external shapes. The pedestrian identification unit 130 then determines what kind of external shape the extracted features of the pedestrian are associated with, matches the extracted features with the corresponding features of the target pedestrian, and omits the matching operation with the remaining features of the target pedestrian, thereby reducing the amount of calculation in the matching process.Additionally, if the pedestrian identification unit 130 determines that a pedestrian detected in a video frame cannot be matched with various appearance characteristics of a target pedestrian, the pedestrian identification unit 130 may assign a new identity to the detected pedestrian.

[0035] Alternatively, the feature update unit may update the target appearance feature of the target pedestrian based on the identification result of the pedestrian identification unit 130. Specifically, when the pedestrian detected in the video frame can be matched with the target pedestrian, the feature update unit may update the target appearance feature of the target pedestrian using various appearance features associated with the detected pedestrian (for example, features associated with the entire frontal part of the detected pedestrian, features associated with the entire side part of the detected pedestrian, features associated with the entire back part of the detected pedestrian, features associated with the front half of the body, features associated with the side half of the body, features associated with the back half of the body, etc.). Details of the operation of the feature update unit for the target appearance feature of the target pedestrian are the same as those described above, so the description here is omitted. Also, when the pedestrian identification unit 130 specifies that the pedestrian detected in the video frame cannot be matched with each feature of the target pedestrian, the feature update unit may construct a target appearance feature and provide it for subsequent feature matching.

[0036] As described above, the pedestrian re-identification device 100 according to an embodiment of the present disclosure identifies a target pedestrian using at least a first appearance feature and a second appearance feature associated with the target pedestrian's external shape, and can effectively track different characteristics associated with different external shapes of the target pedestrian, thereby improving the accuracy of pedestrian re-identification in situations where the pedestrian's posture changes significantly or the pedestrian is identified across cameras.

[0037] The block diagrams used in the description of the above embodiments show functional blocks. These functional blocks (components) are realized by any combination of hardware and / or software. The means for realizing each functional block is not particularly limited. That is, each functional block may be realized by one device that is physically and / or logically coupled, or may be realized by the above-mentioned multiple devices that are two or more physically and / or logically separated devices that are directly and / or indirectly (e.g., wired and / or wirelessly) connected to each other.

[0038] 5 is a hardware block diagram showing an exemplary computing device for implementing the pedestrian re-identification device according to an embodiment of the present disclosure. The pedestrian re-identification device 100 described above may be physically configured as a computer device including a processor 501, a memory 502, a storage 503, a communication device 504, an input device 505, an output device 506, a bus 507, etc.

[0039] In the following description, the term "apparatus" can be read as a circuit, a device, a unit, etc. The hardware configuration of the pedestrian re-identification device 100 may include one or more of the devices shown in the figures, or may not include some of the devices.

[0040] For example, although only one processor 501 is shown, there may be multiple processors. Furthermore, the processes may be performed by one processor, or by one or more processors simultaneously, sequentially, or in other manners. Additionally, the processor 501 may be implemented on one or more chips.

[0041] Each function of the pedestrian re-identification device 100 is realized, for example, by loading a specific software (program) onto hardware such as a processor 501 and a memory 502, so that the processor 501 performs calculations and controls communication via a communication device 504 and the reading and / or writing of data in the memory 502 and the storage 503.

[0042] The processor 501 controls the entire computer by running, for example, an operating system. The processor 501 may be configured as a central processing unit (CPU) including an interface with peripheral devices, a control device, an arithmetic unit, a register, etc. For example, the above-mentioned pedestrian detection unit 110, feature extraction unit 120, and pedestrian identification unit 130, etc. may be realized by the processor 501.

[0043] Furthermore, the processor 501 reads out programs (program codes), software modules, data, etc. from the storage 503 and / or the communication device 504 into the memory 502, and executes various processes according to these. As the programs, programs that cause a computer to execute at least a part of the operations described in the above-mentioned embodiments are used.

[0044] The memory 502 is a computer-readable storage medium and may be configured with at least one of, for example, a Read Only Memory (ROM), an Erasable Programmable ROM (EPROM), an Electrically Erasable Programmable ROM (EEPROM), a Random Access Memory (RAM), and other suitable storage media. The memory 502 may also be called a register, a cache, a main memory (primary storage device), and the like. The memory 502 can store an executable program (program code), a software module, and the like for implementing a pedestrian re-identification method according to an embodiment of the present invention.

[0045] Storage 503 is a computer readable storage medium and may be comprised of at least one of, for example, a flexible disk, a floppy disk, a magneto-optical disk (e.g., a Compact Disc ROM (CD-ROM), a digital versatile disk, a Blu-ray disk), a removable magnetic disk, a hard disk drive, a smart card, a flash memory device (e.g., a card, stick, key drive), a magnetic strip, a database, a server, or other suitable storage medium. Storage 503 may also be referred to as a secondary storage device.

[0046] The communication device 504 is hardware (transmitting / receiving device) for performing communication between computers via a wired and / or wireless network, and is also referred to as, for example, a network device, a network controller, a network card, a communication module, etc. The communication device 504 may include, for example, a high-frequency switch, a duplexer, a filter, a frequency synthesizer, etc., to realize Frequency Division Duplex (FDD) and / or Time Division Duplex (TDD). For example, a video sequence to be analyzed may be received via the communication device 504.

[0047] The input device 505 is an input device (e.g., a keyboard, a mouse, a microphone, a switch, a button, a sensor, etc.) that receives input from the outside. The output device 506 is an output device (e.g., a display, a speaker, an LED lamp, etc.) that performs output to the outside. The input device 505 and the output device 506 may be integrated into one device (e.g., a touch panel).

[0048] Furthermore, each device such as the processor 501 and the memory 502 is connected by a bus 507 for communicating information. The bus 507 may be configured as a single bus, or may be configured as different buses between the devices.

[0049] Furthermore, the pedestrian re-identification device 100 may be configured to include hardware such as a microprocessor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a programmable logic device (PLD), or a field programmable gate array (FPGA), and some or all of the functional blocks may be realized by the hardware. For example, the processor 501 may be implemented by at least one of these pieces of hardware.

[0050] Next, a flowchart of a pedestrian re-identification method according to an embodiment of the present disclosure will be described with reference to Fig. 6. Each step of this pedestrian re-identification method substantially corresponds to the operation of each component of the pedestrian re-identification device 100 described above with reference to Figs. 2 to 5. Therefore, to avoid duplication, the method will be briefly described below, and detailed descriptions of similar contents will be omitted.

[0051] As shown in FIG. 6, in step S601, pedestrian detection is performed in each video frame of a video sequence. As described above, unlike general pedestrian tracking, the pedestrian re-identification technique according to the present disclosure can identify a specific target pedestrian in images or video sequences captured across different cameras, and realize long-term tracking and monitoring of the specific target pedestrian. This video sequence is an analysis target video sequence from which it is necessary to identify whether a target pedestrian exists or not, and this video sequence may be captured by a camera different from the video frame in which the target pedestrian was captured and its appearance features were extracted, or may be captured by the same camera at a different time. The video sequence may include one or more video frames, and each video frame may be a continuous or non-continuous video frame captured by a single camera or by multiple cameras with overlapping or non-overlapping fields of view.

[0052] In this step, pedestrians may be detected from each video frame of the video sequence by any suitable image detection technique in the art, and the present disclosure is not limited thereto. For each video frame, processing such as foreground segmentation, edge extraction, motion detection, etc. may be performed to identify each sub-image region corresponding to each pedestrian appearing in the video frame. For example, the sub-image region of a video frame may be represented by a rectangular frame circumscribing the body outline of the detected pedestrian. Also, for example, machine learning methods such as neural networks, support vector machines, etc. may be used to perform pedestrian detection for each video frame using a pre-trained pedestrian detection classifier to determine the location of each pedestrian appearing in the video frame.

[0053] In step S602, for each pedestrian detected from the video frame, its appearance features are extracted. The appearance features may include features reflecting the appearance morphology of the pedestrian, such as color features, texture features, shape features, and face features, but the present disclosure is not limited thereto. This step may use any feature extraction method known in the art to extract the appearance features of each detected pedestrian, and the description thereof will be omitted here.

[0054] In step S603, the appearance features of each detected pedestrian are matched with the target appearance features of the target pedestrian, and the target pedestrian is identified from the video frame based on the matching result. In this step, the appearance features of each detected pedestrian are matched with the appearance features of the target pedestrian, and based on the similarity between the two, the pedestrian re-identification may be performed by identifying whether the pedestrian detected in the video frame is the target pedestrian. Alternatively, the number of target pedestrians may be multiple, and accordingly, in this step, the appearance features of each detected pedestrian are matched with the appearance features of each constructed target pedestrian, and based on the similarity between the two, the pedestrian re-identification may be identified whether the pedestrian detected in the video frame corresponds to any of the target pedestrians, thereby performing pedestrian re-identification. It will be understood that the target appearance features of the target pedestrian may be included in a pedestrian appearance feature library constructed in advance, and the pedestrian appearance feature library may be stored locally in the pedestrian re-identification device or in a server accessible by the pedestrian re-identification device. Alternatively, the target appearance features of the target pedestrian may not be stored in advance in the pedestrian appearance feature library, but may be generated from an image when necessary and input from outside.

[0055] In this embodiment, it is possible to identify the target pedestrian from the video frame using various feature matching methods. For example, in this step S603, the feature distance between the appearance feature of each detected pedestrian and the target appearance feature of the target pedestrian, for example, Manhattan distance, Euclidean distance, Pearl distance, etc., may be calculated and compared with a preset threshold to identify whether the detected pedestrian corresponds to the target pedestrian. Alternatively, the number of target pedestrians may be multiple, and in this case, for each detected pedestrian, each feature distance from each target pedestrian may be calculated, and the closest feature distance among them may be identified, and if the closest feature distance is smaller than a preset threshold, the detected pedestrian may be identified as the target pedestrian corresponding to the closest feature distance. Then, based on the matching result, a pedestrian re-identification result may be output. For example, if the detected pedestrian corresponds to a certain target pedestrian, the detected pedestrian is associated with the identity of the target pedestrian, and if the detected pedestrian cannot be matched with the target pedestrian, a new identity may be assigned to the detected pedestrian.

[0056] As described above, in the present disclosure, the target appearance feature of the target pedestrian includes at least a first appearance feature and a second appearance feature associated with the external shape of the target pedestrian, thereby enabling a plurality of pre-constructed appearance features to fully reflect different characteristics associated with different external shapes of the target pedestrian, thereby improving the accuracy of pedestrian re-identification. Note that the external shape in the present disclosure may refer to the form in which the target pedestrian appears on the screen, for example, the form in which the whole body appears, the half body appears, the front side appears, the side appears, the back side appears, etc., but it will be understood that the present disclosure is not limited thereto. Therefore, the first appearance feature and the second appearance feature are two appearance features associated with different external shapes of the target pedestrian. Hereinafter, the pedestrian re-identification method will be specifically described for different examples of the first appearance feature and the second appearance feature.

[0057] According to an embodiment of the present disclosure, the first appearance feature of the target pedestrian may be a feature associated with the whole body, and the second appearance feature of the target pedestrian may be a feature associated with the half body. It should be understood that the half body in the present disclosure is not strictly limited to half the size of the whole body, and may be any appropriate ratio of the whole body, such as 1 / 4, 1 / 3, etc., as necessary. The specific operation of the pedestrian re-identification method in this case will be described in detail below.

[0058] As described above, step S601 may include detecting pedestrians in each video frame of the video sequence and identifying sub-image regions corresponding to each pedestrian appearing in the video frame. Step S601 may include identifying whether the pedestrian appears in the video frame as a full body or a half body based on an aspect ratio of the detected pedestrian in the video frame (e.g., the aspect ratio of a circumscribing rectangle of the pedestrian), a full-body / half-body image classifier trained based on a neural network, or other methods.

[0059] Therefore, if it is determined in step S601 that the detected pedestrian appears in the video frame with its entire body, step S602 may include extracting features associated with the entire body of the detected pedestrian, for example, extracting appearance features such as color features, texture features, shape features, facial features, etc., from a sub-image region in which the pedestrian is located. Step S602 may further include cropping a half-body image of the detected pedestrian from the video frame and extracting features associated with the half-body, for example, cropping an upper half of the sub-image region in which the pedestrian is located, and extracting appearance features such as color features, texture features, shape features, facial features, etc., associated with the half-body, from the cropped half of the sub-image region.

[0060] Next, step S603 may include matching features associated with the entire body of the detected pedestrian with features associated with the entire body of a target pedestrian, matching features associated with the half body with features associated with the half body of the target pedestrian, and associating the detected pedestrian in the video frame with an identity of the target pedestrian if at least one of the features associated with the entire body of the detected pedestrian and the features associated with the half body of the target pedestrian can be matched with the target pedestrian. Step S603 may also include assigning a new identity to the detected pedestrian when it is determined that the detected pedestrian in the video frame cannot be matched with the target pedestrian, i.e., when it is determined that the features associated with the entire body of the detected pedestrian cannot be matched with the features associated with the entire body of the target pedestrian and the features associated with the half body of the detected pedestrian cannot be matched with the features associated with the half body of the target pedestrian.

[0061] Alternatively, the pedestrian re-identification method may further include a step of updating the target appearance features of the target pedestrian based on the pedestrian identification result. Specifically, when it is determined in step S603 that the pedestrian detected in the video frame and the target pedestrian can be matched, the target appearance features of the target pedestrian may be updated using features associated with the whole body of the detected pedestrian and features associated with the half body. For example, when it is determined in step S603 that the pedestrian detected in the video frame corresponds to a certain target pedestrian and the detected pedestrian is associated with the identity of the target pedestrian, if the target pedestrian lacks features associated with its half body or features associated with its whole body, the features associated with the whole body or its half body of the detected pedestrian in the video frame are directly added to the missing part of the target pedestrian, and for the features associated with the existing whole body of the pedestrian, the features associated with the existing whole body of the target pedestrian are replaced with the features associated with the whole body of the detected pedestrian in the video frame, or the features associated with the detected pedestrian in the video frame are updated. The features associated with the entire body of the target pedestrian and the existing features associated with the entire body of the target pedestrian may be integrated into an updated feature associated with the entire body of the target pedestrian, and for the features associated with the existing half of the target pedestrian, the features associated with the existing half of the target pedestrian may be replaced by the features associated with the half of the target pedestrian extracted after cropping the video frame, or the extracted features associated with the half of the target pedestrian and the features associated with the existing half of the target pedestrian may be integrated into an updated feature associated with the half of the target pedestrian, so that the updated features can be tracked over time to characteristics corresponding to the different external shapes of the target pedestrian.Also, when the detected pedestrian in a video frame is identified as unable to be matched with a target pedestrian and a new identity is assigned to the detected pedestrian, the feature update unit may construct target appearance features, i.e., the features associated with the whole body of the detected pedestrian in the video frame are the features associated with the whole body of the target pedestrian, and the features associated with the half body extracted after cropping the video frame are the features associated with the half body of the target pedestrian, for subsequent feature matching.

[0062] The above describes the pedestrian identification operation corresponding to the case where the detected pedestrian is determined to appear in the video frame with its entire body. Next, the following describes the pedestrian identification operation corresponding to the case where the detected pedestrian is determined to appear in the video frame with only half of its body.

[0063] Specifically, if it is determined in step S601 that the detected pedestrian appears in the video frame with only half of its body, step S602 may include extracting features associated with the half of the pedestrian, e.g., extracting appearance features, such as color features, texture features, shape features, and facial features, associated with the half of the body from the sub-image region in which the pedestrian is present. Thus, step S603 may include matching features associated with the half of the detected pedestrian with features associated with the half of a target pedestrian, and associating the detected pedestrian in the video frame with an identity of the target pedestrian if the features associated with the half of the pedestrian match with the target pedestrian. Also, if it is determined in step S603 that the detected pedestrian in the video frame cannot be matched with the target pedestrian, a new identity may be assigned to the detected pedestrian.

[0064] Alternatively, the pedestrian re-identification method may further include updating target appearance features of the target pedestrian using features associated with the detected pedestrian's body when it is determined in step S603 that the features associated with the detected pedestrian's body in the video frame match with the features associated with the target pedestrian's body. For example, the features associated with the detected pedestrian's body extracted from the video frame may replace the features associated with the existing body of the target pedestrian, or the features associated with the extracted body and the features associated with the existing body of the target pedestrian may be integrated into an updated feature of the target pedestrian's body, so that the updated feature can track characteristics corresponding to a different external shape of the target pedestrian. Also, when it is determined in step S603 that the detected pedestrian in the video frame cannot be matched with the target pedestrian and a new identity is assigned to the detected pedestrian, a target appearance feature may be constructed, i.e., the features associated with the detected pedestrian's body in the video frame may be used as the features associated with the target pedestrian's body for subsequent feature matching.

[0065] In the above, for convenience of explanation, the first appearance feature of the target pedestrian is a feature associated with the whole front body, and the second appearance feature is a feature associated with the front half body. However, the features associated with the whole body and the half body in the present disclosure are not limited to features extracted from the front target pedestrian. For example, the first appearance feature may be a feature associated with the whole side body of the target pedestrian, and the second appearance feature may be a feature associated with the half side body of the target pedestrian, or the first appearance feature may be a feature associated with the whole back body of the target pedestrian, and the second appearance feature may be a feature associated with the half back body of the target pedestrian. The pedestrian re-identification method may identify a pedestrian from a video frame by matching the features associated with the whole body / half body of the side / back of the detected pedestrian with the corresponding features of the target pedestrian, and the present disclosure is not limited thereto.

[0066] According to another embodiment of the present disclosure, the first and second appearance features of the target pedestrian may be any two of a feature associated with the front face, a feature associated with the side face, and a feature associated with the back face. For example, the first appearance feature of the target pedestrian may be a feature associated with the front face, and the second appearance feature may be a feature associated with the side face. Specific operations of the pedestrian re-identification method in this case will be described in detail below.

[0067] As described above, in step S601, pedestrians may be detected in each video frame of the video sequence, and each sub-image region corresponding to each pedestrian appearing in the video frame may be identified. Correspondingly, in step S602, appearance features such as color features, texture features, and shape features associated with the front side, color features, texture features, and shape features associated with the side side, or color features, texture features, and shape features associated with the rear side may be extracted from the sub-image region in which the pedestrian is located, using feature extraction methods such as color features, texture features, and shape features. Next, in step S603, the features associated with the front side, the features associated with the side side, or the rear side of the detected pedestrian may be matched with the first and second appearance features (e.g., the features associated with the front side and the features associated with the side side) of a target pedestrian, and when the features associated with the front side, the features associated with the side side, or the rear side of the pedestrian can be matched with the target pedestrian, the pedestrian detected in the video frame may be matched with the identity of the target pedestrian. That is, since a pedestrian has a strong mobility and its posture changes from moment to moment, a target pedestrian has a plurality of appearance features associated with its plurality of external shapes, if a feature actually extracted from the detected pedestrian and associated with a certain external shape can be matched with either the first appearance feature or the second appearance feature of the target pedestrian, the pedestrian can be accurately identified without being identified as another pedestrian. Also, if it is determined in step S603 that the pedestrian detected in the video frame cannot be matched with the target pedestrian, that is, if it is determined that the feature associated with the front face, the feature associated with the side face, or the feature associated with the back face of the detected pedestrian cannot be matched with any of the features associated with the front face of the target pedestrian and the features associated with the side face of the target pedestrian, a new identity may be assigned to the detected pedestrian.

[0068] Alternatively, the pedestrian re-identification method may further include a step of updating the target appearance feature of the target pedestrian based on the pedestrian identification result. Specifically, when it is determined in step S603 that the pedestrian detected in the video frame and the target pedestrian are matched, the target appearance feature of the target pedestrian may be updated using a feature associated with the front face, a feature associated with the side face, or a feature associated with the back face of the detected pedestrian. For example, when it is determined in step S603 that the pedestrian detected in the video frame corresponds to a certain target pedestrian and the detected pedestrian is associated with the identity of the target pedestrian, for example, a feature associated with the front face of the detected pedestrian and a feature associated with the front face of the target pedestrian match, the feature associated with the front face of the detected pedestrian in the video frame may replace the existing feature associated with the front face of the target pedestrian, or the feature associated with the front face of the detected pedestrian in the video frame and the feature associated with the existing front face of the target pedestrian may be integrated as an updated feature associated with the front face of the target pedestrian. In addition, in step S603, if a pedestrian detected in a video frame is identified as unable to be matched with a target pedestrian and a new identity is assigned to the detected pedestrian, the feature update unit may construct target appearance features for subsequent feature matching.

[0069] In addition, in the present disclosure, the appearance features of the target pedestrian are not limited to two types, and may be more diverse. For example, a plurality of appearance features associated with each target pedestrian may be set as the target appearance features of the target pedestrian. Here, the first appearance feature may be a feature associated with the entire front of the target pedestrian, the second appearance feature may be a feature associated with the entire side of the target pedestrian, the third appearance feature may be a feature associated with the entire back of the target pedestrian, the fourth appearance feature may be a feature associated with the front half of the target pedestrian, the fifth appearance feature may be a feature associated with the side half of the target pedestrian, the sixth appearance feature may be a feature associated with the back half of the target pedestrian, and so on. It will be understood that any of the above various appearance features may be set as the appearance features of the target pedestrian to perform subsequent feature matching. In this case, in step S602, the detected pedestrian may have an appearance feature, such as a color feature, a texture feature, or a shape feature, associated with the detected pedestrian's full front view, full side view, full back view, half front view, half side view, half back view, or other appearance. Thus, in step S603, the extracted feature may be matched with a plurality of appearance features of a target pedestrian, and if the appearance feature of the detected pedestrian can be matched with any of the plurality of appearance features of the target pedestrian, the detected pedestrian in the video frame may be matched with the identity of the target pedestrian, thereby accurately identifying the pedestrian without identifying the pedestrian as another pedestrian. Alternatively, in step S602, it may be further determined whether the pedestrian appears in the video frame as a full front view, full side view, full back view, half front view, half side view, half back view, or other appearance. Then, in step S603, the system determines what kind of external shape the extracted features of the pedestrian are associated with, matches the extracted features with the corresponding features of a target pedestrian, and omits the matching operation with the remaining features of the target pedestrian, thereby reducing the amount of calculation in the matching process.Also, in step S603, if a pedestrian detected in a video frame is identified as unable to match various appearance characteristics of the target pedestrian, the detected pedestrian may be assigned a new identity.

[0070] Alternatively, the pedestrian re-identification method may further include a step of updating the target appearance feature of the target pedestrian based on the pedestrian identification result. Specifically, in step S603, if it is determined that the pedestrian detected in the video frame and the target pedestrian can be matched, the target appearance feature of the target pedestrian may be updated using various appearance features associated with the detected pedestrian (e.g., features associated with the front whole body, the side whole body, the back whole body, the front half body, the side half body, the back half body, etc.) associated with the detected pedestrian. In addition, in step S603, if it is determined that the pedestrian detected in the video frame cannot be matched with each feature of the target pedestrian and a new identity is assigned to the detected pedestrian, a target appearance feature may be constructed and used for subsequent feature matching.

[0071] As described above, the pedestrian re-identification method according to the embodiments of the present disclosure identifies a target pedestrian using at least a first appearance feature and a second appearance feature associated with the external shape of the target pedestrian, thereby enabling effective tracking of different characteristics associated with different external shapes of the target pedestrian, and improving the accuracy of pedestrian re-identification in situations where the pedestrian's posture changes significantly or the pedestrian is identified across cameras.

[0072] In addition to the methods and devices described above, embodiments of the present disclosure may include a computer-readable storage medium storing computer program instructions that may be executed by a processor to cause the processor to perform the pedestrian re-identification method described above.

[0073] Result verification The following describes the results of verifying two pedestrian re-identification methods: (1) pedestrian re-identification based on a single appearance feature of a target pedestrian without considering the appearance factor of the target pedestrian, and (2) pedestrian re-identification based on a first appearance feature and a second appearance feature associated with the appearance of the target pedestrian. Specifically, in the exemplary verification example, a target pedestrian library including appearance features of multiple target pedestrians is constructed in advance, pedestrians are detected for one or more video frames in an analysis target video sequence, and the target pedestrian is identified from the video frame by calculating the feature distance between the appearance feature of each pedestrian detected in the video frame and the appearance feature of the constructed target pedestrian. In this verification, the accuracy of pedestrian re-identification by the above two types of pedestrian re-identification methods is evaluated using the following two indicators.

[0074] Rank1: This index indicates the probability that the target pedestrian with the highest classification result (i.e., the target pedestrian with the closest feature distance) among the target pedestrians is the correct result after matching the appearance features of the detected pedestrian with the appearance features of each target pedestrian. The formula is as follows:

[0075] (Formula 1) TIFF0007612348000001.tif19141

[0076] Accuracy: This metric indicates the average probability that the same pedestrian has the same ID in an image sequence, and its formula is:

[0077] (Formula 2) TIFF0007612348000002.tif15134

[0078] Based on the above two indicators, the testing results for the two pedestrian re-identification methods are shown in Table 1.

[0079] [Table 1]

[0080] As shown in Table 1, the pedestrian re-identification method based on the first appearance feature and the second appearance feature associated with the outer shape of the target pedestrian according to the present disclosure achieved better results in both indices.

[0081] 7(a) and 7(b) are schematic diagrams showing the results of pedestrian re-identification using the above-mentioned two types of pedestrian re-identification methods. Specifically, FIG. 7(a) shows the results of pedestrian re-identification based on a single appearance feature of a target pedestrian, and for example, the same pedestrian circled is identified as P7 and P3 in different video frames due to changes in shooting angle and posture. FIG. 7(b) shows the results of pedestrian re-identification based on a first appearance feature and a second appearance feature associated with the target pedestrian's external shape, and since the target pedestrian has multiple appearance features associated with its external shape, accurate identification results were obtained under different shooting angles and pedestrian postures.

[0082] The basic principles of the present disclosure have been described above based on specific examples, but the advantages, advantages, effects, etc. mentioned in the present disclosure are merely illustrative and not limiting, and these advantages, advantages, effects, etc. should not be considered essential to each embodiment of the present disclosure. The specific details disclosed above are merely illustrative and for the convenience of understanding, and are not intended to be limiting. The above details do not limit the present disclosure to the extent that it must be realized by the above specific details.

[0083] As used herein, the phrase "based on" does not mean "based only on," unless expressly stated otherwise. In other words, the phrase "based on" means both "based only on" and "based at least on."

[0084] As used herein, any reference to an element using a designation such as "first," "second," etc., does not generally limit the quantity or order of those elements. These designations may be used herein as a convenient method of distinguishing between two or more elements. Thus, a reference to a first and a second element does not imply that only two elements may be employed therein or that the first element must precede the second element in some way.

[0085] As used herein, the term "determining" may encompass a wide variety of actions. For example, "determining" may be considered to be "determining" calculating, computing, processing, deriving, investigating, looking up (e.g., looking up in a table, database, or another data structure), ascertaining, etc. Also, "determining" may be considered to be "determining" receiving (e.g., receiving information), transmitting (e.g., sending information), input, output, accessing (e.g., accessing data in a memory), etc. Also, "determining" may be considered to be "determining" resolving, selecting, choosing, establishing, comparing, etc. In other words, "determining (identifying)" may be considered as "determining (identifying)" some action.

[0086] As used herein, the terms "connected," "coupled," or any variation thereof, refer to any direct or indirect connection or coupling between two or more elements, and may include the presence of one or more intermediate elements between two elements that are "connected" or "coupled" to one another. The coupling or connection between elements may be physical, logical, or a combination thereof. For example, "connected" may be read as "access." As used herein, two elements may be considered to be "connected" or "coupled" to one another by using one or more wires, cables, and / or printed electrical connections, as well as by using electromagnetic energy, such as electromagnetic energy having wavelengths in the radio frequency range, microwave range, and optical (both visible and invisible) range, as some non-limiting and non-exhaustive examples.

[0087] When the terms "including," "comprising," and variations thereof are used in this specification or the claims, these terms are intended to be inclusive, as is the term "comprising." Further, when used in the specification or the claims, the term "or" is not intended to be an exclusive or.

[0088] Each aspect / embodiment described herein may be used alone, in combination, or switched depending on the implementation. In addition, the processing procedures, sequences, flow charts, etc. of each aspect / embodiment described herein may be rearranged unless inconsistent. For example, the methods described herein present elements of various steps in an example order, and are not limited to the particular order presented.

[0089] Software shall be construed broadly to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executable files, threads of execution, procedures, functions, etc., whether referred to as software, firmware, middleware, microcode, hardware description language, or otherwise.

[0090] Additionally, software, instructions, information, etc. may be transmitted or received over a transmission medium. For example, if the software is transmitted from a website, server, or other remote source using wired technologies (such as coaxial cable, fiber optic cable, twisted pair, Digital Subscriber Line (DSL)), and / or wireless technologies (such as infrared, microwave, etc.), these wired and / or wireless technologies are included within the definition of transmission media.

[0091] Although the present invention has been described in detail above, it is clear to those skilled in the art that the present invention is not limited to the embodiments described in this specification. The present invention can be implemented in modified and altered forms without departing from the spirit and scope of the present invention as defined by the claims. Therefore, the description in this specification is intended to be illustrative and does not have any limiting meaning on the present invention.

Claims

1. a pedestrian detector configured to detect a pedestrian in each video frame of a video sequence and to identify whether the pedestrian appears in the video frame with its entire body or with a half-body; a feature extraction unit configured to extract appearance features of each pedestrian detected in a video frame, the feature extraction unit extracting features associated with the entire body of the detected pedestrian when the detected pedestrian appears in the video frame, and cutting out a half-body image of the detected pedestrian from the video frame to extract features associated with the half-body of the detected pedestrian; a pedestrian identification unit configured to match appearance features of each detected pedestrian with target appearance features of a target pedestrian in order to identify a target pedestrian from the video frames, the pedestrian identification unit matching features associated with the whole body of the detected pedestrian with features associated with the whole body of a target pedestrian, matching features associated with a half body of the detected pedestrian with features associated with a half body of a target pedestrian, and associating the pedestrian detected in the video frames with an identity of the target pedestrian when at least one of the features associated with the whole body of the detected pedestrian and the features associated with the half body of the detected pedestrian can be matched with a target pedestrian; Including, The target appearance feature of the target pedestrian includes at least a first appearance feature and a second appearance feature associated with an external shape of the target pedestrian, The first appearance feature is a feature associated with a whole body, and the second appearance feature is a feature associated with a half body.

2. The pedestrian re-identification device of claim 1, further comprising a feature updating unit that, when a match is made between the pedestrian detected in the video frame and a target pedestrian, updates target appearance features of the target pedestrian using features associated with the entire body of the detected pedestrian and features associated with the half body of the detected pedestrian.

3. the feature extraction unit is further configured to extract features associated with a half-body of the detected pedestrian if the detected pedestrian appears in the video frames with a half-body; the pedestrian identification unit is further arranged to match features associated with the half body of the detected pedestrian with features associated with the half body of a target pedestrian, and, if a match is found between the features associated with the half body of the pedestrian and the target pedestrian, to associate the pedestrian detected in the video frame with an identity of the target pedestrian; The pedestrian re-identification device of claim 1 , further comprising a feature updating unit configured to, when a match is made between the pedestrian detected in the video frames and a target pedestrian, update target appearance features of the target pedestrian using features associated with a half-body of the detected pedestrian.

4. A processor; a memory in which computer program instructions are stored; The computer program instructions, when executed by the processor, cause the processor to: performing pedestrian detection in each video frame of a video sequence and determining whether the pedestrian appears in the video frame as a full body or a half body; extracting appearance features of each pedestrian detected in a video frame, where if the detected pedestrian appears in the video frame with its entire body, extracting features associated with the entire body of the detected pedestrian, and cutting out a half-body image of the detected pedestrian from the video frame to extract features associated with the half-body of the detected pedestrian; and matching appearance features of each detected pedestrian with target appearance features of a target pedestrian to identify a target pedestrian from the video frames, by matching features associated with the entire body of the detected pedestrian with features associated with the entire body of the target pedestrian, matching features associated with a half-body of the detected pedestrian with features associated with a half-body of the target pedestrian, and associating the detected pedestrian in the video frames with an identity of the target pedestrian if at least one of the features associated with the entire body of the detected pedestrian and the features associated with the half-body of the detected pedestrian can be matched with a target pedestrian; The target appearance feature of the target pedestrian includes at least a first appearance feature and a second appearance feature associated with an external shape of the target pedestrian, The first appearance feature is a feature associated with a whole body, and the second appearance feature is a feature associated with a half body.

Images