A Pedestrian Recognition Method Based on Video Data

Abstract

The invention discloses a pedestrian recognition method based on video data, which belongs to the field of data processing. The method comprises the following steps: setting a plurality of pedestrian IDs and data representations of the corresponding pedestrian IDs in a cloud database; obtaining video data, selecting key frames, and identifying whether contour features of the key frames are missing; obtaining a first representation and directly matching the data representation if the key frames are not missing; if the key frames are missing, marking the key frames as sensitive images, taking N consecutive frames associated with sequence information of the sensitive images to obtain N extended images, dividing the extended image grid into a plurality of local image blocks, obtaining a first image set and a second image set, and then integrating feature vectors of the sensitive images into a plurality of vector sets, fusing the first image set and the second image set with the plurality of vector sets according to fusion attributes to obtain a second representation, and matching through the second representation; it is expected that the processing method of video data is optimized on the basis of existing person re-identification, and the problem of feature fusion deviation of video data in pedestrian re-identification is improved.

Description

Technical Field

[0001] The invention relates to video data processing, and in particular to a pedestrian recognition method based on video data. Background Art

[0002] Existing pedestrian re-identification technology plays an increasingly important role in public security and surveillance systems. With the advancement of technology and the deepening of application, modern pedestrian re-identification systems rely on advanced data collection and big data analysis technologies to extract pedestrian data from the previously collected database and associate each data item with a unique ID to identify and record specific people. By capturing image data of pedestrians in various environments, extracting features from these data, and then matching these features with the information stored in the database, specific people can be identified and tracked.

[0003] There are many ways to obtain pedestrian features, including but not limited to facial recognition, gait analysis, clothing color, body shape and other biological and behavioral features. Ideally, these representative features are clearly identifiable, making the recognition and matching process relatively smooth. However, in practice, the collection of pedestrian features faces many challenges, especially due to the diverse environments in which pedestrians are located during the sampling process and the unpredictable sampling conditions, which makes pedestrian features easily affected by external factors, resulting in incomplete or inconsistent feature data quality.

[0004] On the one hand, due to the difference between the camera installation angle and the pedestrian sampling angle, some features of the pedestrian, such as part of the face or body, may be blocked by other pedestrians or environmental objects. On the other hand, lighting conditions also have a great impact on the quality of pedestrian images. For example, strong light or dark light may cause the image to be overexposed or too dark, making the facial features or overall outline of the pedestrian unclear. This directly affects the accuracy of pedestrian recognition. In addition, some pedestrians are in dynamic changes during the sampling process, which increases the difficulty of feature recognition and easily leads to the inability of feature extraction methods to accurately extract necessary information from pedestrian images, thereby affecting the efficiency and accuracy of the pedestrian re-identification system. Therefore, it is worth studying how to optimize the recognition method based on existing video data and reduce environmental impact. Summary of the invention

[0005] The purpose of the present invention is to provide a pedestrian recognition method based on video data, hoping to optimize the processing method of video data on the basis of existing person re-identification and improve the problem of feature fusion deviation of video data in pedestrian re-identification.

[0006] In order to solve the above-mentioned technical problems, the present invention adopts the following technical scheme, a pedestrian recognition method based on video data, comprising the following steps: S100, a cloud database sets a number of pedestrian IDs and data representations of corresponding pedestrian IDs; the server obtains video data and selects key frames, identifies whether the contour features of the key frames are missing, if the contour features are not missing, converts the feature vector of the key frames into a first representation and executes S600, if the contour features are missing, the server adds sequence information to each frame of the image, the above sequence information includes a frame sequence number and a timestamp; obtains the timestamp association between the key frames and the consecutive frames and executes S200.

[0007] S200, the server presets a processing coefficient N, marks the key frame as a sensitive image, takes N consecutive frames associated with the sequence information of the sensitive image to obtain N extended images; sorts the extended images according to timestamps, and executes S300.

[0008] S300, dividing the extended image grid into a number of local image blocks, comparing the two extended images in order and obtaining the repeated area and the deviation area; identifying the local image blocks of the repeated area and the deviation area to obtain a first image set and a second image set, respectively. The first image set includes a number of first fuzzy vectors; the second image set includes a number of second fuzzy vectors.

[0009] S400, marking the feature vectors of the sensitive images according to the attribute sensitivity and integrating them into a first vector set and a second vector set; integrating the second fuzzy vectors into a third vector set.

[0010] S500, determining a fusion attribute of the first fuzzy vector; performing feature fusion of the first fuzzy vector with the first vector set, the second vector set and the third vector set respectively according to the fusion attribute to obtain a second representation and execute S600.

[0011] S600, the server inputs the first representation or the second representation into the cloud database to match the data representation; if the first representation or the second representation matches the result in the cloud database, the server retrieves the corresponding pedestrian ID in the cloud database for time stamping. If the first representation does not match the result, the first representation is stored in the cloud database through the server and a unique pedestrian ID is established. If the second representation does not match the result, the server increases the processing coefficient N and re-executes S200, wherein the server records the number of times S200 is executed. When the number of executions is greater than the set number, the second representation is stored in the cloud database through the server and a unique pedestrian ID is established.

[0012] Preferably, when executing S500, the specific steps for determining the fusion attribute of the first fuzzy vector are: S510, the server presets a similarity threshold K, and pairs each first fuzzy vector with the first vector set and the second vector set for similarity. S520, obtain the highest similarity between the first fuzzy vector and any vector in the first vector set as P1. S530, obtain the highest similarity between the first fuzzy vector and any vector in the second vector set as P2. When P1>P2, and P1≥K, the fusion attribute of the above-mentioned first fuzzy vector is a high-sensitivity attribute, and the high-sensitivity attribute is used to be fused into the first vector set. When P1<P2, and P2≥K, the fusion attribute of the above-mentioned first fuzzy vector is a medium-sensitivity attribute, and the medium-sensitivity attribute is used to be fused into the second vector set. When P1 and P2 are both less than K, the fusion attribute of the above-mentioned first fuzzy vector is a low-sensitivity attribute, and the low-sensitivity attribute is used to be fused into the third vector set.

[0013] A further technical solution is to calculate the similarity between the first fuzzy vector and each vector feature in the first vector set and the second vector set. The calculation formula is:

[0014] ;

[0015] In the formula, is any i vector in the first vector set or the second vector set, is the Euclidean norm of any i vector in the first vector set or the second vector set; is the first fuzzy vector; is the Euclidean norm of the first fuzzy vector.

[0016] A further technical solution is to perform feature fusion on all first fuzzy vectors according to high-sensitivity attributes, medium-sensitivity attributes and low-sensitivity attributes with the corresponding first vector set, second vector set and third vector set respectively; and obtain first fusion features, second fusion features and third fusion features.

[0017] Among them, the first fusion feature, the second fusion feature and the third fusion feature are respectively set with fusion weights, and the vectors of the first fusion feature, the second fusion feature and the third fusion feature are weighted fused to obtain the second data representation.

[0018] Preferably, the first fuzzy vector with high sensitivity is fused with the first vector set feature to obtain a first fused feature; the calculation formula of the first fused feature F1 is:

[0019] ;

[0020] In the formula, is the first fuzzy vector with high sensitivity attribute, is the i-th vector in the first vector set, is the weight coefficient of the first fuzzy vector with high sensitivity attribute, is the weight coefficient of the first vector set, and n1 is the number of vectors in the first vector set.

[0021] The first fuzzy vector with medium sensitivity attribute is fused with the second vector set feature to obtain the second fused feature; the calculation formula of the second fused feature F2 is:

[0022] ;

[0023] In the formula, is the first fuzzy vector with medium sensitivity attribute, is the i-th vector in the second vector set, is the weight coefficient of the first fuzzy vector with medium-sensitive attribute, is the weight coefficient of the second vector set, and n2 is the number of vectors in the second vector set.

[0024] The first fuzzy vector with low sensitivity attribute is fused with the third vector set feature to obtain the third fused feature. The calculation formula of the third fused feature F3 is:

[0025] ;

[0026] In the formula, is the first fuzzy vector with low sensitivity attribute, is the i-th vector in the third vector set, is the weight coefficient of the first fuzzy vector with low sensitivity attribute, is the weight coefficient of the third vector set, n3 is the number of vectors in the third vector set;

[0027] The calculation formula for weighted fusion of the vectors of the first fusion feature, the second fusion feature, and the third fusion feature is:

[0028] ;

[0029] In the formula, is the vector sensitivity weight of the first fusion feature, is the vector sensitivity weight of the second fusion feature, is the vector sensitivity weight of the third fusion feature; F1, F2 and F3 are the first fusion feature, the second fusion feature and the third fusion feature respectively.

[0030] Preferably, the step of marking the feature vector of the above-mentioned sensitive image according to the attribute sensitivity is: the above-mentioned server configures the attribute marking, and the above-mentioned attribute marking includes low-sensitivity attributes, medium-sensitivity attributes and high-sensitivity attributes in order according to the attribute sensitivity; the server configures the attribute marking according to predetermined standards, and performs preliminary classification using the attribute sensitivity to provide a basis for subsequent feature fusion and matching.

[0031] Obtain the target contour in the sensitive image, identify and extract the target contour of the pedestrian in the sensitive image, so as to obtain relevant data including the pedestrian's posture, size and movement through the contour information; divide the target contour into areas according to the physiological contour, and determine the salient area through the physiological contour; through further division of the target contour, the division helps to capture the specific characteristics of each area in detail, thereby giving corresponding sensitivity levels to different physiological contours.

[0032] The divisions may be into different physiological regions, such as the head, limbs, and the like.

[0033] The feature vectors corresponding to the salient areas are extracted to generate the first vector set, and the feature vectors of the remaining areas are generated to generate the second vector set. Based on the results of the zoning, the system identifies areas with high recognition, namely salient areas. They will be marked as highly sensitive attributes because they usually contain key individual recognition information, such as acquired facial features, and their corresponding feature vectors are assigned to the first vector set. The remaining feature vectors are assigned to the second vector set because there is a risk of identification difference.

[0034] It should be noted that, through the setting of the first vector set and the second vector set, in principle, high-sensitivity attributes can be processed preferentially during the recognition process, so that when performing data characterization matching, the search range can be quickly narrowed, which helps to reduce the system load during matching and thus improve the overall processing speed.

[0035] Preferably, the step of the server obtaining video data to select key frames is: the server obtains a continuous video stream or video file, and performs consistency processing on the video data to obtain format data; decomposes the format data according to a continuous frame sequence, and divides it according to a fixed number of frames, until it becomes a plurality of dynamic frames; performs clarity detection on each frame image of the dynamic frame; obtains the clarity of each frame; selects in advance the frame whose clarity meets the requirements in each dynamic frame to obtain a valid frame; obtains the valid frame and sets the efficiency value, extracts the characteristic value of the valid frame, and selects the frame whose characteristic value is greater than the efficiency value as the key frame.

[0036] Preferably, the server obtains video data through a local area terminal, there are multiple local area terminals, and each local area terminal is provided with a unique device code; wherein, when the local area terminal sends video data to the server, the local area terminal adds a character code of the corresponding device code to the video data; when the server obtains the video data of the local area terminal, the server obtains the device code of the corresponding local area terminal, and verifies whether the character code of the video data is consistent with the device code of the local area terminal; wherein, if the character code of the video data is consistent with the device code of the local area terminal, the server records the video data; wherein, if the character code of the video data is inconsistent with the device code of the local area terminal, the server does not record the video data.

[0037] A further technical solution is that when the character code of the video data is inconsistent with the device code of the local terminal, the server records the device code corresponding to the character code and sends an abnormal warning to the staff.

[0038] Compared with the prior art, the beneficial effects of the present invention are at least one of the following:

[0039] The present invention selects high-quality image frames as key frames from video data through a server and performs preliminary recognition and confirmation. If the recognition can be made, pedestrian matching is performed directly. If the recognition cannot be made directly, the key frames and the associated continuous frames are expanded to obtain more information in the video data. According to the key frames and the corresponding continuous frames, the vector features of different frames are distinguished according to their importance, and different feature vectors are given different weights. In the feature fusion process, the negative impact caused by video deviation can be reduced, so as to amplify significant features in the fusion process, effectively reduce the information deviation caused by changes in video quality or differences in camera angles, and improve the robustness of video data recognition of pedestrians. At the same time, by dynamically adjusting the processing coefficient N, the system can adaptively improve the acquisition of effective information according to actual conditions, thereby improving the recognition rate of video data in a variety of environments.

[0040] The present invention fuses features of different sensitivities and assigns weights again, assigns different weights (high sensitivity, medium sensitivity, and low sensitivity attributes) to each feature vector, identifies areas with higher recognition value in sensitive images, and processes the feature vectors of the corresponding areas separately from other features. This makes it possible to quickly narrow the search scope and reduce the computing resources required for matching when performing data matching. On the premise of ensuring the expressiveness of the recognition system for pedestrian features, through the processing of video data and the matching of static data in the cloud database, the processing coefficient N and the similarity threshold K can be adjusted according to actual conditions, so as to achieve optimal recognition performance in different environments and conditions and improve the adaptability of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 It is a schematic diagram of the process steps of the present invention. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solution and advantages of the present invention clearer, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention. It should be noted that all directional indications (such as up, down, left, right, front, back...) in the embodiments of the present invention are only used to explain the relative position relationship, movement status, etc. in a certain specific working state. If the specific posture changes, the directional indication also changes accordingly. In the present invention, unless otherwise clearly specified and limited, the term "connection" and the like should be understood in a broad sense. For example, "connection" can be an electrical signal connection or a signal connection; it can also be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly defined. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to the specific circumstances.

[0043] If there are descriptions involving "first", "second", etc. in the embodiments of the present invention, the descriptions of "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the meaning of "and / or" appearing in the full text includes three parallel schemes. Taking "A and / or B" as an example, it includes scheme A, or scheme B, or a scheme that satisfies both A and B. In addition, the technical solutions between the various embodiments can be combined with each other, but it must be based on the ability of ordinary technicians in the field to implement. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0044] refer to Figure 1 As shown, one embodiment of the present invention is a method for pedestrian recognition based on video data, wherein the method mainly processes the video data to obtain different feature vectors in the data, selectively fuses the feature vectors according to their importance, and inputs the fused feature data into the data representation of the cloud database for processing and recognition. If necessary, the cloud database is used to store and read the data representation. The above method includes step S100, wherein the cloud database sets a number of pedestrian IDs and data representations corresponding to the pedestrian IDs; the above data representations are obtained by fusing vector features extracted from the pedestrian's historical data, and by extracting and fusing feature vectors from the historical data, the data representation can more accurately reflect whether the pedestrian in the video data is a known pedestrian. Among them, the cloud data defines a corresponding data representation for each ID, and generates a unique fingerprint representing the pedestrian through the appearance, action, and clothing features of the corresponding pedestrian in the data representation.

[0045] In order to reduce the overall data processing volume, the server obtains the video data and selects key frames to identify whether the contour features of the key frames are missing. The server extracts pictures containing important information from the video data by selecting key frames, and then determines the information integrity of the key frames by identifying whether the contour features of the key frames are missing.

[0046] On the one hand, if the contour feature is not missing, it usually indicates that the image of the frame has sufficient information for accurate pedestrian analysis. Since the feature vector of the key frame usually includes identification information such as the position, posture, and clothing of the pedestrian, by converting the feature vector of the key frame into the first representation, the image feature can be used to represent the state of the pedestrian and directly used for accurate matching with the data representation in the cloud database. Therefore, when the contour feature is not missing, the feature vector of the key frame is converted into the first representation and the matching work of S600 is performed.

[0047] On the one hand, if the contour feature is missing, it usually indicates that there is missing data in the image of the frame, and the missing data may be due to camera occlusion, lighting problems or image quality. In order to reduce the impact of missing data on pedestrian recognition, sequence information will be added to each frame of the image through the server, including frame sequence number and timestamp. Each frame of the image is marked with a sequence number and a timestamp recording the shooting time, so as to ensure that even if the real-time video data is processed in segments during the processing process, the sequence information can still maintain the original order and time relationship between single frames, so as to facilitate the use of coherent frames to reconstruct or make up for the missing data. Therefore, when the contour feature is missing, the server adds sequence information to each frame of the image, and the above sequence information includes a frame sequence number and a timestamp; obtains the timestamp association between the key frame and the coherent frame and executes S200 to complete the missing features.

[0048] The method further includes step S200, considering that in the case where the key frame contour feature is missing, the server needs to complete the information by comparing and analyzing with the continuous frames, and the server presets a processing coefficient N, wherein the processing coefficient N is the number of continuous frames before and after the key frame information required to complete, N is a positive integer, and N≥2, and the preset processing coefficient N is set to facilitate incorporating different numbers of continuous frames into the missing information of the key frame. The above server obtains N continuous frames before and after the key frame by comparing the timestamp of the key frame with the timestamp of the adjacent frame.

[0049] In order to better obtain relevant information of the missing part of the key frame, the key frame is marked as a sensitive image, and N consecutive frames associated with the sequence information of the sensitive image are taken to obtain N extended images; the selected N consecutive frames associated with the sensitive image through sequence information ensure that N frames close to the key frame in the sequence are selected, and the image time of the frames is close, so as to obtain relevant information of the missing part of the key frame according to the difference.

[0050] Sort the extended images by timestamp and execute S300. Sort the extended images by timestamp, and the timestamp can help determine the difference between images. The size of the processing coefficient N directly affects the scope of data processing. When the key frame has more missing data, the value of N needs to be increased, so that the amount of data processed increases, the more information covered by the acquired vector features, the longer the processing time span, and the greater the resource consumption. When the key frame has less missing data, the value of N needs to be reduced, so that the amount of data processed is reduced. When the acquired vector features meet the recognition requirements, they can also meet the recognition requirements and reduce the load required for data processing.

[0051] In order to better obtain the missing part through the extended image, the method also includes step S300, dividing the extended image grid into a number of local image blocks, by dividing the extended image grid into a number of local image blocks, using the local image blocks so that the image analysis and comparison can be performed on smaller image units; comparing the two extended images in order and obtaining the repeated area and the deviation area; and obtaining the first image set and the second image set by identifying the local image blocks of the repeated area and the deviation area. By obtaining the local changes and unchanged areas between the extended images, in the case of missing contour features, the deviation and repeated areas of the continuous frames are used to obtain key frame information that can make up for it.

[0052] The first image set includes a number of first fuzzy vectors, which are obtained from the repeated area and used to compensate for the stable and unchanging features in the image. The second image set includes a number of second fuzzy vectors, which are extracted from the deviation area and are mainly used to analyze and understand the changing features in the image.

[0053] Among them, by comparing the extended images sorted by timestamps in sequence, the changes between images adjacent in time in the image sequence can be found, and repeated areas and deviation areas can be obtained; repeated areas refer to the parts that do not change in continuous extended images, and the unchanged parts are usually specific parts of pedestrians; deviation areas refer to the parts that change in continuous extended images, and the changed parts are usually related to the changing areas during the movement of pedestrians.

[0054] The method further includes step S400, marking the feature vectors of the sensitive image according to the attribute sensitivity and integrating them into a first vector set and a second vector set; integrating the second fuzzy vector into a third vector set. The above attribute sensitivity is mainly aimed at the impact of different features on the recognition task, and is set according to the importance. For example, facial details or unique features usually have a higher sensitivity, and a higher sensitivity will contribute to greater efficiency and accuracy in pedestrian recognition.

[0055] Among them, the first vector set is composed of feature vectors with high sensitivity, and the vector features of the first vector set are key features that directly affect the recognition accuracy. The second vector set is composed of feature vectors with medium sensitivity, and the vector features of the second vector set are features that have the accuracy of pedestrian recognition but may be missing in some parts. The third vector set mainly contains feature vectors identified from local deviation areas, and the vector features of the third vector set are mainly used to obtain features that often change dynamically and have certain indicative properties.

[0056] The method further includes step S500, determining the fusion attribute of the first fuzzy vector; the first fuzzy vector is subjected to feature fusion with the first vector set, the second vector set and the third vector set according to the fusion attribute, to obtain a second representation and execute S600. By determining the fusion attribute of the first fuzzy vector and performing feature fusion with the classified vector set, the first fuzzy vector is subjected to feature fusion with the first vector set, the second vector set and the third vector set, respectively, thereby generating a more comprehensive and refined data representation, namely the second representation.

[0057] The vector features in the first fuzzy vector with the same attributes as the first vector set are fused, and the recognition ability of the model for key attributes is enhanced by fusion. The vector features in the first fuzzy vector with the same attributes as the second vector set are fused to improve the recognition ability of missing features. The feature vector suspected of changing in the first fuzzy vector is fused with the vector features of the second vector set, and the fused features are dynamically changing related vector data.

[0058] The method further includes step S600, wherein the server inputs the first representation or the second representation into the cloud database to match the data representation; and matches the first representation or the second representation with the recorded data representation. If the first representation or the second representation is matched in the cloud database, the matching data representation is successful, and the server retrieves the corresponding pedestrian ID in the cloud database for time stamping. If a matching data representation is found in the database, the corresponding pedestrian ID can be directly retrieved and time stamped, thereby recording the appearance and activities of pedestrians at different times.

[0059] If the first representation does not match any result, it means that the matching fails. The first representation is stored in the cloud database through the server and a unique pedestrian ID is established.

[0060] Since the first representation fails to find a match in the cloud database, it means that the pedestrian corresponding to the first representation may be captured by the system for the first time; the features of the first representation are stored in the cloud database, and a unique pedestrian ID is established for it, which helps to expand the database and re-identify the pedestrian corresponding to the first representation later.

[0061] If the second representation does not match any result, it indicates that the match has failed. In addition to the fact that the feature does not exist, there may also be a situation where the feature information is incomplete. Therefore, the server increases the processing coefficient N and re-executes S200. The server again introduces more continuous frames to try to supplement and improve the feature information.

[0062] Considering that the pedestrian may not be recorded in the database, the server records the number of times S200 is executed. When the number of executions is greater than the set number, the second representation is stored in the cloud database through the server and a unique pedestrian ID is established. By limiting the number of identifications, if a match still cannot be made, the system will record the representation, store it in the cloud database, and assign a new pedestrian ID so that the system can keep its information up to date and facilitate future tracking and analysis.

[0063] Based on the above embodiment, another embodiment of the present invention is that when S500 is executed, the specific steps of determining the fusion attribute of the first fuzzy vector are:

[0064] S510: The server presets a similarity threshold K and pairs each first fuzzy vector with the first vector set and the second vector set according to similarity.

[0065] The setting of the similarity threshold K is a benchmark value for judging whether the similarity between two vectors is high enough, and is used to measure whether the fuzzy vector has the conditions to be merged into a specific vector set.

[0066] Each first fuzzy vector is compared with the vectors in the first vector set and the second vector set for similarity, so as to determine the similarity between the vectors in the comparison process and select the vectors with a similarity threshold value K.

[0067] S520: Obtain the highest similarity between the first fuzzy vector and any vector in the first vector set as P1, wherein P1 represents the maximum similarity between the first fuzzy vector and any vector in the first vector set.

[0068] S530, obtaining the highest similarity between the first fuzzy vector and any vector in the second vector set as P2. P2 represents the maximum similarity between the first fuzzy vector and any vector in the second vector set;

[0069] Exemplarily, a first fuzzy vector is compared with the first vector set, and the similarity parameter after the first fuzzy vector is compared with i feature vectors in the first vector set is ; The highest similarity between the first fuzzy vector and any vector in the first vector set At the same time, the similarity parameter after comparing the first fuzzy vector with the r feature vectors in the second vector set is ; The highest similarity between the first fuzzy vector and any vector in the second vector set .

[0070] Among them, when P1>P2, and P1≥K, it means that the first fuzzy vector is very similar to the vector corresponding to P1 in the first vector set, and the fusion attribute of the first fuzzy vector is a high-sensitivity attribute, which is used to be fused into the first vector set.

[0071] For reference, if the similarity between the first fuzzy vector and any vector in the first vector set is higher than that between the first fuzzy vector and any vector in the second vector set, and the similarity is greater than or equal to the similarity threshold, then the first fuzzy vector carries key identification information, and the first fuzzy vector is regarded as a highly sensitive attribute and should be classified into the first vector set for fusion.

[0072] Among them, when P1<P2, and P2≥K, it means that the first fuzzy vector is very similar to the vector corresponding to P2 in the second vector set, and the fusion attribute of the first fuzzy vector is a medium-sensitive attribute, which is used to be fused into the second vector set. If the similarity between the first fuzzy vector and any vector in the first vector set is lower than the similarity with any vector in the second vector set, and the similarity is greater than or equal to the similarity threshold, the first fuzzy vector is regarded as a medium-sensitive attribute and should be classified into the second vector set for fusion.

[0073] Among them, when P1 and P2 are both less than K, the fusion attribute of the first fuzzy vector is a low-sensitivity attribute, and the low-sensitivity attribute is used to be fused into the third vector set. The similarity between the first fuzzy vector and any vector in the first vector set and the second vector set is less than the similarity threshold, indicating that the first fuzzy vector is not very similar to the vectors in the two existing vector sets, and the first fuzzy vector should be regarded as a low-sensitivity attribute, and the first fuzzy vector is classified into the third vector set for fusion.

[0074] A further technical solution is to calculate the similarity between the first fuzzy vector and each vector feature in the first vector set and the second vector set. The calculation formula is:

[0075] ;

[0076] In the formula, is any i vector in the first vector set or the second vector set, is the Euclidean norm of any i vector in the first vector set or the second vector set; is the first fuzzy vector; is the Euclidean norm of the first fuzzy vector.

[0077] By calculating the similarity, the similarity between the first fuzzy vector and the existing feature vector can be determined, which helps to determine whether the first fuzzy vector forms a clear and logically reasonable feature classification, so that the first fuzzy vector can better select a suitable vector set for fusion, ensuring that the information of the first fuzzy vector helps to improve recognition accuracy.

[0078] A further technical solution is to perform feature fusion on all first fuzzy vectors according to high-sensitivity attributes, medium-sensitivity attributes and low-sensitivity attributes with the corresponding first vector set, second vector set and third vector set respectively; and obtain first fusion features, second fusion features and third fusion features.

[0079] Among them, high-sensitivity attributes refer to features that have a significant impact on pedestrian recognition results and have high discrimination capabilities. The first fuzzy vector regarded as a high-sensitivity attribute corresponds to the feature parts that differ greatly between individuals, such as facial features, global body contours, etc. Different pedestrians can be effectively distinguished through the vector features of high-sensitivity attributes.

[0080] Among them, medium-sensitive attributes refer to features that have an impact on pedestrian recognition results, but are vectors with limited discrimination ability. The first fuzzy vector regarded as medium-sensitive attributes corresponds to the feature part with distinguishing power, such as clothing, partial body contours, etc.; the vector features of medium-sensitive attributes can improve the pedestrian discrimination effect.

[0081] Among them, low-sensitivity attributes refer to features that may affect the possible results of pedestrian recognition. The first fuzzy vector of the low-sensitivity attribute corresponds to features that often change in the entire continuous frame, such as decorations, hairstyles, etc. The vector features of the low-sensitivity attribute can be used as extended information for pedestrian data representation.

[0082] Among them, the first fusion feature, the second fusion feature and the third fusion feature are respectively set with fusion weights, and the vectors of the first fusion feature, the second fusion feature and the third fusion feature are weighted fused to obtain the second data representation.

[0083] For reference, since the first fuzzy vector is subjected to feature fusion with the corresponding first vector set, second vector set, and third vector set, each vector set generates corresponding fusion features during the fusion process, namely, the first fusion feature, the second fusion feature, and the third fusion feature. Different weights are set for the first fusion feature, the second fusion feature, and the third fusion feature. The weight setting is based on the importance of the feature and the contribution factor to the final recognition result. The weighted first fusion feature, the second fusion feature, and the third fusion feature are further combined to generate a second data representation. The second data representation constructs a data structure that integrates all important features and their contributions to ensure that the second data representation has high accuracy and identifiability.

[0084] One setting method is to fuse the first fuzzy vector with high sensitivity attribute with the first vector set feature to obtain the first fused feature; the calculation formula of the first fused feature F1 is:

[0085] ;

[0086] In the formula, is the first fuzzy vector with high sensitivity attribute, is the i-th vector in the first vector set, is the weight coefficient of the first fuzzy vector with high sensitivity attribute, is the weight coefficient of the first vector set, and n1 is the number of vectors in the first vector set. Among them, vector features with high sensitivity attributes are usually features that can significantly improve feature accuracy. Therefore, by and The weights are set respectively mainly to weigh the contribution of the elements in the first fuzzy vector and the second vector.

[0087] Among them, the first fuzzy vector with medium sensitivity attribute is fused with the second vector set feature to obtain the second fused feature; the calculation formula of the second fused feature F2 is:

[0088] ;

[0089] In the formula, is the first fuzzy vector with medium sensitivity attribute, is the i-th vector in the second vector set, is the weight coefficient of the first fuzzy vector with medium-sensitive attribute, is the weight coefficient of the second vector set, and n2 is the number of vectors in the second vector set.

[0090] Among them, the vector features with medium-sensitivity attributes have a large number of effective vector features, and their vector features can complement each other in the process of fusing the first fuzzy vector and the second vector set.

[0091] The third fusion feature is obtained by fusing the first fuzzy vector with low sensitivity attribute with the third vector set feature. The calculation formula of the third fusion feature F3 is:

[0092] ;

[0093] In the formula, is the first fuzzy vector with low sensitivity attribute, is the i-th vector in the third vector set, is the weight coefficient of the first fuzzy vector with low sensitivity attribute, is the weight coefficient of the third vector set, and n3 is the number of vectors in the third vector set.

[0094] Among them, the calculation formula for weighted fusion of the vectors of the first fusion feature, the second fusion feature and the third fusion feature is: ;

[0095] In the formula, is the vector sensitivity weight of the first fusion feature, is the vector sensitivity weight of the second fusion feature, is the vector sensitivity weight of the third fusion feature; F1, F2 and F3 are the first fusion feature, the second fusion feature and the third fusion feature respectively.

[0096] Furthermore, the step of marking the feature vector of the above-mentioned sensitive image according to the attribute sensitivity is: the above-mentioned server configures the attribute marking, and the above-mentioned attribute marking includes low-sensitivity attributes, medium-sensitivity attributes and high-sensitivity attributes in order according to the attribute sensitivity; obtaining the target contour in the sensitive image, dividing the target contour into areas according to the physiological contour, and determining the salient area through the physiological contour.

[0097] Furthermore, the server obtains video data and selects key frames in the following steps:

[0098] Step 1: The server obtains a continuous video stream or video file, and performs uniform processing on the video data to obtain format data; the server obtains a continuous video stream from a video source, pre-processes the video data, and outputs the data in a unified output format to obtain format data.

[0099] For reference, the above preprocessing is to adjust the resolution, frame rate and color format of the video stream so that the format of all video streams is consistent.

[0100] Step 2: Decompose the format data according to a continuous frame sequence, and divide it according to a fixed number of frames until multiple dynamic frames are obtained; perform clarity detection on each frame image of the dynamic frame; obtain the clarity of each frame; and advance the frame whose clarity meets the requirements in each dynamic frame to obtain a valid frame.

[0101] The preprocessed format data is decomposed into static frames in chronological order, and the sequence is divided into multiple frame groups according to a fixed number of frames, each of which is a dynamic frame. The clarity detection is performed on the single-frame image of each dynamic frame. The clarity detection can use the existing image edge detection and contrast detection to obtain the visual quality of the single-frame image. According to the visual quality, the frames with the required clarity are selected and marked as valid frames.

[0102] Step 3: Get valid frames and set efficiency values, extract feature values ​​of valid frames, and select frames with feature values ​​greater than efficiency values ​​as key frames. Get relevant feature values ​​from each valid frame. Feature values ​​are generally facial ranges, body contour ranges, or other identification features, so as to determine the analysis value of the frame, set an efficiency value as a threshold, and select frames with sufficient feature information as key frames.

[0103] Furthermore, the extraction of feature values ​​of valid frames includes determining the feature categories to be extracted, extracting corresponding features from the valid frames, and quantifying the extracted features according to the categories. By selecting appropriate feature types, important information in the frame can be better represented.

[0104] For reference, the feature category includes color features, and the quantization of the color features is as follows: quantize the color distribution of the image into a histogram, divide the RGB space or HSV space into several bins, obtain the number of pixels in each bin, and calculate the quantized value Hi of the color feature. The calculation formula is:

[0105] ;

[0106] Where Hi is the histogram value of the i-th color interval, F(pj) is the indicator function that pixel pj belongs to the i-th interval, and Q is the total number of pixels.

[0107] For reference, the feature category includes shape edges, and the feature quantization of the shape edge is as follows: using the edge detection operator Canny to calculate the gradient direction and intensity, and quantizing the result in the form of a histogram; calculating the shape edge quantization value Ei, the calculation formula is:

[0108] ;

[0109] Where Ei is the edge histogram value of the i-th gradient direction, is the gradient direction at the pixel position; is the indicator function that the pixel gradient direction belongs to interval i.

[0110] The complex image features are converted into numerical forms that can be directly operated and compared through quantization methods to ensure that a single frame has enough information. The frames with feature values ​​greater than or equal to the efficiency value are selected as key frames. It should be noted that the setting of each efficiency value needs to correspond to the feature category so that the efficiency value can determine the lower limit of the information of the same feature category in the image.

[0111] Based on the above embodiment, another embodiment of the present invention is that the above server obtains video data through a local area terminal, there are multiple local area terminals, each of which is provided with a unique device code, wherein when the local area terminal sends video data to the server, the local area terminal adds a character code of the corresponding device code to the video data. The local area terminal may be an existing surveillance camera terminal. When the server obtains the video data of the local area terminal, the server obtains the device code of the corresponding local area terminal, and verifies whether the character code of the video data is consistent with the device code of the local area terminal; the source of the data is verified by the server.

[0112] Among them, the character code of the video data is consistent with the device code of the local area terminal, indicating that the data is normal, and the server records it; among them, the character code of the video data is inconsistent with the device code of the local area terminal, and the server does not record it. For reference, each local area terminal has a unique device code, which is pre-configured by the local area terminal, and the above device code is used to identify the identity of each local area terminal; when the local area terminal captures the video data, it will attach its own device code to the video data in the form of a character code before sending it to the server, so that when the server receives the video data from the local area terminal, it can read the character code device code attached to the video data.

[0113] A further technical solution is that when the character code of the above-mentioned video data is inconsistent with the device code of the local area terminal, since the character code of the video data does not match the device code of the local area terminal, the data may be tampered with or come from an unauthorized device. Therefore, the above-mentioned server records the device code corresponding to the character code and sends an abnormal warning to the staff.

[0114] Through the server device coding verification mechanism, the server verifies the received video data. For abnormal data, the verification mechanism set by the server can identify illegal data input and reduce the risk of forgery or tampering. Improve the authenticity and reliability of data and protect the overall security of the system. The abnormal warning function of the server is that when the device coding is inconsistent, the system automatically sends abnormal warnings to the mobile terminals of nearby staff, which can timely discover and deal with potential problems and ensure data quality.

[0115] The "one embodiment", "another embodiment", "embodiment", "preferred embodiment" and the like mentioned in this specification refer to the specific features, structures or characteristics described in conjunction with the embodiment as included in at least one embodiment generally described in this application. The same expression appearing in multiple places in the specification does not necessarily refer to the same embodiment. Further, when a specific feature, structure or characteristic is described in conjunction with any embodiment, it is claimed that the realization of such feature, structure or characteristic in conjunction with other embodiments also falls within the scope of the present invention.

[0116] Although the present invention is described herein with reference to a number of illustrative embodiments of the present invention, it will be appreciated that those skilled in the art may devise many other modifications and implementations that fall within the scope and spirit of the principles disclosed herein. More specifically, within the scope of the present disclosure, drawings, and claims, a variety of variations and modifications may be made to the components and / or layout of the subject combination layout. In addition to the variations and modifications made to the components and / or layout, other uses will also be apparent to those skilled in the art.

Claims

1. A pedestrian recognition method based on video data, comprising the following steps: S100, a cloud database sets a number of pedestrian IDs and data representations corresponding to the pedestrian IDs; a server obtains video data and selects key frames, and identifies whether contour features of the key frames are missing; If the contour feature is not missing, convert the feature vector of the key frame into a first representation and execute S600; If the contour feature is missing, the server adds sequence information to each frame of the image, wherein the sequence information includes a frame sequence number and a timestamp; Obtain the timestamp association between the key frame and the consecutive frames and execute S200; S200, the server presets a processing coefficient N, marks the key frame as a sensitive image, obtains N consecutive frames associated with the sequence information of the sensitive image to obtain N extended images; sorts the extended images according to the timestamps, and executes S300; S300, dividing the extended image grid into a number of local image blocks, comparing the two extended images in order and obtaining a repeated area and a deviation area; identifying the local image blocks of the repeated area and the deviation area to obtain a first image set and a second image set respectively; The first image set includes a number of first fuzzy vectors; the second image set includes a number of second fuzzy vectors; S400, marking the feature vectors of the sensitive image according to the attribute sensitivity and integrating them into a first vector set and a second vector set; integrating the second fuzzy vectors into a third vector set; S500, determining a fusion attribute of a first fuzzy vector; performing feature fusion of the first fuzzy vector with the first vector set, the second vector set and the third vector set according to the fusion attribute, obtaining a second representation and executing S600; S600, the server inputs the first representation or the second representation into the cloud database to match the data representation; if the first representation or the second representation is matched in the cloud database, the server retrieves the corresponding pedestrian ID in the cloud database for time stamping; If the first representation does not match a result, the first representation is stored in a cloud database through the server and a unique pedestrian ID is established; If the second representation does not match the result, the server increases the processing coefficient N and re-executes S200, wherein the server records the number of times S200 is executed. When the number of executions is greater than the set number, the second representation is stored in the cloud database through the server and a unique pedestrian ID is established.

2. The method for pedestrian recognition based on video data according to claim 1, characterized in that: When executing S500, the specific steps of determining the fusion attribute of the first fuzzy vector are: S510, the server presets a similarity threshold K, and pairs each first fuzzy vector with the first vector set and the second vector set according to similarity; S520, obtaining the highest similarity between the first fuzzy vector and any vector in the first vector set as P1; S530, obtaining the highest similarity between the first fuzzy vector and any vector in the second vector set as P2; When P1>P2, and P1≥K, the fusion attribute of the first fuzzy vector is a high-sensitivity attribute, and the high-sensitivity attribute is used to be fused into the first vector set; When P1<P2, and P2≥K, the fusion attribute of the first fuzzy vector is a medium-sensitive attribute, and the medium-sensitive attribute is used to be fused into the second vector set; When P1 and P2 are both smaller than K, the fusion attribute of the first fuzzy vector is a low-sensitivity attribute, and the low-sensitivity attribute is used to be fused into the third vector set.

3. The method for pedestrian recognition based on video data according to claim 2, characterized in that: Calculate the similarity between the first fuzzy vector and each vector feature in the first vector set and the second vector set. The calculation formula is: ； In the formula, is any i vector in the first vector set or the second vector set, is the Euclidean norm of any i vector in the first vector set or the second vector set; is the first fuzzy vector; is the Euclidean norm of the first fuzzy vector.

4. The method for pedestrian recognition based on video data according to claim 2, characterized in that: All first fuzzy vectors are subjected to feature fusion with the corresponding first vector set, second vector set and third vector set respectively according to high-sensitivity attribute, medium-sensitivity attribute and low-sensitivity attribute to obtain first fusion feature, second fusion feature and third fusion feature; Among them, the first fuzzy vector with high sensitivity attribute is fused with the first vector set feature to obtain the first fused feature; the calculation formula of the first fused feature F1 is: ； In the formula, is the first fuzzy vector with high sensitivity, is the i-th vector in the first vector set, is the weight coefficient of the first fuzzy vector with high sensitivity attribute, is the weight coefficient of the first vector set, n1 is the number of vectors in the first vector set; Among them, the first fuzzy vector with medium sensitivity attribute is fused with the second vector set feature to obtain the second fused feature; the calculation formula of the second fused feature F2 is: ； In the formula, is the first fuzzy vector with medium sensitivity attribute, is the i-th vector in the second vector set, is the weight coefficient of the first fuzzy vector with medium-sensitive attribute, is the weight coefficient of the second vector set, n2 is the number of vectors in the second vector set; The first fuzzy vector with low sensitivity attribute is fused with the third vector set feature to obtain a third fused feature. The calculation formula of the third fused feature F3 is: ； In the formula, is the first fuzzy vector with low sensitivity attribute, is the i-th vector in the third vector set, is the weight coefficient of the first fuzzy vector with low sensitivity attribute, is the weight coefficient of the third vector set, and n3 is the number of vectors in the third vector set.

5. The method for pedestrian recognition based on video data according to claim 4, characterized in that: The first fusion feature, the second fusion feature and the third fusion feature are respectively set with fusion weights; the vectors of the first fusion feature, the second fusion feature and the third fusion feature are weightedly fused to obtain a second data representation; The vectors of the first fusion feature, the second fusion feature and the third fusion feature are weighted and fused, and the calculation formula of the second data representation D2 after fusion is: ； In the formula, is the vector sensitivity weight of the first fusion feature, is the vector sensitivity weight of the second fusion feature, is the vector sensitivity weight of the third fusion feature; F1, F2 and F3 are the first fusion feature, the second fusion feature and the third fusion feature respectively.

6. The method for pedestrian recognition based on video data according to claim 1, characterized in that: The step of marking the feature vector of the sensitive image according to the attribute sensitivity is: The server configures attribute tags, wherein the attribute tags include low-sensitivity attributes, medium-sensitivity attributes, and high-sensitivity attributes in order of attribute sensitivity; Obtain the target contour in the sensitive image, divide the target contour into regions according to the physiological contour, and determine the significant area through the physiological contour; The feature vectors corresponding to the salient regions are extracted to generate a first vector set, and the feature vectors of the remaining regions are extracted to generate a second vector set.

7. The method for pedestrian recognition based on video data according to claim 1, characterized in that: The steps of the server obtaining video data to select key frames are: The server obtains a continuous video stream or video file, and performs uniform processing on the video data to obtain format data; Decompose the format data according to a continuous frame sequence, and divide it according to a fixed number of frames until there are multiple dynamic frames; Perform clarity detection on each frame image of the dynamic frame; Get the clarity of each frame; Extract the frames whose definition meets the requirements in each dynamic frame to obtain valid frames; Get the valid frame and set the efficiency value, extract the characteristic value of the valid frame, and select the frame whose characteristic value is greater than the efficiency value as the key frame.

8. The method for pedestrian recognition based on video data according to claim 7, characterized in that: The extracting of feature values ​​of the valid frame includes determining the feature class to be extracted, extracting corresponding features from the valid frame, and quantifying the extracted features according to the classification.

9. The method for pedestrian recognition based on video data according to claim 1, characterized in that: The server obtains video data through a local area terminal, wherein there are multiple local area terminals, and each local area terminal is provided with a unique device code; When the local terminal sends video data to the server, the local terminal adds a character code of the corresponding device code to the video data; when the server obtains the video data of the local terminal, the server obtains the device code of the corresponding local terminal and verifies whether the character code of the video data is consistent with the device code of the local terminal; Wherein, if the character code of the video data is consistent with the device code of the local terminal, the server records the video data; If the character code of the video data is inconsistent with the device code of the local terminal, the server will not record the video data.

10. The method for pedestrian recognition based on video data according to claim 9, characterized in that: When the character code of the video data is inconsistent with the device code of the local terminal, the server records the device code corresponding to the character code and sends an abnormal warning to the staff.

Images