A security video monitoring method and system based on hybrid feature recognition

Abstract

The invention discloses a security video monitoring method based on hybrid feature recognition, including: obtaining a reference appearance feature and a reference face feature; obtaining a security video to be recognized and parsing it to obtain multiple frame images; obtaining target person features and corresponding capture frame coordinates based on the frame images; marking different target persons in the frame images based on the target person features to obtain multiple different person IDs; intercepting corresponding target person images based on the frame images corresponding to the person IDs; extracting features based on the target person images to obtain recognition appearance features and recognition face features; determining a first identity based on the recognition appearance features and the reference appearance features; determining a second identity based on the recognition face features and the reference face features; determining a final person identity based on the first identity and the second identity; marking the capture frame determined by the final person identity and its corresponding capture frame coordinates into the corresponding frame image. The person's action trajectory can be captured and the person's identity can be recognized efficiently and accurately.

Description

Technical Field

[0001] The present invention relates to the technical field of security monitoring, and more particularly to a security video monitoring method and system based on hybrid feature recognition. Background Art

[0002] At present, as people's pursuit of quality of life continues to improve, the public's requirements for the safety of the living environment have reached an unprecedented level. Smart security systems that can achieve comprehensive and real-time monitoring of the community, timely warning and handling of various safety hazards, and provide residents with a safer and more comfortable living environment have become products that people are looking forward to.

[0003] However, due to the lack of functions and low efficiency, traditional security systems are unable to accurately and efficiently identify people's identities and track their movements through security videos, and can no longer fully meet people's needs for effective public safety management.

[0004] Therefore, how to track the movement trajectory of people appearing in security videos and identify their identities efficiently and accurately is a problem that technical personnel in this field urgently need to solve. Summary of the invention

[0005] In view of this, the present invention provides a security video monitoring method and system based on hybrid feature recognition, which can capture the movement trajectory of people in security videos and efficiently and accurately identify the identities of people, thereby effectively protecting various security issues in security scenes.

[0006] In order to achieve the above object, the present invention adopts the following technical solution:

[0007] A security video monitoring method based on hybrid feature recognition, comprising:

[0008] Acquire multiple known person images and perform feature extraction to obtain baseline appearance features and baseline face features;

[0009] Obtain the security video to be identified and analyze it to obtain multiple frame images;

[0010] Obtaining target person features and corresponding capture frame coordinates based on the frame image recognition;

[0011] Marking different target persons in the frame image based on the target person features to obtain a plurality of different person IDs;

[0012] Based on the frame image corresponding to the character ID, a corresponding target character image is captured;

[0013] Based on the target person image, feature extraction is performed respectively to obtain the recognized appearance features and the recognized face features;

[0014] Determining a first identity based on the identified appearance feature and the reference appearance feature;

[0015] Determining a second identity based on the identified facial features and the reference facial features;

[0016] Determine a final person identity of the target person feature based on the first identity and the second identity;

[0017] The final person identity and the capture frame determined by the corresponding capture frame coordinates are marked in the corresponding frame image.

[0018] Preferably, obtaining the reference appearance features and reference face features specifically includes:

[0019] Acquire a preset number of known person images from which facial features can be extracted, and name the known person images accordingly;

[0020] Based on the known person image, appearance features and facial features are extracted respectively to obtain the reference appearance features and reference facial features;

[0021] The reference appearance features and the reference facial features are both named after the corresponding characters.

[0022] Preferably, obtaining the target person features and their corresponding capture frame coordinates specifically includes:

[0023] Based on the frame image, all target persons whose confidence levels are greater than a preset value are captured to obtain target person features and capture frame coordinates corresponding to the target person features.

[0024] Preferably, a plurality of different character IDs are obtained, specifically including:

[0025] Determine the corresponding relationship based on comparing the target person features in the current frame image with the target person features in the previous frame image;

[0026] Based on the corresponding relationship, different target persons in different frame images are marked to obtain a plurality of different person IDs.

[0027] Preferably, the method for obtaining the identification appearance feature is:

[0028] Determining the image width of the target person image;

[0029] If the image width meets the first threshold, the target person image is put into the appearance image list; otherwise, the target person image is ignored;

[0030] When the number of images in the appearance image list meets a second threshold, feature extraction is performed on all the target person images in the appearance image list to obtain the identified appearance features.

[0031] Preferably, the method for acquiring the facial features to be recognized is:

[0032] Determine the image width of the target person image and whether there are extractable facial features in the target person image;

[0033] If the image width meets the third threshold and the extractable facial features exist, feature extraction is performed on the target person image to obtain the identified facial features and put them into the facial feature list; otherwise, the target person image is ignored.

[0034] Preferably, determining the first identity based on the identified appearance feature and the reference appearance feature specifically includes:

[0035] Based on all the identified appearance features, the Euclidean distances are calculated with the reference appearance features to obtain a first distance matrix;

[0036] Obtain a row of data with the smallest average value in the first distance matrix, where the row number is denoted as i;

[0037] When the average value of the i-th row of data is less than the fourth threshold value, and the minimum value of the i-th row of data is less than the fifth threshold value, the name of the person corresponding to the i-th reference appearance feature is selected as the first identity and marked;

[0038] Otherwise no marking is done.

[0039] Preferably, determining the second identity based on the recognized facial features and the reference facial features specifically includes:

[0040] When the number of the identified facial features in the facial feature list meets a sixth threshold, calculating the Euclidean distance between all the identified facial features in the facial feature list and the reference facial features to obtain a second distance matrix;

[0041] Obtain the row number where the minimum value of each column in the second distance matrix is ​​located, and obtain a minimum row number set;

[0042] Get the row number that appears most times in the minimum row number set and record it as j best ;

[0043] Get the j best The proportion in the minimum row number set is recorded as the statistical proportion;

[0044] Based on the j best Get the jth distance matrix in the second distance matrixbest The minimum value of the row is denoted as dist min ;

[0045] When the statistical ratio is less than the seventh threshold and the dist min When it is less than the eighth threshold, the jth best The name of the person corresponding to the reference facial features is used as the second identity and marked;

[0046] Otherwise no marking is done.

[0047] Preferably, determining the final person identity of the target person feature specifically includes:

[0048] Determine the identity of the target person based on the first identity and the second identity;

[0049] If the person identities of the first identity and the second identity identification mark are inconsistent, the person identity of the second identity identification mark is used as the final person identity of the target person feature;

[0050] If the first identity and the second identity do not recognize the identity of the marked person, the person is marked as a stranger.

[0051] A security video monitoring system based on hybrid feature recognition, comprising: a reference feature module, a video acquisition and analysis module, an identification and marking module, a feature extraction module, a first identity determination module, a second identity determination module and an identity marking module

[0052] The reference feature module is used to obtain multiple known person images and perform feature extraction to obtain reference appearance features and reference face features;

[0053] The video acquisition and analysis module is used to acquire and analyze the security video to be identified to obtain multiple frame images;

[0054] The identification and marking module is used to obtain target person features and their corresponding capture frame coordinates based on the frame image recognition; mark different target persons in the frame image based on the target person features to obtain multiple different person IDs;

[0055] The feature extraction module is used to intercept the corresponding target person image based on the frame image corresponding to the person ID; perform feature extraction based on the target person image respectively, and obtain the recognized appearance features and the recognized face features accordingly;

[0056] The first identity determination module is used to determine a first identity based on the identification appearance feature and the reference appearance feature;

[0057] The second identity determination module is used to determine a second identity based on the recognized facial features and the reference facial features;

[0058] The identity marking module is used to determine the final person identity of the target person feature based on the first identity and the second identity; and mark the capture frame determined by the final person identity and its corresponding capture frame coordinates into the corresponding frame image.

[0059] It can be seen from the above technical solution that, compared with the prior art, the present invention discloses a security video surveillance method and system based on hybrid feature recognition, which can track the movement trajectory of people appearing in the surveillance videos of various security scenes by marking the same ID for people determined to be the same target in different video frame images; and can efficiently and accurately identify the identity of people by combining appearance features and facial features, thereby effectively protecting various security issues in security scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0061] Figure 1 A flow chart of a security video monitoring method based on hybrid feature recognition provided by the present invention.

[0062] Figure 2 A flow chart of a method for determining a first identity based on identifying appearance features and reference appearance features provided by the present invention.

[0063] Figure 3 A flow chart of a method for determining a second identity based on identifying facial features and reference facial features provided by the present invention.

[0064] Figure 4 A schematic structural diagram of a security video monitoring system based on hybrid feature recognition provided by the present invention. DETAILED DESCRIPTION

[0065] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0066] Example 1

[0067] like Figure 1 As shown, the embodiment of the present invention discloses a security video monitoring method based on hybrid feature recognition, comprising:

[0068] Acquire multiple known person images and perform feature extraction to obtain baseline appearance features and baseline face features;

[0069] Obtain the security video to be identified and analyze it to obtain multiple frame images;

[0070] Obtain the target person's features and their corresponding capture frame coordinates based on frame image recognition;

[0071] Mark different target persons in the frame image based on the target person's features to obtain multiple different person IDs;

[0072] Based on the frame image corresponding to the character ID, the corresponding target character image is intercepted;

[0073] Based on the target person's image, feature extraction is performed respectively, and the appearance recognition feature and the face recognition feature are obtained accordingly;

[0074] Determining a first identity based on the recognized appearance feature and the baseline appearance feature;

[0075] Determining a second identity based on the recognized facial features and the reference facial features;

[0076] Determining a final person identity of the target person's characteristics based on the first identity and the second identity;

[0077] The capture frame determined by the final person identity and capture frame coordinates is marked in the corresponding frame image.

[0078] Example 2

[0079] The embodiment of the present invention discloses a preferred security video monitoring method based on hybrid feature recognition, comprising:

[0080] Get multiple known person images and perform feature extraction to obtain baseline appearance features and baseline facial features:

[0081] Preferably, obtaining the reference appearance features and reference face features specifically includes:

[0082] Obtain a preset number of known person images from which facial features can be extracted, and name the known person images accordingly;

[0083] Based on the known person image, the appearance features and the face features are extracted respectively, and the reference appearance features and the reference face features are obtained accordingly;

[0084] The baseline appearance features and baseline facial features are named after the corresponding characters.

[0085] Get the security video to be identified and parse it to get multiple frame images:

[0086] Preferably, the security video with a preset frame rate and a preset frame size is transmitted to the server in the form of a video stream, and the server is parsed to obtain a plurality of different frame images with preset sizes.

[0087] Based on frame image recognition, the target person features and their corresponding capture frame coordinates are obtained:

[0088] Preferably, obtaining the target person features and their corresponding capture frame coordinates specifically includes:

[0089] Based on the frame image, all target persons whose confidence level is greater than a preset value are captured to obtain the target person features and the capture frame coordinates corresponding to the target person features.

[0090] Different target persons in the frame image are marked based on the target person's features to obtain multiple different person IDs:

[0091] Preferably, a plurality of different character IDs are obtained, specifically including:

[0092] Based on the comparison between each target person feature in the current frame image and the target person feature in the previous frame image, the corresponding relationship between each target person feature and the target person feature in the previous frame image is determined;

[0093] Based on the corresponding relationship, different target persons in different frame images are marked, and multiple different person IDs are obtained accordingly, thereby realizing target person tracking.

[0094] Based on the frame image corresponding to the character ID, the corresponding target character image is captured:

[0095] Preferably, whenever a new character ID appears, the corresponding capture frame coordinates are determined according to the target character features corresponding to the character ID, and the target character image is captured in the corresponding frame image based on the capture frame coordinates.

[0096] Based on the target person's image, feature extraction is performed respectively, and the corresponding appearance recognition features and face recognition features are obtained:

[0097] Preferably, the method for obtaining the appearance feature is:

[0098] Determine the image width of the target person image;

[0099] If the image width meets the first threshold, the target person image is put into the appearance image list, otherwise the target person image is ignored;

[0100] When the number of images in the appearance image list meets the second threshold, feature extraction is performed on all target person images in the appearance image list to obtain recognized appearance features.

[0101] Preferably, the method for obtaining facial features is:

[0102] Determine the image width of the target person image and whether there are extractable facial features in the target person image;

[0103] If the image width meets the third threshold and there are extractable facial features, feature extraction is performed on the target person image to obtain the recognized facial features and put them into the facial feature list; otherwise, the target person image is ignored.

[0104] Determine the first identity based on the recognized appearance features and the baseline appearance features:

[0105] Preferably, Figure 2 As shown, determining the first identity based on the recognized appearance feature and the reference appearance feature specifically includes:

[0106] Based on all the recognized appearance features, the Euclidean distances are calculated with the reference appearance features to obtain a first distance matrix;

[0107] Get the row of data with the smallest average value in the first distance matrix, and record the row number as i;

[0108] When the average value of the i-th row of data is less than the fourth threshold value, and the minimum value of the i-th row of data is less than the fifth threshold value, the name of the person corresponding to the i-th reference appearance feature is selected as the first identity and marked;

[0109] Otherwise no marking is done.

[0110] Determine the second identity based on the recognized facial features and the baseline facial features:

[0111] Preferably, Figure 3 As shown, determining the second identity based on the recognized facial features and the reference facial features specifically includes:

[0112] When the number of recognized facial features in the facial feature list meets a sixth threshold, calculating the Euclidean distance between all recognized facial features in the facial feature list and the reference facial features to obtain a second distance matrix;

[0113] Get the row number of the minimum value of each column in the second distance matrix to obtain the minimum row number set;

[0114] Get the row number with the most occurrences in the minimum row number set and record it as j best ;

[0115] Get j bestThe proportion in the minimum row number set is recorded as the statistical proportion;

[0116] Based on j best Get the jth distance matrix in the second distance matrix best The minimum value of the row is denoted as dist min ;

[0117] When the statistical ratio is less than the seventh threshold and dist min When it is less than the eighth threshold, the jth best The name of the person corresponding to the benchmark facial features is used as the second identity and marked;

[0118] Otherwise no marking is done.

[0119] The final identity of the target person is determined based on the first identity and the second identity:

[0120] Preferably, determining the final person identity of the target person's characteristics specifically includes:

[0121] Determine the identity of the target person based on the first identity and the second identity;

[0122] If there is only the first identity identification mark character identity, the character identity of the first identity identification mark is used as the final character identity of the target character feature;

[0123] If there is only the second identity identification mark character identity, the character identity of the second identity identification mark is used as the final character identity of the target character feature;

[0124] If both the first identity and the second identity identify the person's identity and the person's identity of the identification mark is inconsistent, the person's identity of the second identity identification mark is used as the final person's identity of the target person's feature;

[0125] If neither the first identity nor the second identity recognizes the identity of the marked person, then the person is marked as a stranger.

[0126] Mark the capture frame determined by the final person identity and capture frame coordinates into the corresponding frame image:

[0127] Preferably, the final person identity and the capture frame determined by the capture frame coordinates corresponding to the person are jointly marked in the corresponding frame image, and after all the frame images are processed, they are spliced ​​in sequence into a complete marked security video.

[0128] Example 3

[0129] The embodiment of the present invention discloses another preferred security video monitoring method based on hybrid feature recognition, comprising:

[0130] Get multiple known person images and perform feature extraction to obtain baseline appearance features and baseline facial features:

[0131] Preferably, the specific implementation method of this embodiment is:

[0132] Get full-body images of N known people (X1, X2, ..., X N ) as appearance pictures and put them into the appearance image library, and name the pictures with the names of the people, read the file names of N appearance pictures, that is, the names of the people (NR1, NR2, ..., NR N ), saved on hard disk;

[0133] Take N full-body images of known people (X1, X2, ..., X N ) is input into the appearance feature extraction model for feature extraction, and N appearance features (R1, R2, …, R N ) as the baseline appearance features and saved on the hard disk;

[0134] Get full-length or half-length photos of N known people with clear faces (Y1, Y2, ..., Y N ) as a face image into the face image library, and name the image after the person's name, read the file name of N face images, that is, the person's name (NP1, NP2, ..., NP N ), saved on hard disk;

[0135] Take the full-length or half-length photos of these N known people (Y1, Y2, ..., Y N ) is input into the face feature extraction model for feature extraction, and N face features (F1, F2, ..., F N ) as the benchmark facial features and save them on the hard disk.

[0136] Get the security video to be identified and parse it to get multiple frame images:

[0137] Preferably, the specific implementation method of this embodiment is:

[0138] Obtain security video with a frame rate of 30 frames per second and a frame size of 1080×1920 and transmit it to the server in the form of video stream;

[0139] Based on the server, multiple frame images with a size of 1080×1920 are parsed.

[0140] Based on frame image recognition, the target person features and their corresponding capture frame coordinates are obtained:

[0141] Preferably, the specific implementation method of this embodiment is:

[0142] Input all frame images frame into the pre-trained target recognition model Y;

[0143] After being processed by the target recognition model Y, all target persons (P1, P2, ..., P i ) to capture and output the corresponding target person features and their corresponding capture frame coordinates. The target person P i The characteristic representation is T i , the corresponding capture frame coordinates are expressed as (x, y, w, h) i .

[0144] Different target persons in the frame image are marked based on the target person's features to obtain multiple different person IDs:

[0145] Preferably, the specific implementation method of this embodiment is:

[0146] A series of target person features (T1, T2, ..., T i ) Input tracking model Tracker;

[0147] The tracking model Tracker compares the features of each target person in the current frame image with the features of the target person in the previous frame image to determine the corresponding relationship between each target person feature and the features of the target person in the previous frame image;

[0148] Based on the corresponding relationship, different target persons in different frame images are marked, and multiple different person IDs are obtained accordingly, thereby realizing target person tracking.

[0149] Based on the frame image corresponding to the character ID, the corresponding target character image is captured:

[0150] Preferably, the specific implementation method of this embodiment is:

[0151] Whenever a new person ID appears, the corresponding capture frame coordinates are determined according to the target person features corresponding to the person ID, and the target person image is captured in the corresponding frame image based on the capture frame coordinates.

[0152] Based on the target person's image, feature extraction is performed respectively, and the corresponding appearance recognition features and face recognition features are obtained:

[0153] Preferably, the method for obtaining the appearance feature recognition in this embodiment is:

[0154] Determine the image width of the target person image;

[0155] If the image width meets the first threshold m, the target person image is put into the appearance image list List reid , otherwise the target person image is ignored;

[0156] When the number of images in the appearance image list meets the second threshold 40, the appearance image list Listreid All target person images in the input appearance feature extraction model Model reid Perform feature extraction and obtain 40 identification features (r1, r2, ..., r 40 ).

[0157] Preferably, the method for acquiring facial features for recognition in this embodiment is:

[0158] Determine the image width of the target person image and whether there are extractable facial features in the target person image;

[0159] If the image width meets the third threshold n and there are extractable facial features, the target person image is input into the facial feature extraction model Model face Perform feature extraction to obtain the facial feature f and put it into the facial feature list List face , otherwise the target person image is ignored.

[0160] Determine the first identity based on the recognized appearance features and the baseline appearance features:

[0161] Preferably, in this embodiment, the first identity is determined based on the recognized appearance feature and the reference appearance feature, and the specific method is as follows:

[0162] N recognition features (r1, r2, ..., r 40 ) are respectively related to the reference appearance features (R1, R2, …, R N ) Calculate the Euclidean distance and get the first distance matrix Dist1 (N×40) ;

[0163] Get the first distance matrix Dist (N×40) The row of data with the smallest average value is denoted by row number i (i=1,2,…,40);

[0164] When the average value of the i-th row of data is less than the fourth threshold avg , and the minimum value of the i-th row of data is less than the fifth threshold min , then in (NR1,NR2,…,NR N ) Select the name NR of the person corresponding to the i-th benchmark appearance feature i As the first identity and mark it, the target person's appearance mark is set to: "Appearance:" + NR i ;

[0165] Otherwise no marking is done.

[0166] Determine the second identity based on the recognized facial features and the baseline facial features:

[0167] Preferably, in this embodiment, the second identity is determined based on the recognition of facial features and the reference facial features, and the specific method is as follows:

[0168] When the face feature list face When the number of recognized facial features in meets the sixth threshold of 20, List face contains 20 facial features (f1, f2, ..., f 20 );

[0169] List the facial features face All the recognized face features (f1,f2,…,f 20 ) are respectively compared with the benchmark face features (F1, F2, ..., F N ) Calculate the Euclidean distance and get the second distance matrix Dist2 (N×20) ;

[0170] Get the second distance matrix Dist2 (N×20) The row number of the minimum value in each column is obtained, and the minimum row number set (j1, j2,,…, j 20 );

[0171] Get the minimum row number set (j1, j2,,…,j 20 ) is recorded as j best ;

[0172] Get j best In the minimum row number set (j1,j2,,…,j 20 ) is recorded as the statistical proportion percentage;

[0173] Based on obtaining the second distance matrix Dist2 (N×20) J best The minimum value of the row is denoted as dist min ;

[0174] When the statistical proportion percentage is less than the seventh threshold and dist min When it is less than the eighth threshold, then in (NP1, NP2,…, NP N ) select the jth best The name of the person corresponding to the benchmark facial features NP jbest As the second identity and mark it, set the target person's facial mark to "Face:" + NP jbest ;

[0175] Otherwise no marking is done.

[0176] The final identity of the target person is determined based on the first identity and the second identity:

[0177] Preferably, the present embodiment determines the final person identity of the target person feature by:

[0178] Determine the person identity of the target person feature based on the marker of the first identity and the marker of the second identity;

[0179] If only the first identity marker identifies the person's identity, the person's identity of the first identity marker is used as the final person's identity of the target person's feature;

[0180] If only the second identity marker identifies the person's identity, the person's identity of the second identity marker is used as the final person's identity of the target person's feature;

[0181] If both the first identity marker and the second identity marker identify the identity of the person and the identity of the person of the identification marker is inconsistent, the identity of the person identified by the second identity marker is used as the final identity of the target person feature;

[0182] If neither the first identity marker nor the second identity marker can identify the person, the person is marked as a stranger.

[0183] Mark the capture frame determined by the final person identity and capture frame coordinates into the corresponding frame image:

[0184] Preferably, the specific implementation method of this embodiment is:

[0185] The final person identity and the capture frame determined by its corresponding capture frame coordinates are jointly marked in the corresponding frame image. After processing all the frame images, they are spliced ​​in sequence into a complete marked security video, completing the effective identification and marking of the person identity in the security video, which can effectively protect various security issues in the security scene.

[0186] Example 4

[0187] like Figure 4 As shown, a security video monitoring system based on hybrid feature recognition includes: a reference feature module, a video acquisition and analysis module, an identification and marking module, a feature extraction module, a first identity determination module, a second identity determination module and an identity marking module

[0188] A reference feature module is used to obtain multiple known person images and perform feature extraction to obtain reference appearance features and reference face features;

[0189] The video acquisition and analysis module is used to acquire the security video to be identified and analyze it to obtain multiple frame images;

[0190] The recognition and marking module is used to obtain the target person features and the corresponding capture frame coordinates based on the frame image recognition; mark different target persons in the frame image based on the target person features to obtain multiple different person IDs;

[0191] The feature extraction module is used to intercept the corresponding target person image based on the frame image corresponding to the person ID; perform feature extraction based on the target person image, and obtain the recognized appearance features and the recognized face features accordingly;

[0192] A first identity determination module, configured to determine a first identity based on the recognized appearance feature and the reference appearance feature;

[0193] A second identity determination module, used to determine a second identity based on the recognized facial features and the reference facial features;

[0194] The identity marking module is used to determine the final person identity of the target person's features based on the first identity and the second identity; and mark the capture frame determined by the final person identity and its corresponding capture frame coordinates into the security video.

[0195] It can be seen from the above technical solution that, compared with the prior art, the present invention discloses a security video surveillance method and system based on hybrid feature recognition, which can track the movement trajectory of people appearing in the surveillance videos of various security scenes by marking the same ID for people determined to be the same target in different video frame images; and can efficiently and accurately identify the identity of people by combining appearance features and facial features, thereby effectively protecting various security issues in security scenes.

[0196] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0197] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A security video monitoring method based on hybrid feature recognition, characterized in that: include: Acquire multiple known person images and perform feature extraction to obtain baseline appearance features and baseline face features; Obtain the security video to be identified and parse it to obtain multiple frame images; Obtaining target person features and their corresponding capture frame coordinates based on the frame image recognition; Marking different target persons in the frame image based on the target person features to obtain a plurality of different person IDs; Based on the frame image corresponding to the character ID, a corresponding target character image is captured; Based on the target person image, feature extraction is performed respectively to obtain the recognized appearance features and the recognized face features; The method for obtaining the facial features to be identified is: Determine the image width of the target person image and whether there are extractable facial features in the target person image; If the image width satisfies the third threshold value and the extractable facial features exist, feature extraction is performed on the target person image to obtain the identified facial features and put them into the facial feature list; otherwise, the target person image is ignored; Determining a first identity based on the identified appearance feature and the reference appearance feature; Determining a second identity based on the identified facial features and the reference facial features; Specifically include: When the number of the identified facial features in the facial feature list meets a sixth threshold, calculating the Euclidean distance between all the identified facial features in the facial feature list and the reference facial features to obtain a second distance matrix; Obtain the row number where the minimum value of each column in the second distance matrix is ​​located, and obtain a minimum row number set; Get the row number that appears most times in the minimum row number set and record it as j best ; Get the j best The proportion in the minimum row number set is recorded as the statistical proportion; Based on the j best Get the jth distance matrix in the second distance matrix best The minimum value of the row is denoted as dist min ; When the statistical ratio is less than the seventh threshold and the dist min When it is less than the eighth threshold, the jth best The name of the person corresponding to the reference facial features is used as the second identity and marked; Otherwise, no marking is done; Determine a final person identity of the target person feature based on the first identity and the second identity; If only the first identity marker identifies the person's identity, the person's identity of the first identity marker is used as the final person's identity of the target person's feature; The final person identity and the capture frame determined by the corresponding capture frame coordinates are marked in the corresponding frame image.

2. A security video monitoring method based on hybrid feature recognition according to claim 1, characterized in that: Obtain the baseline appearance features and baseline facial features, including: Acquire a preset number of known person images from which facial features can be extracted, and name the known person images accordingly; Based on the known person image, appearance features and facial features are extracted respectively to obtain the reference appearance features and reference facial features; The reference appearance features and the reference facial features are both named after the corresponding characters.

3. A security video monitoring method based on hybrid feature recognition according to claim 1, characterized in that: Get the target person features and their corresponding capture frame coordinates, including: Based on the frame image, all target persons whose confidence levels are greater than a preset value are captured to obtain target person features and capture frame coordinates corresponding to the target person features.

4. The security video monitoring method based on hybrid feature recognition according to claim 1 is characterized in that: Get multiple different character IDs, including: Determine the corresponding relationship based on comparing the target person features in the current frame image with the target person features in the previous frame image; Based on the corresponding relationship, different target persons in different frame images are marked to obtain a plurality of different person IDs.

5. The security video monitoring method based on hybrid feature recognition according to claim 2 is characterized in that: The method for obtaining the identification appearance feature is: Determining the image width of the target person image; If the image width meets the first threshold, the target person image is put into the appearance image list; otherwise, the target person image is ignored; When the number of images in the appearance image list meets a second threshold, feature extraction is performed on all the target person images in the appearance image list to obtain the identified appearance features.

6. A security video monitoring method based on hybrid feature recognition according to claim 5, characterized in that: Determining the first identity based on the identified appearance feature and the reference appearance feature specifically includes: Based on all the identified appearance features, the Euclidean distances are calculated with the reference appearance features to obtain a first distance matrix; Obtain a row of data with the smallest average value in the first distance matrix, where the row number is denoted as i; When the average value of the i-th row of data is less than the fourth threshold value, and the minimum value of the i-th row of data is less than the fifth threshold value, the name of the person corresponding to the i-th reference appearance feature is selected as the first identity and marked; Otherwise no marking is done.

7. The security video monitoring method based on hybrid feature recognition according to claim 1 is characterized in that: Determining the final person identity of the target person's characteristics specifically includes: Determine the identity of the target person based on the first identity and the second identity; If the person identities of the first identity and the second identity identification mark are inconsistent, the person identity of the second identity identification mark is used as the final person identity of the target person feature; If the first identity and the second identity do not recognize the identity of the marked person, the person is marked as a stranger.

8. A security video monitoring system based on hybrid feature recognition, characterized in that: include: Benchmark feature module, video acquisition and analysis module, identification and marking module, feature extraction module, first identity determination module, second identity determination module and identity marking module The reference feature module is used to obtain multiple known person images and perform feature extraction to obtain reference appearance features and reference face features; The video acquisition and analysis module is used to acquire and analyze the security video to be identified to obtain multiple frame images; The identification and marking module is used to obtain target person features and their corresponding capture frame coordinates based on the frame image recognition; Marking different target persons in the frame image based on the target person features to obtain a plurality of different person IDs; The feature extraction module is used to intercept the corresponding target person image based on the frame image corresponding to the person ID; perform feature extraction based on the target person image respectively, and obtain the recognized appearance features and the recognized face features accordingly; The method for obtaining the facial features to be identified is: Determine the image width of the target person image and whether there are extractable facial features in the target person image; If the image width satisfies the third threshold value and the extractable facial features exist, feature extraction is performed on the target person image to obtain the identified facial features and put them into the facial feature list; otherwise, the target person image is ignored; The first identity determination module is used to determine a first identity based on the identification appearance feature and the reference appearance feature; The second identity determination module is used to determine a second identity based on the recognized facial features and the reference facial features; Specifically include: When the number of the identified facial features in the facial feature list meets a sixth threshold, calculating the Euclidean distance between all the identified facial features in the facial feature list and the reference facial features to obtain a second distance matrix; Obtain the row number where the minimum value of each column in the second distance matrix is ​​located, and obtain a minimum row number set; Get the row number that appears most times in the minimum row number set and record it as j best ; Get the j best The proportion in the minimum row number set is recorded as the statistical proportion; Based on the j best Get the jth distance matrix in the second distance matrix best The minimum value of the row is denoted as dist min ; When the statistical ratio is less than the seventh threshold and the dist min When it is less than the eighth threshold, the jth best The name of the person corresponding to the reference facial features is used as the second identity and marked; Otherwise, no marking is done; The identity marking module is used to determine the final person identity of the target person feature based on the first identity and the second identity; if only the first identity mark can identify the person identity, the person identity of the first identity recognition mark is used as the final person identity of the target person feature; and the capture frame determined by the final person identity and its corresponding capture frame coordinates is marked in the corresponding frame image.

Images