[0030] In the following description, many technical details are proposed for the reader to better understand this application. However, those of ordinary skill in the art can understand that even without these technical details and various changes and modifications based on the following embodiments, the technical solutions required by the claims of this application can be implemented.
[0031] In order to make the objectives, technical solutions and advantages of the present invention clearer, the embodiments of the present invention will be further described in detail below in conjunction with the accompanying drawings.
[0032] The first embodiment of the present invention relates to a method for automatic target discovery and registration in surveillance video. figure 1 It is a schematic flow diagram of the automatic target discovery and registration method in the surveillance video.
[0033] Specifically, such as figure 1 As shown, the method for automatic target discovery and registration in the surveillance video includes the following steps:
[0034] In step 101, the target is detected from the video stream.
[0035] The system collects video and converts the video signal into a digital signal that can be analyzed through the sensor. The system performs target detection on the digital video signal to facilitate further processing of the system.
[0036] In order to better understand the present invention, as a preferred implementation of the present invention, the following will use the real-time face comparison application as an implementation of the present invention to detail the complete process of system implementation, but it is not used to limit the present invention. invention.
[0037] The image collection and target detection of the system are realized by a face capture machine. The face capture machine is an IP camera with a face capture function, which specifically realizes real-time detection and tracking of faces from the captured video stream, and selects one of the same person. Upload pictures with the best quality. It has the advantages of front-end analysis to ensure real-time performance and reduce the amount of data transmission. At the same time, the camera exposure parameters can be automatically adjusted according to the target, so that a better target image can be obtained. Specifically, face detection can be The Boosting method with Haar features is adopted. Face tracking can be realized by the optical flow method. Face selection can include comprehensive evaluation of two indicators of face image clarity and posture.
[0038] After that, step 102 is entered to perform feature extraction on the detected target to obtain target features.
[0039] This step includes the necessary steps of target image preprocessing, illumination and geometric normalization, feature extraction and conversion necessary for feature extraction.
[0040] For example, the feature extraction step of the face is implemented as follows. First, locate the binocular coordinates in the face image, and then align the binocular coordinates to the coordinates (48, 48) and (112, 48) to obtain the geometrically normalized Then use the entropy image method to normalize the illumination of the face image, and then extract the selected LBP features and Gabor features (the method of feature selection is to use the method of Boosting to select the features that are most conducive to face classification) Finally, the extracted features are further reduced by using the method of Metric Learning to perform further dimension reduction and measurement space conversion, so that the features are most suitable for calculating the similarity of two face images.
[0041] The present invention does not specifically limit the specific implementation technology of target feature extraction and similarity comparison. There are two main reasons: one is that the feature extraction and comparison technology belongs to the existing real-time comparison content, and the other is related feature extraction. There are many variants of the comparison algorithm. For example: This article uses Gabor+LBP composite features to characterize human faces, and uses Metric Learning to obtain a metric matrix to improve the Mahalanobis distance to calculate face similarity.
[0042] Preferably, the detected targets include vehicles and human faces, but are not limited to this. In some other embodiments of the present invention, they may also include the discovery and automatic registration of other targets such as human body, gait, and animals.
[0043] In addition, it can be understood that the target feature obtained by feature extraction of the detected target can be one, such as a license plate number, or a combination of multiple, such as a combination of multiple features in a face, a combination of license plate and vehicle color and many more.
[0044] After that, step 103 is entered to determine whether the target feature already exists in the target list.
[0045] If yes, go to step 104; if not, go to step 105.
[0046] There are two ways to judge whether the target is in the target list or the alarm list: one is to extract target features and calculate the similarity and compare with the set threshold; the other is to change the face to the target If it is a license plate, it can be accurately described (equivalent to features) through license plate recognition technology, and the comparison can be converted to a look-up table to determine whether it is in the target list or the alarm list.
[0047] In step 104, the number of occurrences of the target feature is increased by one.
[0048] Then go to step 106.
[0049] In step 105, the target feature is added to the target list, and the number of occurrences of the target feature is set to 1.
[0050] Then go to step 106.
[0051] Determine whether the target feature already exists in the target list, if yes, add 1 to the number of occurrences of the target feature; if not, add the target feature to the target list, and add the target feature The number of occurrences is set to 1.
[0052] In step 106, it is determined whether the number of occurrences of the target feature exceeds a preset threshold.
[0053] If it exceeds, go to step 107; if not, go back to step 101 again.
[0054] It should be noted that the number of times threshold can be preset in the system, and generally 3-10 times are appropriate.
[0055] In step 107, the target feature is registered in the alarm list.
[0056] This process ends thereafter.
[0057] Target feature registration to the alarm list database can choose real-time registration, that is, once the number of repetitions reaches the upper limit, it will be registered in the alarm list in real time. You can also choose non-real-time registration, that is, set the upper limit of the number of repetitions within the time period T, and accumulate the number of repetitions of all target objects within the period of time. When the time period T is reached, all the conditions are met (the number of repetitions exceeds the upper limit) The target object is registered to the alarm list database, and the system implementation method is convenient and flexible.
[0058] Steps 101 and 102 are the same steps as the general real-time comparison system in the prior art, followed by steps different from the general real-time comparison system, that is, the key target is added to the real-time comparison system. Automatic discovery and alarm list registration.
[0059] On the basis of the real-time comparison system, the recurring key targets are automatically discovered and automatically registered in the alarm list as the object of subsequent key analysis and processing, making the general real-time comparison system more intelligent and practical.
[0060] The second embodiment of the present invention relates to a method for automatic target discovery and registration in surveillance video. figure 2 It is a schematic flow diagram of the automatic target discovery and registration method in the surveillance video.
[0061] The second embodiment is improved on the basis of the first embodiment, specifically, such as figure 2 As shown, the main improvements are:
[0062] First, before registering the target feature in the alarm list in step 107, it also includes:
[0063] Step 108: Determine whether the target feature already exists in the alarm list.
[0064] If not, go to step 107; if yes, then end this process.
[0065] Step 108 confirms whether the target has been registered in the alarm list. If the comparison similarity with all registered pictures in the alarm list is lower than the set threshold, that is, the target feature does not exist, step 107 is executed to register the target feature in the alarm list.
[0066] Secondly, after registering the target feature in the alarm list in step 107, it also includes:
[0067] Step 109: Delete the record of the target feature in the target list.
[0068] After the target object is registered in the alarm list database, deleting the information of the target object in the target list and updating the target list can further save the storage space in the target list and reduce the workload of subsequent comparison.
[0069] To update the target list, you can choose real-time update, that is, when the target feature is registered in the alarm list database, the information of the target feature in the target list is deleted in time; you can also choose non-real-time registration, for example, you can use a timer mechanism to follow a certain The time interval is updated.
[0070] The specific steps to update the target list using the timer mechanism are as follows:
[0071] Step 1. Start the timer.
[0072] Step 2. The timer reaches the set update interval.
[0073] Step 3. The system removes the target features whose existence time reaches the set value from the target list.
[0074] As an implementation of the present invention, the trigger time interval of the timer for updating the target waiting list can be set to 1 minute, and the clearing time of the target feature can be set in the system as needed, for example, it can be set to 30 minutes Or other, but not limited to this.
[0075] The third embodiment of the present invention relates to a method for automatic target discovery and registration in surveillance video. image 3 It is a schematic flow diagram of the automatic target discovery and registration method in the surveillance video.
[0076] The third embodiment is improved on the basis of the first embodiment, specifically, such as image 3 As shown, the main improvements are:
[0077] First, before determining whether the target feature already exists in the target list in step 103, it also includes:
[0078] Step 108: Determine whether the target feature already exists in the alarm list.
[0079] If not, go to step 103; if yes, then end this process.
[0080] Step 108 confirms whether the target has been registered in the alarm list. If the comparison similarity with all registered pictures in the alarm list is lower than the set threshold, that is, the target feature does not exist, then step 103 is executed to determine whether the target feature already exists in the target list.
[0081] Secondly, after registering the target feature in the alarm list in step 107, it also includes:
[0082] Step 109: Delete the record of the target feature in the target list.
[0083] After the target object is registered in the alarm list database, deleting the information of the target object in the target list and updating the target list can further save the storage space in the target list and reduce the workload of subsequent comparison.
[0084] The fourth embodiment of the present invention relates to a method for automatic target discovery and registration in surveillance video. Figure 4 It is a schematic flow diagram of the automatic target discovery and registration method in the surveillance video.
[0085] Specifically, such as Figure 4 As shown, including the following steps:
[0086] Step 1. The system needs to collect video, and convert the video signal into a digital signal that can be analyzed by the sensor.
[0087] Step 2. The system performs target detection on the digital video signal to facilitate further processing of the system.
[0088] Step 3. The system performs feature extraction on the target. This step includes the necessary steps such as target image preprocessing, illumination and geometric normalization, feature extraction and conversion necessary for feature extraction.
[0089] Steps 1 to 3 are the same steps as the general real-time comparison system, followed by steps different from the general real-time comparison system, that is, on the basis of the real-time comparison system, the automatic discovery and alarm list of key figures is added Registered branch process.
[0090] Real-time comparison alarm process branch:
[0091] Step 4: Compare the extracted features with the alarm list one by one, and judge that the comparison similarity reaches the set threshold.
[0092] Step 5. The comparison similarity reaches the set threshold, and the system issues a real-time comparison alarm.
[0093] Automatic discovery of key figures and registration branch of the alarm list:
[0094] Step 4: Perform a one-to-one comparison between the extracted features and the recurring target. When the presence comparison similarity reaches the set recurring confirmation threshold, the target is considered to be recurring.
[0095] It should be noted that the extracted target object is compared with the recurring target for similarity, and there is no specific limitation on "what feature and matching algorithm to use".
[0096] Step 5: Confirm that the target reappears, then accumulate the reappearance counter of the target, and judge whether the accumulating count reaches the set number of times. If there are multiple comparison similarities between the results of the one-to-one comparison with the target list that reach the threshold, only the target with the highest similarity is counted up.
[0097] Step 6. The counter of the target in the target list is accumulated to the set number of times, then it is compared with the alarm list one by one to confirm whether the target has been registered.
[0098] Step 7. If the comparison similarity with all registered pictures in the alarm list is lower than the set confirmation threshold, the target is not registered.
[0099] Step 8. Register the confirmed unregistered targets to the alarm list.
[0100] Steps 5 to 8, specifically, such as Figure 5 Shown.
[0101] Registering to the alarm list further includes: if the comparison similarity with all registered pictures in the alarm list is lower than the set confirmation threshold, the target is not registered; and the confirmed unregistered target is registered to the alarm list.
[0102] The above "target objects exceeding the number of repetitions are registered to the alarm list in real time", and you can also choose to register to the alarm list in non-real-time, for example: set a time period T, for the target objects that exceed the number of repetitions within T Register to the alarm list.
[0103] It should be noted, Figure 4 The alarm list real-time comparison process and the automatic registration alarm list process shown in can be run on one processor or two processors separately.
[0104] As a preferred embodiment, the present invention adopts two processes to run separately, mainly considering that the existing single processor system has limited real-time processing capabilities, and the completion of the two processes in one processor at the same time will affect the processor’s performance.
[0105] It can be seen from the above description that the core of the present invention lies in the automatic discovery and automatic registration of key targets, which can be summarized as achieved through two sets of alarm lists and three comparison methods, as follows:
[0106] Two sets of alarm lists: ①The alarm list used to compare the alarm; ②The target list used to update the alarm list (that is, the target list that appears in the system statistics time period).
[0107] Three comparisons: ①Real-time comparison alarm; ②Confirm whether it is repeated through comparison; ③Confirm whether it has been registered through comparison.
[0108] The fifth embodiment of the present invention relates to a method for automatic target discovery and registration in surveillance video. Image 6 It is a schematic flow diagram of the automatic target discovery and registration method in the surveillance video.
[0109] Specifically, such as Image 6 As shown, it can be seen that compared with the fourth embodiment, the main difference between this embodiment and the fourth embodiment is that the registration order of the alarm list and the database is inconsistent.
[0110] Similarly, in this embodiment, the target object can be registered in the alarm list database in real time, that is, once the number of repetitions reaches the upper limit, it can be registered in the alarm list in real time; it can also be non-real-time registration, that is, within the set time period T The upper limit of the number of repetitions is to accumulate the number of reappearances of all target objects within the period of time. When the time period T is reached, all eligible (target objects whose repetition times exceed the upper limit) are registered to the alarm list database.
[0111] The update of the recurring target to be compared list is updated at a certain time interval using the timer mechanism. The specific steps are as follows:
[0112] Step 1. The system starts the timer.
[0113] Step 2. The timer reaches the set update interval.
[0114] Step 3. The system clears the target that the existence time reaches the set value from the list of recurring targets to be compared.
[0115] In addition, for the update of the recurring target list, you can choose to update in real time, which is different from the above-mentioned "update at a certain time interval", that is, in the above embodiments, the target object is deleted in time when it is registered in the alarm list database. The information in the list to be compared appears repeatedly.
[0116] In order to better understand the present invention, the following will use the real-time face comparison application as an implementation of the present invention to detail the complete process of the system implementation, but it is not used to limit the present invention. Other biological characteristics such as human body and gait are used. Person target discovery and automatic registration should also belong to the scope of protection of the present invention, and the same should be applied to the discovery and automatic registration of other targets such as vehicles (using license plate, model, color, etc.), animals (using features such as coat color, shape, etc.) It belongs to the protection category of the present invention.
[0117] As an implementation of the present invention, the two steps of image acquisition and target detection of the system are realized by a face capture machine. The face capture machine is an IP camera with a face capture function, which specifically realizes real-time from the captured video stream Detect and track human faces, and upload a picture of the same person with the best quality. It has the advantages of front-end analysis to ensure real-time performance and reduce the amount of data transmission. At the same time, the camera exposure parameters can be automatically adjusted according to the target, so that more The best target image, specifically, the face detection is realized by the Boosting method of Haar feature, the face tracking is realized by the optical flow method, and the face selection includes the comprehensive evaluation of the clarity and posture of the face image.
[0118] As an implementation of the present invention, the feature extraction step of the face is implemented as follows: first locate the binocular coordinates in the face image, and then align the binocular coordinates to the coordinates (48, 48) and (112, 48) , Get the geometrically normalized face image, and then use the entropy image method to normalize the illumination, and then extract the selected LBP features and Gabor features (the feature selection method is to use the Boosting method to select the most beneficial Realize the features of face classification), and finally adopt the method of Metric Learning to further reduce the dimension of the features and transform the measurement space, making the features most suitable for calculating the similarity of two face images.
[0119] As an implementation of the present invention, the Mahalanobis distance is used in the similarity calculation step in the three real-time comparisons. However, since the Metric Learning method has been used in the feature extraction step to perform measurement space conversion, the distance can effectively measure the two The similarity of two face images, and the advantage of simple calculation can be obtained.
[0120] As an implementation of the present invention, the real-time comparison alarm threshold can be set in the system, and is usually set to 0.70.
[0121] As an implementation of the present invention, the same value is used for the repeated occurrence confirmation and the confirmation threshold of the registered alarm list. This value can also be set in the system, but it is usually set slightly higher than the real-time comparison alarm threshold. Such as 0.80.
[0122] As an implementation of the present invention, the threshold of the number of times of the recurrence counter can be set in the system, and generally 3-10 times are appropriate.
[0123] As an implementation of the present invention, the trigger event interval of the timer for maintaining the to-be-compared list of recurring targets is set to 1 minute, and the clearing time of the recurring target is set in the system as required, for example, it can be set Set as 30 minutes or other.
[0124] The invention has good application prospects. For example, it can be applied to service places such as automobile 4S shops, real estate agencies, etc. to find potential users (people who buy large items usually appear for inquiries), and applied to sensitive places such as banks, vaults, or schools, stations Suspicious persons are found in public places (the suspect usually appears multiple times). The application methods listed above are only used to help understand the present invention, not to limit the present invention. Other applications that conform to the present invention should also belong to the present invention. Protection category.
[0125] The method implementation manners of the present invention can all be implemented in software, hardware, firmware, and the like. Regardless of whether the present invention is implemented in software, hardware, or firmware, the instruction code can be stored in any type of computer-accessible memory (for example, permanent or modifiable, volatile or non-volatile, solid state Or non-solid, fixed or replaceable media, etc.). Similarly, the memory may be, for example, programmable array logic (Programmable Array Logic, "PAL"), random access memory (Random Access Memory, "RAM"), and programmable read-only memory (Programmable Read Only Memory, "PROM" for short). "), read-only memory (Read-Only Memory, "ROM"), electrically erasable programmable read-only memory (Electrically Erasable Programmable ROM, "EEPROM"), magnetic disks, optical discs, digital versatile discs (Digital Versatile Disc , Referred to as "DVD") and so on.
[0126] The sixth embodiment of the present invention relates to an automatic target discovery and registration system in surveillance video, such as Figure 7 As shown, the automatic target discovery and registration system in the surveillance video includes:
[0127] The detection unit is used to detect the target from the video stream.
[0128] The target feature extraction unit is used to perform feature extraction on the target detected by the detection unit to obtain the target feature.
[0129] The first judging unit is used to judge whether the target feature already exists in the target list.
[0130] The counting unit is used for adding 1 to the number of occurrences of the target feature when the first determining unit determines that the target feature already exists in the target list. When the first determining unit determines that the target feature does not exist in the target list, the target feature is added to the target list, and the number of occurrences of the target feature is set to 1.
[0131] The second judging unit is used to judge whether the number of occurrences of the target feature output by the counting unit exceeds a preset threshold.
[0132] The registration unit is used for registering the target feature in the alarm list when the second judging unit judges that the number of occurrences of the target feature exceeds a preset threshold.
[0133] The first embodiment is a method embodiment corresponding to this embodiment, and this embodiment can be implemented in cooperation with the first embodiment. The related technical details mentioned in the first embodiment are still valid in this embodiment, and in order to reduce repetition, they will not be repeated here. Correspondingly, the related technical details mentioned in this embodiment can also be applied in the first embodiment.
[0134] The seventh embodiment of the present invention relates to a system for automatic target discovery and registration in surveillance video.
[0135] The seventh embodiment is improved on the basis of the sixth embodiment, and the main improvements are that it also includes:
[0136] The third judgment unit is used for judging whether the target feature already exists in the alarm list before the registration unit registers the target feature in the alarm list.
[0137] The update unit is used for deleting the record of the target feature in the target list after the registration unit registers the target feature in the alarm list.
[0138] The timing unit is used to set the working time interval between the registration unit and the update unit.
[0139] The second, third, fourth, and fifth embodiments are method embodiments corresponding to this embodiment, and this embodiment can be implemented in cooperation with the second, third, fourth, and fifth embodiments. The related technical details mentioned in the second, third, fourth, and fifth implementation manners are still valid in this implementation manner. In order to reduce repetition, they will not be repeated here. Correspondingly, the related technical details mentioned in this embodiment can also be applied in the second, third, fourth, and fifth embodiments.
[0140] It should be noted that each unit mentioned in each device embodiment of the present invention is a logical unit. Physically, a logical unit can be a physical unit, a part of a physical unit, or multiple physical units. The combination of units is realized, and the physical realization of these logical units themselves is not the most important. The combination of the functions implemented by these logical units is the key to solving the technical problems proposed by the present invention. In addition, in order to highlight the innovative part of the present invention, the foregoing device embodiments of the present invention do not introduce units that are not closely related to solving the technical problems proposed by the present invention. This does not mean that there are no other devices in the foregoing device embodiments. unit.
[0141] It should be noted that in the claims and specification of this patent, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or It implies that there is any such actual relationship or order between these entities or operations. Moreover, the terms "include", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes those that are not explicitly listed Other elements of, or also include elements inherent to this process, method, article or equipment. If there are no more restrictions, the element defined by the sentence "including one" does not exclude the existence of other same elements in the process, method, article, or equipment including the element.
[0142] Although the present invention has been illustrated and described by referring to certain preferred embodiments of the present invention, those of ordinary skill in the art should understand that various changes can be made in form and details without departing from the present invention. The spirit and scope of the invention.
