Complex indoor scene personnel detection system and method based on image difference and classification

Abstract

The invention relates to the technical field of image difference and classification detection, in particular to a complex indoor scene personnel detection system and method based on image difference and classification, and the system comprises sequence personnel detection, differential motion discrimination and static target classification, the output end of the sequence personnel detection is electrically connected with the input end of the differential motion discrimination, and the output end of the differential motion discrimination is electrically connected with the output end of the static target classification. The output end of the differential motion discriminator is electrically connected with the input end of the static target classifier; through step-by-step differentiation of personnel detection tasks in a complex scene and through combination of deep learning algorithms such as yov7 target detection, image difference and OfficientNet image classification, a coarse-to-fine progressive human body recognition architecture is designed and completed, and a universal, high-accuracy, efficient and reliable personnel intelligent detection system under a complex indoor scene is realized. Reliable personnel positioning basic information is provided for an indoor intelligent monitoring system in difficult scenes such as poor monitoring conditions and unstable data transmission quality.

Description

Technical Field

[0001] The present invention relates to the technical field of image difference and classification detection, and in particular to a complex indoor scene personnel detection system and method based on image difference and classification. Background Art

[0002] Indoor personnel detection is difficult in real indoor scenes due to problems such as incomplete camera acquisition area, low camera resolution, indoor building occlusion, and wide range of personnel activities. Low-resolution indoor personnel detection is used to improve the accuracy of indoor personnel detection and the accuracy of intelligent management of indoor places. Complex indoor scene personnel detection system refers to the use of machine vision technology to locate and identify people in complex indoor scenes using digital images and video sequences collected by indoor low-resolution monitoring equipment.

[0003] Existing indoor personnel detection systems based on deep learning are often developed based on target detection or image recognition technology, and are widely used in edge-side machine vision algorithm systems equipped with monitoring equipment. However, due to the versatility, computational complexity, and real-time requirements of edge-side algorithms, systems based on target detection methods can only meet the basic human body detection in indoor scenes. When the application scenarios encounter difficult conditions or influencing factors such as low resolution, large occlusion, and poor lighting, common target detection problems such as poor recognition accuracy, increased misidentification and missed recognition will arise, resulting in poor performance of intelligent monitoring and management systems based on human body detection data results. Summary of the invention

[0004] The purpose of the present invention is to provide a complex indoor scene personnel detection system and method based on image difference and classification, so as to solve the problems of low resolution, occlusion and so on in complex indoor scenes mentioned in the above background technology.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] A complex indoor scene personnel detection system based on image difference and classification includes sequence personnel detection, differential motion discrimination and stationary target classification, wherein the output end of the sequence personnel detection is electrically connected to the input end of the differential motion discrimination, the output end of the differential motion discrimination is electrically connected to the input end of the stationary target classification, the sequence personnel detection includes sequence personnel target detection and personnel target reduction, and the input ends of the sequence personnel target detection and personnel target reduction are both electrically connected to the output end of the sequence personnel detection.

[0007] A method for detecting people in complex indoor scenes based on image difference and classification, wherein the sequential person detection includes sequential person target detection and person target reduction, and the steps of the sequential person detection are as follows:

[0008] It is mainly responsible for detecting and locating human targets in the image sequence before and after the marked time point image in the time series image using a low threshold, detecting all targets that may be human bodies in each image, and merging all detection frames in each image into the largest bounding frame using the high threshold intersection-and-union method. Finally, the middle point image is cropped according to the positioning information as the data basis for the differential motion discrimination module.

[0009] As a preferred solution of the present invention, the sequence personnel target detection includes step one, step two and step three, and the step one is:

[0010] Including the process of sequential personnel target detection, based on the YoloV7 target detection algorithm to achieve single-class detection of human targets;

[0011] Step 2 includes the sequence personnel target detection process, which serializes the image list video according to the indoor surveillance video, and evenly selects k equally divided point images in the list of video to form a surveillance image list imgs: [[img1, img2, ..., imgk], where k is set to 3 in the system.

[0012] As a preferred solution of the present invention, the step three includes a human detection algorithm that takes a monitoring image list imgs and a target detection threshold thresh_detec as input, detects and locates human bodies and targets that may be human bodies in complex indoor monitoring images, and relaxes the detection of targets according to the detection threshold thresh_detec to obtain a target positioning information list objs_detec in each image: [img_objs1, img_objs1, ..., img_objsk], wherein img_objsi is the target positioning information of imgi, in the form of [pos1, pos2,, ..., posn], wherein posi identifies the positioning information of the target, and posi is in the form of {x, y, w, h}, indicating the positioning coordinates and area size.

[0013] As a preferred solution of the present invention, the personnel target reduction process first flattens objs_detec into a one-dimensional target box list objs_detec: [pos11, pos12, ..., pos1n, pos21, pos22, ..., pos2n, ..., poskn] according to the target positioning information list objs_detec in each image, and calculates the intersection degree value between each target box using the intersection-over-union method, wherein the IOU method formula is expressed as:

[0014]

[0015] After calculating the IOU values ​​of multiple target boxes, multiple boxes with an intersection degree greater than the overlap threshold are grouped according to the overlap threshold iou_thresh, and the target boxes in each group are reduced and integrated according to the maximum bounding box principle, and the target box list objs_detec: [pos1, pos2, ..., posm] is updated, where the system iou_thresh is set to 0.8.

[0016] As a preferred solution of the present invention, the step of differential motion discrimination is:

[0017] In the module, according to the target box list objs_detec and the monitoring image list imgs, the regional images of the same corresponding position of all the time-series target images in the target box are cropped, and the images of the n+ath frame, the nth frame and the nath frame in the video sequence are recorded as fn+a, fn and fn-a respectively. The gray values ​​of the corresponding pixels in the three frames are recorded as fn+1, fn and fn-1, and the difference images Dn and Dn+1 are obtained respectively. The difference images Dn and Dn+1 are calculated according to the following formula to obtain the image Dn*:

[0018] D n *(x,y)=|f n+a (x,y)-f n (x,y)|∩|f n (x,y)-f n-a (x,y)|

[0019] Then, threshold processing and connectivity analysis are performed, and finally the judgment results of the existence of target motion and relative stillness in each target area are obtained.

[0020] As a preferred solution of the present invention, the steps of classifying stationary targets are:

[0021] The module takes the static target frame static_objs and the monitoring image list imgs as input, crops the k / 2th image imgk / 2 in the monitoring image list, that is, the corresponding static target area image static_imgs:[img1,img2,...,imgq] of the nth frame image in the module, where q is the number of target frames judged as static targets in the module, and uses the EfficientNet human recognition model to perform regional image classification, filter out the non-human target parts of the static targets, and combine with the moving target movement_objs part, finally obtain the person detection result res_objs of the image imgk / 2 at the marked time point.

[0022] Compared with the prior art, the present invention has the following beneficial effects:

[0023] 1. In the present invention, by gradually differentiating the personnel detection tasks in complex scenes, and by combining deep learning algorithms such as yolov7 target detection, image difference, and EfficientNet image classification, a progressive human recognition architecture from coarse to fine is designed and completed, realizing a universal, highly accurate, efficient and reliable intelligent detection system for personnel in complex indoor scenes, and providing relatively reliable basic information for personnel positioning for indoor intelligent monitoring systems in difficult scenes such as poor monitoring conditions and unstable data transmission quality.

[0024] 2. In the present invention, the complex indoor scene personnel detection system can quickly and accurately detect the monitoring personnel's location on the images and sequences collected during the inspection, and accurately detect the relatively static ones. The system is based on target detection, traditional image difference algorithm, and image classification ability. Through the low-threshold primary detection and secondary screening for false detection, while improving the number of personnel detections in complex scenes such as occlusion and low resolution, it also reduces the system's false detection errors for human-like targets, so that the system can be applied to more complex and difficult intelligent monitoring scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 This is a system architecture diagram of the present invention;

[0026] Figure 2 The following is the architecture diagram of the complex indoor scene personnel detection system based on image difference and classification;

[0027] Figure 3 This is an example diagram of data visualization of the results of the complex indoor scene personnel detection system;

[0028] Figure 4 This is an example diagram of the calculation results of the three-frame difference algorithm.

[0029] In the figure: 1. Sequential person detection; 101. Sequential person target detection; 102. Person target reduction; 2. Differential motion discrimination; 3. Stationary target classification. DETAILED DESCRIPTION

[0030] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0031] For examples, see Figure 1-4 , the present invention provides a technical solution:

[0032] A complex indoor scene personnel detection system based on image difference and classification comprises a sequence personnel detection 1, a differential motion discrimination 2 and a stationary target classification 3, wherein the output end of the sequence personnel detection 1 is electrically connected to the input end of the differential motion discrimination 2, and the output end of the differential motion discrimination 2 is electrically connected to the input end of the stationary target classification 3. The sequence personnel detection 1 comprises a sequence personnel target detection 101 and a personnel target reduction 102, respectively, and the input ends of the sequence personnel target detection 101 and the personnel target reduction 102 are both electrically connected to the output end of the sequence personnel detection 1. The sequence personnel detection 1 locates the monitoring target in the sequenced image and locates the area, and performs regional image cropping on the sequence image of the location area. The differential motion discrimination 2 performs image difference calculation on each location area in the sequence area image, and determines whether there is target movement in each location area in the time series according to the difference result. The stationary target classification 3 is responsible for performing image classification processing on each location area image with target movement, identifying and screening out the non-personnel part in the stationary target, and completing adaptive and accurate detection of personnel in complex indoor scenes.

[0033] A method for detecting people in complex indoor scenes based on image difference and classification, wherein sequential person detection 1 includes sequential person target detection 101 and person target reduction 102, and the steps of sequential person detection are as follows:

[0034] It is mainly responsible for detecting and locating human targets in the image sequence before and after the image of the marked time point in the time series image using a low threshold, detecting all targets that may be human bodies in each image, and merging all detection frames in each image into the largest bounding frame using the high threshold intersection-union ratio (IOU) method. Finally, the middle point image is cropped according to the positioning information as the data basis of the differential motion discrimination module. The sequence human target detection 101 includes steps 1, 2 and 3. Step 1 is:

[0035] The process of sequential human target detection 101 uses the YoloV7 target detection algorithm as the basis to achieve single-class detection of human targets. Compared with similar target detection algorithms, the YoloV7 algorithm achieves good performance in both detection and classification, and is currently the only detector that can still exceed 30FPS under high precision (optimal accuracy 56.8% AP). The algorithm can adaptively constrain the size range of the detection target to achieve the effect of accelerating detection and reducing false detection. It is also more lightweight and efficient among similar detection algorithms;

[0036] Step 2 includes a sequence of personnel target detection 101 process, which serializes the image list video according to the indoor surveillance video, and selects k equally divided point images in the list of video to form a surveillance image list imgs: [[img1, img2, ..., imgk], where k is set to 3 in the system;

[0037] Step three includes a human detection algorithm that takes a monitoring image list imgs and a target detection threshold thresh_detec as input, detects and locates human bodies and targets that may be human bodies in complex indoor monitoring images, and relaxes the detection of targets according to the detection threshold thresh_detec, and obtains a target positioning information list objs_detec in each image: [img_objs1, img_objs1, ..., img_objsk], where img_objsi is the target positioning information of imgi, in the form of [pos1, pos2,, ..., posn], where posi identifies the positioning information of the target, and posi is in the form of {x, y, w, h}, indicating the positioning coordinates and area size.

[0038] In the process of reducing the personnel target 102, according to the target positioning information list objs_detec in each image, firstly, the objs_detec is flattened and reduced to a one-dimensional target box list objs_detec: [pos11,pos12,...,pos1n,pos21,pos22,...,pos2n,...,poskn], and the intersection degree value between each target box is calculated by the intersection-over-union (IOU) method, where the IOU method formula is expressed as:

[0039]

[0040] After calculating the IOU values ​​of multiple target boxes, multiple boxes with an intersection degree greater than the overlap threshold are grouped according to the overlap threshold iou_thresh, and the target boxes in each group are reduced and integrated according to the maximum bounding box principle, and the target box list objs_detec is updated: [pos1, pos2, ..., posm], where the system iou_thresh is set to 0.8;

[0041] The steps of differential motion discrimination 2 are:

[0042] In the module, according to the target box list objs_detec and the monitoring image list imgs, the regional images of the same corresponding position of all the time-series target images in the target box are cropped, and the images of the n+ath frame, the nth frame and the nath frame in the video sequence are recorded as fn+a, fn and fn-a respectively. The gray values ​​of the corresponding pixels of the three frames are recorded as fn+1x, y, fnx, y and fn-1x, y, respectively, and the differential images Dn and Dn+1 are calculated according to the following formula to obtain the image Dn*:

[0043] D n *(x,y)=|f n+a (x,y)-f n (x,y)|∩|f n (x,y)-fn-a (x,y)|

[0044] Then, threshold processing and connectivity analysis are performed, and finally the judgment results of whether the target motion in each target area exists movement_objs and relative stillness static_objs are obtained. Differential motion discrimination 2 is mainly responsible for cropping all time series target images in the corresponding positioning frame in the monitoring image list imgs according to the target detection and positioning information in the sequence personnel detection stage, calculating the difference with the target image in each positioning frame according to the three-frame difference algorithm, and judging whether there is target displacement in each positioning frame according to the residual calculation result.

[0045] The steps of stationary target classification 3 are:

[0046] The module takes the static target frame static_objs and the monitoring image list imgs as input, crops the k / 2th image imgk / 2 in the monitoring image list, that is, the corresponding static target area image static_imgs:[img1,img2,...,imgq] of the nth frame image in module 2, where q is the number of target frames judged as static targets in module 2, and uses the EfficientNet human recognition model to perform regional image classification, filter out the non-human target parts in the static targets, and combine the moving target movement_objs part to finally obtain the person detection result res_objs of the image imgk / 2 at the marked time point. The static target classification 3 is mainly responsible for the static targets judged in the image differential motion judgment results, and uses the image classification algorithm to verify and classify whether they are human bodies. The purpose is to filter out the false detection targets caused by low threshold target detection in the static targets. The EfficientNet multi-label classification model is selected for image classification. It uses fewer parameters to achieve better recognition. Under the condition of achieving the same efficiency, EfficientNet has the advantages of larger parameters and computational complexity, which is several times less than the parameters of the same-level network. It has the advantages of small model, small computational complexity and high performance. At the same time, in terms of transfer learning, EfficientNet also has a huge advantage with high performance and low parameter volume. This shows that EfficientNet has strong robustness in data set expansion and is easy to expand other computer vision tasks.

[0047] The workflow of the present invention is as follows: When the complex indoor scene personnel detection system and method based on image difference and classification designed by the present invention is working, the target detection, image difference and image classification algorithms in deep learning are used as the system capability basis, and the algorithm flow is combined to realize human body detection in complex scenes; the system first uses the low-threshold yolo target detection algorithm to detect and locate people in complex indoor scenes, and detects targets that may be human bodies in video sequences or continuous multi-frame images; then uses the image difference algorithm to identify whether the detected targets in continuous multi-frame images are moving or remaining still; finally, the EfficientNet image classification algorithm is used to The detected targets that remain stationary are re-identified to see if they are human, and the falsely detected targets that are foreign objects are screened out, completing the accurate detection of people in complex indoor scenes. The system breaks down the task of detecting people in complex indoor scenes into a coarse-to-fine data processing algorithm flow, and uses the target detection algorithm to detect all possible human bodies in the original image, reducing the system's missed detection rate for human detection. The image difference and image classification algorithms are used to identify and filter the relatively stationary detected targets in the time series, reducing the system's false detection rate while improving the system's accuracy in detecting people, making human body detection information more accurate, and realizing a high-accuracy, low-false-detection intelligent personnel detection system.

[0048] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A complex indoor scene person detection system based on image difference and classification, comprising sequential person detection (1), differential motion discrimination (2) and stationary target classification (3), characterized in that: The output end of the sequential person detection (1) is electrically connected to the input end of the differential motion discrimination (2), and the output end of the differential motion discrimination (2) is electrically connected to the input end of the stationary target classification (3). The sequential person detection (1) includes sequential person target detection (101) and person target reduction (102), and the input ends of the sequential person target detection (101) and the person target reduction (102) are both electrically connected to the output end of the sequential person detection (1).

2. A method for detecting people in complex indoor scenes based on image difference and classification, applicable to the system for detecting people in complex indoor scenes based on image difference and classification as claimed in claim 1, characterized in that: The sequential person detection (1) includes sequential person target detection (101) and person target reduction (102). The steps of the sequential person detection are: It is mainly responsible for detecting and locating human targets in the image sequence before and after the marked time point image in the time series image using a low threshold, detecting all targets that may be human bodies in each image, and merging all detection frames in each image into the largest bounding frame using the high threshold intersection-over-union (IOU) method. Finally, the middle point image is cropped according to the positioning information as the data basis for the differential motion discrimination module.

3. The method for detecting people in complex indoor scenes based on image difference and classification according to claim 2, characterized in that: The sequential human target detection (101) comprises step 1, step 2 and step 3, wherein step 1 is: The process of sequential person target detection (101) is based on the YoloV7 target detection algorithm to achieve single-class detection of human targets; Step 2 includes the process of sequential personnel target detection (101), which serializes the image list video according to the indoor surveillance video, and selects k equally divided point images in the list of video to form a surveillance image list imgs: [[img1, img2, ..., imgk], where k is set to 3 in the system.

4. The method for detecting people in complex indoor scenes based on image difference and classification according to claim 3, characterized in that: The step three includes a human detection algorithm that takes a monitoring image list imgs and a target detection threshold thresh_detec as input, detects and locates human bodies and targets that may be human bodies in complex indoor monitoring images, and relaxes the detection of targets according to the detection threshold thresh_detec, and obtains a target positioning information list objs_detec in each image: [img_objs1, img_objs1, ..., img_objsk], where img_objsi is the target positioning information of imgi, in the form of [pos1, pos2, , ..., posn ], where posi identifies the positioning information of the target, and posi is in the form of {x, y, w, h}, indicating the positioning coordinates and area size.

5. The method for detecting people in complex indoor scenes based on image difference and classification according to claim 2, characterized in that: The person target reduction process (102) first reduces the dimension of objs_detec into a one-dimensional target box list objs_detec: [pos11, pos12, ..., pos1n, pos21, pos22, ..., pos2n, ..., poskn] according to the target positioning information list objs_detec in each image, and calculates the intersection degree value between each target box using the intersection over union (IOU) method, wherein the IOU method formula is expressed as: After calculating the IOU values ​​of multiple target boxes, multiple boxes with an intersection degree greater than the overlap threshold are grouped according to the overlap threshold iou_thresh, and the target boxes in each group are reduced and integrated according to the maximum bounding box principle, and the target box list objs_detec: [pos1, pos2,..., posm] is updated, where the system iou_thresh is set to 0.

8.

6. The method for detecting people in complex indoor scenes based on image difference and classification according to claim 2, characterized in that: The steps of differential motion determination (2) are: In the module, according to the target box list objs_detec and the monitoring image list imgs, the regional images of the same corresponding position of all the time-series target images in the target box are cropped. The images of the n+ath frame, the nth frame and the nath frame in the video sequence are recorded as fn+a, fn and fn-a respectively. The gray values ​​of the corresponding pixels of the three frames are recorded as fn+1 (x, y), fn (x, y) and fn-1 (x, y), respectively, and the differential images Dn and Dn+1 are calculated according to the following formula to obtain the image Dn*: Then, threshold processing and connectivity analysis are performed, and finally the judgment results of the existence of target motion (movement_objs) and relative stillness (static_objs) in each target area are obtained.

7. The method for detecting people in complex indoor scenes based on image difference and classification according to claim 2, characterized in that: The steps of the stationary target classification (3) are: The module takes the static target frame static_objs and the monitoring image list imgs as input, crops the k / 2th image imgk / 2 in the monitoring image list, that is, the corresponding static target region image static_imgs:[img1, img2, ...,imgq] of the nth frame image in module (2), where q is the number of target frames identified as static targets in module (2), and uses the EfficientNet human recognition model to perform regional image classification, filter out the non-human target parts in the static targets, and combine them with the moving target movement_objs part, and finally obtain the person detection result res_objs of the image imgk / 2 at the marked time point.

Images