A power marketing field safety monitoring method and system based on user behavior characteristic analysis

Abstract

The application provides a power marketing field safety monitoring method and system based on user behavior characteristic analysis, which comprises the following steps: step one, power marketing field camera layout and calibration; step two, target dynamic speed basic threshold in the image of the corresponding grid of the calibrated camera; step three, human head recognition; step four, distance estimation according to the visual size of the human head; step five, estimation of the height of the field human according to the layout height of the camera and the ground calibrated in step one and the actual distance of the human head from the camera obtained in step four, and determination of whether it is an adult according to the estimated height of the field human; step six, monitoring of the instantaneous speed of the human body in the grid; and step seven, further assessment of potential safety hazards. The application realizes the estimation of the motion speed of the human body in the field by camera monitoring, multi-camera linkage and grid image processing, and screens out the users who have occurred or exist human hazards and the specific positions thereof.

Description

TECHNICAL FIELD

[0001] The present application relates to a safety monitoring method for power marketing site by image analysis of user behavior, in particular to a safety monitoring method and system for power marketing site based on user behavior feature analysis. BACKGROUND

[0002] The personnel safety guarantee of power marketing site is the top priority of power marketing work. The traditional guarantee method is usually to set security personnel to patrol on site, which brings additional cost to power marketing activities. On the other hand, the safety hazards of power marketing site can be divided into physical hazards and personnel hazards according to the occurrence subject: the physical hazards include the safety hazards caused by on-site instruments and equipment, such as equipment failure, fire points, etc.; the personnel hazards are the safety hazards caused by the behavior of on-site personnel, such as rapid movement in the area where personnel are concentrated, fighting, etc.; on the other hand, for user behavior, it is necessary to analyze different age groups separately, and to monitor the rapid movement of adults (children's movement behavior is not monitored), so it is necessary to classify personnel by accurate assessment of height. SUMMARY

[0003] The present application mainly aims at the personnel safety of power marketing site, and provides a safety monitoring method and system for power marketing site based on user behavior feature analysis, which realizes the estimation of the body movement speed of on-site personnel through the monitoring of on-site camera and the way of multi-camera linkage and grid image processing, screens out the users who have occurred or exist personnel hazards and their specific positions, so as to provide effective data and decision support for the personnel safety monitoring and guarantee of marketing site.

[0004] The above technical problems of the present application are mainly solved by the following technical scheme:

[0005] A safety monitoring method for power marketing site based on user behavior feature analysis, comprising the following steps:

[0006] Step one, camera layout and calibration of power marketing site: layout monitoring camera in power marketing site, and calibrate the layout height of camera and ground;

[0007] Step two, target dynamic speed basic threshold in the grid image corresponding to the calibrated camera: the target dynamic speed is defined as: the instantaneous speed of the target moving in the current time in a certain grid image, the target includes the head, torso and limbs of the user, when the target moves on the plane, the "corresponding relationship between the real moving speed and the target moving speed in the image" of each grid center point position corresponding to each camera is calibrated;

[0008] Step three, head recognition: the monitoring camera arranged in step one recognizes the head contour of the person in the marketing site;

[0009] Step four, estimate the distance according to the visual size of the head of the person: estimate the actual distance of the head of the person from the camera according to the head contour of the person obtained in step three;

[0010] Step five, estimate the height of the person on site according to the layout height of the camera from the ground marked in step one and the actual distance of the head of the person from the camera obtained in step four, and determine whether it is an adult according to the estimated height of the person on site:

[0011] Step six, instantaneous speed monitoring of the body of the person in the grid: obtain the moving speed of the body of the person in each grid, and determine the "potential safety hazard" according to the moving speed and the corresponding relationship marked in step two;

[0012] Step seven, further evaluation of the potential safety hazard: calculate all the heads of the persons within a set distance from the horizontal of the "potential safety hazard" area, and if there is no minor within the set distance, it is defined as a safety hazard; if there is only a minor within the set distance, it is defined as "no safety hazard".

[0013] Further, step one includes:

[0014] Step 1.1, mark the planar positions of the monitoring cameras at the midpoint positions of the four sides of a 20x20 meter marketing area, a total of four;

[0015] Step 1.2, mark the layout height of the camera at a height of 4 meters from the ground;

[0016] Step 1.3, the four cameras monitor the power marketing site area in a top-down manner.

[0017] Further, step two includes:

[0018] Step 2.1, divide the 20x20 meter marketing area into a plurality of 1x1 meter grids, and the entire area has 400 grid areas;

[0019] Step 2.2, mark the target dynamic speed threshold: calculate the footstep moving speed threshold α = 1.2 meters / second at a normal walking speed of 1.2 meters / second; mark the moving speed threshold β = 1.5 meters / second of the torso, head and arm, and then set the comprehensive threshold γ to 2.7 meters / second;

[0020] Step 2.3, number the four cameras as C1, C2, C3, and C4, and number the 400 grid areas as Grid n , wherein n = 1-400;

[0021] Step 2.4, calculate the corresponding to each camera, each grid area, at the grid center point ground position when moving at a speed of γ, corresponding to the moving speed in the frame image of a certain camera, recorded as groundSpeed mn where m is the camera number, and n is the grid number.

[0022] Further, step three includes:

[0023] Step 3.1, through the 4 orientation monitoring cameras, realize the recognition of the head of the on-site figure at a certain time, the realization method is:

[0024] a) select the head recognition of the human body to realize the determination of the number of on-site figures;

[0025] b) make a figure head picture database, the picture shooting angle is a head picture with a downward angle; The head database is further subdivided into hat, hair, and no hair categories from the morphological point of view, and is divided into front, side, and back from the shooting orientation; Each category makes not less than 500 picture samples;

[0026] c) based on the TensorFlow deep learning framework, build a residual neural network model, train the sample set constructed in step b), and obtain a figure head recognition model;

[0027] d) use the figure head recognition model to select the frame image of the current time from the 4 orientation cameras respectively, and perform figure head recognition;

[0028] e) take the maximum value of the figure head count in each grid;

[0029] f) take the camera corresponding to the grid maximum value as the reference picture for the figure head recognition in the grid at that time;

[0030] Step 3.2, use the semantic segmentation model to separate and extract different figure heads in the reference picture;

[0031] Step 3.3, modify the figure head sample set constructed in step 3.1b), and add a sample pair of "figure head local image - complete head portrait";

[0032] Step 3.4, build a generative adversarial network model, and reconstruct all the different figure head images extracted from the reference image into complete figure head images.

[0033] Further, step four includes:

[0034] Step 4.1, set 4 orientation camera model parameters consistent, with fixed size of the object, calibration of different near and far, size in the image: 10 centimeters as an interval, record 2 meters-23 meters range of the actual size and size in the image ratio;

[0035] Step 4.2, after obtaining the profile of a person's head on site, according to the ratio parameters obtained in step 4.1, determine the actual distance of the person's head from each camera, denoted as L a , L b , L c and L d ;

[0036] Step 4.3, take each camera as the center, and the corresponding L a , L b , L c and L d as the radius, draw a three-dimensional sphere;

[0037] Step 4.4, if the intersection of two spheres is in the same area, then the positioning is successful; otherwise, select the frame image again after 1 second and repeat the above steps;

[0038] Step 4.5, if the positioning fails for 5 times in a row, it means that the positioning error is too large, and the administrator is prompted to return to the interactive information;

[0039] Step 4.6, after successful positioning, the actual distance of the person's head from each camera is L a , L b , L c and L d .

[0040] Further, step six includes:

[0041] Step 6.1, extract 10 frames of images at the current time;

[0042] Step 6.2, in the current image and the previous 10 frames of images, extract the "feature area", the feature point area is defined as: the area with special morphological features and color features;

[0043] Step 6.3, estimate the actual speed g-Speed mn of the target movement corresponding to the current grid by the distance of the feature area in the 10 frames of images continuously changing, combined with the corresponding relationship in step 2.4 and the sampling frequency f of the camera;

[0044] Step 6.4, set the error parameter error, if the actual measured g-Speed mn is greater than (2.7 meters / second-error), it is considered as a potential safety hazard.

[0045] Further, step seven includes:

[0046] Step 7.1, calculate all the heads of the figures within 1 meter from the level of the "potential safety hazard" area;

[0047] Step 7.2, if there is no minor within 1 meter, the "potential safety hazard" is determined as a safety hazard;

[0048] Step 7.3, if there is a minor within 1 meter, through multi-camera cooperative judgment, if there is a minor within 1 meter of the "potential safety hazard" area, it is impossible to judge whether the "potential safety hazard" is caused by an adult, and feedback is given to the administrator;

[0049] Step 7.4, if only minors exist within 1 meter, eliminate the monitored "potential safety hazard" and determine that there is no safety hazard.

[0050] A power marketing site safety monitoring system based on user behavior feature analysis, comprising:

[0051] A power marketing site monitoring camera layout and calibration module for laying out monitoring cameras in a power marketing site and calibrating the layout height of the cameras and the ground;

[0052] A grid image target dynamic speed basic threshold calibration module for calibrating the "corresponding relationship between the real moving speed and the target moving speed in the image" of each camera corresponding to each grid center point position when the target moves on a plane, the target dynamic speed being the instantaneous speed of the target moving in a certain grid image at the current time, the target including the head, torso and limbs of the user;

[0053] A figure head recognition module for recognizing the head profile of the figures in the marketing site based on the laid-out monitoring cameras;

[0054] A distance estimation module according to the visual size of the figure head for estimating the actual distance of the figure head from the camera according to the obtained figure head profile;

[0055] A site figure height estimation module for estimating the height of the figures in the site according to the calibrated layout height of the cameras and the ground and the actual distance of the figure head from the camera, and determining whether the figures are adults according to the estimated height of the figures in the site;

[0056] A grid figure limb instantaneous speed monitoring module for obtaining the moving speed of the limbs of the figures in each grid, and determining the "potential safety hazard" according to the moving speed and the calibrated corresponding relationship;

[0057] Further assessment module of potential security risks, used for calculating all the heads of the people within a set distance from the "potential security risks" area level, if all the heads of the people within the set distance do not exist minors, then it is defined as a security risk, if all the heads of the people within the set distance only exist minors, then it is defined as "no security risk".

[0058] The present application has the following beneficial effects:

[0059] 1. By arranging the camera in the power marketing site, the no-corner monitoring method of the personnel in the area is realized.

[0060] 2. Since the body is adult or not, the head size changes little (relative to the height), the real of the body to the camera is evaluated through the head size of the body and the straight line distance between the fixed camera of each position and coordinate.

[0061] 3. By the induction classification of the walking speed and the limb moving speed of the body, the speed threshold is proposed to calculate and evaluate whether the fast moving (potential security risk) behavior occurs in the site area.

[0062] 4. By the linkage of the four direction cameras, the production of the face feature "reference picture" of the body is realized, and the positioning of the people in the site shielding situation is realized.

[0063] 5. By the multi-grid division and the triangular relationship of the camera shooting, the real height range of the target people in the area is calculated, and then whether the object is an adult is judged.

[0064] 6. In the case of considering the limb height, the accurate evaluation of the position and the instantaneous speed of the limb of the people is realized, and then whether the "potential security risk" exists is judged. BRIEF DESCRIPTION OF DRAWINGS

[0065] Figure 1 It is a flowchart of the power marketing site security monitoring method based on user behavior feature analysis of the present application;

[0066] Figure 2 It is a module block diagram of the power marketing site security monitoring system based on user behavior feature analysis of the present application;

[0067] Figure 3 It is a geometric relationship diagram when the height of the people in the site is estimated in the embodiment of the present application. DETAILED DESCRIPTION

[0068] In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application. The whole system is realized in a software manner.

[0069] Referring to Figure 1 The embodiments of the present application provide a power marketing site safety monitoring method based on user behavior characteristic analysis, comprising the following steps:

[0070] Step one, power marketing site monitoring camera layout and calibration: used for conveniently and quickly laying out monitoring cameras in the power marketing site, supporting subsequent all-around accurate identification of user behaviors; taking a 20*20m square marketing area as an example, the specific implementation steps are as follows:

[0071] Step 1.1, calibrating the midpoint positions of the four sides of the marketing area as the planar positions of the monitoring cameras, totally four;

[0072] Step 1.2, calibrating the height of 4m from the ground as the layout height of the cameras;

[0073] Step 1.3, the four cameras all monitor the power marketing site area in a top-down manner.

[0074] Step two, target dynamic speed basic threshold calibration in the grid image: the target dynamic speed is defined as the instantaneous speed of the target (including the head, torso and limbs of the user) moving in a certain grid image at the current time, since the straight-line distances of different grids from the camera are different, the moving speed in the image and the real speed are nonlinear changes; this step assumes that the target moves on the plane (ground), and calibrates the corresponding relationship between the real moving speed and the target moving speed in the image corresponding to each grid center point position of each camera, and the specific implementation steps are as follows:

[0075] Step 2.1, dividing the 20*20m marketing area into multiple 1*1m grids, and the whole area has 400 grid areas;

[0076] Step 2.2, calibrating the target dynamic speed basic threshold: taking the normal walking speed 1.2m / s as the calculation, the step moving speed threshold a is calibrated as 1.2m / s; the moving speed thresholds of the torso, head and arms b are calibrated as 1.5m / s, and then the comprehensive threshold g is set as 2.7m / s (the linear superposition value of the two speeds is the upper limit under the normal walking state of moving other parts of the body at the same time);

[0077] Step 2.3, number 4 cameras as C1, C2, C3, C4, and number 400 grid areas as Grid n where n = 1-400;

[0078] Step 2.4, calculate the moving speed of the ground at the center point of each grid area corresponding to each camera, which is the moving speed corresponding to the frame image of a certain camera, denoted as groundSpeed mn where m is the camera number and n is the grid number.

[0079] Step 3, human head recognition: this step is used to recognize the head profile of the person in the marketing scene from the 4 cameras, and especially, due to the possibility of partial occlusion of the person, the 4 cameras need to be linked to identify the head profile as much as possible. The specific implementation steps are:

[0080] Step 3.1, recognize the head of the person in the scene at a certain time through the 4 monitoring cameras, the implementation method is:

[0081] a) Since the camera is calibrated to be 4 meters high from the ground, which is much higher than the height of the person in the scene, the head of the person is selected for recognition to determine the number of people in the scene;

[0082] b) Make a database of pictures of human heads, the pictures are taken at an angle with a downward angle; The head database can be further divided into categories such as wearing a hat, having hair, and no hair from a morphological perspective, and can be divided into front, side, and back from the shooting direction; Each category makes not less than 500 picture samples;

[0083] c) Based on the TensorFlow deep learning framework, build a residual neural network model, train the sample set constructed in step b), and get a human head recognition model;

[0084] d) Use the human head recognition model to select the frame image of the current time from the 4 cameras, and perform human head recognition;

[0085] e) Since there may be occlusion of the head of a person at a certain angle, in order to minimize errors, the maximum value of the number of human heads in each grid is taken (the number of human heads in the corresponding grid in the 4 direction images);

[0086] f) Take the camera corresponding to the maximum value of the grid as the reference picture for recognizing the human head in the grid at that time.

[0087] Step 3.2, use the semantic segmentation model SAM2.0 of Meta company to separate and extract different human heads in the reference picture;

[0088] Step 3.3, the human head sample set constructed in step 3.1b) is modified to add a sample pair of "human head local image - complete head portrait";

[0089] Step 3.4, construct a generative adversarial network model, and reconstruct different human head images extracted from the reference image into complete human head images (the human head image in the "reference image" may not be complete due to occlusion).

[0090] Step four, estimate the distance according to the visual size of the head: this step is used to estimate the actual distance of the head from the camera; at the same time, since the head circumference of children and adults is small (much smaller than the height difference), such as the head circumference of a 3-year-old child is about 48 centimeters, and the head circumference of an adult is about 54 centimeters, so children and adults can be temporarily distinguished, and the distance from the camera is judged by the head circumference, and in step five, the crowd is classified according to the estimation of height; the specific implementation steps are:

[0091] Step 4.1, assuming that the parameters of the 4-position camera model are consistent, with a fixed size of the object, the size in the image is recorded at different distances: take 10 centimeters as an interval, record the actual size of the object and the size ratio in the image in the range of 2 meters-23 meters (the farthest distance from the camera in the field is 22.7 meters);

[0092] Step 4.2, after obtaining the contour of a person's head in the field, determine the actual distance of the person's head from each camera according to the proportion parameters obtained in step 4.1, denoted as L a , L b , L c and L d ;

[0093] Step 4.3, take each camera as the center, and L a , L b , L c and L d as the radius, draw a three-dimensional sphere;

[0094] Step 4.4, if the intersection of two spheres is in the same area (same grid), then this positioning is successful; otherwise, select a frame image again after 1 second and repeat the above steps;

[0095] Step 4.5, if the positioning fails for 5 times in a row, it means that the positioning error in the field is too large, and the interactive prompt information is returned to the administrator;

[0096] Step 4.6, after successful positioning, the actual distance of the head from each camera is L a , L b , L c and Ld .

[0097] Step five, height estimation of on-site figures: this step is used to estimate the specific height of the figures: due to the on-site obstruction, the full body image of the figure cannot be read to calculate the height of the figure, therefore, the present application proposes to measure the head position and distance of the human body first, and to calculate the distance from the head to the ground in the grid area (C point) in the image taken by the camera, that is, the height; the specific implementation steps are as follows: Figure 3

[0098] Step 5.1, as shown in Figure 3 , the line segment AD is one of the above measured L a , L b , L c and L d , which is assumed to be L a ;

[0099] Step 5.2, as shown in Figure 3 , in the image taken by the camera, the figure is shown in the area at C point in the image, so the length of the line segment AC is known;

[0100] Step 5.3, therefore, the actual height DE of the figure can be obtained as follows:

[0101]

[0102] Step 5.4, set a threshold value (such as 1.5 meters), if DE is greater than 1.5 meters, the figure is defined as an adult.

[0103] Step six, monitoring of the instantaneous speed of the figure's limbs in the grid: the moving speed of the figure's limbs in each grid is obtained, and the "potential safety hazard" is determined according to the moving speed and the corresponding relationship calibrated in step two, and the specific implementation steps are as follows:

[0104] Step 6.1, extract 10 frames of images at the current time;

[0105] Step 6.2, in the current image and the previous 10 frames of images, extract "feature regions", which are defined as regions with special morphological features and color features, such as hands, five-point star patterns, etc.

[0106] Step 6.3, estimate the actual speed (assuming on the grid ground) g-Speed mn corresponding to the target movement in the current grid by the distance of the feature region in the 10 frames of images continuously changing, combined with the corresponding relationship in step 2.4 and the sampling frequency f of the camera (that is, how many frames of images are taken per second);

[0107] ​Step 6.4, set error parameter error, if the actual measured g-Speed mn is greater than (2.7 m / s-error), it is determined as a potential safety hazard; wherein, error is defined because g-Speed mn is ground speed, and the target moves a distance from the ground in a certain grid, which may appear in a further grid in the photographed image, thereby causing "speed misjudgment too slow"; error can be dynamically valued according to different grids, and since the dynamic area height is unknown, an average value (assuming the speed error when the action occurs at a distance of 1 m from the ground) can be taken.

[0108] Step seven, further evaluation of potential safety hazards: combine the specific location of the safety hazard area with the matching human head to determine whether the safety person is an adult, and further determine whether the "potential safety hazard" in the foregoing step six belongs to an adult: if it is an adult, it is determined as a "safety hazard"; if it is a minor, it is determined as "no safety hazard". The specific implementation method is:

[0109] Step 7.1, since the horizontal distance of the human body action from the head is usually within 1 m, all human heads within 1 m horizontally from the "potential safety hazard" area can be calculated according to the plane set relationship as shown in Figure 3

[0110] Step 7.2, according to the height determination method in step five, if there is no minor within 1 m, it is determined that the "potential safety hazard" is a safety hazard;

[0111] Step 7.3, if there is a minor within 1 m, through multi-camera cooperative judgment, if the same result is obtained (there is a minor within 1 m of the "potential safety hazard" area), it is impossible to determine whether the "potential safety hazard" is caused by an adult, and feedback is given to the administrator;

[0112] Step 7.4, if only minors exist within 1 m, eliminate the monitored "potential safety hazard" and determine it as no safety hazard.

[0113] Please refer to Figure 2 , the embodiment of the present application provides a power marketing field safety monitoring system based on user behavior feature analysis, which is used to execute the above method, the system comprises:

[0114] ​The power marketing field monitoring camera layout and calibration module 10 is used for conveniently and quickly laying out the monitoring camera in the power marketing field, and supports the subsequent all-around accurate identification of user behaviors; taking a 20x20m square marketing area as an example; the midpoint positions of the four sides of the marketing area are calibrated as the planar positions of the monitoring camera, a total of four; the height of 4m from the ground is calibrated as the layout height of the camera; the four cameras all monitor the power marketing field area in a top-down manner.

[0115] The target dynamic speed basic threshold calibration module 20 in the grid image defines the target dynamic speed as the even speed of the target (including the head, torso, limbs of the user) moving in a certain grid image at the current time; since the straight-line distances of different grids from the camera are different, the moving speed in the image and the real speed are nonlinear changes; this module assumes that the target moves on the plane (ground), and calibrates the “corresponding relationship between the real moving speed and the target moving speed in the image” corresponding to each grid center point position of each camera; the 20x20m marketing area is divided into multiple 1x1m grids, and the entire area has a total of 400 grid areas; the real moving speed threshold is calculated at a normal walking speed of 1.2m / s, and the walking speed threshold a is set as 1.2m / s; the moving speed thresholds of the torso, head and arms are set as β=1.5m / s, and the comprehensive threshold γ is set as 2.7m / s (the linear superposition value of the two speeds is the upper limit in the normal walking state); the four cameras are numbered as C1, C2, C3 and C4, and the 400 grid areas are numbered as Grid n , where n=1~400; the moving speed corresponding to each camera and each grid area at the grid center point ground position at the speed of γ is calculated, and is recorded as groundSpeed mn , where m is the camera number and n is the grid number.

[0116] The human head recognition module 30 is used for recognizing the human head profile in the marketing field from the four-direction cameras, and in particular, since local occlusion of the human head may occur in the field, the four cameras need to be linked to recognize the human head profile as much as possible. First, the human head in the field at a certain time is recognized through the four-direction monitoring cameras; then, the different human head parts in the “benchmark picture” are segmented and extracted by using the semantic segmentation model SAM2.0 of Meta company; then, the constructed human head sample set is modified, and the sample pair of “human head local image — complete head portrait” is added; finally, the generative adversarial network model is constructed, and the different human head images extracted from the “benchmark image” are all reconstructed into complete human head images (the human head images in the “benchmark picture” may not be complete due to occlusion).

[0117] According to the character head visual size distance estimation module 40, the module is used to estimate the actual distance of the character head from the camera; at the same time, since the head circumference of children and adults is small (much smaller than the height difference), such as the head circumference of a 3-year-old child of about 48 cm, and the head circumference of an adult of about 54 cm, children and adults can be temporarily distinguished, and the distance from the camera is judged by the head circumference, and the crowd is classified according to the estimation of the height in step five. First, assuming that the four orientation camera models are consistent, with a fixed size object, the size in the image is recorded when the distance is different: take 10 cm as an interval, record the actual size of the object and the size ratio in the image in the range of 2 m-23 m (the farthest distance from the camera in the field is 22.7 m); then, after obtaining the head profile of a person in the field, according to the proportion parameters obtained in step 4.1, the distance of the character head from each camera is determined, denoted as L a , L b , L c and L d ; at the same time, taking each camera as the center and the corresponding L a , L b , L c and L d as the radius, a three-dimensional sphere is drawn If the intersection of two spheres is in the same area (the same grid), the positioning is successful; otherwise, after 1 second, the frame image is selected again, and the above steps are repeated; if the positioning fails for 5 times in a row, it means that the positioning error in the field is too large, and the interactive prompt information is returned to the administrator; after successful positioning, the actual distance of the character head from each camera is L a , L b , L c and L d .

[0118] The on-site character height estimation module 50 is used to estimate the specific height of the character: due to the occlusion in the field, the full body image of the character cannot be read to calculate the height of the character; and to provide support for subsequent division of whether the target character is an adult. First, as shown in Figure 3 , the line segment AD is one of the above measured L a , L b , L c and L d , assuming L a ; in the image captured by the camera, the character is displayed in the area where point C is located in the image, so the length of the line segment AC is known; therefore, the actual height DE of the character can be obtained as:

[0119]

[0120] Next, a threshold (e.g., 1.5 meters) is set. If DE is greater than 1.5 meters, the person is defined as an adult.

[0121] The instantaneous velocity monitoring module 60 for human limbs in a grid is used to obtain the maximum movement speed of human limbs in each grid. First, at a certain moment, 10 frames of images are extracted from the previous frame. Then, "feature regions" are extracted from the current image and the previous 10 frames. Feature regions are defined as areas with special morphological or color characteristics, such as a hand or a five-pointed star pattern. By measuring the continuously changing distance of these feature regions across the 10 frames, and combining the correspondence in step 2.4 with the sampling frequency f defined in step 6.3, the actual movement speed of the target in the current grid (assuming it's on the grid ground) g-Speed ​​is estimated. mn Finally, set the error parameter 'error', if the actual measured g-Speed mn If the speed exceeds (2.7 m / s - error), it is considered a potential safety hazard; where error is defined because of g-speed. mn It is the ground speed. However, if a target moves at a distance from the ground in a certain grid, it may appear in a grid further away in the captured image, thus causing "misjudged speed too slow". The error can be dynamically determined according to different grids, and since the height of the dynamic area is unknown, an average value can be taken (assuming the speed error occurs when the action occurs at a height of 1 meter above the ground).

[0122] The further assessment module 70 for potential safety hazards, combined with the specific location of the safety hazard area and its correlation with a person's head, determines whether the person is an adult. Firstly, since the horizontal distance between a person's limbs and their head is typically within 1 meter, it can be determined based on... Figure 3 The planar set relationship shown is used to calculate the heads of all people within 1 meter horizontally of the "potential safety hazard" area. According to the height judgment method in module 50, if there are no minors within 1 meter, the "potential safety hazard" is identified as a safety hazard. If there are minors within 1 meter, multiple cameras are used for collaborative judgment. If all cameras produce the same result (a minor exists within 1 meter of the "potential safety hazard"), it cannot be determined whether the "potential safety hazard" was caused by an adult, and feedback is sent to the administrator. If only minors exist within 1 meter, the detected "potential safety hazard" is eliminated, and it is considered to be without safety hazard.

[0123] Those skilled in the art can further understand that the units and algorithms described in connection with the embodiments disclosed herein can be realized by electronic hardware, computer software or a combination of both. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of various examples have been described in general terms above. Whether these functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not exceed the scope of the present application.

[0124] The steps of the methods or algorithms described in connection with the embodiments disclosed herein can be directly implemented by hardware, software executed by a processor, or a combination of both. The software module can be placed in a random storage, a memory, a read-only memory, an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0125] It can be understood that, for those skilled in the art, other various corresponding changes and modifications can be made according to the technical concept of the present application, and all these changes and modifications shall fall within the protection scope of the present application.

[0126] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto, and any changes or replacements easily thought of by those skilled in the art within the technical scope disclosed by the present application shall fall within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A power marketing field safety monitoring method based on user behavior feature analysis, characterized in that, Comprising the following steps: Step one, power marketing site camera layout and calibration: layout the monitoring camera in the power marketing site, and calibrate the layout height of the camera and the ground; Step two, calibrate the target dynamic speed threshold in the corresponding grid image of the camera: the target dynamic speed is defined as: the instantaneous speed of the target moving in a certain grid image at the current time, the target includes the head, torso and limbs of the user, when the target moves on the plane, the "corresponding relationship between the real moving speed and the target moving speed in the image" of each camera corresponding to each grid center point position is calibrated; the grid refers to the equal size area formed after the marketing area is divided; Step three, human head recognition: based on the monitoring camera arranged in step one, the human head profile in the marketing site is recognized; Step four, estimate the distance according to the visual size of the human head: estimate the actual distance of the human head from the camera according to the human head profile obtained in step three; Step five, estimate the height of the on-site person according to the layout height of the camera and the ground calibrated in step one and the actual distance of the human head from the camera obtained in step four, and determine whether it is an adult according to the estimated height of the on-site person; Step six, monitoring of the instantaneous speed of the human body in the grid: the moving speed of the human body in each grid is obtained, and the "potential safety hazard" is determined according to the moving speed and the corresponding relationship calibrated in step two; Step seven, further evaluation of potential safety hazards: calculate all human heads within a set distance from the "potential safety hazard" area horizontally, if there is no minor within the set distance, it is defined as a safety hazard; if there is only a minor within the set distance, it is defined as "no safety hazard".

2. The user behavior feature analysis-based power marketing field safety monitoring method of claim 1, wherein, Step one includes: Step 1.1, in The middle points of the four edges of the marketing area of the size of rice are marked as the plane positions of the monitoring cameras, and there are four in total. Step 1.2, the height of 4 meters from the ground is calibrated as the layout height of the camera; Step 1.3, the four cameras monitor the power marketing site area in a top-down manner.

3. The user behavior feature analysis-based power marketing field safety monitoring method of claim 2, wherein, Step two includes: Step 2.1, divide the marketing area of 1m x 1m into a plurality of 1m x 1m grids, and the entire area has 400 grid areas; 1m x 1m grids, and the entire area has 400 grid areas; Step 2.2, Calibrate target dynamic speed base threshold: with normal walking speed 1.2 m / s, calibrate footstep moving speed threshold ; calibrate trunk, head, arm moving speed threshold , then the comprehensive threshold is set to 2.7 m / s; Step 2.3, Number 4 cameras as C1, C2, C3, C4, and number 400 grid areas as Grid n where n = 1~400; Step 2.4, compute the ground speed at the grid center point location for each camera, each grid region, at corresponding to the moving speed in the frame image of a certain camera, denoted as groundSpeed mn where m is the camera number and n is the grid number.

4. The user behavior feature analysis-based power marketing field safety monitoring method of claim 1, wherein, Step three includes: Step 3.1, through the four direction monitoring cameras, the identification of the human head in the site at a certain time is realized, the realization method is: a) Select the head recognition of the human body to realize the determination of the number of on-site persons; b) Make a human head picture database, the picture shooting angle is a head picture with a downward angle; the head database is further subdivided into hat, hair and no hair categories from the morphological point of view, and is divided into front, side and back from the shooting direction; not less than 500 picture samples are made for each category; c) Based on the TensorFlow deep learning framework, a residual neural network model is constructed, the sample set constructed in step b) is trained, and a human head recognition model is obtained; d) Use the human head recognition model to select the frame image of the current time from the four direction cameras respectively to recognize the human head; e) Take the maximum value of the human head count in each grid; f) Take the camera corresponding to the maximum value of the grid as the reference picture for human head recognition in the grid at that time; Step 3.2, use the semantic segmentation model to segment and extract different human heads in the reference picture respectively; Step 3.3, the human head sample set constructed in step 3.1b) is modified, and a sample pair of "human head local image-full head portrait" is added; Step 3.4, a generative adversarial network model is constructed, and different human head images extracted from the reference image are all reconstructed into complete human head images.

5. The user behavior feature analysis-based power marketing field safety monitoring method of claim 1, wherein, Step four includes: Step 4.1, set 4 orientation camera model parameters consistent, with fixed size object, calibration different near and far in the image size: 10 centimeters as an interval, record 2 meters-23 meters range object actual size and size ratio in the image; Step 4.2, after the head contour of the person in the scene is obtained, the actual distance of the head of the person from each camera is determined according to the scale parameter obtained in step 4.1, denoted as L a , L b , L c , and L d ; Step 4.3, draw a three-dimensional sphere with each camera as the center and the corresponding L a , L b , L c , and L d as the radius; Step 4.4, if the sphere intersects each other in the same area, then the positioning is successful; otherwise, select the frame image again after 1 second and repeat the above steps; Step 4.5, if the positioning fails for 5 times in a row, it means that the positioning error is too large, and the interactive prompt information is returned to the administrator; Step 4.6, after positioning successfully, the actual distance between the head of the person and each camera is L a , L b , L c , and L d .

6. The user behavior feature analysis-based power marketing field safety monitoring method of claim 3, wherein, Step six includes: Step 6.1, extract 10 frames of images at the current time; Step 6.2, extract "feature area" from the current image and the previous 10 frames of images, the feature area is defined as: the area with special morphological features and color features; Step 6.3, Estimate the actual speed g-Speed of the target movement in the current grid by the distance of the feature region changing continuously in 10 frames of images, combined with the correspondence in step 2.4 and the sampling frequency f of the camera mn ; Step 6.4, Set error parameter error, if actual measured g-Speed mn greater than then identify as potential safety hazard.

7. The user behavior feature analysis-based power marketing field safety monitoring method of claim 6, wherein, Step seven includes: Step 7.1, calculate all human heads within 1 meter from the "potential safety hazard" area horizontally; Step 7.2, if there is no minor within 1 meter, it is determined that the "potential safety hazard" is a safety hazard; Step 7.3, if there is a minor within 1 meter, through multi-camera cooperative judgment, if there is a minor within 1 meter of the "potential safety hazard" area, it cannot be determined whether the "potential safety hazard" is caused by an adult, and feedback is given to the administrator; Step 7.4, if only minors exist within 1 meter, eliminate the monitored "potential safety hazard" and determine that there is no safety hazard.

8. A power marketing field safety monitoring system based on user behavior feature analysis, characterized in that, It includes: The power marketing site monitoring camera layout and calibration module is used to layout the monitoring camera in the power marketing site and calibrate the layout height of the camera from the ground; The target dynamic speed basic threshold calibration module in the grid image is used to calibrate the "real moving speed and the corresponding relationship between the target moving speed in the image" of each grid center point position corresponding to each camera when the target moves on the plane. The target dynamic speed is the instantaneous speed of the target moving in a certain grid image at the current time. The target includes the user's head, torso, and limbs. The grid refers to the equally sized areas formed after the marketing area is divided; The human head recognition module is used to recognize the human head contour in the marketing site based on the arranged monitoring camera; The distance estimation module according to the visual size of the human head is used to estimate the actual distance of the human head from the camera based on the obtained human head contour; The on-site human height estimation module is used to estimate the on-site human height based on the calibrated layout height of the camera from the ground and the actual distance of the human head from the camera, and to determine whether it is an adult based on the estimated on-site human height; The grid character limb instantaneous speed monitoring module is configured to acquire the moving speed of each grid character limb, and determine the "potential safety hazard" according to the moving speed and the corresponding relationship of the calibration; The further assessment module of the potential safety hazard is configured to calculate all the character heads within a set distance from the "potential safety hazard" area level, and if there is no minor within the set distance, the "potential safety hazard" is defined as a safety hazard; if there is only a minor within the set distance, the "potential safety hazard" is defined as "no safety hazard".

Images