Wall-climbing behavior detection method, device, medium and equipment based on patrol robot

Abstract

The present invention discloses a method for detecting climbing-over-wall behavior based on a patrol robot, including: acquiring image information in different scenarios collected by a camera on the patrol robot; performing wall detection on the image information by means of a semantic segmentation method based on deep learning to obtain the wall area in the image information; analyzing the wall area by means of image processing and statistical methods to obtain the upper edge information and lower edge information of the wall; performing human body detection on the image information by means of a deep learning object detection method to obtain the human body information in the image information; judging the climbing-over-wall behavior according to the human body information, the upper edge information and the lower edge information of the wall; and outputting an alarm message according to the judgment result of the climbing-over-wall behavior. The present invention solves the problems existing in the existing detection of climbing-over-wall behavior, such as fixed detection angle, poor accuracy and high cost, and realizes multi-scenario and multi-angle patrol detection.

Description

Technical Field

[0001] The present invention relates to the field of video surveillance, and in particular to a method, device, medium and equipment for detecting climbing-over-wall behavior based on a patrol robot. Background Art

[0002] When detecting climbing-over-wall behavior in the prior art, a camera is installed on the wall to collect video streams, and then a preset algorithm is used to detect and analyze the video streams frame by frame to capture the illegal climbing-over-wall behavior of suspicious persons and output an alarm message. However, the method of installing a camera can only collect video streams at specific positions and detect climbing-over-wall behavior at specific angles, and cannot achieve multi-scene and multi-angle patrol detection; moreover, the accuracy of climbing-over-wall behavior detection is not good; if it is necessary to detect multiple scenes, cameras must be installed in different areas of the wall, which greatly increases the detection cost. Summary of the Invention

[0003] Embodiments of the present invention provide a method, device, medium and equipment for detecting climbing-over-wall behavior based on a patrol robot, so as to solve the problems of fixed detection angle, poor accuracy and high cost existing in the existing climbing-over-wall behavior detection, and achieve multi-scene and multi-angle patrol detection.

[0004] A method for detecting climbing-over-wall behavior based on a patrol robot, the method comprising:

[0005] Obtaining image information in different scenes collected by a camera on the patrol robot;

[0006] Performing wall detection on the image information based on a semantic segmentation method of deep learning to obtain a wall area in the image information;

[0007] Analyzing the wall area based on an image processing and statistical method to obtain upper edge information and lower edge information of the wall;

[0008] Performing human body detection on the image information based on a deep learning object detection method to obtain human body information in the image information;

[0009] Judging climbing-over-wall behavior according to the human body information, the upper edge information and the lower edge information of the wall;

[0010] Outputting an alarm message according to the climbing-over-wall behavior judgment result.

[0011] Optionally, the performing wall detection on the image information based on a semantic segmentation method of deep learning to obtain a wall area in the image information includes:

[0012] Constructing a wall detection model and training the model with wall images under several different perspectives and environmental factors;

[0013] Use the trained model to detect the walls in the image information to obtain the wall areas in the image information;

[0014] Among them, the wall detection model uses a deep learning-based semantic segmentation method to classify each pixel in the image information into background and wall, and all pixels belonging to the wall classification form a wall mask.

[0015] Optionally, analyzing the wall area based on image processing and statistical methods to obtain the upper edge information and lower edge information of the wall includes:

[0016] Traverse each column mask in the wall area to obtain the abscissa of the mask in each column;

[0017] Obtain the mask with the smallest abscissa in each column mask as the upper edge point of the wall for that column mask, and combine the upper edge points of the wall to obtain the upper edge information of the wall;

[0018] Obtain the mask with the largest abscissa in each column mask as the lower edge point of the wall for that column mask, and combine the lower edge points of the wall to obtain the lower edge information of the wall.

[0019] Optionally, analyzing the wall area based on image processing and statistical methods to obtain the upper edge information and lower edge information of the wall further includes:

[0020] Perform linear fitting on the upper edge information of the wall to obtain the representation of the upper edge line of the wall.

[0021] Optionally, performing human detection on the image information based on the deep learning object detection method to obtain the human information in the image information includes:

[0022] Construct a human detection model and train the model with human images under several different perspectives and environmental factors;

[0023] Use the trained model to perform human detection on the image information to obtain a bounding box containing human information;

[0024] Among them, the human detection model performs human detection on the image information based on a convolutional neural network and uses a bounding box to mark the human information from the image information.

[0025] Optionally, the judgment of climbing-over behavior according to the human information, the upper edge information and the lower edge information of the wall includes:

[0026] Obtain the human information, and calculate the center point coordinates of the bounding box according to the bounding box of the human information;

[0027] Judge whether the abscissa of the center point of the bounding box falls within the abscissa range represented by the upper edge line of the wall;

[0028] If the abscissa of the center point of the bounding box falls within the range of the abscissa represented by the upper edge line of the wall, then obtain the ordinate corresponding to the abscissa of the center point of the bounding box on the upper edge line of the wall;

[0029] Determine whether the ordinate of the upper left corner of the bounding box is greater than the ordinate corresponding to the abscissa of the center point of the bounding box on the upper edge line of the wall;

[0030] If the ordinate of the upper left corner of the bounding box is greater than the ordinate corresponding to the abscissa of the center point of the bounding box on the upper edge line of the wall, then obtain the maximum height of the wall in the image information according to the upper edge information and the lower edge information of the wall;

[0031] Determine whether the maximum height of the wall is greater than the height of the bounding box;

[0032] If the maximum height of the wall is greater than the height of the bounding box, then horizontally magnify the bounding box and calculate the proportion of the wall height in the horizontally magnified bounding box;

[0033] Determine whether the proportion is greater than a preset threshold. If so, determine that there is a behavior of climbing over the wall.

[0034] Optionally, the obtaining the maximum height of the wall in the image information according to the upper edge information and the lower edge information of the wall includes:

[0035] Calculate the height information of all wall positions in the image information according to the upper edge information and the lower edge information of the wall, where the height information Hwi = Diy - Uiy, Diy represents the ordinate of the i-th upper edge point of the wall, and Uiy represents the ordinate of the i-th lower edge point of the wall;

[0036] Obtain the maximum value in the height information as the maximum height of the wall in the image information.

[0037] A device for detecting the behavior of climbing over the wall based on a patrol robot, the device includes:

[0038] An acquisition module, configured to acquire image information of different scenes collected by a camera on the patrol robot;

[0039] A wall detection module, configured to perform wall detection on the image information based on the semantic segmentation method of deep learning to obtain the wall area in the image information;

[0040] An edge information acquisition module, configured to analyze the wall area based on image processing and statistical methods to obtain the upper edge information and the lower edge information of the wall;

[0041] A human body detection module, configured to perform human body detection on the image information based on a deep learning object detection method to obtain human body information in the image information;

[0042] A judgment module, configured to judge the behavior of climbing over the wall according to the human body information, the upper edge information and the lower edge information of the wall;

[0043] An alarm module, configured to output alarm information according to the result of the judgment of the behavior of climbing over the wall.

[0044] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for detecting the behavior of climbing over the wall based on a patrol robot as described above is implemented.

[0045] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method for detecting the behavior of climbing over the wall based on a patrol robot as described above is implemented.

[0046] In the embodiment of the present invention, by acquiring image information in different scenarios collected by a camera on a patrol robot, there is no need to fix the scenario or angle; then, based on a deep learning semantic segmentation method, wall detection is performed on the image information to obtain the wall area in the image information; based on image processing and statistical methods, the wall area is analyzed to obtain the upper edge information and the lower edge information of the wall; based on a deep learning object detection method, human body detection is performed on the image information to obtain human body information in the image information; finally, according to the human body information, the upper edge information and the lower edge information of the wall, the behavior of climbing over the wall is judged; and alarm information is output according to the result of the judgment of the behavior of climbing over the wall; effectively improving the accuracy of detecting the behavior of climbing over the wall, realizing multi-scenario and multi-angle patrol detection, and solving the problems of fixed detection angle, poor accuracy and high cost existing in the existing detection of the behavior of climbing over the wall. Description of the Drawings

[0047] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments of the present invention. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained according to these drawings.

[0048] Figure 1 is a flowchart of a method for detecting the behavior of climbing over the wall based on a patrol robot provided by an embodiment of the present invention;

[0049] Figure 2 is an implementation flowchart of step S103 in the method for detecting the behavior of climbing over the wall based on a patrol robot provided by an embodiment of the present invention;

[0050] Figure 3 This is a perspective diagram of a wrong judgment of circumvention provided by an embodiment of the present invention;

[0051] Figure 4 It is a flowchart for implementing step S105 in the wall-climbing behavior detection method based on a patrol robot provided in one embodiment of the present invention;

[0052] Figure 5 It is a structural schematic diagram of a wall-climbing behavior detection device based on a patrol robot provided by an embodiment of the present invention;

[0053] Figure 6 is a schematic diagram of a computer device in one embodiment of the present invention. DETAILED DESCRIPTION

[0054] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0055] This embodiment obtains image information of different scenes collected by the camera on the patrol robot without fixing the scene or angle; then performs wall detection on the image information based on the deep learning semantic segmentation method to obtain the wall area in the image information; analyzes the wall area based on image processing and statistical methods to obtain the upper edge information and lower edge information of the wall; performs human body detection on the image information based on the deep learning target detection method to obtain human body information in the image information; finally, determines the wall-climbing behavior based on the human body information and the upper and lower edge information of the wall; and outputs alarm information based on the result of the wall-climbing behavior determination; effectively improves the accuracy of wall-climbing behavior detection, realizes patrol detection in multiple scenes and multiple angles, and solves the problems of fixed detection angle, poor accuracy and high cost in existing wall-climbing behavior detection.

[0056] The following is a detailed description of the wall-climbing behavior detection method based on the patrol robot provided in this embodiment. Figure 1 As shown, the wall-climbing behavior detection method based on patrol robots includes:

[0057] In step S101, image information of different scenes captured by a camera on the patrol robot is obtained.

[0058] Here, in the embodiment of the present invention, a camera is installed on a patrol robot, and the patrol robot patrols near the fence to collect video streams, which include image information in different scenes.

[0059] In step S102, a semantic segmentation method based on deep learning is used to detect the enclosure in the image information, and the enclosure area in the image information is obtained.

[0060] Here, semantic segmentation is a basic task in computer vision. Its essence is to classify each pixel in the image. Therefore, each pixel in the image is judged and divided into either the background or the enclosure. All pixels of the same category are called the foreground mask of that category. Detecting the enclosure through the deep learning semantic segmentation method is divided into the training and inference stages. In the embodiment of the present invention, step S102 further includes:

[0061] In step S201, an enclosure detection model is constructed, and the model is trained using enclosure images under several different perspectives and environmental factors.

[0062] In the training stage, this embodiment uses a large number of enclosure images under different perspectives and environmental factors to train the model. To further improve the robustness of the algorithm, data augmentation methods such as randomly changing the brightness of the enclosure images and random cropping are used to further enrich the diversity of the training data.

[0063] The enclosure detection model uses a semantic segmentation method based on deep learning to classify each pixel in the image information into the background and the enclosure. All pixels belonging to the enclosure classification form an enclosure mask, and all pixels belonging to the background classification form a background mask.

[0064] In step S202, the trained model is used to detect the enclosure in the image information, and the enclosure area in the image information is obtained.

[0065] In the inference application stage, the trained model is used for semantic segmentation of the enclosure. The semantic segmentation method includes, but is not limited to, deep learning methods such as U-Net and DeepLab, and is not restricted to other forms of image segmentation methods.

[0066] In step S103, based on image processing and statistical methods, the enclosure area is analyzed to obtain the upper edge information and lower edge information of the enclosure.

[0067] Here, in this embodiment, based on the image processing method, the enclosure mask obtained through semantic segmentation is processed, the point sets of the upper and lower edges of the enclosure are statistically obtained, and a straight line fitting is performed on the point set of the upper edge of the enclosure to obtain the straight line representation of the upper edge of the enclosure. Optionally, as Figure 2 shown, step S103 further includes:

[0068] In step S301, each column mask in the enclosure area is traversed, and the abscissa of the mask in each column mask is obtained.

[0069] In step S302, obtain the mask with the smallest abscissa in each column of masks as the upper edge point of the fence for that column of masks, and combine the upper edge points of the fence to obtain the upper edge information of the fence.

[0070] In step S303, obtain the mask with the largest abscissa in each column of masks as the lower edge point of the fence for that column of masks, and combine the lower edge points of the fence to obtain the lower edge information of the fence.

[0071] Here, the embodiments of the present invention obtain the set of all coordinate points of the fence, and perform statistics for each column of the fence mask as a unit to obtain the abscissa of the fence mask in each column. Compare the abscissas of the fence masks in the same column. The mask with the smallest abscissa is the upper edge point of the fence for the current column, and the mask with the largest abscissa is the lower edge point of the fence for the current column. Combine all the upper edge points of the fence to obtain the upper edge information of the fence in the image information, and combine all the lower edge points of the fence to obtain the lower edge information of the fence in the image information.

[0072] Optionally, as a preferred example of the present invention, after step S302 "obtain the mask with the smallest abscissa in each column of masks as the upper edge point of the fence for that column of masks, and combine the upper edge points of the fence to obtain the upper edge information of the fence", step S103 further includes:

[0073] In step S304, perform linear fitting on the upper edge information of the fence to obtain the representation of the upper edge line of the fence.

[0074] After the above analysis, a set of data of the upper edge points is obtained. Here, the linear fitting uses the least squares method. The fitted line is equivalent to the side line of the wall top. By fitting the points into a straight line, it is more convenient to determine whether a person climbs over the side line of the wall top.

[0075] In step S104, perform human body detection on the image information based on the deep learning object detection method to obtain the human body information in the image information.

[0076] Optionally, in the embodiments of the present invention, the human body detection based on deep learning object detection obtains the target position of the human body in the image information through a convolutional neural network, and marks the target position from the image information with a bounding box. The process can be divided into a training stage and an inference stage. Step S104 further includes:

[0077] In step S401, construct a human body detection model, and use human body images under several different perspectives and environmental factors for model training.

[0078] Similarly, in the training phase, the embodiments of the present invention need to collect a large number of human body images from different perspectives and environments for model training. In order to further enrich the training data and improve the robustness of the algorithm, the embodiments of the present invention also further improve the diversity of the training data by means of data augmentation such as random brightness and random cropping.

[0079] In step S402, the trained model is used to perform human body detection on the image information to obtain a bounding box containing human body information.

[0080] Among them, the human body detection model performs human body detection on the image information based on a convolutional neural network, and uses a bounding box to mark the human body information from the image information.

[0081] In the embodiments of the present invention, the trained model is used for human body detection in the inference application phase. Its essence is to obtain the detection box of the target object through classification recognition and regression calculation. The embodiments of the present invention mainly use Yolov5 as the main framework for object detection, and the methods of object detection include but are not limited to FasterRcnn, Yolo, and SSD series, etc. Among them, FasterRcnn, Yolo, and SSD are open-source object detection algorithm frameworks on the Internet.

[0082] In step S105, a wall-climbing behavior is judged according to the human body information, the upper edge information and the lower edge information of the wall.

[0083] The existing condition for judging wall climbing is whether there is an intersection between the human body and the wall. If there is an intersection between the human body and the wall, there is a wall-climbing behavior. However, for a patrol robot, it is very likely that the judgment will be interfered by different perspective problems, such as Figure 3 shown. This kind of perspective problem is mainly due to the fact that the human body is relatively closer to the acquisition camera with respect to the wall, which is likely to cause a misaligned perspective of the human body climbing over the wall. The manifestation of this incorrect perspective problem is mainly that the human body appears "higher" than the wall in the image.

[0084] The embodiments of the present invention judge the wall-climbing behavior by judging the positional relationship between the human body and the wall. The embodiments of the present invention follow the wall-climbing behavior judgment criteria of "the person intersects with the upper edge of the wall" and "the person is close to the wall and the person is lower than the wall", which can effectively reduce the incorrect judgment caused by perspective problems. Optionally, as Figure 4 shown, step S105 further includes:

[0085] In step S501, the human body information is obtained, and the center point coordinates of the bounding box are calculated according to the bounding box of the human body information.

[0086] As described above, in the embodiments of the present invention, bounding boxes are used to mark the human body information in the image information. According to the bounding boxes, the embodiments of the present invention assume that the position of the bounding box is A(X1, Y1, W1, H1), where (X1, Y1) represents the upper left corner coordinates of the bounding box in the image information, W1 represents the width of the bounding box, and H1 represents the height of the bounding box. The center point coordinates include the abscissa Xc and the ordinate Yc, where Xc = X1 + W1 / 2 and Yc = Y1 + H1 / 2.

[0087] In step S502, it is determined whether the abscissa of the center point of the bounding box falls within the range of the abscissa represented by the upper edge line of the wall.

[0088] Here, it is assumed that the left end point coordinates of the upper edge line L of the wall are (Lx0, Ly0), and the right end point coordinates are (Lx1, Ly1). The embodiments of the present invention determine whether the abscissa Xc of the center point is within the range of [Lx0, Lx1]. If so, it means that the human body is within the wall range, and step S503 is executed; otherwise, it means that the human body is not within the wall range, and it can be determined that there is no behavior of climbing over the wall.

[0089] In step S503, the ordinate corresponding to the abscissa of the center point of the bounding box is obtained on the upper edge line of the wall.

[0090] Here, when the human body is within the wall range, the embodiments of the present invention further find the target point (Lxc, Lyc) on the upper edge line L of the wall when the abscissa is the abscissa Xc of the center point of the bounding box, and obtain the ordinate Lyc of the target point, so as to obtain the ordinate Lyc corresponding to the abscissa of the center point of the bounding box on the upper edge line of the wall.

[0091] In step S504, it is determined whether the ordinate of the upper left corner of the bounding box is greater than the ordinate corresponding to the abscissa of the center point of the bounding box on the upper edge line of the wall.

[0092] The ordinate corresponding to the abscissa of the center point of the bounding box on the upper edge line of the wall refers to the ordinate of the point on the upper edge line of the wall where the abscissa is the same as the abscissa of the center point of the bounding box. The embodiments of the present invention determine whether the ordinate Y1 of the upper left corner of the bounding box is greater than the ordinate Lyc corresponding to the abscissa of the center point of the bounding box on the upper edge line of the wall. If so, it means that part of the human body is higher than the upper edge of the wall, and step S505 is executed; otherwise, it means that part of the human body is not higher than the upper edge of the wall, and it can be determined that there is no behavior of climbing over the wall.

[0093] In step S505, the maximum height of the wall in the image information is obtained according to the upper edge information and the lower edge information of the wall.

[0094] Here, for ease of understanding, in the embodiments of the present invention, it is assumed that the set of upper edge points corresponding to the upper edge information of the wall is {U1, U2,..., Ui}, where Uix represents the abscissa of the i-th upper edge point Ui of the wall, Uiy represents the ordinate of the i-th upper edge point Ui of the wall, the set of lower edge points corresponding to the lower edge information of the wall is {D1, D2,..., Di}, Dix represents the abscissa of the i-th lower edge point Di of the wall, and Diy represents the ordinate of the i-th lower edge point Di of the wall.

[0095] Step S505 further includes:

[0096] In step S601, according to the upper edge information and the lower edge information of the wall, calculate the height information of all wall positions in the image information, where the height information is the difference between the ordinate of the lower edge information and the ordinate of the upper edge information in the i-th column of the wall mask, that is, Hwi = Diy - Uiy.

[0097] In step S602, obtain the maximum value in the height information as the maximum height of the wall in the image information.

[0098] In the embodiments of the present invention, compare the height information Hwi, and select the maximum value among them as the maximum height Hmax of the wall in the image information.

[0099] In step S506, determine whether the maximum height of the wall is greater than the height of the bounding box.

[0100] Here, in the embodiments of the present invention, compare the maximum height Hmax of the wall with the height H1 of the bounding box, and determine whether the maximum height Hmax of the wall is greater than the height H1 of the bounding box. If so, it means that the wall is higher than the human body, and execute step S507; otherwise, it means that the wall is not higher than the human body, and it can be determined that there is no behavior of climbing over the wall.

[0101] In step S507, horizontally magnify the bounding box, and calculate the proportion of the wall height in the horizontally magnified bounding box.

[0102] In practical applications, to determine whether a person is approaching the wall, for example, when a person approaches the wall, the greater the wall height corresponding to the inside of the bounding box of the human body information, and when the person is far from the wall, the smaller the wall height corresponding to the inside of the bounding box of the human body information. In the embodiments of the present invention, the bounding box is appropriately horizontally magnified, and the proportion r of the wall height in the magnified bounding box is calculated. The proportion r represents the ratio of the maximum value of the wall height in the bounding box to the height of the bounding box. By obtaining the maximum value Hwn of the wall height inside the bounding box of the human body information and knowing the height H1 of the bounding box of the human body information, and then calculating the ratio of Hwn to H1, the proportion r = Hwn / H1 is obtained, where the larger r is, the closer to the wall it reflects.

[0103] In step S508, it is determined whether the ratio is greater than a preset threshold. If so, it is determined that there is a behavior of climbing over the wall.

[0104] The preset threshold here refers to the r value corresponding to when a person approaches the wall. That is, when r is greater than the preset threshold, it can be considered that a person approaches the wall. The acquisition method of the preset value needs to be an empirical value obtained through a large number of tests.

[0105] If the ratio is greater than the preset threshold, it indicates that the human body is close to the wall, and it is determined that there is a behavior of climbing over the wall; otherwise, it indicates that the human body is not close to the wall, and it is determined that there is no behavior of climbing over the wall, and a climbing-over-wall detection result is generated.

[0106] In step S106, an alarm message is output according to the result of the climbing-over-wall behavior judgment.

[0107] After obtaining the climbing-over-wall detection result, when the detection result is that there is a behavior of climbing over the wall, an alarm message is generated and the alarm message is output, including but not limited to voice broadcast, buzzer alarm, and sending alarm text messages, emails, etc. to relevant departments or personnel.

[0108] In the embodiment of the present invention, by installing a camera on the patrol robot and combining methods such as semantic segmentation and object detection, wall detection is realized; then, through image processing and statistical methods, geometric representations of the enclosure wall in multiple scenarios and at multiple angles are obtained, such as the upper and lower edge point sets and straight line representations of the enclosure wall, etc., providing effective information for judging whether a person climbs over the wall; finally, following the judgment criteria for the climbing-over-wall behavior of "a person intersects with the upper edge of the wall" and "a person is close to the wall and the person is lower than the wall", the misjudgment of the climbing-over-wall behavior caused by the perspective problem can be effectively reduced, realizing the detection of the climbing-over-wall behavior of personnel in multiple scenarios and at multiple angles, effectively improving the detection efficiency of the climbing-over-wall behavior of personnel in multiple scenarios and at multiple angles, and reducing costs.

[0109] It should be understood that the magnitudes of the sequence numbers of the steps in the above embodiments do not mean the order of execution. The execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.

[0110] In an embodiment, the present invention further provides a climbing-over-wall behavior detection device based on a patrol robot, and the climbing-over-wall behavior detection device based on a patrol robot corresponds one-to-one to the climbing-over-wall behavior detection method based on a patrol robot in the above embodiment. As Figure 5 shown, the climbing-over-wall behavior detection device based on a patrol robot includes an acquisition module 51, an enclosure wall detection module 52, an edge information acquisition module 53, a human body detection module 54, a judgment module 55, and an alarm module 56. The detailed description of each functional module is as follows:

[0111] The acquisition module 51 is used to acquire image information of different scenarios collected by the camera on the patrol robot;

[0112] A fence detection module 52, configured to perform fence detection on the image information based on a semantic segmentation method of deep learning to obtain a fence area in the image information;

[0113] An edge information acquisition module 53, configured to analyze the fence area based on image processing and statistical methods to obtain upper edge information and lower edge information of the fence;

[0114] A human body detection module 54, configured to perform human body detection on the image information based on a deep learning object detection method to obtain human body information in the image information;

[0115] A judgment module 55, configured to perform an over-the-wall behavior judgment according to the human body information, the upper edge information and the lower edge information of the fence;

[0116] An alarm module 56, configured to output an alarm message according to the result of the over-the-wall behavior judgment.

[0117] Optionally, the fence detection module 52 includes:

[0118] A fence detection model training unit, configured to construct a fence detection model and train the model by using fence images under several different perspectives and environmental factors;

[0119] A fence detection unit, configured to perform fence detection on the image information by using the trained model to obtain a fence area in the image information;

[0120] Wherein the fence detection model classifies the background and the fence for each pixel in the image information by using a semantic segmentation method of deep learning, and all pixels belonging to the fence classification form a fence mask.

[0121] Optionally, the edge information acquisition module 53 includes:

[0122] An abscissa acquisition unit, configured to traverse each column mask in the fence area and acquire the abscissa of the mask in each column mask;

[0123] An upper edge information acquisition unit, configured to acquire the mask with the smallest abscissa in each column mask as the upper edge point of the fence of the column mask, and combine the upper edge points of the fence to obtain the upper edge information of the fence;

[0124] A lower edge information acquisition unit, configured to acquire the mask with the largest abscissa in each column mask as the lower edge point of the fence of the column mask, and combine the lower edge points of the fence to obtain the lower edge information of the fence.

[0125] Optionally, the edge information acquisition module 53 further includes:

[0126] Performing linear fitting on the upper edge information of the fence to obtain a representation of the upper edge line of the fence.

[0127] Optionally, the human body detection module 54 includes:

[0128] A human body detection model training unit for constructing a human body detection model and training the model using human body images under several different perspectives and environmental factors;

[0129] A human body detection unit for performing human body detection on the image information using the trained model to obtain a bounding box containing human body information;

[0130] Wherein the human body detection model performs human body detection on the image information based on a convolutional neural network and marks the human body information from the image information using a bounding box.

[0131] Optionally, the judgment module 55 includes:

[0132] A first calculation unit for obtaining the human body information and calculating the coordinates of the center point of the bounding box according to the bounding box of the human body information;

[0133] A first judgment unit for judging whether the abscissa of the center point of the bounding box falls within the range of the abscissa represented by the upper edge line of the wall;

[0134] A second calculation unit for calculating the ordinate corresponding to the abscissa of the center point of the bounding box on the upper edge line of the wall when the judgment result of the first judgment unit is yes;

[0135] A second judgment unit for judging whether the ordinate of the upper left corner of the bounding box is greater than the ordinate corresponding to the abscissa of the center point of the bounding box on the upper edge line of the wall;

[0136] A maximum height obtaining unit for obtaining the maximum height of the wall in the image information according to the upper edge information and the lower edge information of the wall when the judgment result of the second judgment unit is yes;

[0137] A third judgment unit for judging whether the maximum height of the wall is greater than the height of the bounding box;

[0138] A third calculation unit for horizontally magnifying the bounding box when the judgment result of the third judgment unit is yes and calculating the proportion of the wall height of the horizontally magnified bounding box;

[0139] A wall climbing behavior judgment unit for judging whether the proportion is greater than a preset threshold, and if so, determining that there is a wall climbing behavior.

[0140] Optionally, the maximum height obtaining unit includes:

[0141] A height calculation subunit, configured to calculate the height information of all wall positions in the image information according to the upper edge information and the lower edge information of the fence, where the height information Hwi = Diy - Uiy, Diy represents the ordinate of the upper edge point of the i-th fence, and Uiy represents the ordinate of the lower edge point of the i-th fence;

[0142] A maximum height acquisition subunit, configured to obtain the maximum value in the height information as the maximum height of the fence in the image information.

[0143] For the specific limitations of the climbing-over-the-fence behavior detection device based on the patrol robot, reference can be made to the limitations of the climbing-over-the-fence behavior detection method based on the patrol robot in the above text, which will not be elaborated here. Each module in the above climbing-over-the-fence behavior detection device based on the patrol robot can be implemented in whole or in part by software, hardware, and their combination. The above-mentioned modules can be embedded in the processor of the computer device in hardware form or be independent of it, or can be stored in the memory in software form, so as to facilitate the processor to call and execute the operations corresponding to the above-mentioned modules.

[0144] In one embodiment, a computer device is provided. The computer device can be a server, and its internal structure diagram can be as Figure 6 shown. The computer device includes a processor, a memory, a network interface, and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, it implements a climbing-over-the-fence behavior detection method based on a patrol robot.

[0145] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the following steps are implemented:

[0146] Obtain image information of different scenes collected by a camera on the patrol robot;

[0147] Perform fence detection on the image information based on the semantic segmentation method of deep learning to obtain the fence area in the image information;

[0148] Analyze the fence area based on image processing and statistical methods to obtain the upper edge information and the lower edge information of the fence;

[0149] Perform human detection on the image information based on the deep learning object detection method to obtain the human information in the image information;

[0150] Judge the behavior of climbing over the wall according to the human information, the upper edge information and the lower edge information of the wall;

[0151] Output an alarm message according to the judgment result of the behavior of climbing over the wall.

[0152] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database or other medium used in the embodiments provided by the present invention can include non-volatile and / or volatile memories. Non-volatile memories can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memories can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0153] Those skilled in the art can clearly understand that for the convenience and brevity of description, only the above division of each functional unit and module is used as an example. In actual applications, the above functions can be allocated to different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above.

[0154] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A method for detecting climbing-over-wall behavior based on a patrol robot, characterized in that Including: Obtain image information of different scenarios collected by a camera on a patrol robot; Perform wall detection on the image information by using a semantic segmentation method based on deep learning to obtain the wall area in the image information; Analyze the wall area by using image processing and statistical methods to obtain the upper edge information and lower edge information of the wall; Perform human detection on the image information by using a deep learning object detection method to obtain human information in the image information; Obtain the human information, and calculate the coordinates of the center point of the bounding box according to the bounding box of the human information; Judge whether the abscissa of the center point of the bounding box falls within the abscissa range represented by the upper edge line of the wall; If the abscissa of the center point of the bounding box falls within the abscissa range represented by the upper edge line of the wall, then obtain the ordinate corresponding to the abscissa of the center point of the bounding box on the upper edge line of the wall; Judge whether the ordinate of the upper left corner of the bounding box is greater than the ordinate corresponding to the abscissa of the center point of the bounding box on the upper edge line of the wall; If the ordinate of the upper left corner of the bounding box is greater than the ordinate corresponding to the abscissa of the center point of the bounding box on the upper edge line of the wall, then obtain the maximum height of the wall in the image information according to the upper edge information and lower edge information of the wall; Judge whether the maximum height of the wall is greater than the height of the bounding box; If the maximum height of the wall is greater than the height of the bounding box, then horizontally magnify the bounding box and calculate the proportion of the wall height in the horizontally magnified bounding box; Judge whether the proportion is greater than a preset threshold. If so, determine that there is a behavior of climbing over the wall; Output an alarm message according to the judgment result of the behavior of climbing over the wall.

2. The method for detecting over-the-wall behavior based on a patrol robot according to claim 1, wherein The performing wall detection on the image information by using a semantic segmentation method based on deep learning to obtain the wall area in the image information includes: Construct a wall detection model, and use wall images under several different perspectives and environmental factors for model training; Use the trained model to perform wall detection on the image information to obtain the wall area in the image information; Wherein the wall detection model classifies each pixel in the image information into background and wall by using a semantic segmentation method based on deep learning, and all pixels belonging to the wall classification constitute a wall mask.

3. The method for detecting over-the-wall behavior based on a patrol robot according to claim 2, wherein The analyzing the wall area by using image processing and statistical methods to obtain the upper edge information and lower edge information of the wall includes: Traverse each column mask in the wall area, and obtain the abscissa of the mask in each column mask; Obtain the mask with the smallest abscissa in each column mask as the upper edge point of the wall of the column mask, and combine the upper edge points to obtain the upper edge information of the wall; Obtain the mask with the largest abscissa in each column mask as the lower edge point of the wall of the column mask, and combine the lower edge points to obtain the lower edge information of the wall.

4. The method for detecting over-the-wall behavior based on a patrol robot according to claim 3, wherein The analyzing the wall area by using image processing and statistical methods to obtain the upper edge information and lower edge information of the wall further includes: Perform linear fitting on the upper edge information of the wall to obtain the representation of the upper edge line of the wall.

5. The method for detecting wall-climbing behavior based on a patrol robot according to claim 1, wherein The performing human detection on the image information by using a deep learning object detection method to obtain human information in the image information includes: Build a human detection model and use human body images under several different perspectives and environmental factors for model training; Use the trained model to perform human detection on the image information to obtain a bounding box containing human information; Among them, the human detection model performs human detection on the image information based on a convolutional neural network, and uses a bounding box to mark the human information from the image information.

6. The method for detecting over-the-wall behavior based on a patrol robot according to claim 5, wherein, The obtaining of the maximum height of the wall in the image information according to the upper edge information and the lower edge information of the wall includes: Calculate the height information of all wall positions in the image information according to the upper edge information and the lower edge information of the wall, where the height information Hwi = Diy - Uiy, Diy represents the ordinate of the upper edge point of the i-th wall, and Uiy represents the ordinate of the lower edge point of the i-th wall; Obtain the maximum value in the height information as the maximum height of the wall in the image information.

7. A wall-climbing behavior detection device based on a patrol robot, characterized in that, Includes: An acquisition module for acquiring image information in different scenarios collected by a camera on a patrol robot; A wall detection module for performing wall detection on the image information based on a semantic segmentation method of deep learning to obtain the wall area in the image information; An edge information acquisition module for analyzing the wall area based on image processing and statistical methods to obtain the upper edge information and the lower edge information of the wall; A human detection module for performing human detection on the image information based on a deep learning object detection method to obtain human information in the image information; A judgment module for judging the over-the-wall behavior according to the human information, the upper edge information and the lower edge information of the wall; The judgment of the over-the-wall behavior according to the human information, the upper edge information and the lower edge information of the wall includes: Obtain the human information and calculate the center point coordinates of the bounding box according to the bounding box of the human information; Judge whether the abscissa of the center point of the bounding box falls within the abscissa range represented by the upper edge line of the wall; If the abscissa of the center point of the bounding box falls within the abscissa range represented by the upper edge line of the wall, then obtain the ordinate corresponding to the abscissa of the center point of the bounding box on the upper edge line of the wall; Judge whether the ordinate of the upper left corner of the bounding box is greater than the ordinate corresponding to the abscissa of the center point of the bounding box on the upper edge line of the wall; If the ordinate of the upper left corner of the bounding box is greater than the ordinate corresponding to the abscissa of the center point of the bounding box on the upper edge line of the wall, then obtain the maximum height of the wall in the image information according to the upper edge information and the lower edge information of the wall; Judge whether the maximum height of the wall is greater than the height of the bounding box; If the maximum height of the wall is greater than the height of the bounding box, then horizontally enlarge the bounding box and calculate the proportion of the wall height of the horizontally enlarged bounding box; Judge whether the proportion is greater than a preset threshold. If so, determine that there is an over-the-wall behavior; An alarm module for outputting an alarm message according to the result of the over-the-wall behavior judgment.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the over-the-wall behavior detection method based on a patrol robot according to any one of claims 1 to 6.

9. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method for detecting over-the-wall behavior based on a patrol robot according to any one of claims 1 to 6.

Images