A lawn trampling detection method, medium and robot

Abstract

The application provides a lawn trampling detection method, S1, acquiring a lawn picture or video; S2, detecting a human body in the picture or video and outputting a target detection frame; S3, acquiring human ankle key point prediction according to the target detection frame, and acquiring ankle part coordinates; S4, performing lawn segmentation on the picture or video to acquire lawn information; S5, correcting the ankle part coordinates to acquire foot bottom coordinates; and S6, judging whether a trampling behavior occurs according to the foot bottom coordinates and the lawn information. The ankle part coordinates are corrected to acquire the foot bottom coordinates, and whether the trampling behavior occurs is judged according to the foot bottom coordinates and the lawn information, so that the accuracy of trampling behavior prediction is improved.

Description

Technical Field

[0001] This invention relates to the field of robotics, and more specifically, to a method, medium, and robot for detecting lawn trampling. Background Technology

[0002] Many lawns and green spaces have very prominent signs prohibiting trampling on the grass, but trampling still occurs. Since the signs are only static reminders, some tourists may not notice them, meaning the reminders are not strong enough.

[0003] The background description provided herein is for the purpose of generally presenting the context of this disclosure. Unless otherwise indicated herein, the material described in this section is not prior art to the claims of this application and should not be acknowledged as prior art by virtue of its inclusion in this section. Summary of the Invention

[0004] To address the aforementioned technical problems in related technologies, this invention proposes a method for detecting lawn trampling, comprising the following steps:

[0005] S1, retrieve images or videos of the lawn;

[0006] S2, detect the human body in the image or video and output the target detection box;

[0007] S3, based on the target detection box, obtain the predicted key points of the human ankle and obtain the coordinates of the ankle.

[0008] S4, perform grassland segmentation on the image or video to obtain grassland information;

[0009] S5, correct the coordinates of the ankle area to obtain the coordinates of the sole of the foot;

[0010] S6 determines whether trampling has occurred based on the coordinates of the feet and the grass information.

[0011] Specifically, it also includes step S7, which uses voice to broadcast when a stampede is detected.

[0012] Specifically, step S5 is as follows: keeping the horizontal coordinate position of the ankle unchanged, the vertical coordinate position is corrected by averaging the vertical coordinate of the ankle with the vertical coordinate of the bottom edge of the corresponding target detection box to obtain the coordinate of the foot position.

[0013] Specifically, step S6 involves: drawing a horizontal line segment with the same length as the target detection box width based on the foot coordinates; counting the number of grass pixels within the horizontal line segment; obtaining the proportion of grass pixels to all pixels in the horizontal line segment; and then comparing it with a set threshold. If the proportion is greater than a certain threshold, it is determined to be a grass trampling behavior.

[0014] Specifically, step S2 involves using a YOLOv3-tiny network to detect human bodies in the image or video and outputting the target detection box; or step S4 involves using FCN16s to perform grass segmentation on the image or video to obtain grass information.

[0015] Secondly, another embodiment of the present invention discloses a lawn trampling detection device, which includes the following modules:

[0016] The lawn acquisition module is used to acquire lawn images or videos;

[0017] A human target detection module is used to detect human bodies in the image or video and output the target detection box.

[0018] The human ankle key point recognition module is used to obtain predictions of human ankle key points based on the target detection box and obtain the coordinates of the ankle.

[0019] The grassland segmentation module is used to segment the image or video to obtain grassland information;

[0020] The foot coordinate acquisition module is used to correct the coordinates of the ankle area and obtain the foot coordinates.

[0021] The trampling behavior detection module is used to determine whether trampling behavior has occurred based on the coordinates of the feet and the grass information.

[0022] Specifically, it also includes a voice broadcast module, which is used to broadcast a message when a stampede is detected.

[0023] Specifically, the foot coordinate acquisition module works by keeping the horizontal coordinate position of the ankle unchanged and correcting the vertical coordinate position by averaging the vertical coordinate of the ankle with the vertical coordinate of the bottom edge of the corresponding target detection box to obtain the coordinates of the foot position.

[0024] Specifically, the trampling behavior judgment module works as follows: draw a horizontal line segment with the same length as the target detection box width based on the foot coordinates, count the number of grass pixels within the horizontal line segment, obtain the proportion of grass pixels to all pixels in the horizontal line segment, and then compare it with a set threshold. If it is greater than a certain threshold, it is judged as trampling behavior.

[0025] Thirdly, another embodiment of the present invention discloses a non-volatile memory that stores instructions which, when executed by a processor, are used to implement the above-described lawn trampling detection method.

[0026] Fourthly, another embodiment of the present invention discloses a robot, the robot comprising: a processing module, a chassis, a storage module, a camera, and a voice playback device, wherein the storage module stores instructions, which, when executed, are used to implement the above-described lawn trampling detection method.

[0027] The lawn trampling detection method of this invention utilizes a YOLOv3-Tiny network for target detection to determine whether a human body exists in the image. If no target is found, the next frame is processed; otherwise, the human target detection bounding box information is returned and fed into a human keypoint recognition network to obtain the ankle coordinates. Then, an FCN network is used to obtain a mask image of the grass. Finally, combining the ankle keypoint coordinate information and the segmented grass mask, a certain logical algorithm is used to determine whether grass trampling has occurred. In this embodiment, a variable-length line segment thresholding method is used to determine whether trampling has occurred. For target detection bounding boxes of different sizes, different length detection line segments are preset, which effectively solves the problem of detecting targets of different sizes and improves the detection accuracy.

[0028] Furthermore, the lawn trampling detection method of the present invention provides reminders to lawn tramplers through robot voice prompts, which is even more effective. Attached Figure Description

[0029] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0030] Figure 1 This is a flowchart of the lawn trampling detection method provided in an embodiment of the present invention;

[0031] Figure 2 This is a schematic diagram of the human body detection module provided in an embodiment of the present invention;

[0032] Figure 3 This is a schematic diagram of the grassland segmentation module provided in an embodiment of the present invention;

[0033] Figure 4 This is a schematic diagram of ankle coordinate correction provided in an embodiment of the present invention;

[0034] Figure 5 This is a schematic diagram of the variable length line segment entry threshold method provided in an embodiment of the present invention;

[0035] Figure 6 This is a schematic diagram of the lawn trampling detection device provided in an embodiment of the present invention;

[0036] Figure 7 This is a schematic diagram of the lawn trampling detection device provided in an embodiment of the present invention. Detailed Implementation

[0037] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention are within the scope of protection of the present invention.

[0038] Example 1

[0039] refer to Figure 1 This embodiment discloses a method for detecting lawn trampling, which includes the following steps:

[0040] S1, retrieve images or videos of the lawn;

[0041] Specifically, the robot in this embodiment includes a camera that can acquire images or videos of the lawn in real time.

[0042] Specifically, the robot in this embodiment can patrol around the lawn and take corresponding photos or videos.

[0043] S2, detect the human body in the image or video and output the target detection box;

[0044] The robot-based lawn trampling detection method in this embodiment includes a human target detection module to detect the presence of human bodies. Since it will be deployed on an embedded device, the YOLOv3-tiny network, which is more lightweight than YOLOv3, is used. Its model size is much smaller, the network is more lightweight, and the calculation speed is faster. This is because the subsequent segmentation network and key point detection network of this network have a much larger computational load than the target detection stage. Therefore, the less time spent in the detection stage, the better.

[0045] The YOLOv3-tiny network has only 24 layers, a significant reduction compared to the 107 layers of YOLOv3. This tiny network contains only YOLO16 and YOLO23 layers, with sizes of 13×13 and 26×26 respectively, and each YOLO layer corresponds to only 3 anchors. The network takes a 416×416×3 image as input and generates feature maps of different scales for multi-scale prediction.

[0046] Taking a branch with an output size of 13×13×255 as an example, its meaning is as follows: First, YOLOv3-tiny divides the image into a 13x13 grid. Each grid generates three anchor box coordinates (x,y,w,h), (x,y,w,h), and (x,y,w,h), along with one confidence score. In addition, there are prediction values ​​for 80 categories. Therefore, the final output shape is 13×13×3×(5+80). Then, the NMS algorithm is used to filter the 13×13×3 output boxes.

[0047] refer to Figure 2 , Figure 2 This is a schematic diagram of the object detection network module of this application: it includes an input module, a first convolutional module, wherein the first convolutional module includes 5 convolutional kernels (conv) and 4 pooling layers (pool); a first intermediate module, a second convolutional module, wherein the second convolutional module includes 3 convolutional kernels (conv) and 2 pooling layers (pool); a second intermediate module, a third convolutional module, wherein the third convolutional module includes 2 convolutional kernels; a fourth convolutional module (last), the output of the fourth convolutional module (last) is fed to YOLO; a fifth upsampling convolutional module, the third intermediate module, wherein the fifth upsampling convolutional module upsamples the output of the second intermediate module and feeds it to the third intermediate module, the outputs of the second intermediate module and the third intermediate module are concatenated (concat), and then a fifth convolution is performed, wherein the fifth convolution includes 2 convolutional kernels; and a sixth convolutional module (last), the output of the sixth convolutional module (last) is fed to YOLO.

[0048] Specifically, if no human body is detected, it is assumed that no lawn trampling has occurred.

[0049] S3, based on the target detection box, obtain the predicted key points of the human ankle and obtain the coordinates of the ankle.

[0050] After the acquired images are processed by the human target detection module, if no human body is detected, the next frame is processed directly. Otherwise, the generated target detection box is sent to the human keypoint recognition network to predict the keypoints of the ankle area. This ultimately generates information on 13 keypoints, but we are only interested in the ankle keypoints. Therefore, we take the last two coordinates from the final detection result as the ankle coordinates. This embodiment uses a top-down, heatmap-based keypoint detection method. First, human targets are detected, then human keypoints are detected within the target detection box to obtain the ankle coordinates. The backbone network uses ResNet50 for feature extraction.

[0051] Specifically, if key points of the human ankle are not obtained, it is assumed that no grass trampling has occurred.

[0052] S4, perform grassland segmentation on the image or video to obtain grassland information;

[0053] refer to Figure 3 The grass segmentation module is used to segment the grass to obtain grass information. The performance of the segmentation network directly affects the accuracy of subsequent grass trampling judgment. The grass segmentation network in this embodiment adopts the FCN16s network, which has good performance and fast speed. The input image size is set to 224*384. Due to the basic characteristics of the FCN network, the output is the same size as the input image. However, since the image fed into the network for training has been compressed, the generated grass mask image needs to be resized back to the original image size. This is because the coordinates returned by key point detection are based on the original image size. For subsequent logical judgment processing, a unified standard is required.

[0054] Specifically, if information about the grassland cannot be obtained, it is assumed that no grass trampling has occurred.

[0055] S5, correct the coordinates of the ankle area to obtain the coordinates of the sole of the foot;

[0056] Since keypoint recognition networks can only identify the ankle position and cannot provide accurate coordinates of the sole, while the lawn trampling detection method in this embodiment requires sole coordinates to determine whether grass has been trampled, further correction of the ankle coordinates is needed to obtain the sole coordinates. Specifically, the horizontal coordinate of the ankle remains unchanged, while the vertical coordinate is corrected by averaging the vertical coordinate of the ankle with the vertical coordinate of the bottom edge of the corresponding target detection box. This yields approximate sole coordinates. See the comparison image before and after correction. Figure 4 As shown, where Figure 4 (a) is uncorrected. Figure 4 (b) is the corrected version; it can be seen that the corrected version can more accurately represent the position of the sole of the foot.

[0057] S6 determines whether trampling has occurred based on foot coordinates and grass information;

[0058] After correcting the ankle coordinates, relatively accurate foot position coordinates can be obtained, which can be used as a reference for determining whether grass has been stepped on. Figure 5 This embodiment is based on the "variable length line segment entry threshold method" of the target detection box size. That is, a horizontal line segment with the same length as the width of the target detection box is drawn with the foot position coordinates. The number of grass pixels in the horizontal line segment is counted to obtain the proportion of grass pixels to all pixels in the horizontal line segment. Then, it is compared with a set threshold. If it is greater than a certain threshold, it is judged as trampling on grass.

[0059] Furthermore, this embodiment also includes:

[0060] The S7 uses voice to announce when it detects a stampede.

[0061] For example, the robot in this embodiment includes a voice broadcasting device that can play messages such as "Please do not trample on the grass".

[0062] The lawn trampling detection method in this embodiment uses a YOLOv3-Tiny network for target detection to determine whether a human body is present in the image. If no target is found, the next frame is processed; otherwise, the human target detection bounding box information is returned and fed into a human keypoint recognition network to obtain the ankle coordinates. Then, an FCN network is used to obtain a mask image of the grass. Finally, combining the ankle keypoint coordinates and the segmented grass mask, a certain logical algorithm is used to determine whether grass trampling has occurred. This embodiment employs a variable-length line segment thresholding method to determine whether trampling has occurred. For target detection bounding boxes of different sizes, different length detection line segments are preset, effectively solving the detection problem of targets of different sizes and improving detection accuracy.

[0063] Furthermore, the lawn trampling detection method in this embodiment provides reminders to lawn tramplers via robot voice prompts, which is more effective.

[0064] Example 2

[0065] refer to Figure 6 This embodiment discloses a lawn trampling detection device, which includes the following modules:

[0066] The lawn acquisition module is used to acquire lawn images or videos;

[0067] Specifically, the robot in this embodiment includes a camera that can acquire images or videos of the lawn in real time.

[0068] Specifically, the robot in this embodiment can patrol around the lawn and take corresponding photos or videos.

[0069] A human target detection module is used to detect human bodies in the image or video and output the target detection box.

[0070] The robot-based lawn trampling detection method in this embodiment includes a human target detection module to detect the presence of human bodies. Since it will be deployed on an embedded device, the YOLOv3-tiny network, which is more lightweight than YOLOv3, is used. Its model size is much smaller, the network is more lightweight, and the calculation speed is faster. This is because the subsequent segmentation network and key point detection network of this network have a much larger computational load than the target detection stage. Therefore, the less time spent in the detection stage, the better.

[0071] The YOLOv3-tiny network has only 24 layers, a significant reduction compared to the 107 layers of YOLOv3. This tiny network contains only YOLO16 and YOLO23 layers, with sizes of 13×13 and 26×26 respectively, and each YOLO layer corresponds to only 3 anchors. The network takes a 416×416×3 image as input and generates feature maps of different scales for multi-scale prediction.

[0072] Taking a branch with an output size of 13×13×255 as an example, its meaning is as follows: First, YOLOv3-tiny divides the image into a 13x13 grid. Each grid generates three anchor box coordinates (x,y,w,h), (x,y,w,h), and (x,y,w,h), along with one confidence score. In addition, there are prediction values ​​for 80 categories. Therefore, the final output shape is 13×13×3×(5+80). Then, the NMS algorithm is used to filter the 13×13×3 output boxes.

[0073] The human ankle key point recognition module is used to obtain predictions of human ankle key points based on the target detection box and obtain the coordinates of the ankle.

[0074] After the acquired images are processed by the human target detection module, if no human body is detected, the next frame is processed directly. Otherwise, the generated target detection box is sent to the human keypoint recognition module to predict the keypoints of the ankle area. This ultimately generates information on 13 keypoints, but we are only interested in the ankle keypoints. Therefore, we take the last two coordinates from the final detection result as the ankle coordinates. This embodiment uses a top-down, heatmap-based keypoint detection method. First, human targets are detected, then human keypoints are detected within the target detection box to obtain the ankle coordinates. The backbone network uses ResNet50 for feature extraction.

[0075] The grassland segmentation module is used to segment the image or video to obtain grassland information;

[0076] The grass segmentation module is used to segment the grass to obtain grass information. The performance of the segmentation module directly affects the accuracy of subsequent grass trampling judgment. The grass segmentation module in this embodiment adopts the FCN16s network, which has good performance and fast speed. The input image size is set to 224*384. Due to the basic characteristics of the FCN network, the output is the same size as the input image. However, since the image fed into the network for training has been compressed, the generated grass mask image needs to be resized back to the original image size. This is because the coordinates returned by key point detection are based on the original image size. For subsequent logical judgment processing, a unified standard is required.

[0077] The foot coordinate acquisition module is used to correct the coordinates of the ankle area and obtain the foot coordinates.

[0078] Since key point recognition networks can only identify the position of the ankle and cannot provide accurate coordinates of the foot position, and the lawn trampling detection method in this embodiment needs to determine whether the grass has been trampled based on the foot coordinates, it is necessary to further correct the ankle coordinates to obtain the foot coordinates. The specific method is as follows: the horizontal coordinate position of the ankle remains unchanged, and the vertical coordinate position is corrected. That is, the vertical coordinate of the ankle is averaged with the vertical coordinate of the bottom edge of the corresponding target detection box, so as to obtain the approximate coordinates of the foot position.

[0079] The trampling behavior detection module is used to determine whether trampling behavior has occurred based on the coordinates of the feet and the grass information.

[0080] After correcting the ankle coordinates, relatively accurate foot coordinates can be obtained. To determine whether grass has been trampled, this embodiment uses a "variable length line segment entry threshold method" based on the target detection box size. That is, a horizontal line segment with the same length as the target detection box width is drawn using the foot coordinates. The number of grass pixels within the horizontal line segment is counted to obtain the proportion of grass pixels to all pixels in the horizontal line segment. Then, it is compared with a set threshold. If it is greater than a certain threshold, it is determined that grass has been trampled.

[0081] Furthermore, this embodiment also includes:

[0082] The voice broadcast module is used to broadcast a message via voice when a stampede is detected.

[0083] For example, the robot in this embodiment includes a voice broadcasting device that can play messages such as "Please do not trample on the grass".

[0084] The lawn trampling detection device in this embodiment uses a YOLOv3-Tiny network for target detection to determine whether a human body is present in the image. If no target is found, the next frame is processed; otherwise, the human target detection bounding box information is returned and fed into a human keypoint recognition network to obtain the ankle coordinates. Then, an FCN network is used to obtain a mask image of the grass. Finally, combining the ankle keypoint coordinates and the segmented grass mask, a certain logical algorithm is used to determine whether grass trampling has occurred. This embodiment employs a variable-length line segment thresholding method to determine whether trampling has occurred. For target detection bounding boxes of different sizes, different length detection line segments are preset, effectively solving the detection problem of targets of different sizes and improving detection accuracy.

[0085] Furthermore, the lawn trampling detection method in this embodiment provides reminders to lawn tramplers via robot voice prompts, which is more effective.

[0086] Example 3

[0087] This embodiment discloses a robot, which includes: a processing module, a chassis, a storage module, a camera, and a voice playback device. The storage module stores instructions, which, when executed, are used to implement the lawn trampling detection method as described in Embodiment 1.

[0088] Specifically, the robot in this embodiment also includes the lawn trampling detection device as described in Embodiment 2.

[0089] Example 4

[0090] refer to Figure 7 , Figure 7 This is a schematic diagram of the structure of a lawn trampling detection device according to this embodiment. The lawn trampling detection device 20 of this embodiment includes a processor 21, a memory 22, and a computer program stored in the memory 22 and executable on the processor 21. When the processor 21 executes the computer program, it implements the steps in the above method embodiments. Alternatively, when the processor 21 executes the computer program, it implements the functions of each module / module in the above device embodiments.

[0091] For example, the computer program can be divided into one or more modules, which are stored in the memory 22 and executed by the processor 21 to complete the present invention. The one or more modules can be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program in the lawn trampling detection device 20. For example, the computer program can be divided into the modules shown in Embodiment 2. The specific functions of each module are described in the working process of the device described in the above embodiments, and will not be repeated here.

[0092] The lawn trampling detection device 20 may include, but is not limited to, a processor 21 and a memory 22. Those skilled in the art will understand that the schematic diagram is merely an example of the lawn trampling detection device 20 and does not constitute a limitation on the device. It may include more or fewer components than shown in the diagram, or combine certain components, or use different components. For example, the lawn trampling detection device 20 may also include input / output devices, network access devices, buses, etc.

[0093] The processor 21 can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor. The processor 21 is the control center of the lawn trampling detection device 20, connecting all parts of the device through various interfaces and lines.

[0094] The memory 22 can be used to store the computer program and / or modules. The processor 21 implements various functions of the lawn trampling detection device 20 by running or executing the computer program and / or modules stored in the memory 22 and calling the data stored in the memory 22. The memory 22 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, phonebook, etc.). In addition, the memory 22 may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0095] If the integrated modules / modules of the lawn trampling detection device 20 are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by the processor 21, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content contained in the computer-readable medium may be appropriately added to or subtracted from the content as required by the legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium may not include electrical carrier signals and telecommunication signals.

[0096] It should be noted that the device embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the device embodiments provided by this invention, the connection relationships between modules indicate that they have communication connections, which can be specifically implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.

[0097] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for detecting lawn trampling, comprising the following steps: S1, retrieve images or videos of the lawn; S2, detect the human body in the image or video and output the target detection box; S3, based on the target detection box, obtain the predicted key points of the human ankle and obtain the coordinates of the ankle. S4, perform grassland segmentation on the image or video to obtain grassland information; S5, correct the coordinates of the ankle area to obtain the coordinates of the sole of the foot; S6 determines whether trampling has occurred based on foot coordinates and grass information; Step S5 is as follows: Keep the horizontal coordinate position of the ankle unchanged, and correct the vertical coordinate position, that is, take the average of the vertical coordinate of the ankle and the vertical coordinate of the bottom edge of the corresponding target detection box to obtain the coordinate of the foot position. Step S6 specifically involves: drawing a horizontal line segment with the same length as the target detection box width based on the foot coordinates; counting the number of grass pixels within the horizontal line segment; obtaining the proportion of grass pixels to all pixels in the horizontal line segment; and then comparing it with a set threshold. If the proportion is greater than a certain threshold, it is determined to be a grass trampling behavior.

2. The method according to claim 1 further includes step S7, which involves broadcasting a message using voice when a trampling incident is detected.

3. The method according to claim 1, wherein step S2 specifically comprises: using a YOLOv3-tiny network to detect human bodies in the image or video and outputting the target detection box; or step S4 specifically comprises: using FCN16s to perform grass segmentation on the image or video to obtain grass information.

4. A lawn trampling detection device, comprising the following modules: The lawn acquisition module is used to acquire lawn images or videos; The human target detection module is used to detect human bodies in the image or video and output a target detection box. The human ankle key point recognition module is used to obtain predictions of human ankle key points based on the target detection box and obtain the coordinates of the ankle. The grassland segmentation module is used to segment the image or video to obtain grassland information; The foot coordinate acquisition module is used to correct the coordinates of the ankle area and obtain the foot coordinates. The trampling behavior detection module is used to determine whether trampling behavior has occurred based on the coordinates of the feet and the grass information. The foot coordinate acquisition module is as follows: keeping the horizontal coordinate position of the ankle unchanged, the vertical coordinate position is corrected, that is, the vertical coordinate of the ankle is averaged with the vertical coordinate of the bottom edge of the corresponding target detection box to obtain the coordinate of the foot position; The trampling behavior judgment module specifically works as follows: draw a horizontal line segment with the same length as the target detection box width based on the foot coordinates, count the number of grass pixels within the horizontal line segment, obtain the proportion of grass pixels to all pixels in the horizontal line segment, and then compare it with a set threshold. If it is greater than a certain threshold, it is judged as trampling behavior.

5. The apparatus according to claim 4 further includes a voice broadcasting module for broadcasting a message using voice when a trampling incident is detected.

6. A non-volatile memory storing instructions that, when executed by a processor, implement the lawn trampling detection method as described in any one of claims 1-3.

7. A robot, the robot comprising: The system includes a processing module, a chassis, a storage module, a camera, and a voice playback device. The storage module stores instructions, which, when executed, are used to implement the lawn trampling detection method as described in any one of claims 1-3.

Images