Illegal dog walking event detection method and device, electronic equipment and storage medium

Abstract

This invention provides a method, device, electronic device, and storage medium for detecting illegal dog walking incidents. The method fully utilizes the characteristics of the leash, designs a leash detection model to detect inclined target leashes, and designs a human-dog-leash relationship judgment model to detect the interaction between the human and the leash and / or the interaction between the leash and the dog. Thus, the detection results of illegal dog walking incidents determined based on the interaction between the human and the leash and / or the interaction between the leash and the dog are more accurate. In addition, the entire process is completed automatically and has good intelligence.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method, apparatus, electronic device, and storage medium for detecting illegal dog walking incidents. Background Technology

[0002] With the development of society, more and more people are keeping dogs as pets. As a result, uncivilized behaviors such as walking dogs without leashes have seriously disturbed others. In order to protect the normal work and life of others, various places have increased their efforts to detect whether dogs are leashed (i.e., illegal dog walking incidents).

[0003] Currently, there are two main methods for detecting illegal dog walking. One method involves monitoring by personnel, which requires a large amount of manpower, has low efficiency in governance and management, high costs in terms of manpower and equipment, and is labor-intensive, making evidence collection and enforcement difficult. The other method is based on surveillance video. While this method saves manpower, the scenes in surveillance videos are variable, and the angles of people and dogs vary. Determining whether it is an illegal dog walking incident by estimating the distance between the dog and the person has a certain degree of bias and cannot be adapted to complex scenarios with multiple people and dogs. It also cannot effectively determine the final leash status of the dog, meaning the accuracy of detecting illegal dog walking incidents is poor.

[0004] In summary, existing methods for detecting illegal dog walking have technical limitations that prevent them from achieving both intelligence and accuracy. Summary of the Invention

[0005] In view of this, the purpose of the present invention is to provide a method, device, electronic device and storage medium for detecting illegal dog walking incidents, so as to alleviate the technical problem that existing methods for detecting illegal dog walking incidents cannot simultaneously achieve intelligence and accuracy.

[0006] In a first aspect, embodiments of the present invention provide a method for detecting illegal dog walking incidents, including:

[0007] The video stream to be detected is acquired, and the video stream to be detected is preprocessed to obtain video frame images;

[0008] A human-dog detection model is used to detect humans and dogs in the video frame image to obtain human detection boxes and / or dog detection boxes in the video frame image.

[0009] If the video frame image contains only a dog detection box, it is determined to be an illegal dog walking incident;

[0010] If the video frame image includes a human detection box and a dog detection box, then the image region that satisfies the rope length is determined in the video frame image based on the human detection box and the dog detection box.

[0011] A rope detection model is used to detect ropes in the image region that meets the rope length requirement, thereby obtaining information about the rope detection box.

[0012] If the information of the rope detection frame is that there is no rope detection frame, it is determined to be an illegal dog walking incident;

[0013] If the information of the rope detection frame is that there is a rope detection frame, then the human-dog-rope region image is determined in the video frame image based on the human detection frame, the dog detection frame and the rope detection frame;

[0014] A human-dog-leash relationship determination model is used to determine the human-dog-leash relationship in the human-dog-leash region image, thereby obtaining the relationship between the human and the leash and / or the relationship between the leash and the dog.

[0015] Whether an incident constitutes illegal dog walking is determined based on the relationship between the person and the leash and / or the relationship between the leash and the dog.

[0016] Furthermore, the video stream to be detected is preprocessed, including:

[0017] The video stream to be detected is subjected to frame extraction and decoding to obtain the video frame image after frame extraction and decoding;

[0018] The video frame image obtained by the frame extraction and decoding is scaled proportionally to obtain the video frame image.

[0019] Furthermore, based on the human detection box and the dog detection box, an image region satisfying the rope length is determined in the video frame image, including:

[0020] The length of the rope in the video frame image is calculated based on the width of the human body detection frame in the video frame image and the reference width of the ergonomic human body.

[0021] Using the center point of the dog detection frame as the origin and the length of the rope in the video frame image as the radius, draw a circle to obtain multiple target human detection frames that are closest to the center point of the dog detection frame within the corresponding circular area;

[0022] The image region satisfying the rope length is determined in the video frame image based on the maximum outer bounding box of the plurality of target human body detection boxes.

[0023] Furthermore, determining the human-dog-leash region image in the video frame image based on the human detection frame, the dog detection frame, and the leash detection frame includes:

[0024] The human-dog leash region image is determined in the video frame image based on the outer frame formed by the human detection frame and the dog detection frame closest to the center of the leash detection frame.

[0025] Furthermore, the human-dog-leash relationship determination model includes: a feature extraction module, an interaction generation module, and an interaction matching module. The model is used to determine the human-dog-leash relationship in the human-dog-leash region image, obtaining the relationship between the person and the leash and / or the relationship between the leash and the dog, including:

[0026] The image of the human-dog leash region is input into the feature extraction module, and the features of the human-dog leash region image are output.

[0027] The features of the human-dog leash region image are input into the interaction generation module, and the interaction points and interaction vectors are output. The interaction points include: the center point between the leash and the human, and the center point between the leash and the dog. The interaction vector is the vector of the interaction point toward the center point of the leash.

[0028] The information of the rope detection frame, the human body detection frame, the dog detection frame, the interaction point, and the interaction vector are input into the interaction matching module to predict the relationship between the human and the rope and / or the relationship between the rope and the dog.

[0029] Furthermore, the method also includes:

[0030] Obtain video frame image samples of illegal dog walking behavior;

[0031] The trained human-dog detection model is used to detect humans and dogs in the video frame image samples to obtain human detection boxes and / or dog detection boxes in the video frame image samples.

[0032] The length of the rope in the video frame image sample is calculated based on the width of the human detection frame in the video frame image sample and the reference width of the ergonomic human body.

[0033] Using the center point of the dog detection box as the origin and the length of the rope in the video frame image sample as the radius, draw a circle to obtain multiple target human detection boxes that are closest to the center point of the dog detection box within the corresponding circular area;

[0034] Based on the maximum bounding box of the plurality of target human body detection boxes, an image region sample that satisfies the rope length is determined in the video frame image sample;

[0035] The initial rope detection model is trained based on the image region samples that meet the rope length and the information of the rope detection boxes labeled on the image region samples to obtain the rope detection model. The information of the rope detection boxes includes: category, coordinates of the center point of the rope detection box, length of the long side of the rope detection box, length of the short side of the rope detection box, and the angle through which the long side is encountered when the X-axis is rotated clockwise. The backbone of the rope detection model includes a CBMA attention mechanism.

[0036] Furthermore, the method also includes:

[0037] Based on the human detection box, the dog detection box, and the rope detection box obtained by the trained human-dog detection model, the human-dog-rope region image sample is determined in the video frame image sample;

[0038] The initial human-dog-leash relationship judgment model is trained based on the human-dog-leash region image samples and the relationships between the human and the leash and / or the leash and the dog labeled on the human-dog-leash region image samples, to obtain the human-dog-leash relationship judgment model.

[0039] Secondly, embodiments of the present invention also provide a device for detecting illegal dog walking incidents, comprising:

[0040] An acquisition and preprocessing unit is used to acquire the video stream to be detected and preprocess the video stream to be detected to obtain video frame images;

[0041] The human-dog detection unit is used to perform human-dog detection on the video frame image using a human-dog detection model, and obtain human detection boxes and / or dog detection boxes in the video frame image.

[0042] The first determining unit is used to determine an illegal dog walking event if there is only a dog detection box in the video frame image;

[0043] The second determining unit is used to determine an image region in the video frame image that satisfies the rope length based on the human detection box and the dog detection box if the video frame image includes a human detection box and a dog detection box.

[0044] The rope detection unit is used to perform rope detection on the image region that meets the rope length using a rope detection model, and obtain information of the rope detection box;

[0045] The third determining unit is used to determine an illegal dog walking incident if the information of the rope detection frame is that there is no rope detection frame.

[0046] The fourth determining unit is used to determine the human-dog leash region image in the video frame image based on the human body detection frame, the dog detection frame, and the leash detection frame if the information of the leash detection frame is that there is a leash detection frame.

[0047] The human-dog-leash relationship determination unit is used to determine the human-dog-leash relationship in the human-dog-leash region image using a human-dog-leash relationship determination model, and to obtain the relationship between the human and the leash and / or the relationship between the leash and the dog.

[0048] The fifth determining unit is used to determine whether an incident constitutes illegal dog walking based on the relationship between the person and the leash and / or the relationship between the leash and the dog.

[0049] Thirdly, embodiments of the present invention also provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described in any of the first aspects above.

[0050] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing machine-executable instructions, which, when invoked and executed by a processor, cause the processor to perform the method described in any of the first aspects above.

[0051] In this embodiment of the invention, a method for detecting illegal dog walking incidents is provided, comprising: acquiring a video stream to be detected and preprocessing the video stream to obtain video frame images; performing human and dog detection on the video frame images using a human-dog detection model to obtain human detection boxes and / or dog detection boxes in the video frame images; if the video frame images contain only dog ​​detection boxes, then it is determined to be an illegal dog walking incident; if the video frame images contain both human and dog detection boxes, then determining an image region in the video frame images that satisfies the leash length based on the human and dog detection boxes; and performing leash detection... The model performs rope detection on image regions that meet the rope length requirement, obtaining rope detection box information. If the rope detection box information indicates no rope detection box, it is determined to be an illegal dog walking event. If the rope detection box information indicates the presence of a rope detection box, the human-dog-leash region image is determined in the video frame image based on the human detection box, dog detection box, and rope detection box. A human-dog-leash relationship judgment model is used to judge the human-dog-leash relationship in the human-dog-leash region image, obtaining the relationship between the person and the leash and / or the relationship between the leash and the dog. Based on the relationship between the person and the leash and / or the relationship between the leash and the dog, it is determined whether it is an illegal dog walking event. As described above, the method for detecting illegal dog walking incidents in this invention fully utilizes the characteristics of the leash, designs a leash detection model to detect inclined target leashes, and designs a human-dog-leash relationship judgment model to detect the interaction between the human and the leash and / or the interaction between the leash and the dog. Thus, the detection results of illegal dog walking incidents determined based on the interaction between the human and the leash and / or the interaction between the leash and the dog are more accurate. In addition, the entire process is completed automatically, with good intelligence, alleviating the technical problem that existing methods for detecting illegal dog walking incidents cannot balance intelligence and accuracy. Attached Figure Description

[0052] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0053] Figure 1 A flowchart illustrating a method for detecting illegal dog walking incidents provided in an embodiment of the present invention;

[0054] Figure 2 A schematic diagram illustrating the determination of image region samples that satisfy the rope length according to an embodiment of the present invention;

[0055] Figure 3 This is a schematic diagram of the network structure of the Backbone provided in an embodiment of the present invention;

[0056] Figure 4This is a schematic diagram of the human-dog leash relationship determination model provided in an embodiment of the present invention;

[0057] Figure 5 A schematic diagram of interaction points and interaction vectors provided in an embodiment of the present invention;

[0058] Figure 6 A schematic diagram of the interactive matching module provided in an embodiment of the present invention;

[0059] Figure 7 A schematic diagram of a detection device for illegal dog walking incidents provided in an embodiment of the present invention;

[0060] Figure 8 This is a schematic diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0061] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0062] Existing methods for detecting illegal dog walking have technical limitations that prevent them from achieving both intelligence and accuracy.

[0063] Based on this, the method for detecting illegal dog walking incidents in this invention makes full use of the rope's characteristics, designs a rope detection model to detect inclined target ropes, and designs a human-dog-leash relationship judgment model to detect the interaction between the human and the rope, as well as the interaction between the rope and the dog. Thus, the detection results of illegal dog walking incidents determined based on the interaction between the human and the rope, as well as the interaction between the rope and the dog, are more accurate. In addition, the whole process is completed automatically and has good intelligence.

[0064] To facilitate understanding of this embodiment, a method for detecting illegal dog walking incidents disclosed in this embodiment of the invention will first be described in detail.

[0065] Example 1:

[0066] According to an embodiment of the present invention, an embodiment of a method for detecting illegal dog walking incidents is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0067] Figure 1 This is a flowchart of a method for detecting illegal dog walking incidents according to an embodiment of the present invention, such as... Figure 1 As shown, the method includes the following steps:

[0068] Step S102: Obtain the video stream to be detected and preprocess the video stream to be detected to obtain video frame images;

[0069] In this embodiment of the invention, the video stream to be detected can be any video stream about walking a dog, and the preprocessing can include frame extraction decoding and scaling, with the aim of obtaining a fixed-size video frame image.

[0070] Step S104: Use a human-dog detection model to detect humans and dogs in the video frame image to obtain human detection boxes and / or dog detection boxes in the video frame image.

[0071] Specifically, the video frame image is input into the human-dog detection model for human-dog detection, and the output is the human detection box and / or dog detection box in the video frame image. The human detection box and / or dog detection box are used to represent the position of the human body and the position of the dog in the video frame image.

[0072] Step S106: If there is only a dog detection box in the video frame image, it is determined to be an illegal dog walking event;

[0073] Specifically, if no dog detection box is detected in the video frame image, it returns to the normal state; if there is only a dog detection box and no human detection box in the video frame image, it is determined to be an illegal dog walking event, that is, an unleashed state, and then input to the alarm module to trigger an alarm.

[0074] Step S108: If the video frame image includes a human detection box and a dog detection box, then determine the image region in the video frame image that satisfies the rope length based on the human detection box and the dog detection box.

[0075] Specifically, in order to reduce the interference of the environment on rope detection, the human detection box and dog detection box obtained in step S104 are used to determine the image area in the video frame image that meets the rope length, and this image area is used for rope detection.

[0076] Step S110: Use the rope detection model to detect ropes in the image region that meets the rope length requirement, and obtain the information of the rope detection box.

[0077] The information of the rope detection frame mentioned above includes: whether there is a rope detection frame; if there is a rope detection frame, the information of the rope detection frame also includes: the coordinates of the center point of the rope detection frame, the length of the long side of the rope detection frame, the length of the short side of the rope detection frame, and the angle through which the X-axis rotates clockwise to encounter the long side.

[0078] Step S112: If the information of the leash detection frame is that there is no leash detection frame, then it is determined to be an illegal dog walking incident.

[0079] Specifically, if there is no leash detection frame, it is judged as "unleashed state", which is an illegal dog walking incident, and the information is entered into the alarm module to trigger an alarm.

[0080] Step S114: If the information of the rope detection box is that there is a rope detection box, then determine the human-dog-rope region image in the video frame image based on the human detection box, the dog detection box and the rope detection box.

[0081] Specifically, if there is a rope detection box, the video frame image is divided into several human-dog-rope region images containing the human, dog, and rope based on the human detection box, dog detection box, and rope detection box, which are used to determine the interaction relationship between the human, dog, and rope.

[0082] Step S116: Use the human-dog-leash relationship judgment model to judge the human-dog-leash relationship in the human-dog-leash region image to obtain the relationship between the human and the leash and / or the relationship between the leash and the dog.

[0083] Step S118: Determine whether it is an illegal dog walking incident based on the relationship between the person and the leash and / or the relationship between the leash and the dog.

[0084] Specifically, if both the "human holding leash" and "dog holding leash" relationships exist simultaneously, it can be determined that the dog in the image of the human-dog leash area is leashed and that it is a legal dog walking event. Otherwise, it is determined that the dog in the image of the human-dog leash area is unleashed, which is an illegal dog walking event, and the information is input to the alarm module to trigger an alarm.

[0085] In this embodiment of the invention, a method for detecting illegal dog walking incidents is provided, comprising: acquiring a video stream to be detected and preprocessing the video stream to obtain video frame images; performing human and dog detection on the video frame images using a human-dog detection model to obtain human detection boxes and / or dog detection boxes in the video frame images; if the video frame images contain only dog ​​detection boxes, then it is determined to be an illegal dog walking incident; if the video frame images contain both human and dog detection boxes, then determining an image region in the video frame images that satisfies the leash length based on the human and dog detection boxes; and performing leash detection... The model performs rope detection on image regions that meet the rope length requirement, obtaining rope detection box information. If the rope detection box information indicates no rope detection box, it is determined to be an illegal dog walking event. If the rope detection box information indicates the presence of a rope detection box, the human-dog-leash region image is determined in the video frame image based on the human detection box, dog detection box, and rope detection box. A human-dog-leash relationship judgment model is used to judge the human-dog-leash relationship in the human-dog-leash region image, obtaining the relationship between the person and the leash and / or the relationship between the leash and the dog. Based on the relationship between the person and the leash and / or the relationship between the leash and the dog, it is determined whether it is an illegal dog walking event. As described above, the method for detecting illegal dog walking incidents in this invention fully utilizes the characteristics of the leash, designs a leash detection model to detect inclined target leashes, and designs a human-dog-leash relationship judgment model to detect the interaction between the human and the leash and / or the interaction between the leash and the dog. Thus, the detection results of illegal dog walking incidents determined based on the interaction between the human and the leash and / or the interaction between the leash and the dog are more accurate. In addition, the entire process is completed automatically, with good intelligence, alleviating the technical problem that existing methods for detecting illegal dog walking incidents cannot balance intelligence and accuracy.

[0086] In an optional embodiment of the present invention, step S102 above, which preprocesses the video stream to be detected, specifically includes the following steps:

[0087] (1) Perform frame extraction and decoding on the video stream to be detected to obtain the video frame image after frame extraction and decoding.

[0088] (2) Scale the video frame image after frame extraction and decoding to obtain the video frame image.

[0089] The scaling ratio can be 640*640, which helps to improve the speed of human and dog detection. This embodiment of the invention does not impose specific limitations on the above size.

[0090] To reduce environmental interference, this invention does not directly use video frame images as input to the rope detection model. Instead, it uses image regions that satisfy the rope length as input to detect the rope. This invention provides a method for determining image regions that satisfy the rope length. In an optional embodiment of this invention, step S108, which determines the image regions that satisfy the rope length in the video frame image based on the human detection box and the dog detection box, specifically includes the following steps:

[0091] (1) Calculate the length of the rope in the video frame image based on the width of the human body detection box in the video frame image and the reference width of the ergonomic human body;

[0092] Specifically, according to regulations, the length of the rope should not exceed 2 meters, so the longest rope length drope = 2 meters is taken. According to ergonomics, the baseline width of the human body is dperson = 0.4 meters. The width of the detected human body detection frame in the video frame image is w. The length of the rope in the video frame image can be approximately calculated as D = w * drope / dperson. Considering the influence of factors such as posture, it can be appropriately extended by 10% as the final length of the rope in the video frame image.

[0093] (2) Using the center point of the dog detection box as the origin and the length of the rope in the video frame image as the radius, draw a circle to obtain multiple target human detection boxes that are closest to the center point of the dog detection box within the corresponding circular area.

[0094] Specifically, the human body detection box whose center is located within the circular area is used as the target human body detection box.

[0095] (3) Determine the image region that meets the rope length in the video frame image based on the maximum outer bounding box of multiple target human body detection boxes.

[0096] Specifically, the region in the video frame image enclosed by the aforementioned maximum bounding box is taken as the image region that satisfies the rope length. The video frame image is then cut along the aforementioned maximum bounding box to obtain the image region that satisfies the rope length.

[0097] In an optional embodiment of the present invention, step S114 above, which determines the human-dog-leash region image in the video frame image based on the human detection box, the dog detection box, and the leash detection box, specifically includes the following steps:

[0098] The human-dog-leash region image is determined in the video frame image based on the bounding box formed by the human detection box and the dog detection box that are closest to the center of the rope detection box.

[0099] Specifically, the area within the video frame image enclosed by the aforementioned bounding box is designated as the human-dog leash region image. The video frame image is then segmented along this bounding box, resulting in several human-dog leash region images, each used to determine the human-dog leash relationship. A dog is considered leashed only when both a human-holding and dog-held relationships are simultaneously present in a human-dog leash region image. In cases where multiple dogs are present in a single video frame image, if one human-dog leash region image identifies that dog as leashed, then that dog in the entire video frame can be considered leashed, and the other dogs can then be identified.

[0100] In an optional embodiment of the present invention, the human-dog-leash relationship determination model includes: a feature extraction module, an interaction generation module, and an interaction matching module. Step S116 above, which uses the human-dog-leash relationship determination model to determine the human-dog-leash relationship in the human-dog-leash region image, to obtain the relationship between the person and the leash and / or the relationship between the leash and the dog, specifically includes the following steps:

[0101] (1) Input the human-dog leash region image into the feature extraction module and output the features of the human-dog leash region image;

[0102] (2) Input the features of the human-dog leash region image into the interaction generation module and output the interaction points and interaction vectors. The interaction points include: the center point between the leash and the human, and the center point between the leash and the dog. The interaction vector is the vector of the interaction point toward the center point of the leash.

[0103] The center point between the rope and the person specifically refers to the center point between the center point of the rope detection frame and the center point of the human body detection frame. The center point between the rope and the dog specifically refers to the center point between the center point of the rope detection frame and the center point of the dog detection frame.

[0104] (3) Input the information of the rope detection box, human body detection box, dog detection box, interaction point and interaction vector into the interaction matching module to predict the relationship between the human and the rope and / or the relationship between the rope and the dog.

[0105] The training process of the human-dog detection model, the rope detection model, and the human-dog-rope relationship judgment model is described below.

[0106] In an optional embodiment of the present invention, the method further includes:

[0107] (1) Obtain video frame image samples of illegal dog walking behavior;

[0108] Specifically, the data comes from video frame image samples containing various illegal dog-walking behaviors in real-world application scenarios.

[0109] (2) Use the trained human-dog detection model to perform human-dog detection on video frame image samples to obtain human detection boxes and / or dog detection boxes in the video frame image samples.

[0110] The data and annotation format required for the human-dog detection model are as follows: The target is labeled in the form of a rectangle, using the industry-standard object detection dataset annotation method, and divided into two categories: person and dog.

[0111] The human-dog detection model can employ a general object detection network. The input is video frame image samples, and the output is the detection results of human and dog bounding boxes. This invention uses YOLOv7 as the training model and a pre-trained model from the COCO dataset, combined with fine-turning on this dataset. This invention uses a (640*640) input during training and testing, tailored to practical application scenarios, which helps improve the speed of object detection. The trained human-dog detection model can then prepare for the training of the rope detection model in the next step.

[0112] In real-world dog-walking scenarios, the leash is in a variable state and is often tilted. Conventional target detection methods tend to learn excessive redundant features, negatively impacting leash detection accuracy. This invention improves the YOLOv7 network structure and adds training based on the tilt angle to achieve accurate leash detection. Improvements are made primarily in four areas: image input (not video frame samples, but image region samples that match the leash length), data loading (adding angle information θ dimension), backbone (adding a CBAM attention mechanism), and loss function.

[0113] (3) Calculate the length of the rope in the video frame image sample based on the width of the human body detection box in the video frame image sample and the reference width of the ergonomic human body;

[0114] Specifically, according to regulations, the rope length should not exceed 2 meters, so the longest length, drope, is taken as 2 meters. Based on ergonomics, the baseline width of the human body is dperson = 0.4 meters. The width of the human detection frame in the video frame image sample is w. The rope length in the video frame image sample can be approximately calculated as D = w * drope / dperson. Considering the influence of factors such as posture, this can be appropriately extended by 10% to obtain the final rope length in the video frame image sample.

[0115] (4) Using the center point of the dog detection box as the origin and the length of the rope in the video frame image sample as the radius, draw a circle to obtain the multiple target human detection boxes that are closest to the center point of the dog detection box within the corresponding circular area.

[0116] Specifically, such as Figure 2As shown, the human detection box whose center is located within the circular area is taken as the target human detection box.

[0117] (5) Determine the image region sample that satisfies the rope length in the video frame image sample based on the maximum outer bounding box of multiple target human body detection boxes;

[0118] Specifically, the region in the video frame image sample enclosed by the maximum bounding box is taken as the image region sample that satisfies the rope length. The image region sample is cut along the maximum bounding box to obtain the image region sample that satisfies the rope length.

[0119] Since the rope is relatively thin and long, in order to further improve the detection effect of small targets, an image region sample of size (1280*1280) is used as the input of the network in the rope detection model stage.

[0120] (6) The initial rope detection model is trained based on the image region samples that meet the rope length and the information of the rope detection boxes labeled on the image region samples to obtain the rope detection model. The information of the rope detection boxes includes: category, coordinates of the center point of the rope detection box, length of the long side of the rope detection box, length of the short side of the rope detection box, and the angle through which the long side is encountered when the X-axis is rotated clockwise. The backbone of the rope detection model includes the attention mechanism CBMA.

[0121] Specifically, due to the variable shape of ropes and their tendency to tilt during movement, conventional rectangular bounding box detection results in excessive redundant features and low detection accuracy. This invention improves rope detection by adding tilt angle prediction to the traditional target detection network. The data annotation method required for the rope detection model differs from that of the human-dog detection model. This invention adds angle information to the rectangular bounding box, with the annotation format being (category, x, y, longside, shortside, θ). In this invention, there is only one category: rope. x and y are the coordinates of the center point of the rope detection box, longside is the length of the long side of the rope detection box, shortside is the length of the short side of the rope detection box, and θ is the angle traversed by clockwise rotation of the X-axis to reach the long side (θ∈[0, 180)).

[0122] For data loading and preprocessing, this invention adds an angle information θ dimension to the original target detection labels for network training and inference of angles. At the same time, an θ dimension is also added for data augmentation, and mosaic, affine matrix augmentation, and ordinary data augmentation methods (scaling, translation, etc.) are used to enrich sample features. Rotation and deformation affine data augmentation methods are removed to avoid causing changes in the target angle.

[0123] Backbone: This invention adds an attention mechanism, CBAM, to the original backbone, such as... Figure 3 As shown in the dashed line, the attention mechanism integrates high-level and low-level features from both channel attention and spatial attention perspectives to enhance the extraction of rope features.

[0124] The loss function incorporates an θ-angle loss in addition to the existing confidence loss, classification loss, and bounding box regression loss. This invention treats the prediction of the θ-angle as a classification, decoupling the angle information from the bounding box loss and category loss. After processing the θ-angle using the CLS function, the same loss function BCEWithLogitsLoss as the classification loss is applied.

[0125]

[0126] Where g(x) is a window function, which can be an impulse function, a rectangular function, a trigonometric function, etc., and the window radius is controlled by r.

[0127] In an optional embodiment of the present invention, the method further includes:

[0128] (1) Determine the human-dog-rope region image samples in the video frame image samples based on the human detection box, dog detection box and rope detection box obtained by the trained human-dog detection model;

[0129] Specifically, a video frame image sample is divided into several human and dog leash region image samples by taking the rope detection frame as the center and forming an outer frame composed of the nearest human detection frame and dog detection frame.

[0130] (2) The initial human-dog-leash relationship judgment model is trained based on the human-dog-leash region image samples and the relationship between the human and the leash and / or the relationship between the leash and the dog labeled on the human-dog-leash region image samples, so as to obtain the human-dog-leash relationship judgment model.

[0131] Specifically, each sample needs to be labeled as a three-tuple, including a rope detection box, a human (dog) detection box, and an interaction behavior. The interaction behavior is divided into two categories: hold and held. The specific formats are (human, hold, rope) and (dog, held, rope).

[0132] Specifically, the human-dog leash relationship determination model is divided into three modules: feature extraction, interaction generation, and interaction matching. Figure 4 As shown.

[0133] Feature Extraction Module: The backbone network of this invention can employ networks such as Hourglass, DLA, and ResNet to extract appearance features from images. The features extracted from the backbone are input to the interaction generation module to obtain interaction points and interaction vectors. Interaction points are the center points between the rope and the target (human, dog) pair, representing the interaction category, such as hold or held, and are also the starting point of the interaction vector. The interaction points and interaction vectors, along with the detected human, dog, and rope bounding boxes, are input to the interaction matching module to obtain the final triplet (rope, action, human or dog) prediction.

[0134] The interaction generation module comprises two parallel branches: a rope interaction point branch and an interaction vector branch. Both branches use features extracted from the backbone as input. The interaction point branch generates an interaction point heatmap using a 3x3 convolutional layer to obtain human-rope and dog-rope interaction points, such as... Figure 5 As shown. The interaction vector branches are as follows. Figure 5 As shown, based on the interaction points, the interaction vector branch aims to predict the interaction vector toward the center point of the rope. This branch is trained to predict the magnitude of the unsigned interaction vector. The branch uses a 3x3 convolutional layer to generate a two-channel unsigned interaction vector map, one representing the horizontal length of the interaction vector and the other its vertical length.

[0135] The interactive matching module establishes the relationship between the person, the dog, and the leash. For example... Figure 4 As shown, its input consists of three parts: a rope detection box (rbox), a target (human or dog) detection box (obox), an interaction point P from the interaction heatmap, and an interaction vector V; the output is the prediction result of the triple. The rules for interaction matching are as follows:

[0136] 1. Based on the interaction points and interaction vectors, the possible coordinates of the rope's center point for the four vertices of the dashed box in the diagram are calculated using the following formula (these are predictions; the solid box represents the true values). Figure 6 As shown;

[0137]

[0138] This leads to the interactive box ibox (the dashed box in the diagram);

[0139] 2. If the interaction box, target (person or dog) box / rope box, and 4 vectors satisfy the following conditions:

[0140]

[0141] d tl d tr d bl d brLet d be the length of the vector (the vertices of the dashed box are obtained from the predicted interaction points and interaction vectors through the above calculations, while the vertices of the solid box are the true values, so the length of the vector can be calculated). τ The threshold for filtering is set in advance.

[0142] The final predicted interaction score for the current triple is obtained as follows:

[0143] score = r score *o score *p score

[0144] Where, r score Indicates the confidence level of the rope detection frame, o score p represents the confidence level of the target detection box (dog detection box or human detection box). score This represents the confidence level of whether a person holds the leash or a dog holds the leash (after interactive matching, a predicted p will be obtained). score ).

[0145] If the score exceeds a preset threshold, it can be determined that there is interactive behavior (hold or held) between the current target box and the rope box.

[0146] 3. Based on the triplet relationship pairs obtained in step 2, establish the relationship between the person, the dog, and the leash, and then determine the dog walking behavior. If there are two interaction pairs in the same image, one showing the person holding the leash and the other showing the dog on a leash, the dog can be identified as being on a leash.

[0147] It should be noted that the training stages of the three models described above employed general deep learning network model training methods. Specifically, to improve the model's detection performance and generalization ability, a warm-up combined with cosine annealing was used to update the learning strategy. Initially, a small learning rate was chosen, and it was linearly increased with the number of iterations until the model was relatively stable, at which point the pre-set learning rate was switched. After the warm-up phase, the learning rate was gradually updated using cosine annealing, resulting in a more stable loss reduction. This approach makes the network model training smoother and converges faster.

[0148] The method for detecting illegal dog walking incidents of the present invention has the following advantages:

[0149] 1. This invention makes full use of the rope's features and designs a rope detection model to detect ropes on inclined targets. In addition, it incorporates an attention mechanism to address the characteristics of small targets, which can effectively improve the detection effect of ropes on small targets, reduce interference from irrelevant scenes, and improve the detection accuracy of the model.

[0150] 2. This invention designs a relationship judgment model between humans, dogs, and leashes. By utilizing the detection results of the three, the interaction relationship between humans, leashes, and dogs is constructed to realize the relationship judgment among the three, which can then be used to judge illegal dog walking incidents, making the detection results more accurate.

[0151] 3. This invention combines the characteristics of application scenarios and adopts different input methods and business logics in different detection and recognition models, which can effectively improve the detection effect in complex scenarios such as multiple people and multiple dogs.

[0152] 4. This invention can automatically identify the illegal behavior of walking dogs without a leash without human intervention, thus reducing workload.

[0153] In summary, this invention makes full use of the relationship between the person and the dog leash to efficiently and accurately identify whether there are illegal dog-walking incidents in videos.

[0154] Example 2:

[0155] This invention also provides a device for detecting illegal dog walking incidents. This device is mainly used to execute the method for detecting illegal dog walking incidents provided in Embodiment 1 of this invention. The following is a detailed description of the device for detecting illegal dog walking incidents provided in this invention.

[0156] Figure 7 This is a schematic diagram of a detection device for illegal dog walking according to an embodiment of the present invention, as shown below. Figure 7 As shown, the device mainly includes: an acquisition and preprocessing unit 10, a human-dog detection unit 20, a first determination unit 30, a second determination unit 40, a leash detection unit 50, a third determination unit 60, a fourth determination unit 70, a human-dog leash relationship determination unit 80, and a fifth determination unit 90, wherein:

[0157] The acquisition and preprocessing unit is used to acquire the video stream to be detected and preprocess the video stream to be detected to obtain video frame images;

[0158] The human-dog detection unit is used to perform human-dog detection on video frame images using a human-dog detection model, and to obtain human detection boxes and / or dog detection boxes in the video frame images.

[0159] The first determining unit is used to determine an illegal dog walking event if there is only a dog detection box in the video frame image;

[0160] The second determining unit is used to determine the image region that satisfies the rope length in the video frame image based on the human detection box and the dog detection box if the video frame image includes a human detection box and a dog detection box.

[0161] The rope detection unit is used to perform rope detection on image regions that meet the rope length requirements using a rope detection model, and obtain information about the rope detection box.

[0162] The third determining unit is used to determine an illegal dog walking incident if the information of the rope detection frame is that there is no rope detection frame.

[0163] The fourth determining unit is used to determine the human-dog-rope region image in the video frame image based on the human body detection box, the dog detection box, and the rope detection box if the rope detection box information is that there is a rope detection box.

[0164] The human-dog-leash relationship determination unit is used to determine the human-dog-leash relationship in the human-dog-leash region image using a human-dog-leash relationship determination model, and to obtain the relationship between the human and the leash and / or the relationship between the leash and the dog.

[0165] The fifth determining unit is used to determine whether an incident constitutes illegal dog walking based on the relationship between the person and the leash and / or the relationship between the leash and the dog.

[0166] In this embodiment of the invention, a device for detecting illegal dog walking incidents is provided, comprising: acquiring a video stream to be detected and preprocessing the video stream to obtain video frame images; performing human and dog detection on the video frame images using a human-dog detection model to obtain human detection boxes and / or dog detection boxes in the video frame images; if the video frame images contain only dog ​​detection boxes, then it is determined to be an illegal dog walking incident; if the video frame images contain both human and dog detection boxes, then determining an image region in the video frame images that satisfies the leash length based on the human and dog detection boxes; and performing leash detection... The model performs rope detection on image regions that meet the rope length requirement, obtaining rope detection box information. If the rope detection box information indicates no rope detection box, it is determined to be an illegal dog walking event. If the rope detection box information indicates the presence of a rope detection box, the human-dog-leash region image is determined in the video frame image based on the human detection box, dog detection box, and rope detection box. A human-dog-leash relationship judgment model is used to judge the human-dog-leash relationship in the human-dog-leash region image, obtaining the relationship between the person and the leash and / or the relationship between the leash and the dog. Based on the relationship between the person and the leash and / or the relationship between the leash and the dog, it is determined whether it is an illegal dog walking event. As described above, the illegal dog-walking detection device of the present invention fully utilizes the characteristics of the rope and designs a rope detection model to detect inclined target ropes. Furthermore, it designs a human-dog-leash relationship judgment model, which can detect the interaction between the human and the rope and / or the interaction between the rope and the dog. Thus, the detection results of illegal dog-walking events determined based on the interaction between the human and the rope and / or the interaction between the rope and the dog are more accurate. In addition, the entire process is completed automatically, exhibiting high intelligence, thus alleviating the technical problem that existing illegal dog-walking detection methods cannot simultaneously achieve both intelligence and accuracy.

[0167] Optionally, the acquisition and preprocessing unit is further configured to: perform frame-by-frame decoding on the video stream to be detected to obtain a video frame image after frame-by-frame decoding; and scale the video frame image after frame-by-frame decoding to obtain a video frame image.

[0168] Optionally, the second determining unit is further configured to: calculate the length of the rope in the video frame image based on the width of the human detection box in the video frame image and the reference width of the ergonomic human body; draw a circle with the center point of the dog detection box as the origin and the length of the rope in the video frame image as the radius to obtain multiple target human detection boxes that are closest to the center point of the dog detection box within the corresponding circular area; and determine the image area in the video frame image that satisfies the rope length based on the maximum bounding box of the multiple target human detection boxes.

[0169] Optionally, the fourth determining unit is further configured to: determine the human-dog leash region image in the video frame image based on the outer frame composed of the human detection frame and the dog detection frame closest to the center of the leash detection frame.

[0170] Optionally, the human-dog-leash relationship determination model includes: a feature extraction module, an interaction generation module, and an interaction matching module. The human-dog-leash relationship determination unit is further used to: input the human-dog-leash region image into the feature extraction module and output the features of the human-dog-leash region image; input the features of the human-dog-leash region image into the interaction generation module and output the interaction points and interaction vectors, wherein the interaction points include: the center point between the leash and the human, and the center point between the leash and the dog, and the interaction vector is the vector from the interaction point toward the center point of the leash; input the information of the leash detection box, the human detection box, the dog detection box, the interaction points, and the interaction vectors into the interaction matching module to predict the relationship between the human and the leash and / or the relationship between the leash and the dog.

[0171] Optionally, the device is also used to: acquire video frame image samples of illegal dog walking behavior; perform human and dog detection on the video frame image samples using a trained human-dog detection model to obtain human detection boxes and / or dog detection boxes in the video frame image samples; calculate the length of the rope in the video frame image samples based on the width of the human detection box in the video frame image samples and the reference width of the ergonomic human body; draw a circle with the center point of the dog detection box as the origin and the length of the rope in the video frame image samples as the radius to obtain multiple target human detection boxes that are closest to the center point of the dog detection box within the corresponding circular area; Based on the maximum bounding box of multiple target human detection boxes, image region samples that meet the rope length requirements are determined in the video frame image samples. The initial rope detection model is trained based on the image region samples that meet the rope length requirements and the information of the rope detection boxes labeled on the image region samples. The rope detection box information includes: category, coordinates of the center point of the rope detection box, length of the long side of the rope detection box, length of the short side of the rope detection box, and the angle through which the long side is encountered when rotating clockwise along the X-axis. The backbone of the rope detection model includes a CBMA attention mechanism.

[0172] Optionally, the device is also used to: determine human-dog-leash region image samples in video frame image samples based on human detection boxes, dog detection boxes and rope detection boxes detected by the trained human-dog detection model and the trained rope detection model; and train an initial human-dog-leash relationship judgment model based on the human-dog-leash region image samples and the relationships between people and rope and / or rope and dogs labeled on the human-dog-leash region image samples to obtain a human-dog-leash relationship judgment model.

[0173] The device provided in this embodiment of the invention has the same implementation principle and technical effect as the aforementioned method embodiment. For the sake of brevity, any parts not mentioned in the device embodiment can be referred to the corresponding content in the aforementioned method embodiment.

[0174] like Figure 8 As shown in the embodiment of this application, an electronic device 600 includes a processor 601, a memory 602, and a bus. The memory 602 stores machine-readable instructions executable by the processor 601. When the electronic device is running, the processor 601 communicates with the memory 602 via the bus, and the processor 601 executes the machine-readable instructions to perform the steps of the above-described method for detecting illegal dog walking incidents.

[0175] Specifically, the memory 602 and processor 601 mentioned above can be general-purpose memory and processor, without any specific limitations. When the processor 601 runs the computer program stored in the memory 602, it can execute the above-mentioned method for detecting illegal dog walking incidents.

[0176] The processor 601 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the processor 601 or by instructions in software form. The processor 601 may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly manifested as execution by a hardware decoding processor, or execution by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory 602, and processor 601 reads the information from memory 602 and, in conjunction with its hardware, completes the steps of the above method.

[0177] Corresponding to the above-mentioned method for detecting illegal dog walking incidents, this application embodiment also provides a computer-readable storage medium storing machine-executable instructions. When the machine-executable instructions are called and run by a processor, the machine-executable instructions cause the processor to perform the steps of the above-mentioned method for detecting illegal dog walking incidents.

[0178] The illegal dog-walking detection device provided in this application embodiment can be specific hardware on a device or software or firmware installed on the device. The implementation principle and technical effects of the device provided in this application embodiment are the same as those in the foregoing method embodiments. For the sake of brevity, any parts not mentioned in the device embodiment can be referred to the corresponding content in the foregoing method embodiments. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can all be referred to the corresponding processes in the above method embodiments, and will not be repeated here.

[0179] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0180] For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0181] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0182] In addition, the functional units in the embodiments provided in this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0183] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the vehicle marking method described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0184] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In addition, the terms "first", "second", "third", etc. are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0185] Finally, it should be noted that the above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The protection scope of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this application; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application. All should be covered within the protection scope of this application. Therefore, the protection scope of this application should be determined by the protection scope of the claims.

Claims

1. A method for detecting illegal dog walking incidents, characterized in that, include: The video stream to be detected is acquired, and the video stream to be detected is preprocessed to obtain video frame images; A human-dog detection model is used to detect humans and dogs in the video frame image to obtain human detection boxes and / or dog detection boxes in the video frame image. If the video frame image contains only a dog detection box, it is determined to be an illegal dog walking incident; If the video frame image includes a human detection box and a dog detection box, then the image region that satisfies the rope length is determined in the video frame image based on the human detection box and the dog detection box. A rope detection model is used to detect ropes in the image region that meets the rope length requirement, thereby obtaining information about the rope detection box. If the information of the rope detection frame is that there is no rope detection frame, it is determined to be an illegal dog walking incident; If the information of the rope detection frame is that there is a rope detection frame, then the human-dog-rope region image is determined in the video frame image based on the human detection frame, the dog detection frame and the rope detection frame; A human-dog-leash relationship determination model is used to determine the human-dog-leash relationship in the human-dog-leash region image, thereby obtaining the relationship between the human and the leash, as well as the relationship between the leash and the dog. Whether it is an illegal dog walking incident is determined based on the relationship between the person and the leash, and the relationship between the leash and the dog; Specifically, determining the image region satisfying the rope length in the video frame image based on the human detection box and the dog detection box includes: The length of the rope in the video frame image is calculated based on the width of the human body detection frame in the video frame image and the reference width of the ergonomic human body. Using the center point of the dog detection frame as the origin and the length of the rope in the video frame image as the radius, draw a circle to obtain multiple target human detection frames that are closest to the center point of the dog detection frame within the corresponding circular area; The image region that satisfies the rope length is determined in the video frame image based on the maximum outer bounding box of the plurality of target human body detection boxes; The human-dog-leash relationship determination model includes a feature extraction module, an interaction generation module, and an interaction matching module. The model is used to determine the human-dog-leash relationship in the human-dog-leash region image, obtaining the relationship between the person and the leash, and the relationship between the leash and the dog, including: The image of the human-dog leash region is input into the feature extraction module, and the features of the human-dog leash region image are output. The features of the human-dog leash region image are input into the interaction generation module, and the interaction points and interaction vectors are output. The interaction points include: the center point between the leash and the human, and the center point between the leash and the dog. The interaction vector is the vector of the interaction point toward the center point of the leash. The information of the rope detection box, the human body detection box, the dog detection box, the interaction point, and the interaction vector are input into the interaction matching module to predict the relationship between the human and the rope, as well as the relationship between the rope and the dog.

2. The detection method according to claim 1, characterized in that, Preprocessing the video stream to be detected includes: The video stream to be detected is subjected to frame extraction and decoding to obtain the video frame image after frame extraction and decoding; The video frame image obtained by the frame extraction and decoding is scaled proportionally to obtain the video frame image.

3. The detection method according to claim 1, characterized in that, Determining the human-dog-leash region image in the video frame image based on the human detection frame, the dog detection frame, and the leash detection frame includes: The human-dog leash region image is determined in the video frame image based on the outer frame formed by the human detection frame and the dog detection frame closest to the center of the leash detection frame.

4. The detection method according to claim 1, characterized in that, The method further includes: Obtain video frame image samples of illegal dog walking behavior; The trained human-dog detection model is used to detect humans and dogs in the video frame image samples to obtain human detection boxes and / or dog detection boxes in the video frame image samples. The length of the rope in the video frame image sample is calculated based on the width of the human body detection frame in the video frame image sample and the reference width of the ergonomic human body. Using the center point of the dog detection box as the origin and the length of the rope in the video frame image sample as the radius, draw a circle to obtain multiple target human detection boxes that are closest to the center point of the dog detection box within the corresponding circular area; Based on the maximum bounding box of the plurality of target human body detection boxes, an image region sample that satisfies the rope length is determined in the video frame image sample; The initial rope detection model is trained based on the image region samples that meet the rope length and the information of the rope detection boxes labeled on the image region samples to obtain the rope detection model. The information of the rope detection boxes includes: category, coordinates of the center point of the rope detection box, length of the long side of the rope detection box, length of the short side of the rope detection box, and the angle through which the long side is encountered when the X-axis is rotated clockwise. The backbone of the rope detection model includes a CBMA attention mechanism.

5. The detection method according to claim 4, characterized in that, The method further includes: Based on the human detection box, the dog detection box, and the rope detection box obtained by the trained human-dog detection model, the human-dog-rope region image sample is determined in the video frame image sample; The initial human-dog-leash relationship judgment model is trained based on the human-dog-leash region image samples and the relationships between the human and the leash, as well as the relationship between the leash and the dog, as labeled on the human-dog-leash region image samples, to obtain the human-dog-leash relationship judgment model.

6. A device for detecting illegal dog walking incidents, characterized in that, include: An acquisition and preprocessing unit is used to acquire the video stream to be detected and to preprocess the video stream to be detected to obtain video frame images; The human-dog detection unit is used to perform human-dog detection on the video frame image using a human-dog detection model, and obtain human detection boxes and / or dog detection boxes in the video frame image. The first determining unit is used to determine an illegal dog walking event if there is only a dog detection box in the video frame image; The second determining unit is used to determine an image region in the video frame image that satisfies the rope length based on the human detection box and the dog detection box if the video frame image includes a human detection box and a dog detection box. The rope detection unit is used to perform rope detection on the image region that meets the rope length using a rope detection model, and obtain information of the rope detection box; The third determining unit is used to determine an illegal dog walking incident if the information of the rope detection frame is that there is no rope detection frame. The fourth determining unit is used to determine the human-dog leash region image in the video frame image based on the human body detection frame, the dog detection frame, and the leash detection frame if the information of the leash detection frame is that there is a leash detection frame. The human-dog-leash relationship determination unit is used to determine the human-dog-leash relationship in the human-dog-leash region image using a human-dog-leash relationship determination model, thereby obtaining the relationship between the human and the leash, as well as the relationship between the leash and the dog. The fifth determining unit is used to determine whether an incident constitutes illegal dog walking based on the relationship between the person and the leash, and the relationship between the leash and the dog. The second determining unit is further configured to: calculate the length of the rope in the video frame image based on the width of the human detection box in the video frame image and the reference width of the ergonomic human body; draw a circle with the center point of the dog detection box as the origin and the length of the rope in the video frame image as the radius to obtain a plurality of target human detection boxes within the corresponding circular area that are closest to the center point of the dog detection box; and determine the image area in the video frame image that satisfies the rope length based on the maximum bounding box of the plurality of target human detection boxes. The human-dog-leash relationship determination model includes a feature extraction module, an interaction generation module, and an interaction matching module. The human-dog-leash relationship determination unit is further configured to: input the human-dog-leash region image into the feature extraction module and output the features of the human-dog-leash region image; input the features of the human-dog-leash region image into the interaction generation module and output interaction points and interaction vectors, wherein the interaction points include the center point between the leash and the human, and the center point between the leash and the dog, and the interaction vector is the vector from the interaction point toward the center point of the leash; input the information of the leash detection box, the human detection box, the dog detection box, the interaction points, and the interaction vectors into the interaction matching module to predict the relationship between the human and the leash, and the relationship between the leash and the dog.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores machine-executable instructions that, when invoked and executed by a processor, cause the processor to perform the method according to any one of claims 1 to 5.

Images