Image-based pet dog excretion monitoring method and device, equipment and medium

Abstract

The invention relates to the technical field of pet nursing, solves the problem that accurate non-contact pet dog excretion monitoring cannot be realized in the prior art, and provides an image-based pet dog excretion monitoring method, device, equipment and medium. The method comprises the steps that a first target image sequence is acquired, the target image sequence comprises a plurality of frames of target images, and each target image comprises a to-be-monitored pet dog; preprocessing the first target image sequence to obtain a second target image sequence; according to the second target image sequence and a deep learning algorithm, obtaining posterior limb posture information of the pet dog; and judging whether the pet dog has an excretion behavior or not according to the posterior limb posture information. By identifying the postures of the hind limbs of the pet dog, the accuracy and the intelligent level of pet dog excretion monitoring are improved.

Description

Technical Field

[0001] The present invention relates to the field of pet care technology, and in particular to an image-based pet dog excretion monitoring method, device, equipment and medium. Background Art

[0002] Pet dogs are one of the most common companion animals for humans and are widely popular for their loyalty, friendliness and lively personalities. Pet dogs play an important role in the family, but their daily lives also require the care and management of their owners. Especially for the management of excretion behavior, pet owners need to ensure that the excrement of pet dogs is cleaned up in time to maintain household hygiene and the health of pets. Timely monitoring and cleaning of pet dog excrement helps to keep the home environment clean. By monitoring excretion behavior, pet owners can promptly detect health problems of pets, such as constipation, diarrhea, etc., and take corresponding treatment measures.

[0003] At present, the existing technology mainly monitors the excretion behavior of pet dogs through the following methods: For example, the pet owner personally observes the behavior of the pet dog to determine whether the pet is performing excretion activities. This method requires the owner to always pay attention to the pet's dynamics, which consumes energy and may lead to missed judgments. Some devices can be worn on pet dogs to determine their excretion behavior by monitoring indicators such as the pet's movement and posture. This method requires the pet dog to wear the device, but it may cause discomfort or inconvenience to the pet.

[0004] Manual monitoring is prone to missed or misjudgment situations and cannot ensure accurate monitoring of pet excretion behavior. Smart wearable devices may cause discomfort to pet dogs and require regular maintenance and battery replacement, which brings inconvenience to pet owners. Summary of the invention

[0005] In view of this, the embodiments of the present invention provide an image-based pet dog excretion monitoring method, device, equipment and medium to solve the problem that the prior art cannot achieve accurate non-contact pet dog excretion monitoring.

[0006] In a first aspect, an embodiment of the present invention provides an image-based pet dog excretion monitoring method, the method comprising: Acquire a first target image sequence, wherein the target image sequence includes a plurality of frames of target images, each of which includes a pet dog to be monitored; Preprocessing the first target image sequence to obtain a second target image sequence; Acquire the hind limb posture information of the pet dog according to the second target image sequence and the deep learning algorithm; According to the hind limb posture information, it is determined whether the pet dog has defecation behavior.

[0007] Preferably, the acquiring of the first target image sequence comprises: Acquire video stream data of the activity scene where the pet dog is located in real time; Identifying the pet dog in each frame of the video stream data by using a bounding box according to a target detection algorithm; The pet dog marked by the bounding box is tracked according to a target tracking algorithm to obtain the first target image sequence.

[0008] Preferably, the preprocessing of the first target image sequence to obtain the second target image sequence includes: Performing filtering processing on each frame of the target image in the first target image sequence according to a preset filtering algorithm to obtain a filtered image sequence, wherein the preset filtering algorithm includes one or more of a Gaussian filtering algorithm, a median filtering algorithm, and a bilateral filtering algorithm; Deduplication processing is performed on the filtered image sequence to obtain the second target image sequence.

[0009] Preferably, performing deduplication processing on the filtered image sequence to obtain the second target image sequence includes: Evenly dividing the filtered image sequence to obtain a preset number of filtered image groups; According to the optical flow field of the bounding box of each frame image in the filtered image group, obtaining the optical flow field gradient mean of each bounding box; Obtaining the motion amount of the bounding box of each frame image in the filtered image group according to the mean value of the optical flow field gradient; Selecting an image corresponding to a bounding box with the smallest amount of motion in each of the filtered image groups as a key frame; The second target sequence is obtained according to the key frames of each of the filtered image groups.

[0010] Preferably, the acquiring of the hind limb posture information of the pet dog according to the second target image sequence and the deep learning algorithm includes: Identify the hind limbs of the pet dog in the second target image sequence according to a deep learning algorithm to obtain coordinate information of key points of the hind limbs, wherein the key points of the hind limbs include at least three left hind limb key points and at least three right hind limb key points; According to the coordinate information of the left hind limb key point, obtaining the left hind limb direction vector of the pet dog; According to the coordinate information of the right hind limb key point, obtaining the direction vector of the right hind limb of the pet dog; The hind limb posture information is obtained according to the left hind limb direction vector and the right hind limb direction vector.

[0011] Preferably, the excretion behavior includes urination behavior, and judging whether the pet dog has excretion behavior according to the hind limb posture information includes: Obtaining a hind limb angle according to the left hind limb direction vector and the right hind limb direction vector; Calculate the variance of the X coordinate values ​​of each of the left hind limb key points to obtain a first variance; Calculate the variance of the X coordinate values ​​of each of the right hind limb key points to obtain a second variance; If the hind limb angle is greater than a preset angle, and the first variance or the second variance is less than a preset variance, then the pet dog has urinated.

[0012] Preferably, the excretion behavior also includes defecation behavior, the left hind limb key points include a first left hind limb key point, a second left hind limb key point and a third left hind limb key point distributed in sequence from top to bottom, the right hind limb key points include a first right hind limb key point, a second right hind limb key point and a third right hind limb key point distributed in sequence from top to bottom, and judging whether the pet dog has excretion behavior according to the hind limb posture information also includes: Acquire a first left hind limb direction vector according to the first left hind limb key point and the second left hind limb key point; Acquire a second left hind limb direction vector according to the second left hind limb key point and the third left hind limb key point; Acquire a first right hind limb direction vector according to the first right hind limb key point and the second right hind limb key point; Acquire a second right hind limb direction vector according to the second right hind limb key point and the third right hind limb key point; Calculating the angle between the first left hind limb direction vector and the second left hind limb direction vector to obtain a left hind limb angle; Calculating the angle between the first right hind limb direction vector and the second left hind limb direction vector to obtain a right hind limb angle; If the left hind limb angle or the right hind limb angle belongs to a preset angle range, and the ratio of the left hind limb angle or the right hind limb angle belongs to a preset ratio range, then the pet dog has defecation behavior.

[0013] In a second aspect, an embodiment of the present invention provides an image-based pet dog excretion monitoring device, the device comprising: An image acquisition module, used for acquiring a first target image sequence, wherein the target image sequence includes a plurality of frames of target images, each of which includes a pet dog to be monitored; A preprocessing module, used for preprocessing the first target image sequence to obtain a second target image sequence; A posture acquisition module, used to acquire the posture information of the hind limbs of the pet dog according to the second target image sequence and a deep learning algorithm; The judgment module is used to judge whether the pet dog has defecation behavior according to the hind limb posture information.

[0014] In a third aspect, an embodiment of the present invention provides an electronic device, comprising: at least one processor, at least one memory, and computer program instructions stored in the memory, which, when executed by the processor, implement the method of the first aspect in the above-mentioned embodiment.

[0015] In a fourth aspect, an embodiment of the present invention provides a storage medium having computer program instructions stored thereon, which implements the method of the first aspect in the above-mentioned embodiment when the computer program instructions are executed by a processor.

[0016] In summary, the beneficial effects of the present invention are as follows: The image-based pet dog excretion monitoring method, device, equipment and medium provided in the embodiments of the present invention can continuously monitor the behavior of the pet dog by acquiring a first target image sequence including the pet dog in real time, thereby providing a reliable data source for subsequent analysis and monitoring; the first target image sequence is preprocessed to obtain a second target image sequence, which provides clearer and more representative data for the deep learning algorithm and enhances the accuracy of the analysis; the deep learning algorithm identifies the pet dog's hind limbs in the second target image sequence and can accurately extract the hind limb posture information, which is an important basis for judging its excretion behavior. The excretion behavior judgment method based on posture information is more scientific and reliable than the traditional observation method, thereby improving the overall efficiency of the monitoring system. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solution of the embodiment of the present invention, the drawings required for use in the embodiment of the present invention will be briefly introduced below. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work, and these are all within the protection scope of the present invention.

[0018] Figure 1 It is a flow chart of an image-based pet dog excretion monitoring method according to an embodiment of the present invention.

[0019] Figure 2 It is another flow chart of the method for monitoring pet dog excretion based on images according to an embodiment of the present invention.

[0020] Figure 3 It is another flow chart of the method for monitoring pet dog excretion based on images according to an embodiment of the present invention.

[0021] Figure 4Schematic diagram of the structure of an image-based pet dog excretion monitoring device according to an embodiment of the present invention.

[0022] Figure 5 It is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0023] The features and exemplary embodiments of various aspects of the present invention will be described in detail below. In order to make the purpose, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and Examples. It should be understood that the specific embodiments described herein are only configured to explain the present invention and are not configured to limit the present invention. For those skilled in the art, the present invention can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present invention by illustrating examples of the present invention.

[0024] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0025] Example 1

[0026] See also Figure 1 , an embodiment of the present invention provides an image-based pet dog excretion monitoring method, the method comprising: S1. Acquire a first target image sequence, wherein the target image sequence includes a plurality of frames of target images, each of which includes a pet dog to be monitored; Specifically, images of the activity scene where the pet dog is located are obtained in real time through a camera or other image acquisition device. These images are captured continuously to record the activities of the pet dog at different times; since the pet dog has various behaviors in the activity scene, the first target image sequence should include images of the pet dog at different times and scenes. This data diversity is helpful for the subsequent comprehensive analysis of the pet dog's behavior.

[0027] In one embodiment, see Figure 2 , the acquiring of the first target image sequence comprises: S11, acquiring video stream data of the activity scene where the pet dog is located in real time; Specifically, through imaging devices such as cameras or other video acquisition devices, video stream data of the activity scene where the pet dog is located is obtained in real time. This data can be continuous and real-time, and include the behavior of the pet dog at different time points and in different activity scenes; real-time acquisition of video stream data is the basis for monitoring the behavior of the pet dog. Real-time ensures that the activity status and behavior changes of the pet dog can be monitored immediately.

[0028] In a specific embodiment, the imaging device is arranged on a movable cart, and the shooting angle of the imaging device is forward-looking, which means that the imaging device is not fixed and can move with the movement of the cart, providing continuous monitoring of the activity area of ​​the pet dog. This mobility ensures that the imaging device can follow the activity trajectory of the pet dog. The shooting angle of the imaging device is forward-looking, which means that the imaging device faces the front of the cart and can see the area in front of the cart. This setting helps to monitor the activities and behaviors of the pet dog in front of the cart. Through the real-time monitoring of the forward-looking imaging device, the system can observe the behavior of the pet dog in front of the cart in real time. This real-time performance is very important for timely detection and identification of activities such as the excretion behavior of the pet dog.

[0029] S12, identifying the pet dog in each frame of the video stream data by using a bounding box according to a target detection algorithm; In each frame of the video image, an object detection algorithm is used to identify the presence of the pet dog and mark the location of the pet dog with a bounding box. The object detection algorithm is usually based on deep learning and can identify the target objects in the image and enclose them with a bounding box. The bounding box is the output of the object detection algorithm and is used to mark the location and range of the pet dog in the image. Each bounding box corresponds to a target object. In this embodiment, the target object is the pet dog. S13. Track the pet dog marked by the bounding box according to a target tracking algorithm to obtain the first target image sequence.

[0030] Based on target detection, a target tracking algorithm is used to track the pet dog marked by the detection frame. This algorithm can continuously track the pet dog in continuous image frames to ensure the continuity of its position and behavior. Common target tracking algorithms include DeepSort, SORT, Kalman Filter, etc. They predict and update the movement of the pet dog in continuous images based on the position, size, speed and other information of the detection frame. Through target tracking, the system can track the movement trajectory of the pet dog in continuous video stream data and obtain the first target image sequence. This sequence contains several frames of target images including the pet dog.

[0031] S2, preprocessing the first target image sequence to obtain a second target image sequence; Specifically, this step preprocesses the first target image sequence to obtain the second target image sequence, and the preprocessing includes but is not limited to filtering, deduplication and image enhancement, etc. Filtering can reduce the noise in the image. Commonly used filtering algorithms include Gaussian filtering, median filtering and bilateral filtering. These filtering methods help to remove noise in the image and improve the clarity and quality of the image; the purpose of deduplication is to remove continuous and repeated image frames and retain representative and differential key frames to reduce the amount of data and improve processing efficiency. Image enhancement enhances the image in terms of contrast, brightness, sharpness, etc. These processes can improve the visual effect of the image and improve the visibility and recognition of the pet dog in the image.

[0032] In one embodiment, the preprocessing of the first target image sequence to obtain the second target image sequence includes: S21, filtering each frame of the target image in the first target image sequence according to a preset filtering algorithm to obtain a filtered image sequence, wherein the preset filtering algorithm includes one or more of a Gaussian filtering algorithm, a median filtering algorithm, and a bilateral filtering algorithm; Specifically, each frame of the first target image sequence is processed according to a preset filtering algorithm. This helps to reduce noise in the image, smooth the image, and improve the accuracy of subsequent analysis. The preset filtering algorithm includes one or more of a Gaussian filtering algorithm, a median filtering algorithm, and a bilateral filtering algorithm. Among them, the Gaussian filter smoothes the image by using a Gaussian kernel, the median filter replaces the central pixel value by taking the median in the field of the image, and the bilateral filter smoothes the image by combining spatial and intensity weights, which can smooth the image while retaining edge information; by filtering each frame of the image, a filtered image sequence is finally obtained. These images are of high quality and less noise, providing a better basis for subsequent deduplication and analysis.

[0033] S22: Perform deduplication processing on the filtered image sequence to obtain the second target image sequence.

[0034] Specifically, in the filtered image sequence, there may be repeated or highly similar image frames between consecutive frames. Through deduplication processing, these repeated images can be eliminated. The deduplication process is usually implemented by selecting key frames, for example, by analyzing the optical flow field, motion, visual features and other indicators of the image, and selecting the most representative key frame in each group of images. Through deduplication processing, the second target image sequence is finally obtained. This sequence contains fewer key frames, but still retains the main features of the pet dog's activities.

[0035] In one embodiment, see Figure 3 , performing deduplication processing on the filtered image sequence to obtain the second target image sequence, comprising: S221, evenly dividing the filtered image sequence to obtain a preset number of filtered image groups; The filtered image sequence is a continuous sequence of image frames. In this step, the filtered image sequence is evenly divided into a preset number of filtered image groups. For example, if the preset number is 10 groups, the entire sequence is evenly divided into 10 equal parts; by even division, it is ensured that each image group contains approximately the same number of image frames. In this way, the data distribution is more uniform; when dividing the image sequence, the time order in the original image sequence is maintained. This means that in each group of images, the order of the image frames is consistent with the order in the original sequence.

[0036] Grouping the entire filtered image sequence helps reduce the amount of data for subsequent processing. By selecting key frames in each group, duplicate and redundant information can be removed, improving data processing efficiency. After being divided into multiple image groups, representative key frames can be selected in each group. This helps retain the most important information in each group and discard unnecessary frames. Grouping can help the system better analyze local features, such as a specific behavior or state of a pet dog in a certain period of time. This is important for behavior recognition and monitoring.

[0037] S222, obtaining a mean value of the optical flow field gradient of each bounding box according to the optical flow field of the bounding box of each frame image in the filtered image group; Specifically, the bounding box in each frame of the image is the target area marked by the target detection algorithm. In this case, the bounding box is usually used to identify the position and range of the pet dog to be monitored. The optical flow field is a field used to describe the direction and speed of pixel movement in the image. It reflects the movement changes of pixels in the image between consecutive frames. In this step, the optical flow field of the bounding box area in each frame of the image needs to be calculated. By calculating the mean value of the optical flow field gradient in each bounding box area, the degree of movement change of pixels in the area is quantified. The gradient mean can reflect the smoothness or volatility of the movement in the area. The calculation of the optical flow field gradient mean is usually achieved by calculating the mean of the horizontal and vertical components of the optical flow field (for example, by the Sobel operator) in the bounding box area. The smaller the gradient mean, the smoother the movement change of pixels in the image area, which may indicate that the target is relatively stationary in the area. Calculating the optical flow field gradient mean is one of the important bases for judging the behavior of pet dogs. A smaller optical flow field gradient mean is usually associated with stationary or relatively stable behavior, which helps to accurately identify the excretion behavior of pet dogs.

[0038] S223, obtaining the motion amount of the bounding box of each frame image in the filtered image group according to the mean value of the optical flow field gradient; Specifically, the amount of motion indicates the degree of motion of the target in the image. In this step, the amount of motion of the bounding box is calculated based on the mean value of the optical flow field gradient. The amount of motion of the target in the bounding box area can be inferred based on the mean value of the optical flow field gradient. Specifically, a smaller mean gradient usually means that the movement of the pixels in the area is relatively smooth, while a larger mean gradient indicates a larger amplitude of motion. The calculation of the amount of motion is usually based on the mean value of the optical flow field gradient. The mean gradient can be regarded as an approximate measure of the amount of motion, that is, the larger the mean gradient, the greater the amount of motion. The calculation of the amount of motion is to determine the activity state of the pet dog in the area. A higher amount of motion may be associated with the active behavior of the pet dog, while a lower amount of motion may mean that the pet dog is stationary or relatively stationary. Based on the amount of motion of the bounding box, the behavioral state of the pet dog can be better judged. A lower amount of motion may indicate that the pet dog is stationary during excretion behavior.

[0039] S224, selecting an image corresponding to a bounding box with the smallest amount of motion in each of the filtered image groups as a key frame; Specifically, in each filtered image group, the motion of all frame images is compared, and the frame image with the smallest motion is selected. The frame image with the smallest motion usually means that the target has the smallest motion in the frame, which may indicate that the pet dog is stationary or relatively stationary in the frame. Images with small motion usually mean that the pet dog within the bounding box is in a relatively stationary or slightly moving state. When monitoring the excretion behavior of a pet dog, a key frame with small motion may represent the key behavior of the pet dog, such as urination or defecation. These behaviors are usually accompanied by the pet dog remaining stationary or moving slightly.

[0040] S225. Obtain the second target sequence according to the key frames of each of the filtered image groups.

[0041] Specifically, after obtaining the key frames of each filtered image group, the key frames selected from each filtered image group are merged to form a second target image sequence. This sequence consists of the most representative key frames in each group. Although the amount of data is reduced, by selecting the key frames with the smallest amount of motion, the second target image sequence retains the main behavioral characteristics of the pet dog in the active scene, especially the important behaviors in the static state.

[0042] S3. Acquire the hind limb posture information of the pet dog according to the second target image sequence and the deep learning algorithm; Specifically, a suitable deep learning algorithm, such as a convolutional neural network or OpenPose, is selected to analyze the posture information of the pet dog's hind limbs in the second target image sequence. The deep learning algorithm extracts the posture information of the pet dog's hind limbs by analyzing and identifying the pet dog in the image. This includes the position, angle, curvature and other features of the pet dog's hind limbs. The deep learning model can identify the key points of the pet dog's hind limbs, such as joints, feet, etc. The position and angle of these key points are important bases for evaluating the posture of the pet dog's hind limbs.

[0043] In one embodiment, the step of obtaining the hind limb posture information of the pet dog according to the second target image sequence and the deep learning algorithm includes: S31, identifying the hind limbs of the pet dog in the second target image sequence according to a deep learning algorithm to obtain coordinate information of hind limb key points, wherein the hind limb key points include at least three left hind limb key points and at least three right hind limb key points; Specifically, in this embodiment, the deep learning algorithm is OpenPose, through which the coordinate information of the key points of the hind limbs can be obtained, and the identified key points should include at least three left hind limb key points and at least three right hind limb key points, corresponding to the joints and parts of the left hind limb and the right hind limb respectively; In a specific embodiment, the key points of the hind limbs include at least the hip joint, knee, and ankle. The hip joint is the part where the pet dog's hind limbs are connected to the trunk, and plays a supporting and connecting role. The knee joint is located in the middle of the hind legs and is one of the main joints of the hind limbs. It plays a key role in the walking and jumping of the pet dog. The ankle joint is located in the lower part of the hind legs and connects the calf and the foot. This is another important joint of the hind limbs.

[0044] S32, obtaining the direction vector of the left hind limb of the pet dog according to the coordinate information of the left hind limb key point; Specifically, the direction vector of the left hind limb of the pet dog is calculated based on the coordinate information of the identified left hind limb key points. This is usually determined by connecting the line segments between the key points. The left hind limb direction vector describes the direction and angle of the left hind limb of the pet dog, which is helpful for subsequent analysis of the behavior and status of the pet dog.

[0045] S33, obtaining the direction vector of the right hind limb of the pet dog according to the coordinate information of the key point of the right hind limb; Similar to step S32, the direction vector of the right hind limb of the pet dog is calculated based on the coordinate information of the identified right hind limb key points. The right hind limb direction vector describes the direction and angle of the right hind limb of the pet dog, providing a reference for analyzing the behavior of the pet dog.

[0046] S34. Obtain the hind limb posture information according to the left hind limb direction vector and the right hind limb direction vector.

[0047] Specifically, by comprehensively analyzing the left hind limb direction vector and the right hind limb direction vector, the overall hind limb posture information of the pet dog is obtained. This includes the position, angle and movement state of the pet dog's hind limbs. The hind limb posture information is an important basis for judging the behavior of the pet dog. By analyzing the hind limb posture information, the excretion behavior of the pet dog can be more accurately identified.

[0048] S4. Determine whether the pet dog has defecation behavior based on the hind limb posture information.

[0049] Specifically, pet dogs usually show specific hind limb postures during urination or defecation. For example, when urinating, one hind leg may be raised and the hind limbs may be slightly separated; when defecating, the dog may squat and maintain a stable posture. By matching these specific posture features, it can be determined whether the pet dog is performing defecation behavior. Based on the posture information of the pet dog's hind limbs, it is determined whether it has defecation behavior. This can be done by analyzing the posture features of the pet dog's hind limbs, such as the squatting angle, changes in leg position, stability of the hind limbs, etc.

[0050] Preferably, the excretion behavior includes urination behavior, and judging whether the pet dog has excretion behavior according to the hind limb posture information includes: S41, acquiring a hind limb angle according to the left hind limb direction vector and the right hind limb direction vector; Specifically, in this embodiment, the excretion behavior includes urination behavior. Since a pet dog usually lifts one hind leg and slightly separates the hind legs when urinating, in this step, the angle between the left and right hind legs is first obtained through the direction vectors of the left and right hind legs; In a specific embodiment, the calculation process is as follows: Note that the left hind limb key points include the first left hind limb key point Left1 (x, y), the second left hind limb key point Left2 (x, y) and the third left hind limb key point Left3 (x, y), and the right hind limb key points include the first right hind limb key point Right1 (x, y), the second right hind limb key point Right2 (x, y) and the third right hind limb key point Right3 (x, y); Then the left hind limb direction vector can be expressed as: V_Left=(Left3(x)-Left1(x),Left3(y)-Left1(y)) The right hind limb direction vector can be expressed as: V_Right=(Right3(x)-Right1(x),Right3(y)-Right1(y)) The angle θ is calculated by the following formula: ; S42, calculating the variance of the X coordinate values ​​of each of the left hind limb key points to obtain a first variance; Specifically, the X coordinate values ​​of the key points of the left hind limb are analyzed and their variances are calculated. The variance of the X coordinate values ​​of the key points of the left hind limb can reflect the stability and horizontal movement degree of the left hind limb. The calculation of the variance belongs to conventional technical means and will not be elaborated here.

[0051] S43, calculating the variance of the X coordinate values ​​of each of the right hind limb key points to obtain a second variance; Analyze the X coordinate values ​​of the key points of the right hind limb and calculate their variance. The variance of the X coordinate values ​​of the key points of the right hind limb can reflect the stability and horizontal movement of the right hind limb.

[0052] S44: If the hind limb angle is greater than a preset angle, and the first variance or the second variance is less than a preset variance, the pet dog has urinating behavior.

[0053] Specifically, when a pet dog urinates, it usually adjusts the posture of its hind limbs to maintain stability, resulting in a larger angle between the left and right hind limbs of the pet dog (i.e., the hind limb angle). This is because the pet dog needs to lift or swing one hind leg or maintain a wider standing posture to defecate when urinating. Therefore, a hind limb angle greater than the preset angle may be an important feature of the pet dog's urination behavior.

[0054] Furthermore, since the pet dog needs to lift one of its hind legs when urinating, one of its left hind limb or right hind limb needs to maintain a stable posture, so the variance of the X coordinate value of the key point of its left hind limb or right hind limb should be small.

[0055] When these two conditions (the angle of the hind limbs is greater than the preset angle, and the variance of the X coordinate value of the left hind limb or the right hind limb is less than the preset variance) are met at the same time, it can be more accurately judged that the pet dog is urinating. This judgment is based on the typical posture characteristics of the pet dog when urinating, which are significantly different from the posture of the pet dog in other activities.

[0056] Preferably, the excretion behavior also includes defecation behavior, the left hind limb key points include a first left hind limb key point, a second left hind limb key point and a third left hind limb key point distributed in sequence from top to bottom, the right hind limb key points include a first right hind limb key point, a second right hind limb key point and a third right hind limb key point distributed in sequence from top to bottom, and judging whether the pet dog has excretion behavior according to the hind limb posture information also includes: S041. Acquire a first left hind limb direction vector according to the first left hind limb key point and the second left hind limb key point; Specifically, in this embodiment, the left hind limb key points include the first left hind limb key point Left1 (x, y), the second left hind limb key point Left2 (x, y) and the third left hind limb key point Left3 (x, y) distributed from top to bottom, and the right hind limb key points include the first right hind limb key point Right1 (x, y), the second right hind limb key point Right2 (x, y) and the third right hind limb key point Right3 (x, y) distributed from top to bottom; The first left hind limb direction vector V_left1 is calculated by the following formula: This vector describes the posture of the upper part of the pet dog's left hind limb; S042, acquiring a second left hind limb direction vector according to the second left hind limb key point and the third left hind limb key point; Specifically, the second left hind limb direction vector V_left2 is calculated by the following formula: This vector describes the posture of the lower half of the pet dog's left hind limb; S043. Acquire a first right hind limb direction vector according to the first right hind limb key point and the second right hind limb key point; Specifically, the first right hind limb direction vector V_right1 is calculated by the following formula: This vector describes the posture of the upper part of the pet dog's right hind limb; S044. Acquire a second right hind limb direction vector according to the second right hind limb key point and the third right hind limb key point; Specifically, the right hind limb direction vector V_right2 is calculated by the following formula: This vector describes the posture of the lower half of the pet dog's right hind limb; S045, calculating the angle between the first left hind limb direction vector and the second left hind limb direction vector to obtain a left hind limb angle; Specifically, the angle between the first left hind limb direction vector and the second left hind limb direction vector is calculated to obtain the left hind limb angle θ left The left hind limb angle can be calculated by the following formula: This angle describes the degree of joint bending and posture changes of the pet dog's left hind limb; S046, calculating the angle between the first right hind limb direction vector and the second left hind limb direction vector to obtain a right hind limb angle; Specifically, the angle between the first right hind limb direction vector and the second right hind limb direction vector is calculated to obtain the right hind limb angle θ right , the left hind limb angle can be calculated by the following formula: This describes the degree of joint flexion and posture changes in the pet dog's right hind limb.

[0057] S047. If the left hind limb angle or the right hind limb angle belongs to a preset angle range, and the ratio of the left hind limb angle or the right hind limb angle belongs to a preset ratio range, then the pet dog has defecation behavior.

[0058] Specifically, the left hind limb angle and the right hind limb angle are obtained by calculating the angle between the direction vectors of the pet dog's hind limbs. These angles reflect the degree of bending and extension of the pet dog's hind limbs. When the left hind limb angle or the right hind limb angle falls within the preset angle range, it means that the degree of bending of the pet dog's hind limbs conforms to the typical defecation behavior. This angle usually indicates a specific posture maintained by the pet dog during defecation behavior. By comparing the ratio of the left hind limb angle and the right hind limb angle, the system can determine the symmetry of the pet dog's hind limbs. This symmetry may be more obvious during defecation behavior because the pet dog needs to maintain a stable and symmetrical posture. If the ratio of the left hind limb angle or the right hind limb angle falls within the preset ratio range, it means that the left and right hind limbs of the pet dog maintain appropriate symmetry and proportion. This symmetry is usually related to defecation behavior. When the left hind limb angle or the right hind limb angle conforms to the preset angle range, and the angle ratio conforms to the preset ratio range, it can be comprehensively judged that the pet dog may have defecation behavior. This judgment is based on the typical posture characteristics of the pet dog during defecation.

[0059] In a specific embodiment, the preset ratio range is (0.9, 1.1), and this ratio range is used to evaluate the symmetry of the left and right hind limbs of a pet dog. A ratio close to 1 indicates that the angles of the left and right hind limbs are similar and relatively symmetrical. During defecation behavior, pet dogs usually need to maintain the symmetry of the left and right hind limbs to maintain balance and stability. This range allows a small range of differences between the left and right hind limbs, but the overall symmetry is maintained. Therefore, when the angle ratio of the left and right hind limbs is within this range, it can be considered that the posture of the pet dog meets the typical characteristics of defecation behavior.

[0060] In a specific embodiment, the preset angle range is 90°~155°. This angle range is used to evaluate the bending angle of the left and right hind limbs of the pet dog. The range of 90°~155° covers the hind limb postures from moderate bending to large bending. During defecation behavior, pet dogs usually present a posture with bent hind limbs to maintain stability and support. This range includes the typical hind limb postures that pet dogs may present when defecating. Therefore, when the left and right hind limb angles are within this range, it can be considered that the posture of the pet dog meets the characteristics of defecation behavior.

[0061] In the above embodiment, the urination behavior of the pet dog is determined only by the two conditions that the angle of the hind limbs is greater than the preset angle and the first variance or the second variance is less than the preset variance, that is, one leg is raised and the other leg is standing, but this posture is usually the posture of male dogs urinating, and cannot completely cover the urination behavior of female dogs. Therefore, in another embodiment, the hind limb key points also include at least two back key points and at least two tail key points, and the step S4, after the step S44, further includes: S45, obtaining a back vector of the pet dog according to the back key points; Specifically, in this embodiment, the posture information of the spine and tail is added to more comprehensively identify the urination behavior of the pet dog, especially the behavior judgment of the female dog; the spine key points refer to specific coordinate points distributed along the spine of the pet dog, usually including points on the upper back and waist, and the coordinates of these points are obtained by the skeleton detection algorithm in the image or video. The spine vector refers to the direction vector determined by the spine key points, which represents the posture of the pet dog's back. Specifically, it can be expressed as a direction vector from "back 1" to "back 2", describing the tilt direction of the spine; Usually, when a female dog urinates, her back will be slightly bent or remain horizontal, so obtaining the back vector is the basis for identifying this posture. The back vector is calculated through the coordinates of two key points on the back. This vector represents the direction of the back. If the angle between the back and the ground is small, that is, the back of the female dog is closer to horizontal, it indicates that the female dog may be in a crouching posture, which is consistent with the characteristics of urination behavior.

[0062] S46, obtaining a tail vector of the pet dog according to the tail key point; Specifically, the tail key points refer to the coordinate points at the base and end of the tail, and the tail vector is a direction vector determined by the tail key points, describing the bending or extension state of the tail. The direction of the vector indicates the posture of the tail relative to the ground; the purpose of this step is to capture the posture of the pet dog's tail, and the tail position is also a key feature for judging the urination behavior of the female dog. When the female dog urinates, the tail is usually kept low or slightly raised, while the tail will be significantly raised when defecating. Therefore, the analysis of the tail key points can be increased.

[0063] S47, obtaining the angle between the back of the pet dog and the ground according to the back vector, recorded as the back angle; Specifically, by calculating the angle between the back and the ground, it helps to further identify the back posture of the pet dog when urinating. Usually, the urination posture of a female dog involves a certain degree of back bending. By using the back angle, it can be determined whether the pet dog's body presents a typical female dog urinating posture. This angle is very critical in excretion behavior because there are significant differences in the back posture during different excretion behaviors.

[0064] S48, obtaining an angle between the pet dog's tail and the ground according to the tail vector, recorded as a tail angle; Specifically, the tail angle refers to the angle between the tail vector and the ground, reflecting the height or inclination of the tail relative to the ground. The purpose of this step is to determine the position of the female dog's tail when urinating by the angle between the tail and the ground. During urination, the tail usually remains at a low position or at a certain angle to the ground.

[0065] S49. If the hind limb angle is within the preset hind limb angle range, the back angle is less than or equal to the preset back angle range, and the tail angle is less than or equal to the preset tail angle, then the pet dog has urinating behavior.

[0066] This step identifies the urination behavior of female dogs by combining the posture characteristics of the hind limbs, back and tail to ensure the accuracy and completeness of the detection. Specifically, when female dogs urinate, their hind limbs are usually not raised as high as male dogs, but are kept within a relatively small angle range. For example, female dogs will slightly bend their hind limbs, spread their hind legs, and maintain a low posture. The preset hind limb angle can be adjusted according to the urination posture of female dogs and set to a smaller range, such as between 90° and 120°, which covers the common hind limb postures of female dogs when urinating. When female dogs urinate, their backs are usually horizontal or slightly tilted forward. Unlike the posture of significantly bent backs when defecating, the back angle of female dogs when urinating is smaller, and the back is basically parallel to the ground or slightly bent. The preset back angle range can be set from 0° to 20°, covering the relatively straight back posture of female dogs when urinating, ensuring that this parameter can effectively identify the urination behavior of female dogs. Moreover, when a female dog urinates, the tail usually remains low or parallel to the ground. The tail may be slightly upturned, but it will not be lifted up as obviously as when defecating. When defecating, the tail is often lifted higher to maintain body balance and smooth excretion of feces. The preset tail angle can be set to 15 to 30°, reflecting the lower position of the tail when urinating. The relationship between the judgment condition of this step and the judgment condition of step S44 is "or", that is, if any of the conditions in S44 and S49 is met, it can be determined that the pet dog is urinating.

[0067] Through comprehensive judgment of the angles of the hind limbs, the back angle and the tail angle, the system can accurately identify the female dog's urination behavior and distinguish it from defecation behavior. The posture characteristics of female dogs when urinating are different from defecation behavior in three aspects. This multi-dimensional posture analysis allows the system to more accurately detect different excretion behaviors, especially making specific adjustments for the female dog's urination posture.

[0068] Example 2

[0069] See also Figure 4 , an embodiment of the present invention provides an image-based pet dog excretion monitoring device, the device comprising: An image acquisition module, used for acquiring a first target image sequence, wherein the target image sequence includes a plurality of frames of target images, each of which includes a pet dog to be monitored; A preprocessing module, used for preprocessing the first target image sequence to obtain a second target image sequence; A posture acquisition module, used to acquire the posture information of the hind limbs of the pet dog according to the second target image sequence and a deep learning algorithm; The judgment module is used to judge whether the pet dog has defecation behavior according to the hind limb posture information.

[0070] It should be noted that the modules and units in the image-based pet dog excretion monitoring device in this embodiment correspond one-to-one to the steps in the image-based pet dog excretion monitoring method in the aforementioned embodiment. Therefore, the specific implementation of this embodiment can refer to the aforementioned image-based pet dog excretion monitoring implementation, which will not be repeated here.

[0071] Example 3

[0072] In addition, combined Figure 1 The image-based pet dog excretion monitoring method described in the embodiment of the present invention can be implemented by an electronic device. Figure 5 A schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present invention is shown.

[0073] The electronic device may include a processor and a memory storing computer program instructions.

[0074] Specifically, the processor may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiment of the present invention.

[0075] The memory may include a large capacity memory for data or instructions. By way of example and not limitation, the memory may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. Where appropriate, the memory may include a removable or non-removable (or fixed) medium. Where appropriate, the memory may be inside or outside the data processing device. In a particular embodiment, the memory is a non-volatile solid-state memory. In a particular embodiment, the memory includes a read-only memory (ROM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM) or a flash memory or a combination of two or more of these.

[0076] The processor implements any one of the image-based pet dog excretion monitoring methods in the above embodiments by reading and executing computer program instructions stored in the memory.

[0077] In one example, the electronic device may further include a communication interface and a bus. Figure 5 As shown, the processor 401, the memory 402, and the communication interface 403 are connected via a bus 410 and communicate with each other.

[0078] The communication interface is mainly used to implement communication between the modules, devices, units and / or equipment in the embodiments of the present invention.

[0079] The bus includes hardware, software or both, and the components of the electronic device are coupled to each other. For example, but not limitation, the bus may include an accelerated graphics port (AGP) or other graphics bus, an enhanced industrial standard architecture (EISA) bus, a front-side bus (FSB), a hypertransport (HT) interconnection, an industrial standard architecture (ISA) bus, an infinite bandwidth interconnection, a low pin count (LPC) bus, a memory bus, a microchannel architecture (MCA) bus, a peripheral component interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a serial advanced technology attachment (SATA) bus, a video electronics standard association local (VLB) bus or other suitable buses or a combination of two or more of these. Where appropriate, the bus may include one or more buses. Although the embodiments of the present invention describe and illustrate a specific bus, the present invention contemplates any suitable bus or interconnection.

[0080] Example 4

[0081] In addition, in combination with the above-mentioned pet dog excretion monitoring method based on images, the present invention can provide a computer-readable storage medium for implementation. The computer-readable storage medium stores computer program instructions; when the computer program instructions are executed by a processor, any one of the above-mentioned pet dog excretion monitoring methods based on images is implemented.

[0082] In summary, the embodiments of the present invention provide an image-based pet dog excretion monitoring method, device, equipment and medium.

[0083] It should be clear that the present invention is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present invention.

[0084] The functional blocks shown in the structural block diagram described above can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present invention are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier. "Machine-readable medium" may include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROMs, flash memories, erasable ROMs (EROMs), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0085] It should also be noted that the exemplary embodiments mentioned in the present invention describe some methods or systems based on a series of steps or devices. However, the present invention is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiments, or in a different order from the embodiments, or several steps can be performed simultaneously.

[0086] The above is only a specific implementation of the present invention. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the system, module and unit described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. It should be understood that the protection scope of the present invention is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be covered within the protection scope of the present invention.

Claims

1. A method for monitoring pet dog excretion based on images, characterized in that: The method comprises: Acquire a first target image sequence, wherein the target image sequence includes a plurality of frames of target images, each of which includes a pet dog to be monitored; Preprocessing the first target image sequence to obtain a second target image sequence; Acquire the hind limb posture information of the pet dog according to the second target image sequence and the deep learning algorithm; According to the hind limb posture information, it is determined whether the pet dog has defecation behavior.

2. The method for monitoring pet dog excretion based on images according to claim 1, characterized in that: The step of acquiring a first target image sequence comprises: Acquire video stream data of the activity scene where the pet dog is located in real time; Identifying the pet dog in each frame of the video stream data by using a bounding box according to a target detection algorithm; The pet dog marked by the bounding box is tracked according to a target tracking algorithm to obtain the first target image sequence.

3. The method for monitoring pet dog excretion based on images according to claim 1, characterized in that: The preprocessing of the first target image sequence to obtain a second target image sequence includes: Performing filtering processing on each frame of the target image in the first target image sequence according to a preset filtering algorithm to obtain a filtered image sequence, wherein the preset filtering algorithm includes one or more of a Gaussian filtering algorithm, a median filtering algorithm, and a bilateral filtering algorithm; Deduplication processing is performed on the filtered image sequence to obtain the second target image sequence.

4. The method for monitoring pet dog excretion based on images according to claim 3, characterized in that: The performing deduplication processing on the filtered image sequence to obtain the second target image sequence includes: Evenly dividing the filtered image sequence to obtain a preset number of filtered image groups; According to the optical flow field of the bounding box of each frame image in the filtered image group, obtaining the optical flow field gradient mean of each bounding box; Obtaining the motion amount of the bounding box of each frame image in the filtered image group according to the mean value of the optical flow field gradient; Selecting an image corresponding to a bounding box with the smallest amount of motion in each of the filtered image groups as a key frame; The second target sequence is obtained according to the key frames of each of the filtered image groups.

5. The method for monitoring pet dog excretion based on images according to any one of claims 1 to 4, characterized in that: The step of obtaining the hind limb posture information of the pet dog according to the second target image sequence and the deep learning algorithm includes: Identify the hind limbs of the pet dog in the second target image sequence according to a deep learning algorithm to obtain coordinate information of key points of the hind limbs, wherein the key points of the hind limbs include at least three left hind limb key points and at least three right hind limb key points; According to the coordinate information of the left hind limb key point, obtaining the left hind limb direction vector of the pet dog; According to the coordinate information of the right hind limb key point, obtaining the direction vector of the right hind limb of the pet dog; The hind limb posture information is obtained according to the left hind limb direction vector and the right hind limb direction vector.

6. The method for monitoring pet dog excretion based on images according to claim 5, characterized in that: The excretion behavior includes urination behavior, and judging whether the pet dog has excretion behavior according to the hind limb posture information includes: Obtaining a hind limb angle according to the left hind limb direction vector and the right hind limb direction vector; Calculate the variance of the X coordinate values ​​of each of the left hind limb key points to obtain a first variance; Calculate the variance of the X coordinate values ​​of each of the right hind limb key points to obtain a second variance; If the hind limb angle is greater than a preset angle, and the first variance or the second variance is less than a preset variance, then the pet dog has urinated.

7. The method for monitoring pet dog excretion based on images according to claim 6, characterized in that: The excretion behavior also includes defecation behavior, the left hind limb key points include a first left hind limb key point, a second left hind limb key point and a third left hind limb key point distributed in sequence from top to bottom, the right hind limb key points include a first right hind limb key point, a second right hind limb key point and a third right hind limb key point distributed in sequence from top to bottom, and judging whether the pet dog has excretion behavior according to the hind limb posture information also includes: Acquire a first left hind limb direction vector according to the first left hind limb key point and the second left hind limb key point; Acquire a second left hind limb direction vector according to the second left hind limb key point and the third left hind limb key point; Acquire a first right hind limb direction vector according to the first right hind limb key point and the second right hind limb key point; Acquire a second right hind limb direction vector according to the second right hind limb key point and the third right hind limb key point; Calculating the angle between the first left hind limb direction vector and the second left hind limb direction vector to obtain a left hind limb angle; Calculating the angle between the first right hind limb direction vector and the second left hind limb direction vector to obtain a right hind limb angle; If the left hind limb angle or the right hind limb angle belongs to a preset angle range, and the ratio of the left hind limb angle or the right hind limb angle belongs to a preset ratio range, then the pet dog has defecation behavior.

8. An image-based pet dog excretion monitoring device, characterized in that: The device comprises: An image acquisition module, used for acquiring a first target image sequence, wherein the target image sequence includes a plurality of frames of target images, each of which includes a pet dog to be monitored; A preprocessing module, used for preprocessing the first target image sequence to obtain a second target image sequence; A posture acquisition module, used to acquire the posture information of the hind limbs of the pet dog according to the second target image sequence and a deep learning algorithm; The judgment module is used to judge whether the pet dog has defecation behavior according to the hind limb posture information.

9. An electronic device, characterized in that: include: At least one processor, at least one memory and computer program instructions stored in the memory, when the computer program instructions are executed by the processor, implement the method according to any one of claims 1 to 7.

10. A storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images