Pet behavior recognition and detection system

Through multimodal analysis of images and audio, accurate identification of pet eating status and behaviors has been solved, and the problem of accurate identification of existing systems has been improved, and the accuracy and practicality of pet behavior detection has been improved.

CN119091512AActive Publication Date: 2024-12-06XINGCHONG KINGDOM (BEIJING) TECH CO LTD
View PDF 14 Cites 0 Cited by

Patent Information

Application Number
CN202411305059.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-19
Publication Date
2024-12-06
Estimated Expiration
2044-09-19

AI Technical Summary

Technical Problem

The existing pet behavior detection system cannot accurately identify the pet's eating status, and there is a problem of low accuracy in behavior recognition.

Method used

The image data acquisition module is used to obtain grayscale images, and the pixel gradient amplitude is constructed through the feature extraction module, edge pixel points are extracted and outlines are formed. The position analysis module is used to determine the pet position status, the behavior recognition module recognizes pet behavior, and the audio analysis module evaluates Call it the volume and frequency of sound, optimize the behavior recognition results.

Benefits of technology

It realizes comprehensive and accurate identification of pet behavior, improves the accuracy and reliability of identification, can identify pet location and behavior status, and discovers abnormal situations through sound analysis, providing comprehensive monitoring support for pet owners.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119091512A_ABST
    Figure CN119091512A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of image processing, in particular to a pet behavior recognition and detection system, which comprises an image data acquisition module for acquiring a grayscale image of a camera image; the feature extraction module is used for extracting the edge pixel points and forming a preselected contour according to the edge pixel points; the contour extraction module is used for extracting the contour of the pet and the contour of the feeder; the position analysis module is used for constructing the mass center of the contour of the pet and the mass center of the contour of the feeder and analyzing the position state of the pet according to the construction result; the behavior recognition module is used for recognizing the behavior state of the pet according to the analysis result of the position state of the pet in the monitoring period; and the audio analysis module is used for evaluating the abnormality of the audio according to the average sound volume and the average sound frequency of the pet in the monitoring period. According to the invention, the accuracy of pet behavior recognition is improved, and meanwhile, the health and safety of the pet are guaranteed.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a pet behavior recognition and detection system. Background Art

[0002] Pet eating behavior is one of the important indicators of pet health status. Traditional methods of monitoring pet eating behavior rely mostly on manual observation, which is not only inefficient but also difficult to achieve round-the-clock monitoring.

[0003] Chinese patent publication number CN109757395A discloses a pet behavior detection and monitoring system and method, the system includes a video acquisition unit, a target recognition and feature extraction unit, a behavior analysis unit, a storage unit, a control processing unit, an alarm unit, an audio unit, a communication unit and a mobile phone, the video acquisition unit is connected to the target recognition and feature extraction unit, the target recognition and feature extraction unit is connected to the behavior analysis unit; the video acquisition unit, the target recognition and feature extraction unit, and the behavior analysis unit are respectively connected to the storage unit; the behavior analysis unit is respectively connected to the storage unit, the alarm unit, and the audio unit through the control processing unit, and the control processing unit is connected to the mobile phone through the communication unit; when in use, the system can monitor the behavior of the pet and identify and analyze its behavior; if the pet exhibits alarm behavior, the owner is alerted through the mobile phone, and the owner can issue commands to the pet through the communication unit; it can be seen that this solution cannot achieve accurate recognition of the pet's eating status, and there is a problem of low accuracy in pet behavior recognition. Summary of the invention

[0004] The object of the present invention is to provide a pet behavior recognition and detection system to solve at least one of the problems existing in the prior art.

[0005] To achieve the above object, the present invention adopts the following technical solutions: An image data acquisition module, used to acquire a grayscale image of a camera image; A feature extraction module, used to construct the gradient amplitude of the grayscale image pixel points of the camera image, extract the pre-selected edge pixel points according to the construction result of the pixel gradient direction, extract the edge pixel points according to the gradient amplitude of the pre-selected edge pixel points, and form a pre-selected contour according to the edge pixel points; A contour extraction module, for analyzing the centroid of the pre-selected contours, and judging the pet contours and the feeder contours according to the analysis results, and for extracting the pet contours and the feeder contours according to the areas and aspect ratios of the pet contours and the feeder contours; A position analysis module is used to construct the centroid of the pet's outline and the centroid of the feeder's outline, and analyze the pet's position status based on the construction results; A behavior recognition module is used to recognize the behavior state of the pet based on the analysis results of the pet's position state during the monitoring period; The audio analysis module is used to evaluate the abnormality of the audio based on the average volume and average frequency of the pet's calls during the monitoring period.

[0006] Furthermore, the feature extraction module is provided with an edge detection unit, a first feature extraction unit and a second feature extraction unit; the edge detection unit is used to construct the gradient amplitude of the pixel point according to the gradient of the pixel point in the horizontal direction and the gradient in the vertical direction, and set: C(x, y) = (Tix 2 +Tiy 2 ) 1 / 2 ; The edge detection unit constructs the gradient direction of the pixel point according to the gradient of the pixel point in the horizontal direction and the gradient of the pixel point in the vertical direction, and sets: θ(x,y)=arctan(Tiy / Tix), Tix is ​​the horizontal gradient of I(x,y) in the grayscale image, I(x,y) is the pixel with abscissa x and ordinate y in the grayscale image, Tiy is the vertical gradient of I(x,y) in the grayscale image, C(x,y) is the gradient amplitude of I(x,y) in the grayscale image, and θ(x,y) is the gradient direction of I(x,y) in the grayscale image.

[0007] Furthermore, the first feature extraction unit extracts pre-selected edge pixels according to the construction result of the pixel gradient direction, wherein: When a1≤θ(x,y)<a2, if C(x,y)>C(x-1,y) and C(x,y)>C(x+1,y), the first feature extraction unit determines I(x,y) as a pre-selected edge pixel point, and if C(x,y)≤C(x-1,y) or C(x,y)≤C(x+1,y), the first feature extraction unit determines I(x,y) as a non-edge pixel point; When a2≤θ(x,y)<a3, if C(x,y)>C(x-1,y+1) and C(x,y)>C(x+1,y-1), the first feature extraction unit determines I(x,y) as a pre-selected edge pixel point, and if C(x,y)≤C(x-1,y+1) or C(x,y)≤C(x+1,y-1), the first feature extraction unit determines I(x,y) as a non-edge pixel point; When a3≤θ(x,y)<a4, if C(x,y)>C(x,y-1) and C(x,y)>C(x,y+1), the first feature extraction unit determines I(x,y) as a pre-selected edge pixel point, and if C(x,y)≤C(x,y-1) or C(x,y)≤C(x,y+1), the first feature extraction unit determines I(x,y) as a non-edge pixel point; When a4≤θ(x,y)≤a5, if C(x,y)>C(x+1,y+1) and C(x,y)>C(x-1,y-1), the first feature extraction unit determines I(x,y) as a pre-selected edge pixel point; if C(x,y)≤C(x+1,y+1) or C(x,y)≤C(x-1,y-1), the first feature extraction unit determines I(x,y) as a non-edge pixel point; The first feature extraction unit records the pre-selected edge pixel point as YX(x, y), and records the gradient amplitude of the pre-selected edge pixel point as YXC(x, y); Among them, a1 is the first preset angle, a2 is the second preset angle, a3 is the third preset angle, a4 is the fourth preset angle, and a5 is the fifth preset angle.

[0008] Furthermore, the second feature extraction unit compares the gradient amplitude of the preselected edge pixel points with each preset gradient amplitude to extract the edge pixel points, wherein: If YXC(x,y)≤T1, the second feature extraction unit determines that YX(x,y) is a non-edge pixel; if YXC(x,y)≥T2, the second feature extraction unit determines that YX(x,y) is an edge pixel; when T1<YXC(x,y)<T2, the second feature extraction unit determines that YX(x,y) is a weak edge pixel. If YXC(x-1,y) or YXC(x+1,y) or YXC(x,y-1) Or YXC(x,y+1) or YXC(x-1,y+1) or YXC(x-1,y-1) or YXC(x+1,y-1) or YXC(x+1,y+1) is an edge point, and the second feature extraction unit determines that the weak edge point is an edge pixel point. Otherwise, the second feature extraction unit determines that the weak edge pixel point is a non-edge pixel point. The second feature extraction unit connects adjacent edge pixels to form a continuous edge line as the pre-selected contour.

[0009] Furthermore, the contour extraction module is provided with a centroid analysis unit and a contour extraction unit; the centroid analysis unit analyzes the centroid of each preselected contour, and sets the centroid of the i-th preselected contour as Zi, setting: , xni is the horizontal coordinate of the ni-th edge pixel point of the ith pre-selected contour, yni is the vertical coordinate of the ni-th edge pixel point of the ith pre-selected contour, and Ni is the number of edge pixels of the ith pre-selected contour; like , the centroid analysis unit determines that the i-th pre-selected contour is the pet candidate contour; if , the centroid analysis unit determines that the i-th pre-selected contour is the feeder candidate contour; if and , the centroid analysis unit determines that the i-th preselected contour is an invalid contour, D1 is the first preset area, and D2 is the second preset area; The contour extraction unit compares the area of ​​each pet's to-be-selected contour with each preset area to screen each pet's contour. If sk<s1 or sk>s2, the contour extraction unit determines that the kth pet's to-be-selected contour is an invalid contour. If s1≤Sk≤s2 and bk<b1, the contour extraction unit determines that the kth pet's to-be-selected contour is an invalid contour. If s1≤Sk≤s2 or bk>b2, the contour extraction unit determines that the kth pet's to-be-selected contour is an invalid contour. If s1≤Sk≤s2 or b1≤bk≤b2, the contour extraction unit determines that the kth pet's to-be-selected contour is a pre-selected pet contour. When the number of pre-selected pet contours mc=1, the contour extraction unit determines that the pre-selected pet contour is a pet contour. When mc>1, the contour extraction unit constructs a recommendation coefficient for each pre-selected pet contour. The contour extraction unit sets the recommendation coefficient of the ycth pre-selected pet contour to Wyc, and sets: Wyc=-{|SWyc-(s1+s2) / 2| / [(s1+s2) / 2]+|bWyc-(b1+b2) / 2| / [b1+b2) / 2]}; Wherein, sk is the area of ​​the kth pet to be selected, s1 is the preset minimum pet area, s2 is the preset maximum pet area, bk is the aspect ratio of the kth pet to be selected, b1 is the preset minimum pet aspect ratio, b2 is the preset maximum pet aspect ratio, SWyc is the area of ​​the ycth pre-selected pet outline, and bWyc is the aspect ratio of the ycth pre-selected pet outline; The profile extraction unit sorts the recommendation coefficients of the pre-selected pet profiles and uses the pre-selected pet profile with the largest recommendation coefficient as the pet profile; The contour extraction unit extracts the contour of the feeder.

[0010] Furthermore, the position analysis module calculates the distance J between the pet's contour centroid and the feeder's contour centroid, and sets: , CWx is the horizontal coordinate of the centroid of the pet's outline, CWy is the vertical coordinate of the centroid of the pet's outline, WFx is the horizontal coordinate of the feeder's outline, and WFy is the vertical coordinate of the feeder's outline; When J≤j0, the position analysis module determines that the pet is at the feeding distance, and when J>j0, the position analysis module determines that the pet is not at the feeding distance, and j0 is a preset distance.

[0011] Furthermore, the behavior recognition module recognizes the behavior state of the pet according to the analysis result of the pet's position state during the monitoring period, wherein: If t1 / t0≤α, the behavior recognition module determines that the pet is not in the "want to eat" state in the current monitoring period; if t1 / t0>α, the behavior recognition module determines that the pet is in the "want to eat" state in the current monitoring period, and α is a preset state coefficient.

[0012] Furthermore, the volume analysis module is provided with a volume analysis unit, a frequency analysis unit, an audio analysis unit and a noise analysis unit; the volume analysis unit is used to compare the average call volume p0 of the pet during the monitoring period with the preset volume p1 to construct a volume coefficient; The frequency analysis unit compares the pet's average calling frequency u0 during the monitoring period with the preset frequency u1 to construct a frequency coefficient.

[0013] Further, the audio analysis unit constructs the audio coefficient YP of the pet's call according to the construction results of the volume coefficient and the frequency coefficient in the monitoring period, and sets YP=E1×volume coefficient+E2×frequency coefficient, where E1 is the volume weight and E2 is the frequency weight; The audio analysis unit compares the audio coefficient YP with the preset audio coefficient YP0 to evaluate the abnormality of the pet's call audio during the monitoring period, and adjusts the pet status recognition process according to the evaluation result, wherein: If YP≤YP0, the audio analysis unit determines that the audio of the pet's call in the current monitoring period is normal and no adjustment is made. If YP>YP0, the audio analysis unit determines that the audio of the pet's call in the current monitoring period is abnormal and adjusts the pet's status recognition process, and sets the adjusted preset status coefficient to α'.

[0014] Furthermore, the noise analysis unit compares the average ambient noise zb obtained during the monitoring period with the preset ambient noise zb0 to optimize the adjustment process of the pet status recognition process. If zb≤zb0, the noise analysis unit determines that the ambient noise in the current monitoring period is normal and no optimization is performed. If zb>zb0, the noise analysis unit determines that the ambient noise in the current monitoring period is abnormal, and optimizes the adjustment process of the pet status recognition process, and sets the optimized preset volume to p1'.

[0015] The beneficial effects of the present invention are as follows: The image data acquisition module acquires grayscale images to provide basic data for subsequent processing. The feature extraction module extracts image edge and contour features through gradient analysis to effectively identify key information in the image. The contour extraction module analyzes contour centroid, area and other features to accurately extract the contours of the pet and the feeder. The position analysis module calculates the relative position of the pet and the feeder to determine whether the pet is within the feeding range. The behavior recognition module determines whether the pet is in a "want to eat" state based on the position analysis results. The audio analysis module analyzes the volume and frequency of the pet's calls, evaluates abnormal situations, and optimizes the behavior recognition results. Each module works in series, from image acquisition, feature extraction to behavior analysis, and then combines audio information to achieve comprehensive and accurate pet behavior recognition. The system can not only identify the position and behavior state of the pet, but also detect abnormal situations through sound analysis, providing comprehensive monitoring for pet owners. At the same time, the system takes into account the influence of environmental noise, further improving the accuracy and reliability of recognition. This multi-modal and multi-level analysis method greatly improves the accuracy and practicality of pet behavior recognition and provides strong support for pet intelligent monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0017] Figure 1 Schematic diagram of the structure of the pet behavior recognition and detection system of this embodiment.

[0018] Figure 2 Schematic diagram of the structure of the feature extraction module of this embodiment.

[0019] Figure 3 Schematic diagram of the structure of the contour extraction module of this embodiment.

[0020] Figure 4 Schematic diagram of the structure of the audio analysis module of this embodiment. DETAILED DESCRIPTION

[0021] In order to more clearly illustrate the present invention, the present invention is further described below in conjunction with preferred embodiments and accompanying drawings. Similar components in the accompanying drawings are represented by the same reference numerals. It should be understood by those skilled in the art that the content specifically described below is illustrative rather than restrictive, and should not be used to limit the scope of protection of the present invention.

[0022] It should be noted that, although the terms first, second, third, etc. may be used to describe in the embodiments of the present application, these descriptions should not be limited to these terms. These terms are only used to distinguish the descriptions. For example, without departing from the scope of the embodiments of the present application, the first may also be referred to as the second, and similarly, the second may also be referred to as the first.

[0023] See also Figure 1 As shown, it is a structural diagram of the pet behavior recognition and detection system of this embodiment, and the system includes: An image data acquisition module is used to acquire a grayscale image of a camera image; the camera image is defined as an image of the area near the feeder taken by a camera device. In this embodiment, the camera device is not specifically limited, and those skilled in the art can freely set it, wherein the camera device can be set as a camera sensor; In this embodiment, the method for acquiring the grayscale image of the camera image is not specifically limited, and those skilled in the art can freely set it, and it only needs to meet the acquisition requirements of the grayscale image of the camera image, wherein the camera image can be acquired through the camera sensor, and the grayscale image can be acquired through OpenCV in Python; The contour extraction module is used to construct the gradient amplitude of the grayscale image pixel points of the camera image, extract the pre-selected edge pixel points according to the construction result of the pixel gradient direction, and extract the edge pixel points according to the gradient amplitude of the pre-selected edge pixel points, and form a pre-selected contour according to the edge pixel points. The contour extraction module is connected to the image data acquisition module.

[0024] See also Figure 2 As shown, the contour extraction module includes an edge detection unit, which is used to construct the gradient amplitude of the pixel point according to the gradient of the pixel point in the horizontal direction and the gradient in the vertical direction to obtain the direction information of each edge to improve the accuracy of feature extraction; The construction process of the gradient amplitude of a pixel is as follows: C(x,y)=(Tix 2 +Tiy 2 ) 1 / 2 ; The edge detection unit constructs the gradient direction of the pixel point according to the gradient of the pixel point in the horizontal direction and the gradient of the pixel point in the vertical direction, and sets: θ(x,y)=arctan(Tiy / Tix), Tix is ​​the horizontal gradient of I(x,y) in the grayscale image, I(x,y) is the pixel with abscissa x and ordinate y in the grayscale image, Tiy is the vertical gradient of I(x,y) in the grayscale image, C(x,y) is the gradient amplitude of I(x,y) in the grayscale image, and θ(x,y) is the gradient direction of I(x,y) in the grayscale image; the edge detection unit can significantly improve the accuracy and efficiency of image analysis through effective gradient calculation and edge detection, and provide a reliable basis for subsequent image processing.

[0025] Specifically, the edge detection unit establishes a plane rectangular coordinate system with the pixel point in the lower left corner of the grayscale image of the camera image as the origin, the positive direction of the y-axis from bottom to top, and the positive direction of the x-axis from left to right, and the position of each pixel point is represented by the coordinates of the pixel point; in this embodiment, there is no specific limitation on the method for obtaining the gradient of the pixel point in the horizontal direction and the gradient in the vertical direction, and technical personnel in this field can set it freely as long as the requirements for obtaining the gradient of the pixel point are met, wherein the gradient of the pixel point can be obtained by applying a 3x3 convolution kernel through a Sobel edge detection algorithm.

[0026] Please continue reading Figure 2 As shown, the feature extraction module also includes a first feature extraction unit, which extracts pre-selected edge pixels according to the construction result of the pixel gradient direction to accurately identify the pre-selected edge pixels, and the first feature extraction unit is connected to the edge detection unit; The extraction process of pre-selected edge pixels is as follows: When a1≤θ(x,y)<a2, if C(x,y)>C(x-1,y) and C(x,y)>C(x+1,y), the first feature extraction unit determines I(x,y) as a pre-selected edge pixel point, and if C(x,y)≤C(x-1,y) or C(x,y)≤C(x+1,y), the first feature extraction unit determines I(x,y) as a non-edge pixel point; When a2≤θ(x,y)<a3, if C(x,y)>C(x-1,y+1) and C(x,y)>C(x+1,y-1), the first feature extraction unit determines I(x,y) as a pre-selected edge pixel point, and if C(x,y)≤C(x-1,y+1) or C(x,y)≤C(x+1,y-1), the first feature extraction unit determines I(x,y) as a non-edge pixel point; When a3≤θ(x,y)<a4, if C(x,y)>C(x,y-1) and C(x,y)>C(x,y+1), the first feature extraction unit determines I(x,y) as a pre-selected edge pixel point, and if C(x,y)≤C(x,y-1) or C(x,y)≤C(x,y+1), the first feature extraction unit determines I(x,y) as a non-edge pixel point; When a4≤θ(x,y)≤a5, if C(x,y)>C(x+1,y+1) and C(x,y)>C(x-1,y-1), the first feature extraction unit determines I(x,y) as a pre-selected edge pixel point; if C(x,y)≤C(x+1,y+1) or C(x,y)≤C(x-1,y-1), the first feature extraction unit determines I(x,y) as a non-edge pixel point; The first feature extraction unit records the pre-selected edge pixel point as YX(x, y), and records the gradient amplitude of the pre-selected edge pixel point as YXC(x, y); Among them, a1 is the first preset angle, a2 is the second preset angle, a3 is the third preset angle, a4 is the fourth preset angle, and a5 is the fifth preset angle; the first feature extraction unit improves the detection accuracy and enhances the resistance to noise through comprehensive judgment of the gradient direction and amplitude, thereby improving the efficiency of subsequent image processing and analysis.

[0027] Specifically, in this embodiment, the values ​​of each preset angle are not specifically limited, and technical personnel in this field can set them freely as long as the setting requirements of each preset angle are met. Among them, the best value of a1 is 0°, the best value of a2 is 22.5°, the best value of a3 is 67.5°, the best value of a4 is 112.5°, and the best value of a5 is 157.5°.

[0028] Please continue reading Figure 2 As shown, the feature extraction module also includes a second feature extraction unit, which is used to compare the gradient amplitude of the preselected edge pixel point with each preset gradient amplitude to extract the edge pixel point, thereby improving the accuracy and precision of edge detection, and the second feature extraction unit is connected to the first feature extraction unit; The process of extracting edge pixels is as follows: When YXC(x, y)≤T1, the second feature extraction unit determines that YX(x, y) is a non-edge pixel point; When YXC(x,y)≥T2, the second feature extraction unit determines that YX(x,y) is an edge pixel point; When T1<YXC(x,y)<T2, the second feature extraction unit determines that YX(x,y) is a weak edge pixel point, and if YXC(x-1,y) or YXC(x+1,y) or YXC(x,y-1) or YXC(x,y+1) or YXC(x-1,y+1) or YXC(x-1,y-1) or YXC(x+1,y-1) or YXC(x+1,y+1) is an edge point, the second feature extraction unit determines the weak edge point as an edge pixel point, otherwise, the second feature extraction unit determines the weak edge pixel point as a non-edge pixel point; Wherein, T1 is the preset minimum gradient amplitude, and T2 is the preset maximum gradient; The second feature extraction unit connects adjacent edge pixels to form a continuous edge line as a pre-selected contour; the second feature extraction unit sets upper and lower limits of the gradient amplitude to effectively distinguish edge pixels, weak edge pixels and non-edge pixels, thereby improving the accuracy and precision of edge detection.

[0029] Specifically, in this embodiment, the values ​​of T1 and T2 are not specifically limited, and those skilled in the art can set them freely as long as the value requirements of T1 and T2 are met. Among them, the optimal value of T1 is Tmax×0.2, and the optimal value of T2 is Tmax×0.5. Tmax is the maximum value of the pixel gradient amplitude, which can be obtained by sorting the gradient amplitude of each pixel through the bubble sort method.

[0030] Please continue reading Figure 1 As shown, the system also includes a contour extraction module, which is used to analyze the centroid of the pre-selected contour, and judge the pet's selected contour and the feeder's selected contour based on the analysis result, and is also used to extract the pet's contour and the feeder's contour based on their area and aspect ratio. The contour extraction module is connected to the feature extraction module.

[0031] See also Figure 3 As shown, the contour extraction module includes a centroid analysis unit, which is used to analyze the centroid of each pre-selected contour, and judge the pet's candidate contour and the feeder's candidate contour according to the analysis result, so as to effectively filter out contours that do not conform to the shape or position of the target object, so that the recognition of invalid contours becomes more accurate; The centroid analysis process of the preselected contour is as follows: Set the centroid of the i-th preselected contour to Zi, and set: , xni is the horizontal coordinate of the ni-th edge pixel point of the ith pre-selected contour, yni is the vertical coordinate of the ni-th edge pixel point of the ith pre-selected contour, and Ni is the number of edge pixels of the ith pre-selected contour; The judgment process of the pet candidate outline and the feeder candidate outline is as follows: like , the centroid analysis unit determines that the i-th pre-selected contour is the pet candidate contour; if , the centroid analysis unit determines that the i-th pre-selected contour is the feeder candidate contour; if and The centroid analysis unit determines that the i-th pre-selected contour is an invalid contour, D1 is the first preset area, and D2 is the second preset area; the centroid analysis unit can quickly and accurately identify the target object while reducing the false detection rate and calculation complexity, thereby improving the efficiency of image processing through an efficient contour centroid calculation and classification mechanism.

[0032] Specifically, this embodiment does not make any specific limitation on the setting of the preset areas, and those skilled in the art can set them freely as long as the setting requirements of the preset areas are met, wherein the first preset area is the area where the pet eats and the second preset area is the area of ​​the feeder, such as D1={(x,y)|0.4X≤x≤0.6X, 0.4Y≤y≤0.6Y}, D2={(x,y)|0.72X≤x≤0.8X, 0.45Y≤y≤0.53Y}, where X is the length of the grayscale image of the camera image, and Y is the width of the grayscale image of the camera image.

[0033] Please continue reading Figure 3 As shown, the contour extraction module also includes a contour extraction unit, which is used to obtain the area and aspect ratio of the pet's contour to be selected and the feeder's contour to be selected, and extract the pet's contour and the feeder's contour to effectively screen out valid contours that meet specific standards, the aspect ratio is the aspect ratio of the minimum circumscribed rectangle of the pet's contour to be selected and the feeder's contour to be selected, and the contour extraction unit is connected to the centroid analysis unit; The pet outline and feeder outline extraction process is as follows: The contour extraction unit compares the area of ​​each pet's to-be-selected contour with each preset area to screen each pet's contour. If sk<s1 or sk>s2, the contour extraction unit determines that the kth pet's to-be-selected contour is an invalid contour. If s1≤Sk≤s2 and bk<b1, the contour extraction unit determines that the kth pet's to-be-selected contour is an invalid contour. If s1≤Sk≤s2 or bk>b2, the contour extraction unit determines that the kth pet's to-be-selected contour is an invalid contour. If s1≤Sk≤s2 or b1≤bk≤b2, the contour extraction unit determines that the kth pet's to-be-selected contour is a pre-selected pet contour. When the number of pre-selected pet contours mc=1, the contour extraction unit determines that the pre-selected pet contour is a pet contour. When mc>1, the contour extraction unit constructs a recommendation coefficient for each pre-selected pet contour. The contour extraction unit sets the recommendation coefficient of the ycth pre-selected pet contour to Wyc, and sets: Wyc=-{|SWyc-(s1+s2) / 2| / [(s1+s2) / 2]+|bWyc-(b1+b2) / 2| / [b1+b2) / 2]}; Wherein, sk is the area of ​​the kth pet to be selected, s1 is the preset minimum pet area, s2 is the preset maximum pet area, bk is the aspect ratio of the kth pet to be selected, b1 is the preset minimum pet aspect ratio, b2 is the preset maximum pet aspect ratio, SWyc is the area of ​​the ycth pre-selected pet outline, and bWyc is the aspect ratio of the ycth pre-selected pet outline; The profile extraction unit sorts the recommendation coefficients of the pre-selected pet profiles and uses the pre-selected pet profile with the largest recommendation coefficient as the pet profile; The contour extraction unit extracts the contour of the feeder. In this embodiment, the process of extracting the contour of the feeder is the same as the process of extracting the contour of the pet, which will not be repeated in this embodiment. The contour extraction unit can effectively reduce false detection and missed detection and improve the overall detection accuracy by setting threshold conditions for the area and aspect ratio.

[0034] Specifically, in this embodiment, the method for obtaining the area and aspect ratio of the pet's candidate contour and the feeder's candidate contour is not specifically limited. Those skilled in the art can set it freely, and only need to meet the requirements for obtaining the area and aspect ratio of the pet's candidate contour and the feeder's candidate contour, wherein the aspect ratio can be obtained by using the OpenCV function cv2.minAreaRect(), and the area can be obtained by using the OpenCV function cv2.contourArea(); in this embodiment, the value of the screening threshold is not specifically limited. Those skilled in the art can set it freely, and only need to meet the value requirements of each screening threshold, wherein, when the pet is a small pet, the best value of s1 is 1000 pixels, the best value of s2 is 5000 pixels, the best value of b1 is 1.5, and the best value of b2 is 2; when extracting the feeder contour, the best value of the minimum area threshold is 1000 pixels, the best value of the maximum area threshold is 3000 pixels, the best value of the minimum aspect ratio threshold is 1, and the best value of the maximum aspect ratio threshold is 1.5.

[0035] Please continue reading Figure 1 As shown, the system further includes a position analysis module for constructing the pet contour centroid and the feeder contour centroid, and analyzing the position state of the pet according to the construction result, and the position analysis module is connected to the contour extraction module; the position analysis module constructs the pet contour centroid CW (CWx, CWy) and the feeder contour centroid WF (WFx, WFy), CWx is the horizontal coordinate of the pet contour centroid, CWy is the vertical coordinate of the pet contour centroid, WFx is the horizontal coordinate of the feeder contour, and WFy is the vertical coordinate of the feeder contour; the method for constructing the pet contour centroid and the feeder contour centroid in this embodiment is the same as the method for constructing the pre-selected contour, and this embodiment will not be repeated; The position analysis module calculates the distance J between the pet's contour centroid and the feeder's contour centroid, and sets: ; When J≤j0, the position analysis module determines that the pet is at the feeding distance; when J>j0, the position analysis module determines that the pet is not at the feeding distance, and j0 is a preset distance; the position analysis module improves the accuracy of the pet's position status determination through a reasonable center of mass calculation and distance determination mechanism, thereby improving the accuracy of the pet's status recognition.

[0036] Specifically, this embodiment does not impose any specific limitation on the setting of the preset distance, and those skilled in the art can freely set it as long as the setting requirements of the preset distance are met, wherein the best value of j0 is 80 pixels.

[0037] Please continue reading Figure 1As shown, the system further includes a behavior recognition module, which is used to recognize the behavior state of the pet according to the analysis result of the pet's position state during the monitoring period, and the behavior recognition module is connected to the position analysis module; The process of identifying the pet's behavioral state is as follows: If t1 / t0≤α, the behavior recognition module determines that the pet is not in the "want to eat" state during the current monitoring period; if t1 / t0>α, the behavior recognition module determines that the pet is in the "want to eat" state during the current monitoring period; The behavior recognition module outputs the pet's behavior status to the user; t1 is the length of time the pet is within the feeding distance during the monitoring period, t0 is the length of the monitoring period, and α is the preset state coefficient; the behavior recognition module can accurately determine whether the pet is in the "want to eat" state based on the pet's activity during the monitoring period, thereby optimizing the feeding time. Secondly, through the set monitoring coefficient α, the system can flexibly adapt to the behavioral habits of different pets.

[0038] Specifically, in this embodiment, there is no specific limitation on the setting of the monitoring period, and those skilled in the art can set it freely, as long as the setting requirements of the monitoring period are met, wherein the monitoring period can be set to 2 minutes, 3 minutes, etc.; in this embodiment, there is no specific limitation on the setting of the preset state coefficient, and those skilled in the art can set it freely, as long as the setting requirements of the preset state coefficient are met, wherein the optimal value of α is 0.78.

[0039] Please continue reading Figure 1 As shown, the system also includes an audio analysis module for obtaining the average barking volume and average barking frequency of the pet during the monitoring period, and evaluating the abnormality of the audio based on the average barking volume and average barking frequency of the pet during the monitoring period. The audio analysis module is connected to the behavior recognition module.

[0040] Specifically, in this embodiment, there is no specific limitation on the method for obtaining the average volume and average frequency of the pet's barking during the monitoring period. Technical personnel in this field can set it freely, and it only needs to meet the requirements for obtaining the average volume and average frequency of the pet's barking during the monitoring period. Among them, the average ambient noise can be obtained through a sound sensor.

[0041] See also Figure 4 As shown, the audio recognition module includes a volume analysis unit, which compares the average call volume p0 of the pet during the monitoring period with the preset volume p1 to construct a volume coefficient to effectively monitor the emotional state of the pet; The volume coefficient is constructed as follows: The volume analysis unit compares the average volume p0 of the pet's call during the monitoring period with the preset volume p1 to construct a volume coefficient. If p0≤p1, the volume analysis unit determines that the pet's call volume is normal, and sets the volume coefficient to YL1, setting YL1=0. If p0>p1, the volume analysis unit determines that the pet's call volume is abnormal, and sets the volume coefficient to YL2, setting YL2=ln[(p0-p1) / (p0+p1)+1]. By comparing the average value of the pet's call volume with the preset value, the volume analysis unit can accurately determine whether the pet's call is normal, thereby realizing effective monitoring of the pet's emotional state. When an abnormal volume is detected, the system will generate a corresponding volume coefficient to provide feedback on pet behavior recognition, thereby improving the accuracy of pet behavior recognition.

[0042] Specifically, the setting of the preset volume is not specifically limited in this embodiment, and those skilled in the art can set it freely as long as the setting requirements of the preset volume are met. Among them, when the pet is a small dog, the best value of p1 is 70 decibels.

[0043] Please continue reading Figure 4 As shown, the audio analysis module also includes a frequency analysis unit, which is used to compare the average call frequency u0 of the pet during the monitoring period with the preset frequency u1 to construct a frequency coefficient, so as to effectively determine whether the frequency of the pet's call is normal, and the frequency analysis unit is connected to the volume analysis unit; The frequency coefficients are constructed as follows: The frequency analysis unit compares the pet's average call frequency u0 during the monitoring period with the preset frequency u1 to construct a frequency coefficient. If u0≤u1, the frequency analysis unit determines that the pet's call frequency is normal, and sets the frequency coefficient to PL1, setting PL1=0. If u0>u1, the frequency analysis unit determines that the pet's call frequency is abnormal, and sets the frequency coefficient to PL2, setting PL2=(u0-u1) / (u0+u1). When monitoring the frequency of the pet's call, the frequency analysis unit can effectively determine whether the frequency of the pet's call is normal by comparing it with the preset frequency, so as to provide feedback on the pet's behavior recognition, thereby improving the accuracy of the pet's behavior recognition.

[0044] Specifically, in this embodiment, the setting of the preset frequency is not specifically limited, and those skilled in the art can set it freely as long as the setting requirements of the preset frequency are met. Among them, when the pet is a small dog, the best value of u1 is 400 Hz.

[0045] Please continue reading Figure 4As shown, the audio analysis module also includes an audio analysis unit, which is used to construct an audio coefficient YP of the pet's call according to the construction results of the volume coefficient and the frequency coefficient in the monitoring period, and adjust the recognition process of the pet state to improve the accuracy of the pet behavior analysis, and the audio analysis unit is connected to the volume analysis unit; The adjustment process of the pet status recognition process is as follows: The audio analysis unit constructs the audio coefficient YP of the pet's call according to the construction results of the volume coefficient and the frequency coefficient in the monitoring period, and sets YP=E1×volume coefficient+E2×frequency coefficient, where E1 is the volume weight, E2 is the frequency weight, E1+E2=1, and E1<E2; The audio analysis unit compares the audio coefficient YP with the preset audio coefficient YP0 to evaluate the abnormality of the pet's call audio during the monitoring period, and adjusts the pet status recognition process according to the evaluation result, wherein: If YP≤YP0, the audio analysis unit determines that the audio of the pet's call in the current monitoring period is normal and does not make any adjustments. If YP>YP0, the audio analysis unit determines that the audio of the pet's call in the current monitoring period is abnormal and adjusts the recognition process of the pet's state, and sets the adjusted preset state coefficient to α', setting: α'=α×[1-e (YP-YP0)-1 ) / (e-1)], where e is the natural logarithm; the audio analysis unit combines the volume coefficient and the frequency coefficient to construct the audio coefficient of the pet's call, and evaluates the abnormality of the pet's call by comparing it with the preset audio coefficient, so as to adjust the recognition process of the pet's status in real time, make the system more intelligent, and improve the accuracy of pet behavior analysis.

[0046] Specifically, in this embodiment, there is no specific limitation on the setting of each weight and preset audio coefficient. Technical personnel in this field can set it freely as long as the setting requirements of each weight and preset audio coefficient are met. Among them, the optimal value of E1 is 0.4, the optimal value of E2 is 0.6, and the optimal value of YP0 is 0.15.

[0047] Please continue reading Figure 4 As shown, the audio analysis module also includes a noise analysis unit, which is used to compare the average environmental noise zb obtained during the monitoring period with the preset environmental noise zb0 to optimize the adjustment process of the pet state recognition process, so as to optimize the pet state recognition process, and the noise analysis unit is connected to the audio analysis unit; The noise analysis unit compares the average environmental noise zb obtained during the monitoring period with the preset environmental noise zb0 to optimize the adjustment process of the pet state recognition process. If zb≤zb0, the noise analysis unit determines that the environmental noise in the current monitoring period is normal and does not perform optimization. If zb>zb0, the noise analysis unit determines that the environmental noise in the current monitoring period is abnormal and optimizes the adjustment process of the pet state recognition process. The optimized preset volume is set to p1'. Set: p1'=p1×{1+exp[(lg(zb-zb0) / (zb+zb0)]}; the noise analysis unit optimizes the pet state recognition process by monitoring the ambient noise and comparing it with the preset ambient noise, reduces the interference of environmental factors on the pet audio, and improves the accuracy of pet behavior recognition.

[0048] Specifically, in this embodiment, there is no specific limitation on the method for obtaining the average ambient noise, and those skilled in the art can set it freely, as long as the requirements for obtaining the average ambient noise are met, wherein the average ambient noise can be obtained through a sound sensor; in this embodiment, there is no specific limitation on the value of the preset ambient noise, and those skilled in the art can set it freely, as long as the requirements for the value of the preset ambient noise are met, wherein the optimal value of zb0 is 45 decibels.

[0049] Specifically, the pet behavior recognition detection system described in this embodiment is applied to the recognition of the eating status of a pet. It extracts multi-dimensional features from the grayscale image captured by the camera, analyzes the position of the pet and the feeder based on the extracted features, and recognizes the behavior status of the pet in combination with the position status analysis results and the monitoring period. It evaluates the volume and frequency of the pet's calls through the audio analysis module, and adjusts the behavior recognition process based on the results. This realizes intelligent and accurate recognition of pet behavior and improves the efficiency and accuracy of pet behavior detection.

[0050] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not limitations on the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is impossible to list all the embodiments here. All obvious changes or modifications derived from the technical solution of the present invention are still within the protection scope of the present invention.

Claims

1. A pet behavior recognition and detection system, characterized in that: include, An image data acquisition module, used to acquire a grayscale image of a camera image; A feature extraction module, used to construct the gradient amplitude of the grayscale image pixel points of the camera image, extract the pre-selected edge pixel points according to the construction result of the pixel gradient direction, extract the edge pixel points according to the gradient amplitude of the pre-selected edge pixel points, and form a pre-selected contour according to the edge pixel points; A contour extraction module, for analyzing the centroid of the pre-selected contours, and judging the pet contours and the feeder contours according to the analysis results, and for extracting the pet contours and the feeder contours according to the areas and aspect ratios of the pet contours and the feeder contours; A position analysis module is used to construct the centroid of the pet's outline and the centroid of the feeder's outline, and analyze the pet's position status based on the construction results; A behavior recognition module is used to recognize the behavior state of the pet based on the analysis results of the pet's position state during the monitoring period; The audio analysis module is used to evaluate the abnormality of the audio based on the average volume and average frequency of the pet's calls during the monitoring period.

2. The pet behavior recognition and detection system according to claim 1, characterized in that: The feature extraction module is provided with an edge detection unit, a first feature extraction unit and a second feature extraction unit; the edge detection unit is used to construct the gradient amplitude of the pixel point according to the gradient of the pixel point in the horizontal direction and the gradient in the vertical direction, and set: C(x, y) = (Tix 2 +Tiy 2 ) 1 / 2 ; The edge detection unit constructs the gradient direction of the pixel point according to the gradient of the pixel point in the horizontal direction and the gradient of the pixel point in the vertical direction, and sets: θ(x,y)=arctan(Tiy / Tix), Tix is ​​the horizontal gradient of I(x,y) in the grayscale image, I(x,y) is the pixel with abscissa x and ordinate y in the grayscale image, Tiy is the vertical gradient of I(x,y) in the grayscale image, C(x,y) is the gradient amplitude of I(x,y) in the grayscale image, and θ(x,y) is the gradient direction of I(x,y) in the grayscale image.

3. The pet behavior recognition and detection system according to claim 2, characterized in that: The first feature extraction unit extracts pre-selected edge pixels according to the construction result of the pixel gradient direction, wherein: When a1≤θ(x,y)<a2, if C(x,y)>C(x-1,y) and C(x,y)>C(x+1,y), the first feature extraction unit determines I(x,y) as a pre-selected edge pixel point, and if C(x,y)≤C(x-1,y) or C(x,y)≤C(x+1,y), the first feature extraction unit determines I(x,y) as a non-edge pixel point; When a2≤θ(x,y)<a3, if C(x,y)>C(x-1,y+1) and C(x,y)>C(x+1,y-1), the first feature extraction unit determines I(x,y) as a pre-selected edge pixel point, and if C(x,y)≤C(x-1,y+1) or C(x,y)≤C(x+1,y-1), the first feature extraction unit determines I(x,y) as a non-edge pixel point; When a3≤θ(x,y)<a4, if C(x,y)>C(x,y-1) and C(x,y)>C(x,y+1), the first feature extraction unit determines I(x,y) as a pre-selected edge pixel point, and if C(x,y)≤C(x,y-1) or C(x,y)≤C(x,y+1), the first feature extraction unit determines I(x,y) as a non-edge pixel point; When a4≤θ(x,y)≤a5, if C(x,y)>C(x+1,y+1) and C(x,y)>C(x-1,y-1), the first feature extraction unit determines I(x,y) as a pre-selected edge pixel point; if C(x,y)≤C(x+1,y+1) or C(x,y)≤C(x-1,y-1), the first feature extraction unit determines I(x,y) as a non-edge pixel point; The first feature extraction unit records the pre-selected edge pixel point as YX(x, y), and records the gradient amplitude of the pre-selected edge pixel point as YXC(x, y); Among them, a1 is the first preset angle, a2 is the second preset angle, a3 is the third preset angle, a4 is the fourth preset angle, and a5 is the fifth preset angle.

4. The pet behavior recognition and detection system according to claim 3, characterized in that: The second feature extraction unit compares the gradient amplitude of the preselected edge pixel points with each preset gradient amplitude to extract the edge pixel points, wherein: If YXC(x,y)≤T1, the second feature extraction unit determines that YX(x,y) is a non-edge pixel; if YXC(x,y)≥T2, the second feature extraction unit determines that YX(x,y) is an edge pixel; when T1<YXC(x,y)<T2, the second feature extraction unit determines that YX(x,y) is a weak edge pixel. If YXC(x-1,y) or YXC(x+1,y) or YXC(x,y-1) Or YXC(x,y+1) or YXC(x-1,y+1) or YXC(x-1,y-1) or YXC(x+1,y-1) or YXC(x+1,y+1) is an edge point, and the second feature extraction unit determines that the weak edge point is an edge pixel point. Otherwise, the second feature extraction unit determines that the weak edge pixel point is a non-edge pixel point. The second feature extraction unit connects adjacent edge pixels to form a continuous edge line as the pre-selected contour.

5. The pet behavior recognition and detection system according to claim 4, characterized in that: The contour extraction module is provided with a centroid analysis unit and a contour extraction unit; the centroid analysis unit analyzes the centroid of each pre-selected contour, and sets the centroid of the i-th pre-selected contour as Zi, and sets: , xni is the horizontal coordinate of the ni-th edge pixel point of the ith pre-selected contour, yni is the vertical coordinate of the ni-th edge pixel point of the ith pre-selected contour, and Ni is the number of edge pixels of the ith pre-selected contour; like , the centroid analysis unit determines that the i-th pre-selected contour is the pet candidate contour; if , the centroid analysis unit determines that the i-th pre-selected contour is the feeder candidate contour; if and , the centroid analysis unit determines that the i-th preselected contour is an invalid contour, D1 is the first preset area, and D2 is the second preset area; The contour extraction unit compares the area of ​​each pet's to-be-selected contour with each preset area to screen each pet's contour. If sk<s1 or sk>s2, the contour extraction unit determines that the kth pet's to-be-selected contour is an invalid contour. If s1≤Sk≤s2 and bk<b1, the contour extraction unit determines that the kth pet's to-be-selected contour is an invalid contour. If s1≤Sk≤s2 or bk>b2, the contour extraction unit determines that the kth pet's to-be-selected contour is an invalid contour. If s1≤Sk≤s2 or b1≤bk≤b2, the contour extraction unit determines that the kth pet's to-be-selected contour is a pre-selected pet contour. When the number of pre-selected pet contours mc=1, the contour extraction unit determines that the pre-selected pet contour is a pet contour. When mc>1, the contour extraction unit constructs a recommendation coefficient for each pre-selected pet contour. The contour extraction unit sets the recommendation coefficient of the ycth pre-selected pet contour to Wyc, and sets: Wyc=-{|SWyc-(s1+s2) / 2| / [(s1+s2) / 2]+|bWyc-(b1+b2) / 2| / [b1+b2) / 2]}; Wherein, sk is the area of ​​the kth pet to be selected, s1 is the preset minimum pet area, s2 is the preset maximum pet area, bk is the aspect ratio of the kth pet to be selected, b1 is the preset minimum pet aspect ratio, b2 is the preset maximum pet aspect ratio, SWyc is the area of ​​the ycth pre-selected pet outline, and bWyc is the aspect ratio of the ycth pre-selected pet outline; The profile extraction unit sorts the recommendation coefficients of the pre-selected pet profiles and uses the pre-selected pet profile with the largest recommendation coefficient as the pet profile; The contour extraction unit extracts the contour of the feeder.

6. The pet behavior recognition and detection system according to claim 5, characterized in that: The position analysis module calculates the distance J between the pet's contour centroid and the feeder's contour centroid, and sets: , CWx is the horizontal coordinate of the centroid of the pet's outline, CWy is the vertical coordinate of the centroid of the pet's outline, WFx is the horizontal coordinate of the feeder's outline, and WFy is the vertical coordinate of the feeder's outline; When J≤j0, the position analysis module determines that the pet is at the feeding distance, and when J>j0, the position analysis module determines that the pet is not at the feeding distance, and j0 is a preset distance.

7. The pet behavior recognition and detection system according to claim 6, characterized in that: The behavior recognition module recognizes the behavior state of the pet according to the analysis result of the pet's position state during the monitoring period, wherein: If t1 / t0≤α, the behavior recognition module determines that the pet is not in the "want to eat" state during the current monitoring period; if t1 / t0>α, the behavior recognition module determines that the pet is in the "want to eat" state during the current monitoring period, and α is a preset state coefficient.

8. The pet behavior recognition and detection system according to claim 7, characterized in that: The volume analysis module is provided with a volume analysis unit, a frequency analysis unit, an audio analysis unit and a noise analysis unit; the volume analysis unit is used to compare the average call volume p0 of the pet during the monitoring period with the preset volume p1 to construct a volume coefficient; The frequency analysis unit compares the pet's average calling frequency u0 during the monitoring period with the preset frequency u1 to construct a frequency coefficient.

9. The pet behavior recognition and detection system according to claim 8, characterized in that: The audio analysis unit constructs the audio coefficient YP of the pet's call according to the construction results of the volume coefficient and the frequency coefficient in the monitoring period, and sets YP=E1×volume coefficient+E2×frequency coefficient, where E1 is the volume weight and E2 is the frequency weight; The audio analysis unit compares the audio coefficient YP with the preset audio coefficient YP0 to evaluate the abnormality of the pet's call audio during the monitoring period, and adjusts the pet status recognition process according to the evaluation result, wherein: If YP≤YP0, the audio analysis unit determines that the audio of the pet's call in the current monitoring period is normal and no adjustment is made. If YP>YP0, the audio analysis unit determines that the audio of the pet's call in the current monitoring period is abnormal and adjusts the pet's status recognition process, and sets the adjusted preset status coefficient to α'.

10. The pet behavior recognition and detection system according to claim 9, characterized in that: The noise analysis unit compares the average ambient noise zb obtained during the monitoring period with the preset ambient noise zb0 to optimize the adjustment process of the pet status recognition process. If zb≤zb0, the noise analysis unit determines that the ambient noise in the current monitoring period is normal and does not perform optimization. If zb>zb0, the noise analysis unit determines that the ambient noise in the current monitoring period is abnormal, and optimizes the adjustment process of the pet status recognition process, and sets the optimized preset volume to p1'.

Citation Information

Patent Citations

  • Rapid building extraction method from high-resolution remote sensing image

    CN105719306A

  • Pig individual recognition and water drinking behavior analysis method based on machine vision

    CN106778784A

  • A poultry abnormity identification method and device

    CN109711346A

  • Pig weight measurement and water drinking behavior recognition system and method based on binocular vision

    CN111161214A

  • Intelligent prawn fry counting method and device

    CN111862246A