Automatic pet photography device and pet behavior recognition method

Abstract

The present invention relates to an automatic pet shooting device and a pet behavior recognition method, and belongs to the field of behavior recognition technology. The device includes a motion module, a video data acquisition module, a target tracking module, a behavior recognition module and an upload judgment module. The device monitors and shoots the behavior image or video data of the pet in real time through the video data acquisition module, and tracks the position of the pet in real time through the target tracking module, and guides the movement of the device to achieve autonomous mobile tracking. At the same time, the behavior image of the pet is recognized by the corresponding behavior recognition algorithm, which realizes the remote monitoring of the pet and improves the functionality and practicality of the device. At the same time, the pet behavior recognition result is judged by the upload judgment module using the photo shooting logic and the pet abnormality judgment logic, and the captured video data is uploaded to the remote monitoring terminal according to the pet status judgment result, so that the user can check the pet status conveniently.

Description

Technical Field

[0001] The present invention relates to the field of behavior recognition technology, and in particular to an automatic pet photographing device and a pet behavior recognition method. Background Art

[0002] With the continuous improvement of material life, more and more families have begun to keep small animals such as cats and dogs as pets to enrich family life. However, due to the busy work or going out of the house, some pet owners cannot always pay attention to the status of their pets, which leads to concerns about the welfare of their pets. Therefore, the market urgently needs a device that can automatically shoot pets and identify the status or behavior of pets to achieve remote monitoring of pets and video information storage.

[0003] Existing automatic pet photography devices exist more as a means of monitoring and usually do not have autonomous mobile tracking functions. Although they can obtain image or video data, they do not provide more semantic information, such as the pet's status, behavior, etc. These products have weak value in terms of functionality and practicality. Summary of the invention

[0004] In order to solve the above-mentioned problems in the prior art, the present invention provides an automatic pet photographing device and a pet behavior recognition method.

[0005] In a first aspect, the present invention provides an automatic pet photography device, comprising:

[0006] A motion module, used to control the motion direction and speed of the automatic pet photography device according to the target tracking signal;

[0007] A video data acquisition module, used to capture pet behavior images or video data, and adjust the video data acquisition direction according to the acquisition feedback signal;

[0008] A target tracking module is used to perform pet tracking detection on behavior images or video data through a detection model, and to generate a target tracking signal and collect feedback signals according to the pet tracking detection results;

[0009] The behavior recognition module is used to recognize pet behaviors in behavior images or video data through two parallel channels and output pet behavior recognition results.

[0010] Furthermore, it also includes an upload judgment module, which is used to use the photo shooting logic and the pet abnormality judgment logic to judge the pet status of the pet behavior recognition result, and upload the captured video data to the remote monitoring terminal for user viewing according to the pet status judgment result;

[0011] The photo shooting logic is used to determine whether the pet has multiple behavioral changes within a unit time. If multiple behavioral changes occur, the pet is considered to be in an excited and active state, and the video data shot during the corresponding period of the excited and active state is uploaded to the remote monitoring terminal;

[0012] The pet abnormality judgment logic is used to judge whether the pet is in a lying state for a long time based on a preset time. If the pet is in a lying state for a long time, it is considered that the pet is at risk of physical discomfort. At the same time, the video data shot in the corresponding state is uploaded to the remote monitoring terminal for user viewing and judgment.

[0013] Further, the target tracking module includes a dynamic appearance unit, an occlusion measurement unit and a blur detection unit;

[0014] The dynamic appearance unit is used to detect multiple pet instances in the initialization image taken by the user through a pre-trained detection model, and generate corresponding index value sets X in sequence according to the target detection order:

[0015] ,in, The image used when generating the index; represents the index, which is a binary vector;

[0016] The occlusion measurement unit is used to determine the index value of the pet instance with partial occlusion according to the target detection result;

[0017] The blur detection unit is used to calculate the possibility of occlusion in the target detection frame of the pet instance in the captured image, and to generate a target tracking signal and a collection feedback signal based on the calculation result of the occlusion possibility.

[0018] Furthermore, the step of determining the index value of the pet instance to which the pet instance is partially blocked according to the target detection result includes:

[0019] The overall measurement result Z in the image is modeled by the index value detection results of different pet instances, assuming that the measurement result Z is uniformly distributed:

[0020] ,

[0021] The measurement result Z represents the judgment result of the index value to which the pet instance belongs; , Indicated by image Determines that the instance belongs to the index collection probability; represents the mean of Poisson distribution; is the volume value of the space formed by the measurement result Z; represents the probability distribution function; Represents an associative mapping, specifically an index value Whether it is detected, if detected, it is 1, if not detected, it is 0; Represents a single object likelihood function, expressing the index value The possibility of identifying the measurement result Z as the index set X;

[0022] According to the index judgment model obtained by the modeling results, the target detection results are input into the index judgment model for index value judgment to obtain the index value of the pet instance with partial occlusion.

[0023] Furthermore, the calculating of the possibility of occlusion of the target detection frame of the pet instance in the captured image, and generating a target tracking signal and a collection feedback signal according to the calculation result of the occlusion possibility, includes:

[0024] Given a pet instance with a measurement result of Z1, calculate the overlap between the pet instance detection box with the measurement result of Z1 and the pet instance detection box with the state of Z2. The overlap is expressed as the area intersection score IoA(Z1, Z2):

[0025] ,

[0026] IoA represents the area intersection score, Area(Z1) represents the area occupied by the pet instance detection box with the measurement result Z1, and IntersectionArea(Z1,Z2) represents the intersection area of ​​the pet instance detection box with the measurement result Z1 and the pet instance detection box with the state Z2;

[0027] Introducing the occlusion probability factor To balance the accuracy of the detection frame of multiple pet instances in judging the occlusion of adjacent objects, the occlusion probability size factor It is expressed as:

[0028] ,

[0029] Where n represents the number of pet instances;

[0030] The area intersection score IoA(Z1,Z2) and the occlusion probability factor Multiply to get the final score ;

[0031] When the score of the detection box When the preset occlusion threshold is exceeded, it is determined that the detection frame of the current pet instance is occluded, and the occluded detection frame is retained, and the moving direction of the unobstructed detection frame in the shooting picture is used as the video data acquisition direction of the video data acquisition module to generate a corresponding acquisition feedback signal;

[0032] When the area of ​​the detection frames of all pet instances in the captured image is lower than the preset monitoring threshold, the moving direction of the detection frame with the largest area in the captured image at the previous N moments is used as the guidance direction of the motion module, and the target tracking signal is continuously generated until the area of ​​the detection frame of any pet instance in the captured image is no lower than the preset monitoring threshold.

[0033] Furthermore, the behavior recognition module includes a parallel first channel and a second channel, as well as a pooling layer and a fully connected layer;

[0034] The first channel uses the behavior image collected by the video data acquisition module as input, extracts spatial features of the input behavior image through ResNet, and obtains the spatial features of the pet behavior;

[0035] The second channel uses the heat map output by the pre-trained skeleton detector for pet skeleton detection on the behavior image as input, and also uses ResNet to capture the temporal features in the heat map, specifically:

[0036] ,

[0037] Where D represents the distance from point (i, j) to the dividing line The distance and For two joint points, represents variance; H represents time characteristics;

[0038] The spatial features of the pet behavior output by the first channel are laterally connected and fused with the temporal features output by the second channel to obtain the fused features, which are then input into the pooling layer and the fully connected layer to predict the pet behavior and output the pet behavior recognition results.

[0039] Furthermore, the motion module is specifically a crawler-type motion device with a bidirectional movement function, and the video data acquisition module is specifically a camera that can rotate 360 ​​degrees to shoot; the camera is installed on the top of the crawler-type motion device.

[0040] Furthermore, the motion module controls the motion direction and motion speed of the automatic pet photography device according to the target tracking signal, including:

[0041] When the target tracking signal is a motion steering, the motion module controls the crawler speed of one side of the crawler motion device to be lower than the crawler speed of the other side, so that the crawler motion device turns to the side with lower crawler speed;

[0042] When the target tracking signal is to turn on the spot, the motion module controls the crawler track on one side of the crawler motion device to move forward and the crawler track on the other side to move backward, so that the crawler motion device turns on the spot.

[0043] In a second aspect, the present invention provides a pet behavior recognition method implemented by the automatic pet photography device of the first aspect, comprising:

[0044] Collecting behavioral images or video data of pets;

[0045] Perform spatial feature extraction and temporal feature extraction on behavioral images or video data through two parallel channels to obtain spatial features and temporal features of pet behavior;

[0046] The spatial features and temporal features of pet behavior are fused, and the fused features are input into the pooling layer and the fully connected layer to predict pet behavior and obtain the pet behavior recognition results.

[0047] Furthermore, it also includes using the photo shooting logic and the pet abnormality judgment logic to judge the pet status based on the pet behavior recognition result, and uploading the shot video data to the remote monitoring terminal for the user to view according to the pet status judgment result;

[0048] The photo shooting logic is used to determine whether the pet has multiple behavioral changes within a unit time. If multiple behavioral changes occur, the pet is considered to be in an excited and active state, and the video data shot during the corresponding period of the excited and active state is uploaded to the remote monitoring terminal;

[0049] The pet abnormality judgment logic is used to judge whether the pet is in a lying state for a long time based on a preset time. If the pet is in a lying state for a long time, it is considered that the pet is at risk of physical discomfort. At the same time, the video data shot in the corresponding state is uploaded to the remote monitoring terminal for user viewing and judgment.

[0050] The beneficial effects of the present invention are reflected in that the video data acquisition module of the automatic pet shooting device monitors and shoots the behavior image or video data of the pet in real time, and the target tracking module tracks the position of the pet in real time, and guides the movement of the device to achieve autonomous mobile tracking. At the same time, the behavior image of the pet is recognized through the corresponding behavior recognition algorithm, thereby realizing remote monitoring of the pet. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 This is a schematic diagram of the working principle of the automatic pet photography device provided by the present invention;

[0052] Figure 2 This is a working flow chart of the automatic pet photography device provided by the present invention;

[0053] Figure 3 This is a unit architecture diagram of the target tracking module provided by the present invention;

[0054] Figure 4 This is a working principle diagram of the behavior recognition module provided by the present invention;

[0055] Figure 5 This is a flow chart of the pet behavior recognition method provided by the present invention. DETAILED DESCRIPTION

[0056] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0057] Embodiment 1:

[0058] Reference Figure 1-Figure 2 , Figure 1 is a schematic diagram of the working principle of the automatic pet photographing device according to an embodiment of the present invention, Figure 2 1 is a flowchart of the automatic pet photography device according to an embodiment of the present invention. Figure 1 As shown, the automatic pet photography device of the embodiment of the present invention mainly includes a motion module, a video data acquisition module, a target tracking module and a behavior recognition module.

[0059] A motion module, used to control the motion direction and speed of the automatic pet photography device according to the target tracking signal;

[0060] A video data acquisition module, used to capture pet behavior images or video data, and adjust the video data acquisition direction according to the acquisition feedback signal;

[0061] A target tracking module is used to perform pet tracking detection on behavior images or video data through a detection model, and to generate a target tracking signal and collect feedback signals according to the pet tracking detection results;

[0062] The behavior recognition module is used to recognize pet behaviors in behavior images or video data through two parallel channels and output pet behavior recognition results.

[0063] Specifically, refer to Figure 2As shown, in an embodiment of the present invention, the behavior image or video data of the pet is first captured by the video data acquisition module and transmitted to the target tracking module and the behavior recognition module respectively. The target tracking module performs pet tracking detection on the behavior image or video data through the detection model, and generates a target tracking signal and an acquisition feedback signal according to the pet tracking detection result. Among them, the target tracking signal is fed back to the motion module, so that the motion module controls the movement direction and movement speed of the automatic pet shooting device according to the target tracking signal. The acquisition feedback signal is fed back to the video data acquisition module, so that the video data acquisition module adjusts the video data acquisition direction according to the acquisition feedback signal. At the same time, the behavior recognition module identifies the pet behavior in the behavior image or video data through two parallel channels, and outputs the pet behavior recognition result, so as to realize remote monitoring of the pet behavior.

[0064] Further, refer to Figure 1 As shown, the automatic pet photography device of the embodiment of the present invention further includes an upload judgment module, which uses the photo shooting logic and the pet abnormality judgment logic to judge the pet status based on the pet behavior recognition result, and uploads the captured video data to the remote monitoring terminal for the user to view according to the pet status judgment result.

[0065] The photo shooting logic is used to determine whether the pet has multiple behavioral changes within a unit of time. If multiple behavioral changes occur, the pet is considered to be in an excited and active state, and the video data shot during the corresponding period of the excited and active state is uploaded to the remote monitoring terminal.

[0066] The pet abnormality judgment logic is used to determine whether the pet is in a lying state for a long time based on a preset time. If the pet is in a lying state for a long time, it is considered that the pet is at risk of physical discomfort. At the same time, the video data shot in the corresponding state is uploaded to the remote monitoring terminal for users to view and judge.

[0067] The remote monitoring terminal can be the user's mobile phone. By installing the corresponding APP that connects to the automatic shooting device on the mobile phone, the pet status can be remotely checked.

[0068] The embodiment of the present invention realizes the judgment of the pet status through the upload judgment module, and uploads the captured video data to the remote monitoring terminal according to the judgment result of the pet status for user viewing and further judgment of the status.

[0069] Further, refer to Figure 3 As shown, in the embodiment of the present invention, the target tracking module includes a dynamic appearance unit, an occlusion measurement unit and a blur detection unit.

[0070] Among them, the dynamic appearance unit is responsible for generating unique index values ​​for multiple targets (mainly for the situation where multiple pets appear in the picture at the same time). When the automatic pet shooting device is turned on for the first time, the user needs to hold the device and use the video data acquisition module to shoot the shooting target (i.e. the pet) to obtain the initialization image information. The dynamic appearance unit performs target detection on multiple pet instances in the initialization image shot by the user through the pre-trained detection model, and generates the corresponding index value set X in sequence according to the target detection order:

[0071] (1)

[0072] in, The image used when generating the index; Represents the index, which is a binary vector.

[0073] The detection model is based on the target detection algorithm and is pre-trained using common pet image data. In addition, since the appearance of the same pet instance changes during movement, in order to improve the robustness of subsequent tracking, there will be multiple indexes (i.e., set X) for a pet instance.

[0074] The occlusion measurement unit is used to determine the index value of the pet instance with partial occlusion based on the target detection result. The occlusion measurement unit is mainly used to improve the detection robustness of pets under partial occlusion (the measurement here refers to the determination of the index of the detected object after the detection result is given by the pre-trained target detection model).

[0075] The blur detection unit is used to calculate the possibility of occlusion in the target detection frame of the pet instance in the captured image, and to generate a target tracking signal and a collection feedback signal based on the calculation result of the occlusion possibility.

[0076] Among them, the working principle of the blur detection unit in the embodiment of the present invention is to assume that the video data acquisition module is always above the ground and the object moves on the same ground plane. Then, the lower the bottom angle of the bounding box, the closer the corresponding object is to the camera, which means that the probability of occluding adjacent objects is higher.

[0077] Further, in the embodiment of the present invention, the occlusion measurement unit determines the index value of the pet instance with partial occlusion according to the target detection result, which specifically includes the following process:

[0078] The overall measurement result Z in the image is modeled by the index value detection results of different pet instances, assuming that the measurement result Z is uniformly distributed:

[0079] (2)

[0080] The measurement result Z represents the judgment result of the index value to which the pet instance belongs; , Indicated by image Determines that the instance belongs to the index collection probability; represents the mean of Poisson distribution; is the volume value of the space formed by the measurement result Z; represents the probability distribution function; Represents an associative mapping, specifically an index value Whether it is detected, if detected, it is 1, if not detected, it is 0; Represents a single object likelihood function, expressing the index value The probability of identifying the measurement result Z as the index set X.

[0081] Finally, based on the index judgment model obtained from the modeling results, the target detection results are input into the index judgment model for index value judgment, and the index value of the partially occluded pet instance is obtained to realize the identity recognition of the partially occluded pet.

[0082] Further, in the embodiment of the present invention, the blur detection unit calculates the possibility of occlusion of the target detection frame of the pet instance in the captured image, and generates a target tracking signal and a collection feedback signal according to the calculation result of the occlusion possibility, specifically including:

[0083] Given a pet instance with a measurement result of Z1, calculate the overlap between the pet instance detection box with the measurement result of Z1 and the pet instance detection box with the state of Z2. The overlap is expressed as the area intersection score IoA(Z1, Z2):

[0084] (3)

[0085] IoA represents the area intersection score, Area(Z1) represents the area occupied by the pet instance detection box with the measurement result Z1, and IntersectionArea(Z1,Z2) represents the intersection area of ​​the pet instance detection box with the measurement result Z1 and the pet instance detection box with the state Z2.

[0086] Since small objects are usually difficult to detect, the embodiment of the present invention introduces an occlusion probability size factor To balance the accuracy of the detection frame of multiple pet instances in judging the occlusion of adjacent objects, the occlusion probability size factor It is expressed as:

[0087] (4)

[0088] Where n represents the number of pet instances.

[0089] The area intersection score IoA(Z1,Z2) and the occlusion probability factor Multiply to get the final score , which gives the probability of occlusion.

[0090] When the score of the detection box When the preset occlusion threshold is exceeded, the detection frame of the current pet instance is judged to be occluded, and the occluded detection frame is retained (the detection frame here is only for the detection frame of the instance determined to be occluded, and the unobstructed detection frame is retained normally), and the moving direction of the unobstructed detection frame in the shooting picture is used as the video data acquisition direction of the video data acquisition module, and a corresponding acquisition feedback signal is generated, and the acquisition feedback signal is fed back to the video data acquisition module.

[0091] In addition, when the areas of all pet instance detection frames in the captured image are lower than the preset monitoring threshold, the moving direction of the detection frame with the largest area in the captured image at the previous N moments is used as the guidance direction of the motion module, and the target tracking signal is continuously generated until the area of ​​the detection frame of any pet instance in the captured image is no lower than the preset monitoring threshold, that is, the motion module controls the device to turn, accelerate, decelerate and other operations so that the size of any detection frame meets the preset ideal size.

[0092] Further, refer to Figure 4 As shown, the behavior recognition module in the embodiment of the present invention is implemented based on the behavior recognition algorithm, and the behavior recognition algorithm includes three processes: feature extraction, feature fusion, and behavior prediction and recognition. The behavior recognition module constructed by the above behavior recognition algorithm includes a parallel first channel and a second channel, as well as a pooling layer and a fully connected layer. Among them,

[0093] The first channel uses the behavior images collected by the video data acquisition module as input, extracts spatial features of the input behavior images through ResNet, and obtains the spatial features of the pet's behavior.

[0094] Among them, ResNet (Residual Network) is a deep neural network structure that is widely used in computer vision tasks such as image classification, target detection, and semantic segmentation.

[0095] The second channel uses the heat map output by the pre-trained skeleton detector for pet skeleton detection on the behavior image as input, and also uses ResNet to capture the temporal features in the heat map, specifically:

[0096] (5)

[0097] Where D represents the distance from point (i, j) to the dividing line The distance and For two joint points, represents variance; H represents time characteristics.

[0098] In some embodiments, the skeleton detector is a model detector obtained by pre-training based on the animal skeleton key point detection network model, which can extract and identify the skeleton key point features of the animal to obtain the animal joint information. Finally, the time feature is extracted based on the animal joint information, and after the time feature is fused with the spatial feature, the specific behavior of the animal can be predicted and judged based on the fused feature.

[0099] Specifically, the spatial features of the pet behavior output by the first channel are laterally connected and fused with the temporal features output by the second channel to obtain fused features, and the fused features are input into the pooling layer and the fully connected layer to predict the pet behavior. The prediction result is the pet behavior recognition result, and finally the pet behavior recognition result is output. In addition, in some other embodiments of the present invention, other behavior recognition algorithms can also be selected according to actual conditions, such as ‌TWO-STREAM CNN, ‌TSN, ‌C3D, ‌RNN, ‌TRN, TSM, SlowFast and other behavior recognition algorithms. ‌

[0100] Furthermore, in the embodiment of the present invention, the motion module is the overall movement dependency of the automatic pet filming device, and is responsible for controlling the movement direction and speed of the crawler, which is specifically a crawler-type motion device with a two-way movement function. The video data acquisition module is specifically a camera that can rotate 360 ​​degrees for shooting, and the camera is installed on the top of the crawler-type motion device, and the rotation of the camera can be controlled according to the feedback result of target tracking.

[0101] Furthermore, in an embodiment of the present invention, the motion module controls the motion direction and motion speed of the automatic pet photography device according to the target tracking signal, specifically including:

[0102] When the target tracking signal is a motion steering, the motion module controls the crawler speed of one side of the crawler motion device to be lower than the crawler speed of the other side, so that the crawler motion device turns to the side with lower crawler speed;

[0103] When the target tracking signal is to turn on the spot, the motion module controls the crawler track on one side of the crawler motion device to move forward and the crawler track on the other side to move backward, so that the crawler motion device turns on the spot.

[0104] In addition, in the embodiment of the present invention, the motion speed control is achieved through the motion module by adjusting the motor power in the motion module, thereby changing the motor speed to achieve the adjustment of the motion speed, such as acceleration and deceleration.

[0105] The embodiment of the present invention uses the video data acquisition module of the automatic pet shooting device to monitor and shoot the behavior image or video data of the pet in real time, and uses the target tracking module to track the position of the pet in real time, and guides the movement of the device to achieve autonomous mobile tracking. At the same time, the behavior image of the pet is recognized through the corresponding behavior recognition algorithm, so as to realize the remote monitoring of the pet and improve the functionality and practicality of the device.

[0106] Embodiment 2:

[0107] Reference Figure 5 As shown, the embodiment of the present invention further provides a pet behavior recognition method based on embodiment 1, and the method includes:

[0108] Collecting pet behavior images or video data. This process is mainly implemented by the video data acquisition module in the automatic pet photography device of Example 1.

[0109] Perform spatial feature extraction and temporal feature extraction on behavioral images or video data through two parallel channels to obtain spatial features and temporal features of pet behavior;

[0110] The spatial features and temporal features of pet behavior are fused, and the fused features are input into the pooling layer and the fully connected layer to predict pet behavior and obtain the pet behavior recognition results.

[0111] Furthermore, the pet behavior recognition method of the embodiment of the present invention also includes: using photo shooting logic and pet abnormality judgment logic to judge the pet status of the pet behavior recognition results, and uploading the captured video data to the remote monitoring terminal for user viewing based on the pet status judgment results.

[0112] Among them, the photo shooting logic is used to determine whether the pet has multiple behavioral changes within a unit of time. If multiple behavioral changes occur, the pet is considered to be in an excited and active state, and the video data shot during the corresponding period of the excited and active state is uploaded to the remote monitoring terminal.

[0113] The pet abnormality judgment logic is used to determine whether the pet is in a lying state for a long time based on a preset time. If the pet is in a lying state for a long time, it is considered that the pet is at risk of physical discomfort. At the same time, the video data shot in the corresponding state is uploaded to the remote monitoring terminal for users to view and judge.

[0114] In the embodiment of the present invention, spatial feature extraction and temporal feature extraction are performed on behavioral images or video data through two parallel channels, and the spatial features and temporal features of the pet behavior are obtained by referring to the working principle of the behavior recognition module in Example 1, and the embodiment of the present invention will not be repeated here.

[0115] The embodiment of the present invention can automatically shoot pets and identify the state or behavior of pets, thereby realizing remote monitoring of pets and video information storage. At the same time, the state of the pet can be further determined according to the animal behavior recognition result, and the corresponding shot video data can be uploaded to the user's remote monitoring terminal according to the actual state, so that the user can view and grasp the state of the pet in real time.

[0116] In the description of the embodiments of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "install", "connect", "connect", and "assemble" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a direct connection, or an indirect connection through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0117] In the description of the embodiments of the present invention, specific features, structures, materials or characteristics may be combined in a suitable manner in any one or more embodiments or examples.

[0118] In the description of the embodiments of the present invention, the term "and / or" herein is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " herein generally indicates that the associated objects before and after are in an "or" relationship.

[0119] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An automatic pet photography device, characterized in that: include: A motion module, used to control the motion direction and speed of the automatic pet photography device according to the target tracking signal; A video data acquisition module, used to capture pet behavior images or video data, and adjust the video data acquisition direction according to the acquisition feedback signal; A target tracking module is used to perform pet tracking detection on the behavior image or video data through a detection model, and generate a target tracking signal and a collection feedback signal according to the pet tracking detection result; the target tracking module includes a dynamic appearance unit, an occlusion measurement unit and a blur detection unit; The dynamic appearance unit is used to detect multiple pet instances in the initialization image taken by the user through a pre-trained detection model, and generate corresponding index value sets in sequence according to the target detection order. X : The index value is associated with the identity of the pet instance in the initialization image; ， in, The image used when generating the index; represents the index, which is a binary vector; The occlusion measurement unit is used to determine the index value of the pet instance with partial occlusion according to the target detection result, including: The overall measurement result Z in the image is modeled by the index value detection results of different pet instances, assuming that the measurement result Z is uniformly distributed: ， The measurement result Z represents the judgment result of the index value to which the pet instance belongs; , Indicated by image Determines that the instance belongs to the index collection probability; represents the mean of Poisson distribution; is the volume value of the space formed by the measurement result Z; represents the probability distribution function; Represents an associative mapping, specifically an index value Whether it is detected, if detected, it is 1, if not detected, it is 0; Represents a single object likelihood function, expressing the index value The measurement result Z is identified as an index set X possibility; According to the index judgment model obtained by the modeling result, the target detection result is input into the index judgment model to judge the index value, and the index value of the pet instance with partial occlusion is obtained; The blur detection unit is used to calculate the possibility of occlusion of the target detection frame of the pet instance in the captured image, and generate a target tracking signal and an acquisition feedback signal according to the calculation result of the occlusion possibility, including: Given a pet instance with a measurement result of Z1, calculate the overlap between the pet instance detection box with the measurement result of Z1 and the pet instance detection box with the state of Z2. The overlap is expressed as the area intersection score IoA(Z1, Z2): ， IoA represents the area intersection score, Area(Z1) represents the area occupied by the pet instance detection box with the measurement result Z1, and IntersectionArea(Z1,Z2) represents the intersection area of ​​the pet instance detection box with the measurement result Z1 and the pet instance detection box with the state Z2; Introducing the occlusion probability factor To balance the accuracy of the detection frame of multiple pet instances in judging the occlusion of adjacent objects, the occlusion probability size factor It is expressed as: ， Where n represents the number of pet instances; The area intersection score IoA(Z1,Z2) and the occlusion probability factor Multiply to get the final score ; When the score of the detection box When the preset occlusion threshold is exceeded, it is determined that the detection frame of the current pet instance is occluded, and the occluded detection frame is retained, and the moving direction of the unobstructed detection frame in the shooting picture is used as the video data acquisition direction of the video data acquisition module to generate a corresponding acquisition feedback signal; When the area of ​​all pet instance detection frames in the captured image is lower than the preset monitoring threshold, the moving direction of the detection frame with the largest area in the captured image at the previous N moments is used as the guidance direction of the motion module, and the target tracking signal is continuously generated until the area of ​​the detection frame of any pet instance in the captured image is no lower than the preset monitoring threshold; The behavior recognition module is used to recognize pet behaviors in behavior images or video data through two parallel channels and output pet behavior recognition results.

2. The automatic pet photographing device according to claim 1, characterized in that: It also includes an upload judgment module, which is used to use the photo shooting logic and the pet abnormality judgment logic to judge the pet status based on the pet behavior recognition result, and upload the shot video data to the remote monitoring terminal for user viewing according to the pet status judgment result; The photo shooting logic is used to determine whether the pet has multiple behavioral changes within a unit time. If multiple behavioral changes occur, the pet is considered to be in an excited and active state, and the video data shot during the corresponding period of the excited and active state is uploaded to the remote monitoring terminal; The pet abnormality judgment logic is used to judge whether the pet is in a lying state for a long time based on a preset time. If the pet is in a lying state for a long time, it is considered that the pet is at risk of physical discomfort. At the same time, the video data shot in the corresponding state is uploaded to the remote monitoring terminal for user viewing and judgment.

3. The automatic pet photographing device according to claim 1, characterized in that: The behavior recognition module includes a parallel first channel and a second channel, a pooling layer and a fully connected layer; The first channel uses the behavior image collected by the video data acquisition module as input, extracts spatial features of the input behavior image through ResNet, and obtains the spatial features of the pet behavior; The second channel uses the heat map output by the pre-trained skeleton detector for pet skeleton detection on the behavior image as input, and also uses ResNet to capture the temporal features in the heat map, specifically: ， Where D represents the point ( i,j ) to the dividing line The distance and For two joint points, represents variance; H represents time characteristics; The spatial features of the pet behavior output by the first channel are laterally connected and fused with the temporal features output by the second channel to obtain fused features, which are then input into the pooling layer and the fully connected layer to predict the pet behavior and output the pet behavior recognition results.

4. The automatic pet photographing device according to claim 1, characterized in that: The motion module is specifically a crawler-type motion device with a bidirectional movement function, and the video data acquisition module is specifically a camera that can rotate 360 ​​degrees to shoot; the camera is installed on the top of the crawler-type motion device.

5. The automatic pet photographing device according to claim 4, characterized in that: The motion module controls the motion direction and motion speed of the automatic pet photography device according to the target tracking signal, including: When the target tracking signal is a motion steering, the motion module controls the crawler speed of one side of the crawler motion device to be lower than the crawler speed of the other side, so that the crawler motion device turns to the side with lower crawler speed; When the target tracking signal is to turn on the spot, the motion module controls the crawler track on one side of the crawler motion device to move forward and the crawler track on the other side to move backward, so that the crawler motion device turns on the spot.

6. A method for identifying pet behavior using the automatic pet photography device according to any one of claims 1 to 5, comprising: Collecting behavioral images or video data of pets; Perform spatial feature extraction and temporal feature extraction on behavioral images or video data through two parallel channels to obtain spatial features and temporal features of pet behavior; The spatial features and temporal features of pet behavior are fused, and the fused features are input into the pooling layer and the fully connected layer to predict pet behavior and obtain the pet behavior recognition results.

7. The pet behavior recognition method according to claim 6, characterized in that: It also includes using the photo shooting logic and the pet abnormality judgment logic to judge the pet status based on the pet behavior recognition result, and uploading the shot video data to the remote monitoring terminal for the user to view according to the pet status judgment result; The photo shooting logic is used to determine whether the pet has multiple behavioral changes within a unit time. If multiple behavioral changes occur, the pet is considered to be in an excited and active state, and the video data shot during the corresponding period of the excited and active state is uploaded to the remote monitoring terminal; The pet abnormality judgment logic is used to judge whether the pet is in a lying state for a long time based on a preset time. If the pet is in a lying state for a long time, it is considered that the pet is at risk of physical discomfort. At the same time, the video data shot in the corresponding state is uploaded to the remote monitoring terminal for user viewing and judgment.

Images