Pet behavior detection method, device, electronic device and storage medium

Abstract

Embodiments of the present invention provide a pet behavior detection method, device, electronic device, and storage medium. The method comprises: acquiring a sequence of images to be detected in a vehicle, wherein the sequence of images to be detected includes a target pet; performing target detection on the sequence of images to be detected to obtain type information of the target pet and a target pet feature sequence; performing behavior detection on the target pet feature sequence in combination with the type information to obtain behavior information of the target pet, wherein the behavior information includes first behavior information based on the current state and / or second behavior information based on a preset future time; and based on the first behavior information and / or the second behavior information, determining whether the target pet currently engages in driving-threatening behavior and / or determining whether the target pet engages in driving-threatening behavior within a preset future time. This can improve the safety of driving with pets.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to a pet behavior detection method, device, electronic device and storage medium. Background Art

[0002] In recent years, with the development of society, more and more people and families have kept pets, and pets have become an inseparable part of people's lives. In addition, as people's economic conditions have improved, the number of vehicles has increased rapidly, and more and more car users are bringing their pets with them when they go out. However, most pets have not received cabin training, and the restraints in cars are mostly designed for humans and have limited effectiveness on pets. This leads to pets easily engaging in uncontrollable behaviors in the car, some of which endanger the driver's driving, such as interfering with the driver's gear shifting, biting objects in the car, and jumping on the driver. However, the driver needs to focus on driving and has no extra energy to pay attention to the pet's behavior. This leads to higher risks and lower safety when driving with pets. Summary of the Invention

[0003] An embodiment of the present invention provides a pet behavior detection method that can automatically detect and / or predict the behavior of pets in a car. It can be determined whether the pet has any behavior that endangers driving without the driver having to pay attention to the pet's behavior, thereby improving the safety of driving with pets.

[0004] In a first aspect, an embodiment of the present invention provides a pet behavior detection method, the method comprising:

[0005] Acquire a sequence of images to be detected in a vehicle, wherein the sequence of images to be detected includes a target pet;

[0006] Performing target detection on the image sequence to be detected to obtain type information of the target pet and a target pet feature sequence;

[0007] Performing behavior detection on the target pet feature sequence in combination with the type information to obtain behavior information of the target pet, the behavior information including first behavior information based on the current state and / or second behavior information based on a preset future time;

[0008] Based on the first behavior information and / or the second behavior information, it is determined whether the target pet currently has dangerous driving behavior, and / or whether the target pet has dangerous driving behavior within a preset future time.

[0009] Optionally, performing target detection on the image sequence to be detected to obtain the type information of the target pet and the target pet feature sequence includes:

[0010] Inputting the current frame image in the image sequence to be detected into a preset first extraction network for feature extraction to obtain a first image feature;

[0011] Inputting the first image feature into a preset detection network for classification to obtain type information of the target pet; and

[0012] Inputting the first image feature into a preset second extraction network for feature extraction to obtain a second image feature of the target pet;

[0013] A target pet feature sequence is obtained based on the second image features of each frame of the image sequence to be detected.

[0014] Optionally, the second extraction network is a pet skeleton feature extraction network, the second image feature is a pet skeleton feature, and the target pet feature sequence is a target pet skeleton feature sequence. Inputting the first image feature into a preset second extraction network for feature extraction to obtain the second image feature of the target pet includes:

[0015] Inputting the first image feature into a preset pet skeleton feature extraction network to extract skeleton features to obtain pet skeleton features of the target pet;

[0016] The step of obtaining a target pet feature sequence based on the second image feature of each frame of the image sequence to be detected includes:

[0017] Based on the target pet features of each frame image of the image sequence to be detected, a target pet skeleton feature sequence is obtained.

[0018] Optionally, performing behavior detection on the target pet feature sequence in combination with the type information to obtain the behavior information of the target pet includes:

[0019] Matching a corresponding pet behavior detection network according to the type information, wherein the pet behavior detection network is a pre-trained sequence detection network, and the output of the pet behavior detection network is the current pet behavior result;

[0020] Inputting the target pet's skeletal feature sequence into the pet behavior detection network to perform behavior detection to obtain first behavior information of the target pet;

[0021] Matching a corresponding pet behavior prediction network according to the type information, wherein the pet behavior prediction network is a pre-trained sequence prediction network, and the output of the pet behavior prediction network is a predicted pet behavior result in the future;

[0022] The target pet's skeletal feature sequence is input into the pet behavior prediction network to perform behavior prediction, and obtain the second behavior information of the target pet within a preset future time.

[0023] Optionally, the pet behavior detection network is trained based on a first data set, and the pet behavior prediction network is trained based on a second data set, wherein the first data set includes a first sample sequence and first label data, and the first sample sequence includes a complete sequence of pet behavior, so that the pet behavior detection network learns the mapping from the first sample sequence to the first label data; the second data set includes a second sample sequence and a third sample sequence, the duration of the third sample sequence is the same as the preset future time, and the third sample sequence serves as the second label data of the second sample sequence, and the second sample sequence and the third sample sequence are combined into a complete sequence of pet behavior.

[0024] Optionally, the method further includes:

[0025] According to the pet type, determining the dangerous driving behavior corresponding to the pet type;

[0026] Based on the dangerous driving behavior, construct a dangerous driving behavior library corresponding to the pet type;

[0027] The determining, based on the first behavior information and / or the second behavior information, whether the target pet currently engages in a dangerous driving behavior, and / or determining whether the target pet engages in a dangerous driving behavior within a preset future time, includes:

[0028] comparing the first behavior information and / or the second behavior information in the dangerous driving behavior database;

[0029] It is determined based on the comparison result whether the target pet currently has dangerous driving behavior, and / or it is determined based on the comparison result whether the target pet has dangerous driving behavior within a preset future time.

[0030] Optionally, the method further includes:

[0031] If the target pet is currently engaging in behaviors that endanger driving, a first prompt message is issued;

[0032] If the target pet has any behavior that endangers driving within a preset future time, a second prompt message is issued.

[0033] In a second aspect, an embodiment of the present invention further provides a pet behavior detection device, comprising:

[0034] An acquisition module, configured to acquire a sequence of images to be detected in the vehicle, wherein the sequence of images to be detected includes a target pet;

[0035] A first detection module is configured to perform target detection on the image sequence to be detected to obtain type information of the target pet and a target pet feature sequence;

[0036] a second detection module, configured to perform behavior detection on the target pet feature sequence in combination with the type information to obtain behavior information of the target pet, the behavior information including first behavior information based on the current state and / or second behavior information based on a preset future time;

[0037] A judgment module is used to judge whether the target pet currently has dangerous driving behavior based on the first behavior information and / or the second behavior information, and / or judge whether the target pet has dangerous driving behavior within a preset future time.

[0038] In a third aspect, an embodiment of the present invention provides an electronic device comprising: a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the pet behavior detection method provided in the embodiment of the present invention are implemented.

[0039] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the pet behavior detection method provided in the embodiment of the invention are implemented.

[0040] In an embodiment of the present invention, a sequence of images to be detected in a vehicle is obtained, wherein the sequence of images to be detected includes a target pet; target detection is performed on the sequence of images to be detected to obtain type information of the target pet and a target pet feature sequence; behavior detection is performed on the target pet feature sequence in combination with the type information to obtain behavior information of the target pet, wherein the behavior information includes first behavior information based on the current state and / or second behavior information based on a preset future time; based on the first behavior information and / or the second behavior information, it is determined whether the target pet currently has behaviors that endanger driving, and / or whether the target pet has behaviors that endanger driving within a preset future time. By performing behavior detection and / or prediction on the pet in the vehicle through the vehicle's image sequence, the behavior information of the pet in the vehicle can be automatically detected, and whether the pet currently and / or within a preset future time can be determined based on the pet's behavior information. The driver does not need to pay attention to the pet's behavior to know whether the pet has behaviors that endanger driving, thereby improving the safety of driving with a pet. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0042] Figure 1 is a flow chart of a pet behavior detection method provided by an embodiment of the present invention;

[0043] Figure 2 1 is a structural diagram of a pet behavior detection device provided by an embodiment of the present invention;

[0044] Figure 3 is a structural diagram of a first detection module provided by an embodiment of the present invention;

[0045] Figure 4 is a structural diagram of a second detection module provided by an embodiment of the present invention;

[0046] Figure 5 Schematic diagram of the structure of a behavior detection submodule provided by an embodiment of the present invention;

[0047] Figure 6 is a schematic structural diagram of another behavior prediction submodule provided by an embodiment of the present invention;

[0048] Figure 7 is a structural diagram of another pet behavior detection device provided by an embodiment of the present invention;

[0049] Figure 8 is a structural diagram of another pet behavior detection device provided by an embodiment of the present invention;

[0050] Figure 9 It is a structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0051] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0052] See Figure 1 , Figure 1 FIG. 1 is a flow chart of a pet behavior detection method provided by an embodiment of the present invention. Figure 1As shown, the method for pet behavior detection and / or prediction comprises the following steps:

[0053] 101. Obtain a sequence of images to be detected inside the vehicle.

[0054] In an embodiment of the present invention, the image sequence to be detected includes the target pet. The image sequence to be detected may be an in-vehicle video stream captured by an on-board camera while the vehicle is in motion. The camera may be a camera dedicated to pet monitoring or an ordinary on-board camera capable of capturing pets.

[0055] The pet behavior detection method of the embodiment of the present invention can be integrated on a terminal device, such as a camera dedicated to monitoring pets, or it can be integrated on a cloud server. For example, when the on-board camera obtains the in-car video stream, it will be uploaded to the server in real time for pet behavior detection.

[0056] In a possible embodiment, when the camera captures the video stream inside the car, third-party passenger detection can be performed first. If it is detected that there are pets and third-party passengers in the video stream inside the car, and the pet is close to the third-party passenger, it can be said that the third-party passenger can take care of the pet. Therefore, when there is a third-party passenger who is close to the pet in the video stream inside the car, pet behavior detection is not required. When only the pet and the driver are present in the video stream inside the car, or the distance between the pet and the third-party passenger in the video stream inside the car is farther than the distance between the pet and the driver, the corresponding video stream is obtained as the image sequence to be detected.

[0057] 102. Perform target detection on the image sequence to be detected to obtain type information of the target pet and a target pet feature sequence.

[0058] In an embodiment of the present invention, target detection refers to detecting pets as targets to detect whether a pet of a corresponding type exists. The target pet type may be a dog, a cat, a pig, or the like. In one possible embodiment, because different pet breeds have different behavioral characteristics, the target pet type may also be a specific breed. For example, for a dog, the type information may be a Golden Retriever, a Husky, or other breed type information.

[0059] The target pet feature sequence corresponds to each frame of the image sequence to be detected, and can be understood as a feature sequence obtained by extracting features of the target pet in each frame of the image sequence to be detected.

[0060] Optionally, the current frame image in the above-mentioned image sequence to be detected can be input into a preset first extraction network for feature extraction to obtain a first image feature; the above-mentioned first image feature is input into a preset detection network for classification to obtain type information of the target pet; and the above-mentioned first image feature is input into a preset second extraction network for feature extraction to obtain a second image feature of the target pet; based on the second image features of each frame image in the image sequence to be detected, a target pet feature sequence is obtained.

[0061] In an embodiment of the present invention, the above-mentioned first extraction network, detection network and second extraction network can be feature extraction networks based on convolutional neural networks. The above-mentioned first extraction network is used to extract the current frame image into an abstract first image feature. The above-mentioned first image feature can also be called a common image feature. The common image feature implicitly contains the type information and target pet behavior information of the target pet.

[0062] The preset detection network includes a classifier corresponding to the pet type, which extracts the target pet type information implicit in the first image feature. The detection network is pre-trained and is trained using sample pet images labeled with different type labels.

[0063] The above-mentioned preset second extraction network is pre-trained and can be trained using sample pet images annotated with pet posture labels.

[0064] Optionally, the second extraction network may be a pet skeletal feature extraction network. Accordingly, the second image feature is a pet skeletal feature, and the target pet feature sequence is a target pet skeletal feature sequence. The pet skeletal feature extraction network may be trained using sample pet images labeled with pet skeletal labels. The pet skeletal features may be obtained by predicting pet joints. Specifically, in the second extraction network, pet joint point estimation (Pose Estimation) may be performed using the first image feature of the target pet.

[0065] Specifically, the first image feature can be input into a preset pet skeletal feature extraction network for skeletal feature extraction to obtain the pet skeletal features of the target pet. Based on the target pet features in each frame of the image sequence to be detected, a target pet skeletal feature sequence is obtained. The pet skeletal features at each moment (frame) correspond to the coordinate position information of K joints of the pet. The target pet skeletal feature sequence is composed of the pet skeletal features corresponding to several frames of images.

[0066] 103. Perform behavior detection on the target pet feature sequence in combination with the type information to obtain the target pet's behavior information.

[0067] In an embodiment of the present invention, the above-mentioned behavior information includes first behavior information based on the current state and / or second behavior information based on a preset future time. The first behavior information based on the current state can be understood as the pet's current behavior. For example, if the pet is biting a seat belt, the first behavior information detected is biting the seat belt. The second behavior information based on the preset future time can be understood as the pet's future behavior. For example, if the pet is currently raising its forelimbs, and it is predicted that the pet is about to touch the barrier with its forelimbs within 5 seconds, the second behavior information predicted is touching the barrier within 5 seconds.

[0068] Optionally, the second extraction network is a pet skeletal feature extraction network, the corresponding second image features are pet skeletal features, and the target pet feature sequence is a target pet skeletal feature sequence. The target pet skeletal feature sequence can be used in conjunction with the aforementioned type information to perform current behavior detection to obtain first behavior information corresponding to the target pet at the current time; and / or the target pet skeletal feature sequence can be used in conjunction with the aforementioned type information to perform behavior prediction to obtain second behavior information corresponding to the target pet within a predetermined future time.

[0069] In an embodiment of the present invention, to improve the accuracy of pet behavior detection and prediction results, multiple pet behavior detection networks and multiple pet behavior prediction networks can be trained based on pet type. After determining the target pet's type information, the corresponding behavior detection network and / or behavior prediction network are matched based on this type information. It should be noted that the pet behavior detection network is a pre-trained sequence detection network, and the output of the pet behavior detection network is the current pet behavior result; the pet behavior prediction network is a pre-trained sequence prediction network, and the output of the pet behavior prediction network is the predicted pet behavior result in the future.

[0070] Furthermore, the target pet's skeletal feature sequence is input into a pet behavior detection network for behavior detection to obtain first behavior information of the target pet. The target pet's skeletal feature sequence is input into a pet behavior prediction network for behavior prediction to obtain second behavior information of the target pet within a preset future time.

[0071] It should be noted that, in the embodiment of the present invention, pet behavior detection and pet behavior prediction can be performed simultaneously based on the target pet's skeletal feature sequence, or pet behavior detection or pet behavior prediction can be performed separately.

[0072] Specifically, the above-mentioned pet behavior detection network can be a sequence detection network based on a recurrent neural network. During the detection process of the target pet skeletal feature sequence through the time dependency in the recurrent neural network, the decoding of the current target pet skeletal feature depends on the detection results of the previous target pet skeletal feature. Since behavior is a series of action state combinations, the sequence detection network based on the recurrent neural network can more accurately detect pet behavior.

[0073] Similarly, the above-mentioned pet behavior prediction network can also be a sequence detection network based on a recurrent neural network. The difference is that the training methods of the pet behavior detection network and the pet behavior prediction network are different. Therefore, the parameters of the pet behavior detection network and the pet behavior prediction network are also different.

[0074] Furthermore, the pet behavior detection network can be trained based on a first dataset, and the pet behavior prediction network can be trained based on a second dataset. The first dataset includes a first sample sequence and first label data, wherein the first sample sequence encompasses a complete sequence of pet behaviors, enabling the pet behavior detection network to learn the mapping from the first sample sequence to the first label data. The second dataset includes a second sample sequence and a third sample sequence, wherein the third sample sequence has a duration equal to a predetermined future time. The third sample sequence serves as the second label data for the second sample sequence, and the second and third sample sequences are combined to form a complete sequence of pet behaviors. Furthermore, the second sample sequence is connected end-to-end with the third parent sequence at the end to form a complete sequence of pet behaviors. In one possible embodiment, the second and third sample sequences can be obtained by splitting the first sample sequence. For example, the latter portion of the first sample sequence is split into the third sample sequence, and the former portion of the first sample sequence is split into the second sample sequence. The second and third sample sequences are then concatenated to form the first sample sequence. It should be noted that the third sample sequence is split based on the predetermined future time.

[0075] For example, during the training process, for a sample sequence (1, 2, 3, 4, 5), the first sample sequence of the pet behavior detection network is (1, 2, 3, 4, 5), and the corresponding first label data is Y, while the second sample sequence of the pet behavior prediction network is (1, 2), (2, 3), (3, 4), and the second label data (third sample sequence) corresponding to the second sample sequence (1, 2) is 3 or (3, 4), specifically 3 or (3, 4) can be determined according to the preset future time, the second label data corresponding to the second sample sequence (2, 3) is 4 or (4, 5), and the second label data corresponding to the second sample sequence (3, 4) is 5 or (5, 6).

[0076] Different pet behavior detection networks and non-scolding pet behavior prediction networks can be trained according to different pet types to detect pet behaviors in the car in a targeted manner and promptly discover dangerous behaviors that endanger vehicle driving.

[0077] 104. Based on the first behavior information and / or the second behavior information, determine whether the target pet currently has dangerous driving behavior, and / or determine whether the target pet has dangerous driving behavior within a preset future time.

[0078] In an embodiment of the present invention, the aforementioned dangerous driving behaviors can be defined based on pet type. For example, if a pet dog's behavior in a car is significantly higher than that of a pet turtle, then the pet dog will correspond to more dangerous driving behaviors, while the pet turtle will correspond to fewer dangerous driving behaviors. Different pet types correspond to different dangerous driving behaviors.

[0079] For example, for pet dogs, the above-mentioned behaviors that endanger driving include biting objects in the car, touching the gear lever, touching the steering wheel, touching the car door, roaring, and touching the driver. For pet parrots, the above-mentioned behaviors that endanger driving include screaming, flying into the driver's seat, etc.

[0080] Optionally, in an embodiment of the present invention, based on the pet type, dangerous driving behaviors corresponding to the above-mentioned pet type can be determined; based on the above-mentioned dangerous driving behaviors, a dangerous driving behavior library corresponding to the above-mentioned pet type can be constructed; the above-mentioned first behavior information and / or the above-mentioned second behavior information can be compared in the above-mentioned dangerous driving behavior library; based on the comparison results, it can be determined whether the above-mentioned target pet currently has dangerous driving behaviors, and / or based on the comparison results, it can be determined whether the above-mentioned target pet has dangerous driving behaviors within a preset future time.

[0081] Optionally, if the target pet is currently engaging in behaviors that endanger driving, a first prompt message is issued; if the target pet is engaging in behaviors that endanger driving within a preset future time, a second prompt message is issued. The above-mentioned first prompt message is a real-time prompt, and the above-mentioned second prompt message is a predictive prompt. For example, when the target pet is a pet dog, if the first behavior information of the pet dog is detected to be biting the seat belt, it means that the pet dog is biting the seat belt. At this time, the first prompt message can be: "Your pet dog is biting the seat belt, please drive safely." If the second behavior information of the pet dog is predicted to be biting the car door handle, it means that the pet dog is about to bite the car door handle within a preset future time. At this time, the second prompt message can be: "Your pet dog is about to bite the car door handle, please drive safely."

[0082] In an embodiment of the present invention, a sequence of images to be detected in a vehicle is obtained, wherein the sequence of images to be detected includes a target pet; target detection is performed on the sequence of images to be detected to obtain type information of the target pet and a target pet feature sequence; behavior detection is performed on the target pet feature sequence in combination with the type information to obtain behavior information of the target pet, wherein the behavior information includes first behavior information based on the current state and / or second behavior information based on a preset future time; based on the first behavior information and / or the second behavior information, it is determined whether the target pet currently has behaviors that endanger driving, and / or whether the target pet has behaviors that endanger driving within a preset future time. By performing behavior detection and / or prediction on the pet in the vehicle through the vehicle's image sequence, the behavior information of the pet in the vehicle can be automatically detected, and whether the pet currently and / or within a preset future time can be determined based on the pet's behavior information. The driver does not need to pay attention to the pet's behavior to know whether the pet has behaviors that endanger driving, thereby improving the safety of driving with a pet.

[0083] It should be noted that the pet behavior detection method provided in the embodiment of the present invention can be applied to devices such as mobile phones, monitors, computers, servers, etc. that can perform pet behavior detection.

[0084] See Figure 2 , Figure 2 FIG. 1 is a structural diagram of a pet behavior detection device provided by an embodiment of the present invention. Figure 2 As shown, the device includes:

[0085] An acquisition module 201 is configured to acquire a sequence of images to be detected in a vehicle, wherein the sequence of images to be detected includes a target pet;

[0086] The first detection module 202 is configured to perform target detection on the image sequence to be detected, and obtain type information of the target pet and a target pet feature sequence;

[0087] The second detection module 203 is configured to perform behavior detection on the target pet feature sequence in combination with the type information to obtain behavior information of the target pet, wherein the behavior information includes first behavior information based on the current state and / or second behavior information based on a preset future time;

[0088] The judgment module 204 is used to judge whether the target pet currently has dangerous driving behavior based on the first behavior information and / or the second behavior information, and / or judge whether the target pet has dangerous driving behavior within a preset future time.

[0089] Optional, such as Figure 3 As shown, the first detection module 202 includes:

[0090] A first extraction submodule 2021 is configured to input a current frame image in the image sequence to be detected into a preset first extraction network for feature extraction, thereby extracting a first image feature;

[0091] a classification submodule 2022, configured to input the first image feature into a preset detection network for classification to obtain type information of the target pet; and

[0092] A second extraction submodule 2023 is configured to input the first image feature into a preset second extraction network for feature extraction to obtain a second image feature of the target pet;

[0093] The determination submodule 2024 is configured to obtain a target pet feature sequence based on the second image feature of each frame of the image sequence to be detected.

[0094] Optionally, the second extraction network is a pet bone feature extraction network, the second image feature is a pet bone feature, and the target pet feature sequence is a target pet bone feature sequence. The second extraction submodule 2023 is further configured to input the first image feature into a preset pet bone feature extraction network to extract bone features, thereby obtaining the pet bone feature of the target pet.

[0095] The determination submodule 2024 is further configured to obtain a target pet skeleton feature sequence based on the target pet features of each frame of the image sequence to be detected.

[0096] Optional, such as Figure 4 As shown, the second detection module 203 includes:

[0097] The behavior detection submodule 2031 is used to perform current behavior detection on the target pet's skeletal feature sequence in combination with the type information to obtain first behavior information currently corresponding to the target pet; and / or

[0098] The behavior prediction submodule 2032 is used to perform behavior prediction on the target pet's skeletal feature sequence in combination with the type information to obtain second behavior information corresponding to the target pet within a preset future time.

[0099] Optional, such as Figure 5 As shown, the behavior detection submodule 2031 includes:

[0100] A first matching unit 20311 is configured to match a corresponding pet behavior detection network according to the type information, wherein the pet behavior detection network is a pre-trained sequence detection network, and the output of the pet behavior detection network is the current pet behavior result;

[0101] The behavior detection unit 20312 is used to input the target pet's skeletal feature sequence into the pet behavior detection network to perform behavior detection and obtain the first behavior information of the target pet.

[0102] Optional, such as Figure 6 As shown, the behavior prediction submodule 2032 includes:

[0103] The second matching unit 20321 is configured to match a corresponding pet behavior prediction network according to the type information, wherein the pet behavior prediction network is a pre-trained sequence prediction network, and the output of the pet behavior prediction network is a predicted pet behavior result in the future;

[0104] The behavior prediction unit 20322 is used to input the target pet's skeletal feature sequence into the pet behavior prediction network to perform behavior prediction and obtain the second behavior information of the target pet within a preset future time.

[0105] Optional, such as Figure 7 As shown, the device further includes:

[0106] A determination module 205 is configured to determine, based on the pet type, a dangerous driving behavior corresponding to the pet type;

[0107] A construction module 206 is configured to construct a dangerous driving behavior library corresponding to the pet type based on the dangerous driving behavior;

[0108] The judgment module 204 includes:

[0109] A comparison submodule 2041 is configured to compare the first behavior information and / or the second behavior information with the dangerous driving behavior database;

[0110] The judgment submodule 2042 is used to judge whether the target pet currently has dangerous driving behavior based on the comparison result, and / or judge whether the target pet has dangerous driving behavior within a preset future time based on the comparison result.

[0111] Optional, such as Figure 8 As shown, the device also includes:

[0112] A first prompting module 207 is configured to issue a first prompt message if the target pet currently has a behavior that endangers driving;

[0113] The second prompt module 208 is configured to issue a second prompt message if the target pet has any behavior that endangers driving within a preset future time.

[0114] It should be noted that the pet behavior detection device provided in the embodiment of the present invention can be applied to devices such as mobile phones, monitors, computers, servers, etc. that can perform pet behavior detection.

[0115] The pet behavior detection device provided in the embodiment of the present invention can implement each process implemented by the pet behavior detection method in the above method embodiment and can achieve the same beneficial effects. To avoid repetition, it will not be described here.

[0116] See also Figure 9 , Figure 9 is a schematic structural diagram of an electronic device provided by an embodiment of the present invention, such as Figure 9 As shown, it includes: a memory 902, a processor 901, and a computer program stored in the memory 902 and executable on the processor 901, wherein:

[0117] The processor 901 is configured to call the computer program stored in the memory 902 and execute the following steps:

[0118] Acquire a sequence of images to be detected in a vehicle, wherein the sequence of images to be detected includes a target pet;

[0119] Performing target detection on the image sequence to be detected to obtain type information of the target pet and a target pet feature sequence;

[0120] Performing behavior detection on the target pet feature sequence in combination with the type information to obtain behavior information of the target pet, the behavior information including first behavior information based on the current state and / or second behavior information based on a preset future time;

[0121] Based on the first behavior information and / or the second behavior information, it is determined whether the target pet currently has dangerous driving behavior, and / or whether the target pet has dangerous driving behavior within a preset future time.

[0122] Optionally, the processor 901 performs target detection on the image sequence to be detected to obtain the type information of the target pet and the target pet feature sequence, including:

[0123] Inputting the current frame image in the image sequence to be detected into a preset first extraction network for feature extraction to obtain a first image feature;

[0124] Inputting the first image feature into a preset detection network for classification to obtain type information of the target pet; and

[0125] Inputting the first image feature into a preset second extraction network for feature extraction to obtain a second image feature of the target pet;

[0126] A target pet feature sequence is obtained based on the second image features of each frame of the image sequence to be detected.

[0127] Optionally, the second extraction network is a pet skeleton feature extraction network, the second image feature is a pet skeleton feature, and the target pet feature sequence is a target pet skeleton feature sequence. The processor 901 inputs the first image feature into a preset second extraction network for feature extraction to obtain the second image feature of the target pet, including:

[0128] Inputting the first image feature into a preset pet skeleton feature extraction network to extract skeleton features to obtain pet skeleton features of the target pet;

[0129] The processor 901 performs the process of obtaining a target pet feature sequence based on the second image features of each frame of the image sequence to be detected, including:

[0130] Based on the target pet features of each frame image of the image sequence to be detected, a target pet skeleton feature sequence is obtained.

[0131] Optionally, the processor 901 performs behavior detection on the target pet feature sequence in combination with the type information to obtain behavior information of the target pet, including:

[0132] Performing current behavior detection on the target pet's skeletal feature sequence in combination with the type information to obtain first behavior information currently corresponding to the target pet; and / or

[0133] The target pet's skeletal feature sequence is subjected to behavior prediction in combination with the type information to obtain second behavior information corresponding to the target pet within a preset future time.

[0134] Optionally, the processor 901 performs current behavior detection on the target pet's skeletal feature sequence in combination with the type information to obtain first behavior information currently corresponding to the target pet, including:

[0135] Matching a corresponding pet behavior detection network according to the type information, wherein the pet behavior detection network is a pre-trained sequence detection network, and the output of the pet behavior detection network is the current pet behavior result;

[0136] The target pet's skeletal feature sequence is input into the pet behavior detection network to perform behavior detection, thereby obtaining first behavior information of the target pet.

[0137] Optionally, the processor 901 performs the behavior prediction on the target pet's skeletal feature sequence in combination with the type information to obtain second behavior information corresponding to the target pet within a preset future time, including:

[0138] Matching a corresponding pet behavior prediction network according to the type information, wherein the pet behavior prediction network is a pre-trained sequence prediction network, and the output of the pet behavior prediction network is a predicted pet behavior result in the future;

[0139] The target pet's skeletal feature sequence is input into the pet behavior prediction network to perform behavior prediction, and obtain the second behavior information of the target pet within a preset future time.

[0140] Optionally, the processor 901 further performs:

[0141] According to the pet type, determining the dangerous driving behavior corresponding to the pet type;

[0142] Based on the dangerous driving behavior, construct a dangerous driving behavior library corresponding to the pet type;

[0143] The processor 901 performs the following steps: determining whether the target pet currently engages in a driving-hazardous behavior based on the first behavior information and / or the second behavior information, and / or determining whether the target pet engages in a driving-hazardous behavior within a preset future time. The steps include:

[0144] comparing the first behavior information and / or the second behavior information in the dangerous driving behavior database;

[0145] It is determined based on the comparison result whether the target pet currently has dangerous driving behavior, and / or it is determined based on the comparison result whether the target pet has dangerous driving behavior within a preset future time.

[0146] Optionally, the processor 901 further performs the following operations:

[0147] If the target pet is currently engaging in behaviors that endanger driving, a first prompt message is issued;

[0148] If the target pet has any behavior that endangers driving within a preset future time, a second prompt message is issued.

[0149] It should be noted that the above-mentioned electronic devices may be mobile phones, monitors, computers, servers and other devices that can be used for pet behavior detection.

[0150] The electronic device provided in the embodiment of the present invention can implement each process implemented by the pet behavior detection method in the above method embodiment and can achieve the same beneficial effects. To avoid repetition, it will not be described here.

[0151] An embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various processes of the pet behavior detection method provided in the embodiment of the present invention are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0152] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing the relevant hardware through a computer program. The program can be stored in a computer-readable storage medium, and when executed, the program can include the processes in the above-described method embodiments. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0153] The above disclosure is merely a preferred embodiment of the present invention and certainly cannot be used to limit the scope of the present invention. Therefore, equivalent changes made according to the claims of the present invention are still within the scope of the present invention.

Claims

1. A pet behavior detection method, characterized in that: The following steps are involved: Acquire a sequence of images to be detected in a vehicle, wherein the sequence of images to be detected includes a target pet; Inputting the current frame image in the image sequence to be detected into a preset first extraction network for feature extraction to obtain a first image feature; inputting the first image feature into a preset detection network for classification to obtain type information of the target pet; and inputting the first image feature into a preset second extraction network for feature extraction to obtain a second image feature of the target pet; Based on the second image features of each frame image of the image sequence to be detected, a target pet feature sequence is obtained; the second extraction network is a pet bone feature extraction network, the second image feature is a pet bone feature, and the target pet feature sequence is a target pet bone feature sequence; the inputting the first image feature into the preset second extraction network for feature extraction to obtain the second image feature of the target pet includes: inputting the first image feature into the preset pet bone feature extraction network for bone feature extraction to obtain the pet bone feature of the target pet; the obtaining the target pet feature sequence based on the second image features of each frame image of the image sequence to be detected includes: obtaining the target pet bone feature sequence based on the target pet feature of each frame image of the image sequence to be detected; Performing behavior detection on the target pet feature sequence in combination with the type information to obtain behavior information of the target pet, the behavior information including first behavior information based on the current state and second behavior information based on a preset future time; specifically matching a corresponding pet behavior detection network according to the type information, the pet behavior detection network being a pre-trained sequence detection network, the output of which is the current pet behavior result, and one pet behavior detection network corresponding to one type information; inputting the target pet skeletal feature sequence into the pet behavior detection network for behavior detection to obtain first behavior information of the target pet; and matching a corresponding pet behavior prediction network according to the type information, the pet behavior prediction network being a pre-trained sequence prediction network, the output of which is the predicted pet behavior result in the future time, and one pet behavior prediction network corresponding to one type information; inputting the target pet skeletal feature sequence into the pet behavior prediction network for behavior prediction to obtain second behavior information of the target pet in a preset future time; Based on the first behavior information and the second behavior information, it is determined whether the target pet currently has dangerous driving behavior, and whether the target pet has dangerous driving behavior within a preset future time.

2. The method according to claim 1, wherein The pet behavior detection network is trained based on a first data set, and the pet behavior prediction network is trained based on a second data set, wherein the first data set includes a first sample sequence and first label data, and the first sample sequence includes a complete sequence of pet behavior, so that the pet behavior detection network learns the mapping from the first sample sequence to the first label data; the second data set includes a second sample sequence and a third sample sequence, the duration of the third sample sequence is the same as the preset future time, the third sample sequence serves as the second label data of the second sample sequence, and the second sample sequence and the third sample sequence are combined into a complete sequence of pet behavior.

3. The method according to claim 1, wherein The method further comprises: According to the pet type, determining the dangerous driving behavior corresponding to the pet type; Based on the dangerous driving behavior, construct a dangerous driving behavior library corresponding to the pet type; The determining, based on the first behavior information and the second behavior information, whether the target pet currently has dangerous driving behavior, and determining whether the target pet has dangerous driving behavior within a preset future time, includes: comparing the first behavior information and the second behavior information in the dangerous driving behavior database; It is determined based on the comparison result whether the target pet currently has dangerous driving behavior, and it is determined based on the comparison result whether the target pet has dangerous driving behavior within a preset future time.

4. The method according to claim 3, wherein The method further comprises: If the target pet is currently engaging in behaviors that endanger driving, a first prompt message is issued; If the target pet has any behavior that endangers driving within a preset future time, a second prompt message is issued.

5. A pet behavior detection device, characterized in that: The device comprises: An acquisition module, configured to acquire a sequence of images to be detected in the vehicle, wherein the sequence of images to be detected includes a target pet; The first detection module is used to input the current frame image in the image sequence to be detected into a preset first extraction network for feature extraction to obtain a first image feature; input the first image feature into a preset detection network for classification to obtain type information of the target pet; and input the first image feature into a preset second extraction network for feature extraction to obtain a second image feature of the target pet; based on the second image features of each frame image of the image sequence to be detected, a target pet feature sequence is obtained; the second extraction network is a pet skeleton feature extraction network, the second image feature is a pet skeleton feature, and the target pet feature sequence is a target pet skeleton feature sequence; the inputting the first image feature into the preset second extraction network for feature extraction to obtain the second image feature of the target pet includes: inputting the first image feature into a preset pet skeleton feature extraction network for skeleton feature extraction to obtain the pet skeleton feature of the target pet; the obtaining of the target pet feature sequence based on the second image features of each frame image of the image sequence to be detected includes: obtaining a target pet skeleton feature sequence based on the target pet features of each frame image of the image sequence to be detected; The second detection module is used to perform behavior detection on the target pet feature sequence in combination with the type information to obtain the behavior information of the target pet, wherein the behavior information includes first behavior information based on the current state and second behavior information based on a preset future time; specifically, the corresponding pet behavior detection network is matched according to the type information, the pet behavior detection network is a pre-trained sequence detection network, the output of which is the current pet behavior result, and one pet behavior detection network corresponds to one type information; the target pet skeletal feature sequence is input into the pet behavior detection network for behavior detection to obtain the first behavior information of the target pet; and the corresponding pet behavior prediction network is matched according to the type information, the pet behavior prediction network is a pre-trained sequence prediction network, the output of which is the predicted pet behavior result in the future time, and one pet behavior prediction network corresponds to one type information; the target pet skeletal feature sequence is input into the pet behavior prediction network for behavior prediction to obtain the second behavior information of the target pet in the preset future time; The judgment module is used to judge whether the target pet currently has dangerous driving behavior and whether the target pet has dangerous driving behavior within a preset future time based on the first behavior information and the second behavior information.

6. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the pet behavior detection method according to any one of claims 1 to 4 when executing the computer program.

7. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the pet behavior detection method according to any one of claims 1 to 4.

Images