Anti-theft method and related products based on image recognition

Abstract

The embodiment of the present application discloses an anti-theft method based on image recognition and related products. The method includes: obtaining a video to be identified shot at a target location; obtaining multiple frames of images to be identified from the video to be identified; performing target detection on each frame of the multiple frames of images to be identified, and obtaining at least one frame of the target image in which a candidate person and a candidate pet appear simultaneously in the multiple frames of images to be identified; performing emotion recognition on the candidate person based on at least one frame of the target image, and obtaining the emotion of the candidate person; performing action recognition on the candidate pet based on at least one frame of the target image, and obtaining the action of the candidate pet; determining whether the candidate person has the behavior of stealing the candidate pet based on the emotion of the candidate person and the action of the candidate pet; if there is the behavior of stealing the candidate pet, sending a prompt message to the target device, and the prompt message is used to prompt that the behavior of stealing the candidate pet has occurred at the target location.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to an anti-theft method based on image recognition and related products. Background Art

[0002] In recent years, more and more people have chosen to keep pets. According to the White Paper on the Pet Industry, the number of pet dogs and cats in urban areas across the country reached tens of millions in 2019. As the number of pets increases, pets are becoming more and more important in people's lives.

[0003] As the value of pets increases, pet thefts often occur. Currently, pet thefts are attracting more and more attention from society. However, after a pet is stolen, the pet owner reports the theft to the police, and relevant personnel conduct manual inspections based on the video, which requires a lot of manpower and physical resources. In addition, this method of preventing pet thefts is relatively simple, and the success rate of preventing thefts is relatively low. Summary of the invention

[0004] The embodiments of the present application provide an anti-theft method and related products based on image recognition, which can accurately determine whether there is pet theft through emotion recognition and action recognition, thereby improving the success rate of anti-theft.

[0005] In a first aspect, an embodiment of the present application provides a pet theft prevention method based on image recognition, comprising:

[0006] Obtain the video to be identified shot at the target location;

[0007] Acquire multiple frames of images to be identified from the video to be identified;

[0008] Performing target detection on each frame of the multiple frames of images to be identified, respectively, to obtain at least one frame of target image in which both the candidate person and the candidate pet appear in the multiple frames of images to be identified;

[0009] Performing emotion recognition on the candidate character according to the at least one frame of target image to obtain the emotion of the candidate character;

[0010] According to the at least one frame of target image, performing action recognition on the candidate pet to obtain the action of the candidate pet;

[0011] Determining whether the candidate character has an act of stealing the candidate pet according to the emotion of the candidate character and the action of the candidate pet;

[0012] If there is an act of stealing the candidate pet, a prompt message is sent to the target device, where the prompt message is used to prompt that the act of stealing the candidate pet has occurred at the target location.

[0013] In a second aspect, an embodiment of the present application provides a pet anti-theft device, comprising: an acquisition unit and a processing unit;

[0014] The acquisition unit is used to acquire the video to be identified shot at the target location;

[0015] The processing unit is used to obtain multiple frames of images to be identified from the video to be identified;

[0016] Performing target detection on each frame of the multiple frames of images to be identified, respectively, to obtain at least one frame of target image in which both the candidate person and the candidate pet appear in the multiple frames of images to be identified;

[0017] Performing emotion recognition on the candidate character according to the at least one frame of target image to obtain the emotion of the candidate character;

[0018] According to the at least one frame of target image, performing action recognition on the candidate pet to obtain the action of the candidate pet;

[0019] Determining whether the candidate character has an act of stealing the candidate pet according to the emotion of the candidate character and the action of the candidate pet;

[0020] If there is an act of stealing the candidate pet, a prompt message is sent to the target device, where the prompt message is used to prompt that the act of stealing the candidate pet has occurred at the target location.

[0021] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor, the processor is connected to a memory, the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the electronic device performs the method described in the first aspect.

[0022] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program enables a computer to execute the method described in the first aspect.

[0023] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer is operable to cause the computer to execute the method described in the first aspect.

[0024] Implementing the embodiments of the present application has the following beneficial effects:

[0025] It can be seen that in the embodiment of the present application, a video of the target location (i.e., a location where pet theft is likely to occur) is first acquired, and then an image to be identified is acquired from the video, and based on the image to be identified, an emotion analysis is performed on the candidate character, and an action analysis is performed on the candidate pet; when the emotion of the candidate character and the action of the candidate pet meet the conditions, it is considered that the candidate character has committed theft against the candidate pet, and a timely warning is issued for the theft, thereby preventing the occurrence of pet theft in a timely manner, and improving the success rate of pet theft prevention. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0027] Figure 1 A schematic diagram of a pet anti-theft system provided in an embodiment of the present application;

[0028] Figure 2 A flowchart of a pet theft prevention method based on image recognition provided in an embodiment of the present application;

[0029] Figure 3 A flowchart of another pet theft prevention method based on image recognition provided in an embodiment of the present application;

[0030] Figure 4 A block diagram of the functional units of a pet anti-theft device provided in an embodiment of the present application;

[0031] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0033] The terms "first", "second", "third" and "fourth" etc. in the specification and claims of the present application and the drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

[0034] Reference to "embodiments" herein means that a particular feature, result, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0035] It should be noted that the anti-theft method of the present application is mainly used to prevent pets from being stolen. Therefore, the candidate characters involved in the present application can be understood as people who have the behavior of stealing pets, and the candidate pets are the stolen pets, and no distinction will be made later.

[0036] See also Figure 1 , Figure 1 A pet anti-theft system is provided for the implementation of the present application. The pet anti-theft system comprises a pet anti-theft device 100, an image acquisition device 200 and a target device 300; wherein the pet anti-theft device 100, the image acquisition device 200 and the target device 300 maintain a communication connection;

[0037] Exemplarily, the image acquisition device 200 is set within a preset range of the target location to achieve image capture of the target location and the preset range of the target location; the target device 300 is a device of the management end, for example, for a pet theft prevention scenario in a residential area, the target device 300 is a device of the residential area management staff;

[0038] Exemplarily, the image acquisition device 200 shoots the target location to obtain a video to be identified, and sends the video to be identified to the anti-theft device 100; the anti-theft device 100 obtains multiple frames of images to be identified from the video to be identified; then, target detection is performed on each frame of the multiple frames of images to be identified to obtain at least one frame of target image in which the candidate character appears simultaneously in the multiple frames of images to be identified; based on the at least one frame of target image, emotion recognition is performed on the candidate character to obtain the emotion of the candidate character; based on the at least one frame of target image, action recognition is performed on the candidate pet to obtain the action of the candidate pet; finally, based on the emotion of the candidate character and the action of the candidate pet, it is determined whether the candidate character has the behavior of stealing the candidate pet; if there is the behavior of stealing the candidate pet, a prompt message is sent to the target device 200, and the user of the target device 200 is prompted by the prompt message that the behavior of stealing the candidate pet has occurred at the target location.

[0039] It can be seen that in the embodiment of the present application, the image acquisition device shoots the target location and generates a video to be identified; then the pet anti-theft device acquires the video to be identified of the target location (i.e., the location where pet theft is likely to occur), and then acquires the image to be identified from the video, and performs emotion analysis on the candidate person based on the image to be identified, and performs action analysis on the candidate pet; when the emotion of the candidate person and the action of the candidate pet meet the conditions, it is considered that the candidate person has a theft behavior towards the candidate pet, and a timely warning is issued for the theft behavior, so as to prevent the occurrence of pet theft in time and improve the success rate of pet theft prevention.

[0040] See also Figure 2 , Figure 2 A flowchart of a pet anti-theft method based on image recognition provided in an embodiment of the present application. The method is applied to the above-mentioned pet anti-theft device. The method includes the following steps:

[0041] 201: Obtain the video to be identified shot at the target location.

[0042] Exemplarily, the pet anti-theft device can receive the video to be identified shot at the target location from the image acquisition device (e.g., camera). Optionally, the image acquisition device can shoot the target location in real time and perform target recognition on the captured content. When the pet and / or person are recognized to appear, the captured video is cached and the pet and / or person is continuously followed. When the object and / or person is detected to disappear, the video to be identified is generated and sent to the pet anti-theft device.

[0043] 202: Acquire multiple frames of images to be recognized from the video to be recognized.

[0044] Exemplarily, multiple frames of images to be identified are obtained from the video to be identified by sampling.

[0045] Optionally, the multiple frames of images to be identified may be acquired from the video to be identified according to a preset sampling frequency.

[0046] Optionally, the sampling frequency corresponding to the current moment can also be obtained, and the multiple frames of images to be identified can be obtained from the video to be identified according to the sampling frequency. Exemplarily, the sampling frequency corresponding to the current moment is determined according to the correspondence between the time period and the sampling frequency, as well as the current moment. For example, different sampling frequencies are preset for different time periods. For example, for time periods where theft is prone to occur (for example, the evening time period), a smaller sampling frequency can be preset, so that a larger number of images to be identified can be sampled for accurate analysis. For time periods where theft is not prone to occur (for example, 8:00-9:00), a larger sampling frequency can be preset, so that a smaller number of images to be identified can be sampled, thereby improving recognition efficiency.

[0047] 203: Performing target detection on each frame of the multiple frames of images to be identified, respectively, to obtain at least one frame of target image in which both the candidate person and the candidate pet appear in the multiple frames of images to be identified.

[0048] Exemplarily, target detection is performed on each frame of the image to be identified to obtain the target in each frame of the image to be identified and the category corresponding to each target, wherein the target in each frame of the image to be identified may be a pedestrian, a pet, a vehicle, etc. For example, target detection can be performed on each frame of the image to be identified by a Sparse R-CNN model to obtain the target in the image to be identified and the category corresponding to each target, which will not be described further.

[0049] Exemplarily, candidate persons and candidate pets are determined according to the category of the target in each frame of the image to be identified.

[0050] Specifically, according to the category of the target in each frame of the image to be identified, the first target in each frame of the image to be identified is determined, wherein the category of the first target is the same as the category of the candidate character, that is, the target of the category of human in each frame of the image to be identified is determined; then, the first targets in the multiple frames of the image to be identified are merged to obtain at least one first target, and the number of times each first target in the at least one first target appears in the multiple frames of the image to be identified is obtained. Finally, according to the number of times each first target appears in the multiple frames of the projection image, the first target a is determined, wherein the first target a is the first target with the largest number of appearances in the at least one target.

[0051] Exemplarily, feature extraction is performed on the first target in each frame of the image to be identified to obtain the contour features of the first target in each frame of the image to be identified, wherein the contour features include the height, width, color, and other features of the first target; for example, an image of the first target can be captured from each frame of the image to be identified, and then feature extraction is performed on the image through a convolutional network to obtain the contour features of the first target. Furthermore, according to the contour features of the first target in each frame of the image to be identified, multiple frames of images to be identified are clustered to obtain at least one first clustering cluster, each first clustering cluster corresponds to a contour feature, that is, corresponds to a first target, so that at least one first target is obtained based on the at least one first clustering cluster; then, the number of images to be identified contained in each first clustering cluster is taken as the number of times the first target corresponding to the first clustering cluster appears in the multiple frames of images to be identified.

[0052] Furthermore, according to the category of each target in each frame of the image to be identified, the second target in each frame of the image to be identified is determined, wherein the category of the second target is the same as the category of the candidate pet, that is, the target of the pet category in each frame of the image to be identified is determined; the second targets in the multiple frames of the image to be identified are merged to obtain at least one second target, and the number of times each second target in the at least one second target appears in the multiple frames of the image to be identified is obtained; according to the number of times each second target appears in the multiple frames of the projection image, the second target b is determined, wherein the second target b is the second target that appears the most times among the at least two targets.

[0053] Specifically, feature extraction is performed on the second target in each frame of the image to be identified to obtain the contour feature of the second target in each frame of the image to be identified, wherein the contour feature includes the height, width, color, hair sparseness, and other features of the second target; for example, an image of the second target can be captured from each frame of the image to be identified, and then feature extraction is performed on the image through a convolutional network to obtain the contour feature of the second target. Furthermore, according to the contour feature of the second target in each frame of the image to be identified, multiple frames of images to be identified are clustered to obtain at least one second cluster, each second cluster corresponds to a contour feature, that is, corresponds to a second target, so that at least one second target is obtained based on the at least one second cluster; then, the number of images to be identified contained in each second cluster is taken as the number of times the second target corresponding to the second cluster appears in the multiple frames of images to be identified.

[0054] Finally, the first object a is taken as a candidate person, and the second object b is taken as a candidate pet.

[0055] Optionally, after the candidate person and the candidate pet are determined, the image to be identified that contains both the candidate task and the candidate pet in the multiple frames of images to be identified is used as at least one frame of target image.

[0056] It should be noted that when a candidate character steals a candidate pet, the route he or she usually walks is a relatively hidden route, so the direction in which the image acquisition device shoots is along the hidden route. Therefore, people who frequently appear in the video to be identified are candidate characters, and pets that frequently appear are candidate pets.

[0057] 204: Perform emotion recognition on the candidate character according to the at least one frame of the target image to obtain the emotion of the candidate character.

[0058] Exemplarily, a facial image of a candidate character in each target image frame is obtained; feature extraction is performed on the facial image of the candidate character in each target image frame to obtain a first feature vector corresponding to each target image frame; then, the first feature vector corresponding to each target image frame is input into a multilayer perceptron to obtain a first emotion of the candidate character in each target image frame, wherein the first emotion may be nervous, flustered, happy, depressed, and sad; at least one first emotion of the candidate character in at least one target image frame is merged to obtain at least one second emotion, that is, at least one first emotion is merged and deduplicated to obtain at least one second emotion; then, the number of target images corresponding to each of the at least one second emotion is obtained, and the second emotion corresponding to the largest number of target images is used as the emotion of the candidate character.

[0059] For example, there are 10 frames of target images, the first emotion in 6 frames of target images is tension, the first emotion in 2 frames of target images is depression, and the first emotion in 2 frames of target images is happiness. Then, at least one second emotion obtained after merging is tension, depression, and happiness. The number of target frame images corresponding to the second emotion tension is 6 frames, the number of target frame images corresponding to depression is 2 frames, and the number of target frame images corresponding to happiness is 2 frames. Therefore, the second emotion tension is taken as the emotion of the candidate character.

[0060] It should be noted that if a certain frame of target image does not contain the facial image of the candidate character, the frame of target image can be ignored, that is, the first expression of the candidate character in the frame of target image is not determined.

[0061] 205: Perform action recognition on the candidate pet according to the at least one frame of target image to obtain the action of the candidate pet.

[0062] Exemplarily, a first image formed by the region of the candidate pet in each frame of the target image is obtained, and the first image of the candidate pet in each frame of the target image is subjected to action recognition to obtain the sub-action of the candidate character in each frame of the target image. Specifically, feature extraction is performed on the first image of the candidate pet in each frame of the target image to obtain a second feature vector of the candidate pet in each frame of the target image. For example, the first image can be subjected to feature extraction through a convolutional layer and a fully connected layer to obtain a second feature vector; then, the second feature vector of the candidate pet in each frame of the target image is matched with a plurality of first action parameters to obtain a matching degree between each frame of the target image and each first action parameter, wherein each first action parameter is obtained by pre-training and is used to characterize a sub-action, and the sub-action may include jumping, running, dragging, pulling, etc.; then, according to the matching degree between each frame of the target image and each first action parameter, the target first action parameter is determined, wherein the target first action parameter is the first action parameter corresponding to the maximum matching degree, and the sub-action represented by the target first action parameter is used as the sub-action of the candidate pet in each frame of the target image.

[0063] Further, the action of the candidate pet is determined according to the sub-actions of the candidate pet in each frame of the target image. That is, the sub-actions of the candidate pet in each frame of the target image are merged to obtain the action of the candidate pet. For example, if the sub-actions of the candidate pet in the target image are jumping, jumping up, and tearing, then after merging the sub-actions, the action of the candidate pet can be escape.

[0064] 206: Determine whether the candidate character has any behavior of stealing the candidate pet based on the emotion of the candidate character and the action of the candidate pet.

[0065] Exemplarily, if the candidate character's emotion is the first type of emotion and / or the candidate pet's action is the first type of action, it is determined that the candidate character has stolen the candidate pet; otherwise, it is determined that the candidate character has not stolen the candidate pet.

[0066] Among them, the first category of emotions includes but is not limited to: tension and panic, and the first category of actions includes but is not limited to: running away, tearing and struggling.

[0067] 207: If there is an act of stealing the candidate pet, a prompt message is sent to the target device, where the prompt message is used to prompt the target location that an act of stealing the candidate pet has occurred.

[0068] Exemplarily, when it is determined that theft occurs, a prompt message is sent to the target device, which may be the device of an administrator, so that the administrator is informed through the prompt message that pet theft occurs at the target location, so as to prevent the pet theft from occurring in time.

[0069] It can be seen that in the embodiment of the present application, a video of the target location (i.e., a location where pet theft is likely to occur) is first acquired, and then an image to be identified is acquired from the video, and based on the image to be identified, an emotion analysis is performed on the candidate character, and an action analysis is performed on the candidate pet; when the emotion of the candidate character and the action of the candidate pet meet the conditions, it is considered that the candidate character has committed theft against the candidate pet, and a timely warning is issued for the theft, thereby preventing the occurrence of pet theft in a timely manner, and improving the success rate of pet theft prevention.

[0070] See also Figure 3 , Figure 3 A flowchart of another pet anti-theft method based on image recognition provided in an embodiment of the present application. The method is applied to the above-mentioned anti-theft device. Figure 2 The same contents as those in the embodiment shown are not described again here. The method of this embodiment may include the following steps:

[0071] 301: Obtain a layout diagram of the area to be identified.

[0072] For example, the area to be identified may be a community, or a park, that is, some areas where pet theft is prone to occur. In this application, the area to be identified is mainly described as a community.

[0073] 302: Determine at least one candidate position in the to-be-identified area according to the layout diagram, wherein the target position is any one of the at least one candidate position.

[0074] Exemplarily, the candidate location may be some gate locations of a community, that is, locations where people can enter and exit the community. Furthermore, the candidate location may be a back door location or a side door location of the community.

[0075] 303: Obtain the video to be identified shot at the target location.

[0076] 304: Acquire multiple frames of images to be recognized from the video to be recognized.

[0077] 305: Performing target detection on each frame of the multiple frames of images to be identified, respectively, to obtain at least one frame of target image in which both the candidate person and the candidate pet appear in the multiple frames of images to be identified.

[0078] 306: Perform emotion recognition on the candidate character according to the at least one frame of the target image to obtain the emotion of the candidate character.

[0079] 307: Perform action recognition on the candidate pet according to the at least one frame of target image to obtain the action of the candidate pet.

[0080] 308: Determine whether the candidate character has any behavior of stealing the candidate pet based on the emotion of the candidate character and the action of the candidate pet.

[0081] 309: If there is an act of stealing the candidate pet, a prompt message is sent to the target device, where the prompt message is used to prompt that the act of stealing the candidate pet has occurred at the target location.

[0082] See also Figure 4 , Figure 4 The functional unit composition block diagram of a pet anti-theft device provided in an embodiment of the present application. The pet anti-theft device 400 includes: an acquisition unit 401 and a processing unit 402;

[0083] An acquisition unit 401 is used to acquire a video to be identified shot at a target location;

[0084] The processing unit 402 is used to obtain multiple frames of images to be identified from the video to be identified;

[0085] Performing target detection on each frame of the multiple frames of images to be identified, respectively, to obtain at least one frame of target image in which both the candidate person and the candidate pet appear in the multiple frames of images to be identified;

[0086] Performing emotion recognition on the candidate character according to the at least one frame of target image to obtain the emotion of the candidate character;

[0087] According to the at least one frame of target image, performing action recognition on the candidate pet to obtain the action of the candidate pet;

[0088] Determining whether the candidate character has an act of stealing the candidate pet according to the emotion of the candidate character and the action of the candidate pet;

[0089] If there is an act of stealing the candidate pet, a prompt message is sent to the target device, where the prompt message is used to prompt that the act of stealing the candidate pet has occurred at the target location.

[0090] In one embodiment of the present application, in terms of performing emotion recognition on the candidate character according to the at least one frame of target image to obtain the emotion of the candidate character, the processing unit 402 is specifically configured to:

[0091] Acquire a facial image of the candidate person in each frame of the target image;

[0092] Performing feature extraction on the facial image of the candidate person in each frame of the target image to obtain a first feature vector corresponding to each frame of the target image;

[0093] Inputting the first feature vector corresponding to each frame of the target image into a multi-layer perceptron to obtain the first emotion of the candidate character in each frame of the target image;

[0094] Merging at least one first emotion of the candidate character in the at least one frame of the target image to obtain at least one second emotion;

[0095] Acquire the number of target images corresponding to each second emotion in the at least one second emotion;

[0096] The second emotion corresponding to the largest number of target images is used as the emotion of the candidate character.

[0097] In one embodiment of the present application, in terms of performing action recognition on the candidate pet according to the at least one frame of target image to obtain the action of the candidate pet, the processing unit 402 is specifically configured to:

[0098] Acquire a first image formed by a region of the candidate pet in each frame of the target image;

[0099] Determine a sub-action of the candidate pet in each frame of the target image according to the first image of the candidate pet in each frame of the target image;

[0100] The action of the candidate pet is determined according to the sub-action of the candidate pet in each frame of the target image.

[0101] In one embodiment of the present application, in determining the sub-action of the candidate pet in each frame of the target image according to the first image of the candidate pet in each frame of the target image, the processing unit 402 is specifically configured to:

[0102] Performing feature extraction on the first image of the candidate pet in each frame of the target image to obtain a second feature vector of the candidate pet in each frame of the target image;

[0103] Matching the second feature vector of the candidate pet in each frame of the target image with a plurality of first action parameters respectively to obtain a matching degree between each frame of the target image and each first action parameter, wherein each first action parameter is used to characterize a sub-action;

[0104] Determine a target first action parameter according to the matching degree between each frame of the target image and each first action parameter, wherein the target first action parameter is the first action parameter corresponding to the maximum matching degree;

[0105] The sub-action represented by the first action parameter of the target is used as the sub-action of the candidate pet in each frame of the target image.

[0106] In one embodiment of the present application, in performing target detection on each frame of the multiple frames of images to be identified to obtain at least one frame of target image in which a candidate person and a candidate pet appear simultaneously in the multiple frames of images to be identified, the processing unit 402 is specifically configured to:

[0107] Performing target detection on each frame of the image to be identified to obtain the category of each target in each frame of the image to be identified;

[0108] Determine the candidate person and the candidate pet according to the category of each target in each frame of the image to be identified;

[0109] The image to be identified that contains both the candidate person and the candidate pet in the multiple frames of images to be identified is used as the at least one frame of target image.

[0110] In one embodiment of the present application, in determining the candidate person and the candidate pet according to the category of each target in each frame of the image to be identified, the processing unit 402 is specifically configured to:

[0111] Determining a first target in each frame of the image to be identified according to the category of each target in each frame of the image to be identified, wherein the category of the first target is the same as the category of the candidate character;

[0112] Determining a second target in each frame of the image to be identified according to the category of each target in each frame of the image to be identified, wherein the category of the second target is the same as the category of the candidate pet;

[0113] Merging the first targets in the multiple frames of images to be identified to obtain at least one first target, and obtaining the number of times each of the at least one first target appears in the multiple frames of images to be identified; determining a first target a according to the number of times each of the first targets appears in the multiple frames of projection images, wherein the first target a is the first target that appears the most times in the at least one target;

[0114] Merging the second targets in the multiple frames of images to be identified to obtain at least one second target, and obtaining the number of times each of the at least one second target appears in the multiple frames of images to be identified; determining a second target b according to the number of times each of the second targets appears in the multiple frames of projection images, wherein the second target b is the second target that appears the most times among the at least two targets;

[0115] The first object a is taken as the candidate person, and the second object b is taken as the candidate pet.

[0116] In one embodiment of the present application, before acquiring multiple frames of images to be identified captured at the target position, the acquisition unit 401 is further used to acquire a layout diagram of the area to be identified;

[0117] The processing unit 402 is further configured to determine at least one candidate position in the to-be-identified area according to the layout diagram, wherein the target position is any one of the at least one candidate position.

[0118] See also Figure 5 , Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 5 As shown, the electronic device 500 includes a transceiver 501, a processor 502 and a memory 503. They are connected via a bus 504. The memory 503 is used to store computer programs and data, and can transmit the data stored in the memory 503 to the processor 502.

[0119] The processor 502 is used to read the computer program in the memory 503 and perform the following operations:

[0120] Controlling the transceiver 501 to obtain the video to be identified shot at the target location;

[0121] Acquire multiple frames of images to be identified from the video to be identified;

[0122] Performing target detection on each frame of the multiple frames of images to be identified, respectively, to obtain at least one frame of target image in which both the candidate person and the candidate pet appear in the multiple frames of images to be identified;

[0123] Performing emotion recognition on the candidate character according to the at least one frame of target image to obtain the emotion of the candidate character;

[0124] According to the at least one frame of target image, performing action recognition on the candidate pet to obtain the action of the candidate pet;

[0125] Determining whether the candidate character has an act of stealing the candidate pet according to the emotion of the candidate character and the action of the candidate pet;

[0126] If there is an act of stealing the candidate pet, a prompt message is sent to the target device, where the prompt message is used to prompt that the act of stealing the candidate pet has occurred at the target location.

[0127] In one embodiment of the present application, in terms of performing emotion recognition on the candidate character according to the at least one frame of target image to obtain the emotion of the candidate character, the processor 502 is specifically configured to perform the following operations:

[0128] Acquire a facial image of the candidate person in each frame of the target image;

[0129] Performing feature extraction on the facial image of the candidate person in each frame of the target image to obtain a first feature vector corresponding to each frame of the target image;

[0130] Inputting the first feature vector corresponding to each frame of the target image into a multi-layer perceptron to obtain the first emotion of the candidate character in each frame of the target image;

[0131] Merging at least one first emotion of the candidate character in the at least one frame of the target image to obtain at least one second emotion;

[0132] Acquire the number of target images corresponding to each second emotion in the at least one second emotion;

[0133] The second emotion corresponding to the largest number of target images is used as the emotion of the candidate character.

[0134] In one embodiment of the present application, in terms of performing action recognition on the candidate pet according to the at least one frame of target image to obtain the action of the candidate pet, the processor 502 is specifically configured to perform the following operations:

[0135] Acquire a first image formed by a region of the candidate pet in each frame of the target image;

[0136] Determine a sub-action of the candidate pet in each frame of the target image according to the first image of the candidate pet in each frame of the target image;

[0137] The action of the candidate pet is determined according to the sub-action of the candidate pet in each frame of the target image.

[0138] In one embodiment of the present application, in determining the sub-action of the candidate pet in each frame of the target image according to the first image of the candidate pet in each frame of the target image, the processor 502 is specifically configured to perform the following operations:

[0139] Performing feature extraction on the first image of the candidate pet in each frame of the target image to obtain a second feature vector of the candidate pet in each frame of the target image;

[0140] Matching the second feature vector of the candidate pet in each frame of the target image with a plurality of first action parameters respectively to obtain a matching degree between each frame of the target image and each first action parameter, wherein each first action parameter is used to characterize a sub-action;

[0141] Determine a target first action parameter according to the matching degree between each frame of the target image and each first action parameter, wherein the target first action parameter is the first action parameter corresponding to the maximum matching degree;

[0142] The sub-action represented by the first action parameter of the target is used as the sub-action of the candidate pet in each frame of the target image.

[0143] In one embodiment of the present application, in terms of performing target detection on each frame of the multiple frames of images to be identified and obtaining at least one frame of the target image in which the candidate person and the candidate pet appear simultaneously in the multiple frames of images to be identified, the processor 502 is specifically configured to perform the following operations:

[0144] Performing target detection on each frame of the image to be identified to obtain the category of each target in each frame of the image to be identified;

[0145] Determine the candidate person and the candidate pet according to the category of each target in each frame of the image to be identified;

[0146] The image to be identified that contains both the candidate person and the candidate pet in the multiple frames of images to be identified is used as the at least one frame of target image.

[0147] In one embodiment of the present application, in determining the candidate person and the candidate pet according to the category of each target in each frame of the image to be identified, the processor 502 is specifically configured to perform the following operations:

[0148] Determining a first target in each frame of the image to be identified according to the category of each target in each frame of the image to be identified, wherein the category of the first target is the same as the category of the candidate character;

[0149] Determining a second target in each frame of the image to be identified according to the category of each target in each frame of the image to be identified, wherein the category of the second target is the same as the category of the candidate pet;

[0150] Merging the first targets in the multiple frames of images to be identified to obtain at least one first target, and obtaining the number of times each of the at least one first target appears in the multiple frames of images to be identified; determining a first target a according to the number of times each of the first targets appears in the multiple frames of projection images, wherein the first target a is the first target that appears the most times in the at least one target;

[0151] Merging the second targets in the multiple frames of images to be identified to obtain at least one second target, and obtaining the number of times each of the at least one second target appears in the multiple frames of images to be identified; determining a second target b according to the number of times each of the second targets appears in the multiple frames of projection images, wherein the second target b is the second target that appears the most times among the at least two targets;

[0152] The first object a is taken as the candidate person, and the second object b is taken as the candidate pet.

[0153] In one embodiment of the present application, before acquiring multiple frames of images to be identified captured at the target location, the processor 502 is further configured to perform the following operations:

[0154] Control the transceiver 501 to obtain a layout diagram of the area to be identified;

[0155] At least one candidate position in the to-be-identified area is determined according to the layout diagram, wherein the target position is any one of the at least one candidate position.

[0156] Specifically, the transceiver 501 may be Figure 4 The acquisition unit 401 of the pet anti-theft device 400 of the embodiment described above, the processor 502 may be Figure 4 The processing unit 402 of the pet anti-theft device 400 of the embodiment described above.

[0157] It should be understood that the electronic devices in this application may include smart phones (such as Android phones, iOS phones, Windows Phone phones, etc.), tablet computers, PDAs, laptops, mobile Internet devices MID (Mobile Internet Devices, referred to as: MID) or wearable devices, etc. The above electronic devices are only examples, not exhaustive, and include but are not limited to the above electronic devices. In practical applications, the above electronic devices may also include: smart vehicle terminals, computer equipment, etc.

[0158] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. The computer program is executed by a processor to implement part or all of the steps of any anti-theft method based on image recognition as recorded in the above method embodiments.

[0159] An embodiment of the present application also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute part or all of the steps of any one of the anti-theft methods based on image recognition as recorded in the above method embodiments.

[0160] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present application.

[0161] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0162] In the several embodiments provided in the present application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of the units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the device or unit can be electrical or other forms.

[0163] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0164] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit may be implemented in the form of hardware or in the form of a software program module.

[0165] If the integrated unit is implemented in the form of a software program module and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a memory, including a number of instructions to enable a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned memory includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or CD-ROM and other media that can store program codes.

[0166] A person skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable memory, and the memory can include: a flash drive, a read-only memory (English: Read-Only Memory, abbreviated as: ROM), a random access memory (English: Random Access Memory, abbreviated as: RAM), a magnetic disk or an optical disk, etc.

[0167] The embodiments of the present application are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core idea of ​​the present application. At the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A pet anti-theft method based on image recognition, characterized in that: include: Obtain the video to be identified shot at the target location; Acquire multiple frames of images to be identified from the video to be identified; Performing target detection on each frame of the multiple frames of images to be identified respectively to obtain at least one frame of target image in which both the candidate person and the candidate pet appear in the multiple frames of images to be identified; comprising: Performing target detection on each frame of the image to be identified to obtain the category of each target in each frame of the image to be identified; Determining a first target in each frame of the image to be identified according to the category of each target in each frame of the image to be identified, wherein the category of the first target is the same as the category of the candidate character; Determining a second target in each frame of the image to be identified according to the category of each target in each frame of the image to be identified, wherein the category of the second target is the same as the category of the candidate pet; Merging the first targets in the multiple frames of images to be identified to obtain at least one first target, and obtaining the number of times each of the at least one first target appears in the multiple frames of images to be identified; determining a first target a according to the number of times each of the first targets appears in the multiple frames of projection images, wherein the first target a is the first target that appears the most times in the at least one target; Merging the second targets in the multiple frames of images to be identified to obtain at least one second target, and obtaining the number of times each of the at least one second target appears in the multiple frames of images to be identified; determining a second target b according to the number of times each of the second targets appears in the multiple frames of projection images, wherein the second target b is the second target that appears the most times among the at least two targets; Taking the first object a as the candidate person, and taking the second object b as the candidate pet; Using the image to be identified that contains both the candidate person and the candidate pet in the multiple frames of images to be identified as the at least one frame of target image; Performing emotion recognition on the candidate character according to the at least one frame of target image to obtain the emotion of the candidate character; According to the at least one frame of target image, performing action recognition on the candidate pet to obtain the action of the candidate pet; Determining whether the candidate character has an act of stealing the candidate pet according to the emotion of the candidate character and the action of the candidate pet; If there is an act of stealing the candidate pet, a prompt message is sent to the target device, where the prompt message is used to prompt that the act of stealing the candidate pet has occurred at the target location.

2. The method according to claim 1, characterized in that The step of performing emotion recognition on the candidate character according to the at least one frame of the target image to obtain the emotion of the candidate character includes: Acquire a facial image of the candidate person in each frame of the target image; Performing feature extraction on the facial image of the candidate person in each frame of the target image to obtain a first feature vector corresponding to each frame of the target image; Inputting the first feature vector corresponding to each frame of the target image into a multi-layer perceptron to obtain the first emotion of the candidate character in each frame of the target image; Merging at least one first emotion of the candidate character in the at least one frame of the target image to obtain at least one second emotion; Acquire the number of target images corresponding to each second emotion in the at least one second emotion; The second emotion corresponding to the largest number of target images is used as the emotion of the candidate character.

3. The method according to claim 1 or 2, characterized in that: The step of performing action recognition on the candidate pet according to the at least one frame of target image to obtain the action of the candidate pet includes: Acquire a first image formed by a region of the candidate pet in each frame of the target image; Determine a sub-action of the candidate pet in each frame of the target image according to the first image of the candidate pet in each frame of the target image; The action of the candidate pet is determined according to the sub-action of the candidate pet in each frame of the target image.

4. The method according to claim 3, characterized in that The step of determining the sub-action of the candidate pet in each frame of the target image according to the first image of the candidate pet in each frame of the target image comprises: Performing feature extraction on the first image of the candidate pet in each frame of the target image to obtain a second feature vector of the candidate pet in each frame of the target image; Matching the second feature vector of the candidate pet in each frame of the target image with a plurality of first action parameters respectively to obtain a matching degree between each frame of the target image and each first action parameter, wherein each first action parameter is used to characterize a sub-action; Determine a target first action parameter according to the matching degree between each frame of the target image and each first action parameter, wherein the target first action parameter is the first action parameter corresponding to the maximum matching degree; The sub-action represented by the first action parameter of the target is used as the sub-action of the candidate pet in each frame of the target image.

5. The method according to claim 1, characterized in that: Before acquiring a plurality of frames of images to be recognized captured at the target position, the method further includes: Obtaining a layout diagram of the area to be identified; At least one candidate position in the to-be-identified area is determined according to the layout diagram, wherein the target position is any one of the at least one candidate position.

6. A pet anti-theft device, characterized in that: include: Acquisition unit and processing unit; The acquisition unit is used to acquire the video to be identified shot at the target location; The processing unit is used to obtain multiple frames of images to be identified from the video to be identified; Performing target detection on each frame of the multiple frames of images to be identified respectively to obtain at least one frame of target image in which both the candidate person and the candidate pet appear in the multiple frames of images to be identified; comprising: Performing target detection on each frame of the image to be identified to obtain the category of each target in each frame of the image to be identified; Determining a first target in each frame of the image to be identified according to the category of each target in each frame of the image to be identified, wherein the category of the first target is the same as the category of the candidate character; Determining a second target in each frame of the image to be identified according to the category of each target in each frame of the image to be identified, wherein the category of the second target is the same as the category of the candidate pet; Merging the first targets in the multiple frames of images to be identified to obtain at least one first target, and obtaining the number of times each of the at least one first target appears in the multiple frames of images to be identified; determining a first target a according to the number of times each of the first targets appears in the multiple frames of projection images, wherein the first target a is the first target that appears the most times in the at least one target; Merging the second targets in the multiple frames of images to be identified to obtain at least one second target, and obtaining the number of times each of the at least one second target appears in the multiple frames of images to be identified; determining a second target b according to the number of times each of the second targets appears in the multiple frames of projection images, wherein the second target b is the second target that appears the most times among the at least two targets; Taking the first object a as the candidate person, and taking the second object b as the candidate pet; Using the image to be identified that contains both the candidate person and the candidate pet in the multiple frames of images to be identified as the at least one frame of target image; Performing emotion recognition on the candidate character according to the at least one frame of target image to obtain the emotion of the candidate character; According to the at least one frame of target image, performing action recognition on the candidate pet to obtain the action of the candidate pet; Determining whether the candidate character has an act of stealing the candidate pet according to the emotion of the candidate character and the action of the candidate pet; If there is an act of stealing the candidate pet, a prompt message is sent to the target device, where the prompt message is used to prompt that the act of stealing the candidate pet has occurred at the target location.

7. An electronic device, characterized in that: include: A processor and a memory, the processor is connected to the memory, the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the electronic device executes the method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the method according to any one of claims 1 to 5.

Images