Intelligent bird feeding method and apparatus

Abstract

An intelligent bird feeding method and apparatus. The method comprises: continuously acquiring images of a bird feeding area by means of a camera; extracting image features by means of a backbone network of a pre-trained bird identification model to form an original feature vector; performing species identification on the original feature vector by means of a detection branch of the pre-trained bird identification model, and mapping identified species data into a continuous vector of a specified dimension by means of an embedding matrix, the dimension of the continuous vector being the same as the dimension of the original feature vector; superposing the continuous vector and the original feature vector to form a new feature vector, and inputting the new feature vector into a re-identification branch of the pre-trained bird identification model to obtain an individual identity of a bird; and when the identified animal species is bird, recording the individual identity and feeding data of the bird, and when the identified animal species is not bird, issuing an expelling signal.

Description

Intelligent bird feeding method and device

[0001] This application claims priority to the invention patent application with application number 202410202526.1 filed with the Patent Office of China on February 23, 2024, and entitled “An interactive intelligent bird feeding method and device based on image recognition”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application belongs to the field of data processing technology and relates to an intelligent bird feeding method and device. Background Art

[0003] As people's living standards improve, bird feeding using bird feeders has attracted significant interest and become a popular hobby. Bird feeders allow people to enjoy a variety of birds and provide a variety of food for them, enriching their lives. Furthermore, bird feeders help bird researchers collect information on various species and conduct research activities such as bird observation. Therefore, the development of intelligent and professional bird feeding devices will further enrich the enjoyment of bird feeding and promote the development of bird research.

[0004] Existing bird feeding devices usually consist of a feeding trough, a tray for birds to stand on, a camera and other components. This simple bird feeding device has the problem of not being able to identify the species of animals, which often leads to rodents such as squirrels eating the food in the bird feeder, causing competition for food for birds. To solve this problem, some bird feeders are equipped with gravity sensors that detect the change in weight on the platform to determine whether it is a bird or a squirrel, and then close the feeding trough. However, this hardware sensor-based design is not accurate enough and has a complex structure and higher cost. Other bird feeders can identify birds through cameras, but they can usually only distinguish between categories and cannot drive away squirrels or identify different individual birds of the same species, so they cannot provide personalized targeted feeding. Technical issues

[0005] In order to solve the above technical problems, the present application provides an intelligent bird feeding method and device. Technical Solutions

[0006] In the first aspect, an intelligent bird feeding method is provided, which mainly includes:

[0007] Step S1, continuously acquiring images of the bird feeding area through a camera;

[0008] Step S2: extracting image features through the backbone network of the pre-trained bird recognition model to form an original feature vector;

[0009] Step S3: The original feature vector is subjected to species identification through the detection branch of the pre-trained bird identification model, and the identified species data is mapped into a continuous vector of a specified dimension through an embedding matrix, where the dimension of the continuous vector is the same as that of the original feature vector;

[0010] Step S4: superimposing the continuous vector and the original feature vector to form a new feature vector, and inputting the new feature vector into the re-identification branch of the pre-trained bird recognition model to obtain the individual identity of the bird;

[0011] Step S5: When the identified animal species is a bird, the individual identity of the bird and its feeding data are recorded; when the identified animal species is non-bird, a driving signal is issued.

[0012] In a second aspect, an intelligent bird feeding method comprises:

[0013] Step 1: Use the camera to obtain image features of the animal activity area;

[0014] Step 2: Build a bird recognition model based on an image neural network, including:

[0015] Extracting the image features through the backbone network of the bird recognition model to form original features;

[0016] The detection branch of the bird recognition model performs species identification on the original features and outputs a category feature; if the category feature is a bird feature, the category feature is mapped into a new feature representation through the category embedding layer of the bird recognition module, the feature representation is added to the sample features of the re-identification branch in the pre-trained bird recognition model, the individual identity of the bird is identified, and a first bird identity feature is output;

[0017] Performing bird individual identity recognition on the original features through the re-identification branch of the bird recognition model, and outputting a second bird identity feature;

[0018] Step 3: Construct a user interaction module for users to interact in real time according to whether the category feature is a non-bird feature, the first bird identity feature, or the second bird identity feature.

[0019] In a third aspect, an intelligent bird feeding device is provided, which mainly includes:

[0020] An image acquisition module, used for continuously acquiring images of the bird feeding area through a camera;

[0021] The original feature vector extraction module is used to extract image features through the backbone network of the pre-trained bird recognition model to form an original feature vector;

[0022] a species identification module, configured to identify the species of the original feature vector through a detection branch of a pre-trained bird identification model, and map the identified species data into a continuous vector of a specified dimension through an embedding matrix, wherein the dimension of the continuous vector is the same as that of the original feature vector;

[0023] A bird individual identity recognition module, configured to superimpose the continuous vector and the original feature vector to form a new feature vector, and input the new feature vector into the re-identification branch of the pre-trained bird recognition model to obtain the individual identity of the bird;

[0024] The feeding control module is used to record the individual identity of the bird and its feeding data when the identified animal species is a bird, and to send a driving signal when the identified animal species is non-bird.

[0025] In a fourth aspect, a computer device is provided, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the intelligent bird feeding method as described in any one of the above items.

[0026] In a fifth aspect, a readable storage medium is provided, wherein the readable storage medium stores a computer program, and when the computer program is executed by a processor, it is used to implement the intelligent bird feeding method as described above. Beneficial effects

[0027] This application can realize the timed and quantitative feeding of different bird individuals, is more intelligent and convenient for interaction, and improves the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or exemplary technical descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0029] FIG1 is a flow chart of an intelligent bird feeding method provided in an embodiment of the present application.

[0030] FIG2 is a diagram of the bird recognition model architecture of the embodiment shown in FIG1 of the present application.

[0031] FIG3 is a schematic diagram of the structure of a computer device of a terminal or server suitable for implementing the implementation mode of the present application, provided in an embodiment of the present application. Modes for Carrying Out the Invention

[0032] In the following description, specific details such as specific system structures and techniques are provided for purposes of illustration rather than limitation to facilitate a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obscuring the description of the present application with unnecessary detail.

[0033] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or collections thereof.

[0034] It will also be understood that the term "and / or" used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0035] As used in this specification and the appended claims, the term "if" can be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0036] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0037] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in conjunction with the embodiment is included in one or more embodiments of the present application. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in yet other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized.

[0038] In order to illustrate the technical solution provided by this application, a detailed description is given below with reference to specific drawings and embodiments.

[0039] According to the first aspect of the present application, as shown in FIG1 , an intelligent bird feeding method mainly includes:

[0040] Step S1: Continuously acquire images of the bird feeding area through a camera.

[0041] In this step, images of the bird feeding area are continuously captured using an image acquisition device, such as a camera, which can capture images or videos of the bird feeding area for subsequent analysis. There may be multiple bird feeding areas, and one or more cameras can be used to capture images of the area.

[0042] This application primarily involves interactive intelligent bird feeding, so a camera is typically mounted above a bird feeder, which primarily includes a feeding trough, a tray for birds to stand on, and a feeding actuator. The camera is mounted to capture images of the feeding area, primarily to determine whether birds or other animals are occupying the trough or tray, allowing for targeted feeding.

[0043] Step S2: extract image features through the backbone network of the pre-trained bird recognition model to form an original feature vector.

[0044] This application constructs a bird recognition model based on an image neural network. In step S2, the backbone network is responsible for extracting image features. In order to improve its image feature extraction capability, it can be pre-trained on the ImageNet dataset.

[0045] Step S3: The original feature vector is subjected to species identification through the detection branch of the pre-trained bird identification model, and the identified species data is mapped into a continuous vector of a specified dimension through an embedding matrix, where the dimension of the continuous vector is the same as that of the original feature vector.

[0046] Step S4: superimpose the continuous vector and the original feature vector to form a new feature vector, and input the new feature vector into the re-identification branch of the pre-trained bird recognition model to obtain the individual identity of the bird.

[0047] The bird recognition model involved in steps S2 to S4 is shown in Figure 2, which mainly includes three parts: a backbone network, a detection branch, and a re-identification branch. After the backbone network outputs the original feature vector, it is transmitted to the detection branch and the re-identification branch respectively. In step S3, the detection branch is used to realize the recognition of various birds or other animals. In step S4, the re-identification branch is used to identify the identity of individual birds.

[0048] The detection and re-identification branches of this application share the original feature vectors output by the backbone network, and a category embedding layer is designed. Its function is to embed the animal category information of the detection branch into the feature representation of the re-identification branch, providing richer and more discriminative features, and enhancing the re-identification branch's ability to identify different birds. The specific implementation of this embedding layer uses an embedding matrix, using the bird category value as input and mapping it to a low-dimensional continuous vector space, as shown in the following formula: bird_embedding = Embedding(bird_class,emd_dim);

[0049] Here, bird_class represents the input bird class information, and the Embedding function embeds it into a vector of emd_dim dimensions. Then, in the re-identification branch, the embedded class information is added to the original feature vector to generate a new feature representation for the re-identification branch to use, as shown in the following formula: new_feature = Add(origin_feature, bird_embedding)

[0050] Here, the category information is fused with the original features through the Add operation, and the dimension of the embedding vector is the same as the dimension of the original feature vector of the re-identification branch.

[0051] For example, assuming that the original feature vector is 1024-dimensional data, step S3 obtains a 3-dimensional category feature vector, such as [1,0,0] represents a magpie, [0,1,0] represents a pigeon, and [0,0,1] represents a sparrow. Then the embedding matrix is ​​a 1024-by-3 matrix. In step S4, the matrix is ​​multiplied by the 3-dimensional category feature vector (converted to a 3-by-1 matrix) to form a 1024-dimensional continuous vector. The 1024-dimensional continuous vector can be superimposed with the 1024-dimensional original feature vector in step S4.

[0052] In this embodiment, each value in the embedding matrix is ​​obtained by training the bird recognition model. In some embodiments, before step S2, the bird recognition model is further trained by the following steps:

[0053] Step S21: obtaining images containing different animal types, different backgrounds, different postures, and different lighting conditions as training data, and labeling the training data with bird target frames, categories, and individual bird identities;

[0054] Step S22: processing the training data based on the bird recognition model to be trained to obtain the coordinates of the detection frame containing the animal, the confidence level, the bird classification, and the individual bird recognition results;

[0055] Step S23: Determine the total loss Lbird of the bird recognition model based on the following formula:

[0056] Lbird=λcoord*Lcoord+λconf*Lconf+λcls*Lcls+λTriplet*LTriplet;

[0057] Among them, Lcoord, Lconf, Lcls are the detection box coordinate regression loss, confidence loss and classification loss calculated by the detection branch, LTriplet is the triplet loss calculated by the re-identification branch, λcoord, λconf, λcls, λTriplet are the weights of the detection box coordinate regression loss, confidence loss, classification loss and triplet loss respectively;

[0058] Step S24: Based on the total loss Lbird, the weight parameters to be optimized and the embedding matrix of the bird recognition model are updated to obtain the finally trained bird recognition model.

[0059] In this embodiment, the bird recognition model can be a ResNet model, or other backbone networks such as VGG, Inception, EfficientNet, MobileNet, ShuffleNet and other models can be used. In this model, the triplet loss (Triplet Loss) is used to optimize the model parameters so that the feature vectors of the same identity are as close as possible and the feature vectors of different identities are as dispersed as possible. The training is iterated multiple times and the weights of the model are updated to reduce the loss function and ensure the convergence of the model. After the training is completed, the original feature vector is extracted by step S2, the animal category is output by step S3, such as magpie, pigeon, and the individual feature vector is output by step S4. The individual identity of the animal that best matches the output individual feature vector is determined, such as magpie 1 and magpie 2.

[0060] In some embodiments, step S4 further comprises:

[0061] Step S41: obtaining an individual identity feature vector of a bird based on a bird recognition model;

[0062] Step S42: Calculate the distances between the individual identity feature vector and each sample feature vector in the database used to characterize different bird individuals, determine the sample feature vector with the smallest distance, and thus determine the individual identity of the bird.

[0063] In this embodiment, the distance is, for example, a cosine distance.

[0064] Step S5: When the identified animal species is a bird, the individual identity of the bird and its feeding data are recorded; when the identified animal species is non-bird, a driving signal is issued.

[0065] In some embodiments, when the identified animal species is a bird, matching user-set parameters are selected for targeted feeding based on the identified individual bird identity, and the parameters include the type of food to be fed, the feeding amount, and the feeding time.

[0066] In some embodiments, step S5 further includes:

[0067] Step S6: According to the set interaction parameters or in response to the user's interaction request, the bird individual identity, the feeding data of the corresponding bird individual, and the monitoring video are remotely sent to the user, and the appearance or feeding frequency of different bird individuals is counted.

[0068] In the above two embodiments, a user interaction module is first constructed, and the user can connect to the bird feeding system through a mobile phone application or a web interface. The main functions include the following:

[0069] (1) Users can view real-time images, for example, they can monitor the images captured by the camera in real time through a mobile phone application or web interface to understand the current bird activity;

[0070] (2) Receive system push notifications to understand the current status of the bird feeder, such as whether squirrels appear, whether new birds or individuals visit, and whether bird feed needs to be replenished;

[0071] (3) Setting feeding targets and food: users can set specific birds or individuals as feeding targets, and select different bird food for different birds. The system will feed them in a targeted manner based on the recognition results.

[0072] (4) View records: users can view system records at any time, including feeding history, frequency of appearance of various birds, and other information to understand the patterns of bird activities;

[0073] (5) Personalized settings: users can make personalized settings based on the records and analysis results of the bird feeding system, and adjust parameters such as feeding amount, feeding food, and feeding time to meet the needs of different birds;

[0074] (6) Socialize with other users, such as sharing your own bird photos or videos to the community, viewing other users' bird observation logs, participating in bird-related topics or activities, etc.

[0075] In some embodiments, in step S5, issuing a driving signal includes:

[0076] The audio device or lighting device installed on the bird feeder is driven to generate sound driving information and light driving signal, and at the same time, the feeding trough baffle is driven to close.

[0077] In this embodiment, in order to prevent rodents such as squirrels from occupying the bird feeder, when a non-avian animal is detected, the system sends a driving signal, which may be a sound, light or other stimulus, and closes the feeding trough baffle.

[0078] In some embodiments, the temperature and humidity of the birds while they are eating are obtained through sensors preset on the feeder.

[0079] In this embodiment, during the interaction with the user, the recording and data transmission of environmental factors, such as temperature, humidity, geographic coordinates, etc., are added to enhance the in-depth understanding of bird behavior.

[0080] The intelligent bird feeding method provided in this application can efficiently realize the identification of squirrels and various birds as well as different individual birds. At the same time, users can use the interactive module to feed birds in a targeted manner in real time, record bird information, and other functions, thereby meeting the needs of different users and birds and realizing intelligent bird feeding.

[0081] In a second aspect of the present application, as shown in FIG2 , an intelligent bird feeding method includes:

[0082] Step 1: Use the camera to obtain image features of the animal activity area;

[0083] Step 2: Build a bird recognition model based on an image neural network, including:

[0084] Extracting the image features through the backbone network of the bird recognition model to form original features;

[0085] Performing category recognition on the original features through the detection branch of the bird recognition model, and outputting category features, wherein the category features include non-bird category features and bird category features;

[0086] Mapping the category features into new feature representations through the category embedding layer of the bird recognition module, embedding the feature representations into the re-identification branch of the bird recognition model and superimposing them with the original features in the re-identification branch, performing bird individual identity recognition on the superimposed feature information through the re-identification branch, and outputting bird identity features;

[0087] Step 3: Construct a user interaction module to allow users to interact in real time according to the category feature being the non-bird category feature or the bird identity feature.

[0088] This application mainly realizes intelligent bird feeding and interaction. The camera is usually installed above the bird feeder, which can capture images or videos of the bird feeding area and continuously obtain images of the bird feeding area for subsequent analysis. The bird feeder mainly includes a feeding trough, a tray for birds to stand on, and a feeding actuator. The camera is installed to be able to obtain images of the feeding area, mainly to be able to know whether there are birds or other animals occupying the feeding trough, tray, etc., so as to feed food in a targeted manner.

[0089] The image features captured in the bird feeding area of ​​this application will involve image features of different birds and non-bird animals, non-bird animals including rodents such as squirrels, and a detection branch is used to perform species identification on the image features to determine whether they belong to different birds or non-bird animals, that is, the category features output by the species identification include non-bird category features and bird category features.

[0090] This application builds a bird recognition model based on an image neural network, in which the backbone network is responsible for extracting image features and can be pre-trained on the ImageNet dataset to improve its image feature extraction capability.

[0091] As shown in Figure 2, the bird recognition model of the present application mainly includes three parts, one is the backbone network (Backbone), one is the detection branch (Detection Branch), and the other is the re-identification branch (Re-identification Branch). Among them, after the original features are output by the backbone network, they are respectively transmitted to the detection branch and the re-identification branch. The detection branch is used to realize the recognition of various birds or other animals, and the re-identification branch is used to identify the identity of individual birds.

[0092] In some embodiments, the category embedding layer maps the category features using an embedding matrix, and step 2 includes:

[0093] Extract the image features through the backbone network and output the original feature vector;

[0094] Performing category identification on the original feature vector through the detection branch, and outputting a category feature vector;

[0095] The category feature vector is mapped into a continuous vector of a specified dimension using the embedding matrix, the continuous vector is embedded into the re-identification branch and superimposed with the original feature vector in the re-identification branch, and the superimposed feature information is used by the re-identification branch to perform individual bird identification and generate bird identity features.

[0096] The detection and re-identification branches of this application share the original feature vectors output by the backbone network, and a category embedding layer is designed. Its function is to embed the animal category information of the detection branch into the feature representation of the re-identification branch, providing richer and more discriminative features, and enhancing the re-identification branch's ability to identify different birds. The specific implementation of this embedding layer uses an embedding matrix, using the bird category value as input and mapping it to a low-dimensional continuous vector space, as shown in the following formula: bird_embedding = Embedding(bird_class,emd_dim);

[0097] Here, bird_class represents the input bird class information, and the Embedding function embeds it into a vector of emd_dim dimensions. Then, in the re-identification branch, the embedded class information is added to the original feature vector to generate a new feature representation for the re-identification branch to use, as shown in the following formula: new_feature = Add(origin_feature, bird_embedding)

[0098] Here, the category information is fused with the original features through the Add operation, and the dimension of the embedding vector is the same as the dimension of the original feature vector of the re-identification branch. The re-identification branch and the detection branch share the original feature vector output by the backbone network.

[0099] For example, assuming that the original feature vector is 1024-dimensional data, the detection branch is used to identify the species of the original feature vector to obtain a 3-dimensional category feature vector, such as [1,0,0] represents a magpie, [0,1,0] represents a pigeon, and [0,0,1] represents a sparrow. The embedding matrix is ​​a 1024-by-3 matrix. After multiplying the matrix with the 3-dimensional category feature vector (converted to a 3-by-1 matrix), the category feature vector is mapped to form a 1024-dimensional continuous vector. The 1024-dimensional continuous vector can be superimposed with the 1024-dimensional original feature vector in the re-identification branch that embeds the continuous vector into the bird recognition model.

[0100] In some embodiments, the step of superimposing the continuous vector with the original feature vector includes:

[0101] Obtaining individual identity feature vectors of birds based on the bird recognition model;

[0102] The distances between the individual identity feature vector and each sample feature vector in the database used to characterize different bird individuals are calculated respectively, and the sample feature vector with the smallest distance is determined, thereby determining the individual identity of the bird.

[0103] In some embodiments, the bird recognition model is trained using a bird image dataset, including:

[0104] Acquire images containing different animal types, different backgrounds, different postures, and different lighting conditions as training data, and annotate the training data with bird target frames, categories, and individual bird identities;

[0105] Processing the training data based on the bird recognition model to be trained to obtain detection frame coordinates containing animals, confidence levels, bird classifications, and individual bird recognition results;

[0106] The total loss Lbird of the bird recognition model is determined based on the following formula: Lbird = λcoord*Lcoord+λconf*Lconf+λcls*Lcls+λTriplet*LTriplet;

[0107] Among them, Lcoord, Lconf, Lcls are the detection box coordinate regression loss, confidence loss and classification loss calculated by the detection branch, LTriplet is the triplet loss calculated by the re-identification branch, λcoord, λconf, λcls, λTriplet are the weights of the detection box coordinate regression loss, confidence loss, classification loss and triplet loss respectively;

[0108] Based on the total loss Lbird, the weight parameters to be optimized of the bird recognition model and the embedding matrix are updated to obtain the final trained bird recognition model.

[0109] The bird recognition model of the present application can be a ResNet model, or other backbone networks such as VGG, Inception, EfficientNet, MobileNet, ShuffleNet and other models can be used. In this model, the triplet loss (Triplet Loss) is used to optimize the model parameters so that the feature vectors of the same identity are as close as possible and the feature vectors of different identities are as dispersed as possible. The training is iterated multiple times and the weights of the model are updated to reduce the loss function and ensure the convergence of the model. During training, by extracting the original feature vector, by outputting the animal category, such as magpie, pigeon, by outputting the individual feature vector, the most consistent individual animal identity is determined according to the output individual feature vector, such as magpie 1 and magpie 2.

[0110] In some embodiments, step 2 includes:

[0111] When the bird species category feature is detected according to the category feature and / or different bird individuals are detected according to the bird identity feature, relevant information is recorded in the database, and the relevant information includes bird classification, bird individual identity, and feeding time.

[0112] In some embodiments, step 3 includes:

[0113] When the category feature is a non-bird category feature, a driving signal is issued, including: driving the audio equipment or lighting equipment installed on the bird feeder to generate sound driving information and light driving signal, and driving the feeding trough baffle to close.

[0114] In some embodiments, step 3 includes:

[0115] When the bird identity feature is identified, the interactive action at least includes:

[0116] Match the individual identities of birds for targeted feeding; or, according to the set interaction parameters or in response to the user's interaction request, send various types of information about individual birds to the user remotely.

[0117] In the user interaction module, users can connect to the bird feeding system through mobile applications or web interfaces, which mainly include the following functions:

[0118] (1) Users can view real-time images, for example, they can monitor the images captured by the camera in real time through a mobile phone application or web interface to understand the current bird activity;

[0119] (2) Receive system push notifications to understand the current status of the bird feeder, such as whether squirrels appear, whether new birds or individuals visit, and whether bird feed needs to be replenished;

[0120] (3) Setting feeding targets and food: users can set specific birds or individuals as feeding targets, and select different bird food for different birds. The system will feed them in a targeted manner based on the recognition results.

[0121] (4) View records: users can view system records at any time, including feeding history, frequency of appearance of various birds, and other information to understand the patterns of bird activities;

[0122] (5) Personalized settings: users can make personalized settings based on the records and analysis results of the bird feeding system, and adjust parameters such as feeding amount, feeding food, and feeding time to meet the needs of different birds;

[0123] (6) Socialize with other users, such as sharing your own bird photos or videos to the community, viewing other users' bird observation logs, participating in bird-related topics or activities, etc.

[0124] In some embodiments, step 3 includes: introducing a voice interaction module to interact with the user through voice commands. In this embodiment, the voice interaction function is introduced into the user interaction module, and the user can interact with the system through voice commands, thereby improving the user experience.

[0125] In some embodiments, step 3 includes:

[0126] Acquiring environmental factors of the bird feeding area through sensors preset on the feeder, including but not limited to temperature, humidity and geographic coordinates;

[0127] The environmental factors are recorded and data transmission is performed on the environmental factors.

[0128] This application provides a deeper understanding of bird behavior through the perception and recording of environmental factors.

[0129] This application uses a camera as an image feature acquisition device and an image neural network as an implementation carrier. After the backbone network extracts image features, it adds a detection branch and a re-identification branch. The detection branch is used to recognize non-birds such as squirrels and various bird species, and the re-identification branch is used to recognize the identities of different bird individuals. Combined with the user interaction module, it can achieve targeted feeding and recording of non-birds such as squirrels and different bird individuals.

[0130] In a third aspect, the present application provides an interactive intelligent bird feeding device based on image recognition corresponding to the above method, which mainly includes:

[0131] An image acquisition module, used for continuously acquiring images of the bird feeding area through a camera;

[0132] The original feature vector extraction module is used to extract image features through the backbone network of the pre-trained bird recognition model to form an original feature vector;

[0133] a species identification module, configured to identify the species of the original feature vector through a detection branch of a pre-trained bird identification model, and map the identified species data into a continuous vector of a specified dimension through an embedding matrix, wherein the dimension of the continuous vector is the same as that of the original feature vector;

[0134] A bird individual identity recognition module, configured to superimpose the continuous vector and the original feature vector to form a new feature vector, and input the new feature vector into the re-identification branch of the pre-trained bird recognition model to obtain the individual identity of the bird;

[0135] The feeding control module is used to record the individual identity of the bird and its feeding data when the identified animal species is a bird, and to send a driving signal when the identified animal species is non-bird.

[0136] In some embodiments, the method further comprises training the animal detection model by:

[0137] A training data labeling unit is used to obtain images containing different animal types, different backgrounds, different postures and different lighting conditions as training data, and label the training data with bird target frames, categories and individual bird identities;

[0138] A model actual output acquisition unit is used to process the training data based on the bird recognition model to be trained to obtain the detection frame coordinates containing the animal, the confidence level, the bird classification, and the individual bird recognition results;

[0139] A loss determination unit is used to determine a total loss Lbird of the bird recognition model based on the following formula: Lbird = λcoord*Lcoord+λconf*Lconf+λcls*Lcls+λTriplet*LTriplet;

[0140] Among them, Lcoord, Lconf, Lcls are the detection box coordinate regression loss, confidence loss and classification loss calculated by the detection branch, LTriplet is the triplet loss calculated by the re-identification branch, λcoord, λconf, λcls, λTriplet are the weights of the detection box coordinate regression loss, confidence loss, classification loss and triplet loss respectively;

[0141] The cyclic optimization unit is used to update the weight parameters to be optimized of the bird recognition model and the embedding matrix based on the total loss Lbird to obtain the finally trained bird recognition model.

[0142] In some embodiments, the bird individual identification module includes:

[0143] An identity feature vector output unit, configured to obtain an individual identity feature vector of a bird based on a bird recognition model;

[0144] The individual identity determination unit is used to respectively calculate the distance between the individual identity feature vector and each sample feature vector in the database used to characterize different bird individuals, determine the sample feature vector with the smallest distance, and thus determine the individual identity of the bird.

[0145] In some embodiments, the feeding control module is configured to connect to and drive an audio device or a lighting device installed on the bird feeder to generate sound driving information and light driving signals, and at the same time drive to close the feeding trough baffle.

[0146] In some embodiments, the feeding control module further comprises:

[0147] The directional feeding control unit is used to select matching parameters set by the user for directional feeding when the identified animal species is a bird, based on the identified individual identity of the bird. The parameters include the type of food to be fed, the feeding amount and the feeding time.

[0148] In some embodiments, the interactive intelligent bird feeding device based on image recognition further includes:

[0149] The bird activity pattern statistics module is used to remotely send the individual bird identity, feeding data of the corresponding individual bird, and monitoring video to the user based on the set interaction parameters or in response to the user's interaction request, and at the same time count the appearance or feeding frequency of different bird individuals.

[0150] In a fourth aspect of the present application, a computer device includes a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement an intelligent bird feeding method.

[0151] In a fifth aspect of the present application, a readable storage medium stores a computer program that, when executed by a processor, implements the intelligent bird feeding method described above. The computer-readable storage medium may be included in the device described in the above embodiments, or it may exist independently and not be incorporated into the device. The computer-readable storage medium carries one or more programs that, when executed by the device, process data according to the method described above.

[0152] 3, which shows a schematic diagram of the structure of a computer device 400 suitable for implementing the embodiments of the present application. The computer device shown in FIG3 is only an example and should not limit the functions and scope of use of the embodiments of the present application.

[0153] As shown in FIG3 , a computer device 400 includes a central processing unit (CPU) 401, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 402 or a program loaded from a storage unit 408 into a random access memory (RAM) 403. Various programs and data required for the operation of the device 400 are also stored in the RAM 403. The CPU 401, ROM 402, and RAM 403 are connected to each other via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0154] The following components are connected to the I / O interface 405: an input section 406 including a keyboard, a mouse, and the like; an output section 407 including devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), and a speaker; a storage section 408 including a hard disk; and a communication section 409 including a network interface card such as a LAN card or a modem. The communication section 409 performs communication processing via a network such as the Internet. A drive 410 is also connected to the I / O interface 405 as needed. Removable media 411, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 410 as needed, so that computer programs read therefrom can be installed into the storage section 408 as needed.

[0155] In particular, according to the embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 409, and / or installed from the removable medium 411. When the computer program is executed by the central processing unit (CPU) 401, the above functions defined in the method of the present application are executed. It should be noted that the computer storage medium of the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, but not limited to, a system, device or component of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection having one or more conductors, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transfer a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code embodied on a computer-readable medium may be transmitted using any suitable medium, including, but not limited to, wireless, wire, optical cable, RF, etc., or any suitable combination thereof.

[0156] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, program segment or a part of code, and the module, program segment or a part of code contains one or more executable instructions for realizing the prescribed logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the prescribed function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0157] The modules or units described in the embodiments of this application may be implemented in software or hardware. The modules or units described may also be provided in a processor, and the names of these modules or units do not, in certain circumstances, limit the modules or units themselves.

[0158] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. An intelligent bird feeding method, wherein: include: Step S1: continuously acquiring images of the bird feeding area through a camera; Step S2: extracting image features through the backbone network of the pre-trained bird recognition model to form an original feature vector; Step S3: The original feature vector is subjected to species identification through the detection branch of the pre-trained bird identification model, and the identified species data is mapped into a continuous vector of a specified dimension through an embedding matrix, where the dimension of the continuous vector is the same as that of the original feature vector; Step S4: superimposing the continuous vector and the original feature vector to form a new feature vector, which is input into the re-identification branch of the pre-trained bird recognition model to obtain the individual identity of the bird; Step S5: When the identified animal species is a bird, the individual identity of the bird and its feeding data are recorded; when the identified animal species is non-bird, a driving signal is issued.

2. The interactive intelligent bird feeding method based on image recognition according to claim 1, wherein: Before step S2, the method further includes training the bird recognition model through the following steps: Step S21: acquiring images containing different animal types, different backgrounds, different postures, and different lighting conditions as training data, and labeling the training data with bird target frames, categories, and individual bird identities; Step S22: processing the training data based on the bird recognition model to be trained to obtain the coordinates of the detection frame containing the animal, the confidence level, the bird classification, and the individual bird recognition results; Step S23: Determine the total loss Lbird of the bird recognition model based on the following formula: Lbird=λcoord*Lcoord+λconf*Lconf+λcls*Lcls+λTriplet*LTriplet; Among them, Lcoord, Lconf, Lcls are the detection box coordinate regression loss, confidence loss and classification loss calculated by the detection branch, LTriplet is the triplet loss calculated by the re-identification branch, λcoord, λconf, λcls, λTriplet are the weights of the detection box coordinate regression loss, confidence loss, classification loss and triplet loss respectively; Step S24: Based on the total loss Lbird, the weight parameters to be optimized of the bird recognition model and the embedding matrix are updated to obtain the final trained bird recognition model.

3. The intelligent bird feeding method according to claim 1, wherein: Step S4 further comprises: Step S41: obtaining an individual identity feature vector of the bird based on a bird recognition model; Step S42: Calculate the distances between the individual identity feature vector and each sample feature vector in the database used to characterize different bird individuals, determine the sample feature vector with the smallest distance, and thus determine the individual identity of the bird.

4. The intelligent bird feeding method according to claim 1, wherein: In step S5, issuing a driving signal includes: The audio device or lighting device installed on the bird feeder is driven to generate sound driving information and light driving signal, and at the same time, the feeding trough baffle is driven to close.

5. The intelligent bird feeding method according to claim 1, wherein: Step S5 further comprises: When the identified animal species is a bird, the user-set parameters that match the identified individual bird are selected for targeted feeding, including the type of food to be fed, the feeding amount, and the feeding time.

6. The intelligent bird feeding method according to claim 1, wherein: After step S5, the method further comprises: Step S6: According to the set interaction parameters or in response to the user's interaction request, the bird individual identity, the feeding data of the corresponding bird individual, and the monitoring video are remotely sent to the user, and the appearance or feeding frequency of different bird individuals is counted.

7. An intelligent bird feeding method, wherein: include: Step 1: Use the camera to obtain image features of the animal activity area; Step 2: Build a bird recognition model based on an image neural network, including: Extracting the image features through the backbone network of the bird recognition model to form original features; Performing category recognition on the original features through the detection branch of the bird recognition model, and outputting category features, wherein the category features include non-bird category features and bird category features; Mapping the category features into new feature representations through the category embedding layer of the bird recognition module, embedding the feature representations into the re-identification branch of the bird recognition model and superimposing them with the original features in the re-identification branch, performing bird individual identity recognition on the superimposed feature information through the re-identification branch, and outputting bird identity features; Step 3: Construct a user interaction module to allow users to interact in real time according to the category feature being the non-bird category feature or the bird identity feature.

8. The intelligent bird feeding method according to claim 7, wherein: The category embedding layer maps the category features using an embedding matrix. Step 2 includes: Extract the image features through the backbone network and output the original feature vector; Performing category identification on the original feature vector through the detection branch, and outputting a category feature vector; The category feature vector is mapped into a continuous vector of a specified dimension using the embedding matrix, the continuous vector is embedded into the re-identification branch and superimposed with the original feature vector in the re-identification branch, and the superimposed feature information is used by the re-identification branch to perform individual bird identification and generate bird identity features.

9. The intelligent bird feeding method according to claim 8, wherein: The step of superimposing the continuous vector and the original feature vector comprises: Obtaining individual identity feature vectors of birds based on the bird recognition model; The distances between the individual identity feature vector and each sample feature vector in the database used to characterize different bird individuals are calculated respectively, and the sample feature vector with the smallest distance is determined, thereby determining the individual identity of the bird.

10. The intelligent bird feeding method according to claim 7, wherein: The bird recognition model is trained using a bird image dataset, including: Acquire images containing different animal types, different backgrounds, different postures, and different lighting conditions as training data, and annotate the training data with bird target frames, categories, and individual bird identities; Processing the training data based on the bird recognition model to be trained to obtain detection frame coordinates containing animals, confidence levels, bird classifications, and individual bird recognition results; The total loss Lbird of the bird recognition model is determined based on the following formula: Lbird=λcoord*Lcoord+λconf*Lconf+λcls*Lcls+λTriplet*LTriplet; Among them, Lcoord, Lconf, Lcls are the detection box coordinate regression loss, confidence loss and classification loss calculated by the detection branch, LTriplet is the triplet loss calculated by the re-identification branch, λcoord, λconf, λcls, λTriplet are the weights of the detection box coordinate regression loss, confidence loss, classification loss and triplet loss respectively; Based on the total loss Lbird, the weight parameters to be optimized of the bird recognition model and the embedding matrix are updated to obtain the final trained bird recognition model.

11. The intelligent bird feeding method according to claim 7, wherein: In step 2, include: When the bird species category feature is detected according to the category feature and / or different bird individuals are detected according to the bird identity feature, relevant information is recorded in the database, and the relevant information includes bird classification, bird individual identity, and feeding time.

12. The intelligent bird feeding method according to claim 7, wherein: Step 3 includes: When the category feature is a non-bird category feature, issuing a driving signal, including: driving the audio device or lighting device installed on the bird feeder to generate sound driving information and light driving signal, and driving the baffle of the feeding trough to close; or When the bird identity feature is identified, the interactive action at least includes: Match the individual identities of birds for targeted feeding; or, according to the set interaction parameters or in response to the user's interaction request, send various types of information about individual birds to the user remotely.

13. The intelligent bird feeding method according to claim 7, wherein: Step 3 includes: introducing a voice interaction module to interact with users through voice commands.

14. The intelligent bird feeding method according to claim 7, wherein: Step 3 includes: Acquiring environmental factors of the bird feeding area through sensors preset on the feeder, including but not limited to temperature, humidity and geographic coordinates; The environmental factors are recorded and data transmission is performed on the environmental factors.

15. An intelligent bird feeding device, wherein: include: An image acquisition module, used for continuously acquiring images of the bird feeding area through a camera; The original feature vector extraction module is used to extract image features through the backbone network of the pre-trained bird recognition model to form an original feature vector; a species identification module, configured to identify the species of the original feature vector through a detection branch of a pre-trained bird identification model, and map the identified species data into a continuous vector of a specified dimension through an embedding matrix, wherein the dimension of the continuous vector is the same as that of the original feature vector; A bird individual identity recognition module, configured to superimpose the continuous vector and the original feature vector to form a new feature vector, and input the new feature vector into the re-identification branch of the pre-trained bird recognition model to obtain the individual identity of the bird; The feeding control module is used to record the individual identity of the bird and its feeding data when the identified animal species is a bird, and to send a driving signal when the identified animal species is non-bird.

16. The intelligent bird feeding device according to claim 15, wherein: Further comprising training the animal detection model by the following units: A training data labeling unit is used to obtain images containing different animal types, different backgrounds, different postures and different lighting conditions as training data, and label the training data with bird target frames, categories and individual bird identities; A model actual output acquisition unit is used to process the training data based on the bird recognition model to be trained to obtain the detection frame coordinates containing the animal, the confidence level, the bird classification, and the individual bird recognition results; The loss determination unit is used to determine the total loss Lbird of the bird recognition model based on the following formula: Lbird=λcoord*Lcoord+λconf*Lconf+λcls*Lcls+λTriplet*LTriplet; Among them, Lcoord, Lconf, Lcls are the detection box coordinate regression loss, confidence loss and classification loss calculated by the detection branch, LTriplet is the triplet loss calculated by the re-identification branch, λcoord, λconf, λcls, λTriplet are the weights of the detection box coordinate regression loss, confidence loss, classification loss and triplet loss respectively; The cyclic optimization unit is used to update the weight parameters to be optimized of the bird recognition model and the embedding matrix based on the total loss Lbird to obtain the finally trained bird recognition model.

17. The intelligent bird feeding device according to claim 15, wherein: The bird individual identification module includes: An identity feature vector output unit, configured to obtain an individual identity feature vector of a bird based on a bird recognition model; The individual identity determination unit is used to respectively calculate the distance between the individual identity feature vector and each sample feature vector in the database used to characterize different bird individuals, determine the sample feature vector with the smallest distance, and thus determine the individual identity of the bird.

18. The intelligent bird feeding device according to claim 15, wherein: The feeding control module is configured to connect to and drive the audio equipment or lighting equipment installed on the bird feeder to generate sound driving information and light driving signals, and at the same time drive to close the feeding trough baffle.

19. The intelligent bird feeding device according to claim 15, wherein: The feeding control module also includes: The directional feeding control unit is used to select matching parameters set by the user for directional feeding when the identified animal species is a bird, based on the identified individual identity of the bird. The parameters include the type of food to be fed, the feeding amount and the feeding time.

20. The intelligent bird feeding device according to claim 15, wherein: The interactive intelligent bird feeding device based on image recognition also includes: The bird activity pattern statistics module is used to remotely send the individual bird identity, feeding data of the corresponding individual bird, and monitoring video to the user based on the set interaction parameters or in response to the user's interaction request, and at the same time count the appearance or feeding frequency of different bird individuals.

Images