An interactive intelligent bird feeding method and device based on image recognition

Abstract

The application provides an interactive intelligent bird feeding method and device based on image recognition. The method comprises the following steps: S1, continuously acquiring images of a bird feeding area through a camera; S2, extracting image features through a backbone network of a pre-trained bird recognition model to form an original feature vector; S3, performing species identification on the original feature vector through a detection branch of the pre-trained bird recognition model, and mapping the identified species data into a continuous vector of a specified dimension through an embedding matrix, wherein the dimension of the continuous vector is the same as that of the original feature vector; S4, superimposing the continuous vector and the original feature vector to form a new feature vector, inputting the new feature vector into a re-identification branch of the pre-trained bird recognition model, and obtaining the individual identity of the bird; and S5, when the identified animal species is a bird, recording the individual identity of the bird and its feeding data, and when the identified animal species is a non-bird, issuing a driving signal.

Description

Technical Field

[0001] This application belongs to the field of data processing technology, and specifically relates to an interactive intelligent bird feeding method and device based on image recognition. Background Technology

[0002] With the improvement of people's living standards, bird feeding via bird feeders has attracted great interest and has become a very popular leisure activity. People can enjoy diverse birdlife and provide various foods to wild birds, enriching their leisure time. Furthermore, for bird research enthusiasts, bird feeders can help them collect information on various bird species and conduct scientific research activities such as wild bird observation. Therefore, developing intelligent and professional bird feeding devices will further enrich the enjoyment of bird feeding activities and promote the development of bird research.

[0003] Existing bird feeders typically consist of a feeding trough, a tray for birds to stand on, and a camera. This simple system suffers from the inability to identify animal species, often allowing rodents like squirrels to also consume the food, creating competition for resources. To address this, some feeders incorporate gravity sensors that detect changes in weight on the platform to determine whether it's a bird or a squirrel, thus closing the feeding trough. However, this hardware-based sensor design is inaccurate, complex, and expensive. Other feeders use cameras to identify birds, but these typically only differentiate species and cannot repel squirrels or distinguish between different individuals of the same species, thus failing to provide personalized, targeted feeding. Summary of the Invention

[0004] To address the aforementioned technical issues, this application provides an interactive intelligent bird-feeding method and apparatus based on image recognition, offering a more intelligent and convenient interactive bird-feeding solution.

[0005] The first aspect of this application is an interactive intelligent bird-feeding method based on image recognition, which mainly includes:

[0006] Step S1: Continuously acquire images of the bird feeding area using a camera;

[0007] Step S2: Extract image features through the backbone network of the pre-trained bird recognition model to form the original feature vector;

[0008] Step S3: The original feature vector is used to identify the species through the detection branch of the pre-trained bird identification model, and the identified species data is mapped into a continuous vector of a specified dimension through an embedding matrix. The dimension of the continuous vector is the same as the dimension of the original feature vector.

[0009] Step S4: Superimpose the continuous vector with the original feature vector to form a new feature vector, and input it into the re-identification branch of the pre-trained bird recognition model to obtain the individual identity of the bird;

[0010] Step S5: When the identified animal species is a bird, record the individual bird's identity and its feeding data; when the identified animal species is not a bird, issue a shooing signal.

[0011] Preferably, prior to step S2, the bird recognition model is further trained through the following steps:

[0012] Step S21: Obtain images containing different animal types, backgrounds, poses, and lighting conditions as training data, and annotate the training data with bird target boxes, categories, and individual bird identities;

[0013] Step S22: Process the training data based on the bird recognition model to be trained to obtain the detection box coordinates, confidence scores, bird classifications, and individual bird recognition results containing the animals;

[0014] Step S23: Determine the total loss L of the bird recognition model based on the following formula. bird :

[0015] L bird =λ coord *L coord +λ conf *L conf +λ cls *L cls +λ Triplet *L Triplet ;

[0016] Among them, L coord L conf L cls The detection branch calculates the bounding box coordinate regression loss, confidence loss, and classification loss, respectively. Triplet The triplet loss calculated for the re-identification branch, λ coord , λ conf , λ cls , λ Triplet These are the weights of the bounding box coordinate regression loss, confidence loss, classification loss, and triplet loss, respectively.

[0017] Step S314, based on total loss L bird Update the weight parameters to be optimized in the bird recognition model and the embedding matrix to obtain the final trained bird recognition model.

[0018] Preferably, step S4 further includes:

[0019] Step S41: Obtain individual bird identification feature vectors based on the bird recognition model;

[0020] Step S42: Calculate the distance between the individual identity feature vector and each sample feature vector in the database used to represent different bird individuals, and determine the sample feature vector with the smallest distance, thereby determining the individual identity of the bird.

[0021] Preferably, in step S5, issuing the driving signal includes:

[0022] The device drives the sound or light equipment installed on the bird feeder to generate sound and light scare signals, and at the same time drives the feeding trough baffle to close.

[0023] Preferably, step S5 further includes:

[0024] When the identified animal species is a bird, the system selects matching parameters set by the user based on the individual bird's identity for targeted feeding. These parameters include the type of food, the amount of food, and the feeding time.

[0025] Preferably, step S5 further includes:

[0026] Step S6: Based on the set interaction parameters or in response to the user's interaction request, remotely send the individual bird's identity, the corresponding feeding data of the individual bird, and the monitoring video to the user, and at the same time, statistically analyze the appearance or feeding frequency of different individual birds.

[0027] A second aspect of this application discloses an interactive intelligent bird-feeding device based on image recognition, which mainly includes:

[0028] The image acquisition module is used to continuously acquire images of the bird feeding area via a camera;

[0029] The original feature vector extraction module is used to extract image features through the backbone network of a pre-trained bird recognition model to form the original feature vector;

[0030] The species identification module is used to identify the species of the original feature vector through the detection branch of a pre-trained bird identification model, and to map the identified species data into a continuous vector of a specified dimension through an embedding matrix, wherein the dimension of the continuous vector is the same as the dimension of the original feature vector.

[0031] The bird individual identification module is used to superimpose the continuous vector with the original feature vector to form a new feature vector, which is then input into the re-identification branch of the pre-trained bird identification model to obtain the individual bird identification.

[0032] The feeding control module is used to record the individual bird's identity and feeding data when the identified animal species is a bird, and to issue a shooing signal when the identified animal species is a non-bird.

[0033] Preferably, the animal detection model is further trained using the following units:

[0034] The training data annotation unit is used to acquire images containing different animal types, backgrounds, poses, and lighting conditions as training data, and to annotate the training data with bird target boxes, categories, and individual bird identities.

[0035] The actual output acquisition unit of the model is used to process the training data based on the bird recognition model to be trained, and obtain the detection box coordinates, confidence scores, bird classification and individual bird recognition results containing the animal;

[0036] The loss determination unit is used to determine the total loss L of the bird recognition model based on the following formula. bird :

[0037] L bird =λ coord *L coord +λ conf *L conf +λ cls *L cls +λ Triplet *L Triplet ;

[0038] Among them, L coord L conf L cls The detection branch calculates the bounding box coordinate regression loss, confidence loss, and classification loss, respectively. Triplet The triplet loss calculated for the re-identification branch, λ coord , λ conf , λ cls , λ Triplet These are the weights of the bounding box coordinate regression loss, confidence loss, classification loss, and triplet loss, respectively.

[0039] A loop optimization unit is used to optimize based on the total loss L. bird Update the weight parameters to be optimized in the bird recognition model and the embedding matrix to obtain the final trained bird recognition model.

[0040] Preferably, the bird individual identification module includes:

[0041] The identity feature vector output unit is used to obtain the individual identity feature vector of a bird based on the bird recognition model;

[0042] The individual identification unit is used to calculate the distance between the individual identification feature vector and each sample feature vector in the database used to represent different bird individuals, and to determine the sample feature vector with the smallest distance, thereby determining the individual identification of the bird.

[0043] Preferably, the feeding control module is configured to connect to and drive the audio or lighting equipment installed on the bird feeder to generate sound and light deterrent signals, while simultaneously driving the feeding trough baffle to close.

[0044] Preferably, the feeding control module further includes:

[0045] The directional feeding control unit is used to select matching parameters set by the user based on the individual bird's identity when the identified animal species is a bird. The parameters include the type of food to be fed, the amount of food to be fed, and the feeding time.

[0046] Preferably, the image recognition-based interactive intelligent bird feeding device further includes:

[0047] The bird activity pattern statistics module is used to remotely send the individual bird's identity, the corresponding bird's feeding data, and monitoring video to the user based on the set interaction parameters or in response to the user's interaction request. At the same time, it can also count the frequency of appearance or feeding of different bird individuals.

[0048] A third aspect of this application is a computer device comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, the processor executing the computer program to implement the image recognition-based interactive intelligent bird feeding method as described in any of the preceding claims.

[0049] A fourth aspect of this application is a readable storage medium storing a computer program that, when executed by a processor, is used to implement the image recognition-based interactive intelligent bird feeding method described above.

[0050] This application enables timed and quantitative feeding of different bird individuals, making it more intelligent and convenient to interact with, thus improving the user experience. Attached Figure Description

[0051] Figure 1 This is a flowchart of a preferred embodiment of the interactive intelligent bird feeding method based on image recognition in this application.

[0052] Figure 2 This application Figure 1 The bird recognition model architecture diagram of the embodiment shown. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions in the embodiments of this application will be described in more detail below with reference to the accompanying drawings. In the drawings, the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The described embodiments are only some, not all, of the embodiments of this application. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application. The embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0054] According to the first aspect of this application, Figure 1 As shown, an interactive intelligent bird-feeding method based on image recognition mainly includes:

[0055] Step S1: Continuously acquire images of the bird feeding area using a camera.

[0056] In this step, images of the bird feeding area are continuously acquired using image acquisition devices, such as cameras, which can capture images or videos of the bird feeding area for subsequent analysis. Multiple bird feeding areas may be included, and comprehensive image acquisition can be performed using one or more cameras.

[0057] This application primarily focuses on interactive intelligent bird feeding. Therefore, the camera is typically mounted above the bird feeder, which mainly includes a feeding trough, a tray for birds to stand on, and a feeding actuator. The camera is installed to capture images of the feeding area, primarily to determine whether birds or other animals are occupying the feeding trough or tray, allowing for targeted food feeding.

[0058] Step S2: Extract image features through the backbone network of the pre-trained bird recognition model to form the original feature vector.

[0059] This application constructs a bird recognition model based on an image neural network. In step S2, the backbone network is responsible for extracting image features. To improve its image feature extraction capability, it can be pre-trained on the ImageNet dataset.

[0060] Step S3: The original feature vector is used to identify the species through the detection branch of the pre-trained bird identification model, and the identified species data is mapped into a continuous vector of a specified dimension through an embedding matrix. The dimension of the continuous vector is the same as the dimension of the original feature vector.

[0061] Step S4: Superimpose the continuous vector with the original feature vector to form a new feature vector, and input it into the re-identification branch of the pre-trained bird recognition model to obtain the individual identity of the bird.

[0062] The bird recognition models involved in steps S2-S4 are as follows: Figure 2 As shown, it mainly includes three parts: a backbone network, a detection branch, and a re-identification branch. The original feature vectors output by the backbone network are sent to the detection branch and the re-identification branch respectively. In step S3, the detection branch is used to identify various birds or other animals. In step S4, the re-identification branch is used to identify the individual bird.

[0063] In this application, the detection branch and the re-identification branch share the original feature vector output by the backbone network. A category embedding layer is designed to embed the animal category information from the detection branch into the feature representation of the re-identification branch, providing richer and more discriminative features and enhancing the re-identification branch's ability to distinguish different birds. The specific implementation of this embedding layer uses an embedding matrix, taking the bird category value as input and mapping it to a low-dimensional continuous vector space, as shown in the following equation.

[0064] bird_embedding=Embedding(bird_class,emd_dim);

[0065] Here, bird_class represents the input bird category information. The Embedding function embeds it into a vector of dimension emd_dim. Then, in the re-identification branch, the embedded category information is added to the original feature vector to generate a new feature representation for use by the re-identification branch, as shown in the following formula.

[0066] new_feature=Add(origin_feature,bird_embedding)

[0067] Here, the category information and the original features are fused through the Add operation, and the dimension of the embedded vector is the same as the dimension of the original feature vector of the re-identification branch.

[0068] For example, assuming the original feature vector is 1024-dimensional data, step S3 obtains a 3-dimensional category feature vector, such as [1,0,0] representing a magpie, [0,1,0] representing a pigeon, and [0,0,1] representing a sparrow. Then the embedding matrix is ​​a 1024-by-3 matrix. In step S4, this matrix is ​​multiplied by the 3-dimensional category feature vector (converted to a 3-by-1 matrix) to form a 1024-dimensional continuous vector. This 1024-dimensional continuous vector can then be superimposed on the 1024-dimensional original feature vector in step S4.

[0069] In this embodiment, each value within the embedded matrix is ​​obtained during the training of the bird recognition model. In some optional embodiments, prior to step S2, the bird recognition model is further trained through the following steps:

[0070] Step S21: Obtain images containing different animal types, backgrounds, poses, and lighting conditions as training data, and annotate the training data with bird target boxes, categories, and individual bird identities;

[0071] Step S22: Process the training data based on the bird recognition model to be trained to obtain the detection box coordinates, confidence scores, bird classifications, and individual bird recognition results containing the animals;

[0072] Step S23: Determine the total loss L of the bird recognition model based on the following formula. bird :

[0073] L bird =λ coord *L coord +λ conf *L conf +λ cls *L cls +λ Triplet *L Triplet ;

[0074] Among them, L coord L conf L cls The detection branch calculates the bounding box coordinate regression loss, confidence loss, and classification loss, respectively. Triplet The triplet loss calculated for the re-identification branch, λ coord , λ conf , λ cls , λ Triplet These are the weights of the bounding box coordinate regression loss, confidence loss, classification loss, and triplet loss, respectively.

[0075] Step S24, based on total loss L birdUpdate the weight parameters to be optimized in the bird recognition model and the embedding matrix to obtain the final trained bird recognition model.

[0076] In this embodiment, the bird recognition model can be a ResNet model, or other backbone networks such as VGG, Inception, EfficientNet, MobileNet, ShuffleNet, etc. In this model, triplet loss is used to optimize the model parameters, making feature vectors of the same identity as close together as possible, and feature vectors of different identities as dispersed as possible. The model is trained iteratively multiple times, and the model weights are updated to reduce the loss function and ensure model convergence. After training, the original feature vectors are extracted in step S2, the animal category (e.g., magpie, pigeon) is output in step S3, and the individual feature vectors are output in step S4. Based on the output individual feature vectors, the most suitable animal individual identity is determined, for example, magpie 1 and magpie 2.

[0077] In some alternative implementations, step S4 further includes:

[0078] Step S41: Obtain individual bird identification feature vectors based on the bird recognition model;

[0079] Step S42: Calculate the distance between the individual identity feature vector and each sample feature vector in the database used to represent different bird individuals, and determine the sample feature vector with the smallest distance, thereby determining the individual identity of the bird.

[0080] In this embodiment, the distance is, for example, a cosine distance.

[0081] Step S5: When the identified animal species is a bird, record the individual bird's identity and its feeding data; when the identified animal species is not a bird, issue a shooing signal.

[0082] In some alternative implementations, when the identified animal species is a bird, the user-defined parameters are selected to provide targeted feeding based on the individual bird's identity. These parameters include the type of food to be fed, the amount of food to be fed, and the feeding time.

[0083] In some alternative implementations, step S5 is further followed by:

[0084] Step S6: Based on the set interaction parameters or in response to the user's interaction request, remotely send the individual bird's identity, the corresponding feeding data of the individual bird, and the monitoring video to the user, and at the same time, statistically analyze the appearance or feeding frequency of different individual birds.

[0085] In the two embodiments described above, a user interaction module is first constructed, allowing users to connect to the bird feeding system via a mobile application or web interface. This module primarily includes the following functions:

[0086] (1) Users can view real-time images. For example, users can monitor the images captured by the camera in real time through a mobile application or network interface to understand the current bird activity.

[0087] (2) Receive system push notifications to understand the current status of the bird feeder, such as whether squirrels have appeared, whether new birds or individuals have visited, and whether bird food needs to be replenished.

[0088] (3) Set feeding targets and food. Users can set specific birds or individuals as feeding targets and select different birdseed for different birds. The system will feed them according to the recognition results.

[0089] (4) View records. Users can view system records at any time, including feeding history, frequency of appearance of various birds, etc., to understand the patterns of bird activity.

[0090] (5) Personalized settings: Users can make personalized settings based on the records and analysis results of the bird feeding system, and adjust parameters such as feeding amount, feeding food, and feeding time to meet the needs of different birds.

[0091] (6) Socialize with other users, such as sharing your bird photos or videos to the community, viewing other users' bird observation logs, and participating in bird-related topics or activities.

[0092] In some alternative implementations, step S5, issuing the eviction signal, includes:

[0093] The device drives the sound or light equipment installed on the bird feeder to generate sound and light scare signals, and at the same time drives the feeding trough baffle to close.

[0094] In this embodiment, in order to prevent rodents such as squirrels from occupying the bird feeder, when a non-avian animal is detected, the system issues a shooing signal, which may be sound, light or other stimuli, and closes the feeding trough baffle.

[0095] In some alternative implementations, the temperature and humidity when the birds are feeding are obtained by sensors pre-installed on the feeder.

[0096] In this embodiment, during the interaction with the user, the recording and data transmission of environmental factors, such as temperature, humidity, and geographic coordinates, are increased to improve the in-depth understanding of bird behavior.

[0097] This device can efficiently identify squirrels, various birds, and different individual birds. At the same time, users can use the interactive module to feed birds in real time and record bird information, thus meeting the needs of different users and birds and realizing intelligent bird feeding.

[0098] A second aspect of this application provides an interactive intelligent bird-feeding device based on image recognition, corresponding to the above-described method, mainly comprising:

[0099] The image acquisition module is used to continuously acquire images of the bird feeding area via a camera;

[0100] The original feature vector extraction module is used to extract image features through the backbone network of a pre-trained bird recognition model to form the original feature vector;

[0101] The species identification module is used to identify the species of the original feature vector through the detection branch of a pre-trained bird identification model, and to map the identified species data into a continuous vector of a specified dimension through an embedding matrix, wherein the dimension of the continuous vector is the same as the dimension of the original feature vector.

[0102] The bird individual identification module is used to superimpose the continuous vector with the original feature vector to form a new feature vector, which is then input into the re-identification branch of the pre-trained bird identification model to obtain the individual bird identification.

[0103] The feeding control module is used to record the individual bird's identity and feeding data when the identified animal species is a bird, and to issue a shooing signal when the identified animal species is a non-bird.

[0104] In some alternative implementations, the animal detection model is further trained using the following units:

[0105] The training data annotation unit is used to acquire images containing different animal types, backgrounds, poses, and lighting conditions as training data, and to annotate the training data with bird target boxes, categories, and individual bird identities.

[0106] The actual output acquisition unit of the model is used to process the training data based on the bird recognition model to be trained, and obtain the detection box coordinates, confidence scores, bird classification and individual bird recognition results containing the animal;

[0107] The loss determination unit is used to determine the total loss L of the bird recognition model based on the following formula. bird :

[0108] L bird =λ coord *L coord +λ conf *L conf +λcls *L cls +λ Triplet *L Triplet ;

[0109] Among them, L coord L conf L cls The detection branch calculates the bounding box coordinate regression loss, confidence loss, and classification loss, respectively. Triplet The triplet loss calculated for the re-identification branch, λ coord , λ conf , λ cls , λ Triplet These are the weights of the bounding box coordinate regression loss, confidence loss, classification loss, and triplet loss, respectively.

[0110] A loop optimization unit is used to optimize based on the total loss L. bird Update the weight parameters to be optimized in the bird recognition model and the embedding matrix to obtain the final trained bird recognition model.

[0111] In some alternative implementations, the bird individual identification module includes:

[0112] The identity feature vector output unit is used to obtain the individual identity feature vector of a bird based on the bird recognition model;

[0113] The individual identification unit is used to calculate the distance between the individual identification feature vector and each sample feature vector in the database used to represent different bird individuals, and to determine the sample feature vector with the smallest distance, thereby determining the individual identification of the bird.

[0114] In some alternative implementations, the feeding control module is configured to connect to and drive an audio or lighting device installed on the bird feeder to generate sound and light deterrent signals, while simultaneously driving the feeding trough baffle to close.

[0115] In some optional embodiments, the feeding control module further includes:

[0116] The directional feeding control unit is used to select matching parameters set by the user based on the individual bird's identity when the identified animal species is a bird. The parameters include the type of food to be fed, the amount of food to be fed, and the feeding time.

[0117] In some alternative embodiments, the image recognition-based interactive intelligent bird feeding device further includes:

[0118] The bird activity pattern statistics module is used to remotely send the individual bird's identity, the corresponding bird's feeding data, and monitoring video to the user based on the set interaction parameters or in response to the user's interaction request. At the same time, it can also count the frequency of appearance or feeding of different bird individuals.

[0119] In a third aspect of this application, a computer device is provided, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement an interactive intelligent bird feeding method based on image recognition.

[0120] In a fourth aspect, this application provides a readable storage medium storing a computer program that, when executed by a processor, implements the image recognition-based interactive intelligent bird-feeding method described above. This computer-readable storage medium may be included in the apparatus described in the above embodiments; or it may exist independently and not incorporated into the apparatus. The aforementioned computer-readable storage medium carries one or more programs that, when executed by the apparatus, process data according to the described method.

[0121] Computer devices include a central processing unit (CPU), which performs various appropriate actions and processes based on programs stored in read-only memory (ROM) or loaded from memory into random access memory (RAM). RAM also stores various programs and data required for device operation. The CPU, ROM, and RAM are interconnected via a bus. Input / output (I / O) interfaces are also connected to the bus.

[0122] The following components are connected to the I / O interface: input sections including keyboards, mice, etc.; output sections including cathode ray tubes (CRTs), liquid crystal displays (LCDs), and speakers; storage sections including hard disks; and communication sections including network interface cards such as LAN cards and modems. The communication sections perform communication processing via networks such as the Internet. Drives are also connected to the I / O interface as needed. Removable media, such as disks, optical disks, magneto-optical disks, semiconductor memories, etc., are installed on the drive as needed so that computer programs read from them can be installed into the storage section as required.

[0123] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication component, and / or installed from a removable medium. When the computer program is executed by a central processing unit (CPU), it performs the functions defined in the methods of this application. It should be noted that the computer storage medium of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on a computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0124] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0125] The modules or units described in the embodiments of this application can be implemented in software or hardware. The described modules or units can also be located in a processor, and the names of these modules or units do not necessarily constitute a limitation on the module or unit itself.

[0126] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An image recognition-based interactive intelligent bird feeding method, characterized in that, The method comprises the following steps: Step S1, continuously acquiring images of a bird feeding area through a camera; Step S2, extracting image features through a backbone network of a pre-trained bird recognition model to form an original feature vector; Step S3, performing species identification on the original feature vector through a detection branch of the pre-trained bird recognition model, and mapping the identified species data into a continuous vector of a specified dimension through an embedding matrix, wherein the dimension of the continuous vector is the same as that of the original feature vector; Step S4, superimposing the continuous vector and the original feature vector to form a new feature vector, inputting the new feature vector into a re-identification branch of the pre-trained bird recognition model, and obtaining an individual identity of the bird; Step S5, when the identified animal species is a bird, recording the individual identity of the bird and its feeding data, and when the identified animal species is a non-bird, issuing a driving signal; Before step S2, the method further comprises training the bird recognition model through the following steps: Step S21, acquiring images containing different animal types, different backgrounds, different postures, and different illumination conditions as training data, and labeling the training data with bird target boxes, categories, and individual identities of birds; Step S22, processing the training data based on the bird recognition model to be trained to obtain detection box coordinates, confidence, bird classification, and bird individual identification results of the animal; Step S23, determine the total loss L of the bird recognition model based on the following formula bird : L bird =λ coord *L coord +λ conf *L conf +λ cls *L cls +λ Triplet *L Triplet ; wherein, L coord , L conf , L cls are respectively a bounding box coordinate regression loss, a confidence loss and a classification loss calculated by the detection branch, L Triplet is a triplet loss calculated by the re-identification branch, and λ coord , λ conf , λ cls , λ Triplet are respectively weights of the bounding box coordinate regression loss, the confidence loss, the classification loss and the triplet loss. Step S24, based on the total loss L bird The to-be-optimized weight parameters of the bird recognition model and the embedding matrix are updated to obtain a final trained bird recognition model.

2. The image recognition based interactive intelligent bird feeding method of claim 1, wherein, Step S4 further comprises: Step S41, obtaining an individual identity feature vector of the bird based on the bird recognition model; Step S42, respectively calculating distances between the individual identity feature vector and each sample feature vector in a database representing different bird individuals to determine a sample feature vector with the smallest distance, thereby determining the individual identity of the bird.

3. The image recognition based interactive intelligent bird feeding method of claim 1, wherein, In step S5, the driving signal comprises: Driving a sound device or a light device installed on the bird feeder to generate sound driving information and light driving signals, and simultaneously driving the closing of the feeding trough shutter.

4. The image recognition based interactive intelligent bird feeding method of claim 1, wherein, Step S5 further comprises: When the identified animal species is a bird, selecting a matching parameter set by a user for targeted feeding according to the identified individual identity of the bird, wherein the parameter comprises a food species, a feeding amount, and a feeding time.

5. The image recognition based interactive intelligent bird feeding method of claim 1, wherein, Step S5 further comprises: Step S6, remotely sending the individual identity of the bird, the feeding data of the corresponding bird individual, and the monitoring video to the user according to the set interaction parameter or in response to the user's interaction request, and simultaneously counting the appearance or feeding frequency of different bird individuals.

6. An image recognition-based interactive intelligent bird feeding device, characterized in that, The method comprises the following steps: An image acquisition module for continuously acquiring images of a bird feeding area through a camera; An original feature vector extraction module for extracting image features through a backbone network of a pre-trained bird recognition model to form an original feature vector; A species identification module for performing species identification on the original feature vector through a detection branch of the pre-trained bird recognition model, and mapping the identified species data into a continuous vector of a specified dimension through an embedding matrix, wherein the dimension of the continuous vector is the same as that of the original feature vector; A re-identification branch of the pre-trained bird recognition model, and obtaining an individual identity of the bird; The bird individual identity recognition module is configured to superimpose the continuous vector and the original feature vector to form a new feature vector, input the new feature vector into a re-identification branch of a pre-trained bird recognition model, and obtain the individual identity of the bird; The feeding control module is configured to record the individual identity of the bird and feeding data of the bird when the identified animal species is the bird, and send a driving signal when the identified animal species is a non-bird; The device trains the animal detection model through the following units: A training data labeling unit is configured to obtain images under different animal types, different backgrounds, different postures, and different light conditions as training data, and label the training data with a bird target frame, a category, and a bird individual identity. A model actual output obtaining unit is configured to process the training data based on a bird recognition model to be trained to obtain a detection frame coordinate, a confidence, a bird classification, and a bird individual recognition result of an animal. A loss determination unit is configured to determine a total loss L of the bird identification model based on the following formula bird : L bird =λ coord *L coord +λ conf *L conf +λ cls *L cls +λ Triplet *L Triplet ; wherein, L coord , L conf , L cls are respectively a bounding box coordinate regression loss, a confidence loss and a classification loss calculated by the detection branch, L Triplet is a triplet loss calculated by the re-identification branch, λ coord , λ conf , λ cls , λ Triplet are respectively weights of the bounding box coordinate regression loss, the confidence loss, the classification loss and the triplet loss. a cycle optimization unit configured to optimize the total loss L bird updating the weight parameters to be optimized of the bird recognition model and the embedding matrix to obtain a final trained bird recognition model.

7. The image recognition based interactive intelligent bird feeding device of claim 6, wherein, The bird individual identity recognition module includes: An identity feature vector output unit is configured to obtain an individual identity feature vector of a bird based on a bird recognition model. An individual identity determination unit is configured to calculate distances between the individual identity feature vector and each sample feature vector in a database for representing different bird individuals, respectively, determine a sample feature vector with a minimum distance, and determine the individual identity of the bird.

8. The image recognition based interactive intelligent bird feeding device of claim 6, wherein, The feeding control module is configured to connect and drive a sound device or a light device installed on the bird feeder to generate sound driving information and light driving signals, and simultaneously drive a closing of a feeding trough shutter.

Images