A low-power bird identification method and system based on millimeter wave radar and image recognition

Abstract

The application discloses a kind of low-power consumption bird identification method and system based on millimeter wave radar and image recognition, it is related to bird monitoring technical field.The method includes: normal control millimeter wave radar low-power value guard, complete suspected bird target preliminary screening by multi-feature fusion trigger model, only preliminary screening qualified when awakening camera;Lightweight model is completed target fine screening by adopting the lightweight model of bird feature contribution degree pruning, mixed precision quantization, directional knowledge distillation, double-template matching optimization, combined with radar-image collaborative verification double check;Invalid trigger is adaptively adjusted radar threshold, combined with three-level power adaptive energy-saving control, realize whole link low-power management and control.The application greatly reduces the invalid power consumption and false alarm rate of system, is suitable for airport, power transmission line and other outdoor scene bird monitoring, guarantees off-grid long time stable value guard.

Description

Technical Field

[0001] This invention relates to the field of bird monitoring technology, and in particular to a low-power bird identification method and system based on millimeter-wave radar and image recognition. Background Technology

[0002] Bird surveillance is a core component of ensuring civil aviation safety, power facility safety, and agricultural production. Current bird surveillance technologies suffer from the following key shortcomings: (1) Pure camera solution has high power consumption and poor reliability: it needs to run continuously for 24 hours and cannot be guarded for a long time in the field of off-grid scenarios; and it is easily affected by light and weather, with a very high false alarm rate.

[0003] (2) The pure radar solution has a high false trigger rate: it can only detect moving targets and cannot distinguish between birds and wild interference objects such as plastic bags, drones, and kites, resulting in a large amount of invalid power consumption and invalid alarms.

[0004] (3) Imbalance between lightweight model and accuracy: Existing large bird recognition models cannot run on embedded devices. After conventional pruning and quantization, the accuracy drops significantly and cannot meet the recognition needs of complex outdoor scenes.

[0005] (4) No adaptive threshold mechanism: The radar trigger threshold is fixed, and it is frequently falsely triggered even when there are no birds for a long time, resulting in high power consumption.

[0006] Therefore, there is an urgent need for a low-power, high-accuracy, strong anti-interference, and adaptive intelligent bird recognition solution to address the core pain points of existing technologies. Summary of the Invention

[0007] The purpose of this invention is to overcome the technical problems of high power consumption and poor reliability of pure camera solutions, high false trigger rate of pure radar solutions, imbalance between lightweight models and accuracy, and lack of adaptive threshold mechanism in the existing technology, and to provide a low-power bird recognition method and system based on millimeter-wave radar and image recognition.

[0008] According to a first aspect of the present invention, a low-power bird identification method based on millimeter-wave radar and image recognition is provided. The method includes: Step S1, under normal conditions, the control layer sends a low-power monitoring command to the perception layer, regulating the millimeter-wave radar mounted on the perception layer to enter a low-power acquisition state; Step S2, the millimeter-wave radar acquires a set of feature parameters in real time and synchronously transmits them back to the control layer cache; Step S3, the control layer calculates a target fusion value based on the set of feature parameters using a pre-installed multi-feature fusion triggering model; Step S4, the control layer completes the initial screening of suspected bird targets by comparing the target fusion value with a pre-stored trigger threshold. After the initial screening is qualified, a wake-up command is issued to trigger the camera mounted on the perception layer to start and acquire target visual image data; Step S5, the camera transmits the target visual image data to an algorithm layer equipped with a customized lightweight image recognition model to complete the identification of birds and interference targets. Step S6: The control layer performs dual verification by substituting the target fusion value and the bird target screening result into a preset radar-image feature collaborative verification model. Based on the verification result, the control layer determines whether the target is a valid bird target or an interfering target. Step S7: If the target is determined to be a valid bird target, the control layer immediately wakes up the communication layer module, performs lightweight compression processing on the target image data, uploads it to the Web monitoring platform of the application layer, and triggers a remote alarm. Step S8: If the target is determined to be an interfering target, or if no valid target is detected after the camera is woken up, the control layer issues a sleep command to the camera in the perception layer. The control layer also automatically adjusts the pre-stored millimeter-wave radar trigger threshold according to preset quantization rules based on the invalid trigger data. Step S9: The control layer collects the remaining SOC power of the power supply battery in real time and performs three-level adaptive energy-saving control based on the SOC value.

[0009] Furthermore, the feature parameter set includes one or a combination of target flight speed, radar cross-section, azimuth change, and range change; the mathematical expression of the multi-feature fusion triggering model is: ; In the formula, The target fusion value; , , and These are the feature weights for target flight speed, radar cross-section, azimuth change, and range change, respectively. v represents the target's flight speed, in m / s; Here, is the average flight speed of birds, in m / s; s is the actual radar cross-section of the target, in m / s. ; This is a reference value for the average reflective area of ​​birds, in units of ; The change in azimuth angle of the target over three consecutive frames; This represents the maximum permissible variation in azimuth. The distance change of the target over three consecutive frames, in meters; This represents the maximum permissible variation in distance, expressed in meters (m). Step S4 further includes: the control layer retrieving its pre-stored consecutive confirmation frame count parameter, and calculating the average fusion value of multiple consecutive frames based on the target fusion value using the anti-shake frame confirmation formula. The formula for confirming the stabilized frame is: ; In the formula, N is the number of consecutive frames. The target fusion value corresponding to the i-th frame; Only when At that time, the control layer determines that the initial screening is qualified and issues a camera wake-up command. Pre-store trigger thresholds for the control layer.

[0010] Furthermore, the customized lightweight image recognition model is a four-layer quantitative optimization process for the lightweight backbone network, using bird feature template vectors as the core benchmark, and involving bird feature contribution pruning, mixed precision quantization, directional knowledge distillation, and dual-template feature matching. The bird feature contribution pruning involves iterating through all convolutional kernels of the lightweight backbone network, calculating the contribution of each convolutional kernel to bird feature extraction based on a preset bird feature contribution pruning formula, and directly discarding convolutional kernels whose contribution is less than or equal to a preset contribution threshold. The bird feature contribution pruning formula is as follows: ; In the formula, The total number of convolution kernels, The number of parameters in a single convolution kernel. For the j-th parameter value of the k-th convolutional kernel, This is a bird feature template vector trained based on a dataset of common wild birds.

[0011] Furthermore, the hybrid precision quantization optimization uses the bird feature template vector as a benchmark to distinguish the importance of network layers in bird feature extraction, thereby achieving hierarchical precision control. The hierarchical precision control includes the following steps: traversing all network layers of the pruned model and extracting the output feature vector of each layer. ,calculate Bird feature template vector The cosine similarity is used to select the network layer output feature vectors with the highest similarity. The normalized similarity of each layer is calculated using a mixed-precision quantization precision weight formula. The formula for the precision weighting of the mixed precision quantization is: ; in This is the function for calculating cosine similarity. Set feature layer decision threshold ,when When this layer is identified as a key feature layer for birds, 16-bit fixed-point quantization is used; when When the layer is determined to be a background-independent layer, 8-bit fixed-point quantization is used; after quantization, calibration is performed using real image data of the monitored scene to correct the quantization error.

[0012] Furthermore, the targeted knowledge distillation adopts a teacher-student model architecture. The teacher model is a high-precision deep learning model trained on a large number of bird and interference samples, and the student model is a customized lightweight model optimized by pruning and mixed precision quantization. The knowledge distillation uses a customized loss function, focusing on the targeted transfer of bird feature recognition experience. The customized loss function is as follows: ; in The total training loss for the student model. For cross-entropy loss, For knowledge distillation loss, Weights for knowledge transfer related to bird features; The bird feature knowledge transfer weight The CIoU loss between birds and the background, calculated using the teacher model, is used to enable the student model to learn the bird feature distribution mastered by the teacher model, rather than the background feature distribution. The calculation formula is as follows: ; in For the teacher model, the CIoU loss for birds, The CIoU loss of the teacher model against the background; The distillation rule is as follows: the loss function of the student model is... Learn cross-entropy loss for the weights, The knowledge distillation loss is learned by weighting the model, focusing only on the learning of bird features. This avoids interference from background learning in student models, ensuring bird recognition accuracy while keeping the model lightweight, and adapting to the low-power operation requirements of embedded low-computing platforms.

[0013] Furthermore, the dual-template feature matching employs quantitative discrimination rules, using a bird-interference target matching formula to determine the target. This includes, after feature extraction is completed by the optimized model, quantitatively distinguishing bird targets from interference targets using the dual-template matching formula, outputting the determination result, and sending it back to the control layer. The bird-interference target matching formula is as follows: ; like If the target is identified as a bird, it is considered a dart; otherwise, it is considered a distracting target. In the formula, To detect the feature matching degree between the target and the bird template; To detect the feature matching degree between the target and the interfering target template; This is the image feature vector of the currently detected target; The feature template vector for the interference target is obtained by training on a dataset of common interference objects in the wild. This is the margin threshold.

[0014] Furthermore, the mathematical expression of the preset radar-image feature co-verification model in step S6 includes: ; when When the target is in the correct position, it is considered a valid bird target; otherwise, it is considered a distracting target. In the formula, and These are radar feature weights and image feature weights, respectively. , , This is the preset verification threshold; The preset quantization rules in step S8 include: ; In the formula, The pre-stored trigger threshold before adjustment, The adjusted pre-stored trigger threshold, Here, t is a preset coefficient, and t is the duration of no effective target. This is the preset base time.

[0015] Furthermore, step S9 includes: when SOC > first threshold, the system operates in normal mode, the radar maintains a preset normal acquisition frequency, and the camera maintains a preset wake-up delay; when second threshold < SOC ≤ first threshold, the system enters intermediate power-saving mode, the control layer reduces the radar acquisition frequency and extends the camera wake-up delay to reduce the overall power consumption of the system; when SOC ≤ second threshold, the system enters deep power-saving mode, retaining only the low-power acquisition function of the millimeter-wave radar, and prohibiting the wake-up of the camera, algorithm layer and communication layer to achieve minimum power consumption operation.

[0016] Furthermore, the method also includes: real-time acquisition of the actual average brightness of the image; based on the actual average brightness of the image, determining whether the scene is in low light or backlight; the control layer sends an image enhancement command to the camera of the perception layer; and adjusts the camera gain through a contrast correction formula to improve image recognition accuracy. The contrast correction formula includes: ; In the formula, For the corrected gain, Gain for the original image. This represents the actual average brightness of the image. This is a normal brightness reference value. This is the enhancement coefficient.

[0017] According to a second aspect of the present invention, a low-power bird identification system based on millimeter-wave radar and image recognition is provided, implemented using the aforementioned low-power bird identification method based on millimeter-wave radar and image recognition. The system comprises five collaborative modules: a perception layer, a control layer, an algorithm layer, a communication layer, and an application layer. The perception layer is equipped with a millimeter-wave radar and a camera. The millimeter-wave radar is used to collect a set of feature parameters in real time, including target flight speed, radar cross-section, azimuth change, and distance change. The camera is used to collect target visual image data. The control layer is equipped with an embedded main control module, which integrates a multi-feature fusion triggering model. This model calculates a target fusion value based on the feature parameter set, compares the target fusion value with a pre-stored trigger threshold to complete the initial screening of suspected bird targets, and issues a wake-up command after the initial screening is successful, triggering the camera on the perception layer to start. The system also substitutes the target fusion value and the bird target fine screening result into a preset radar-image feature collaborative verification model for double verification, and determines whether the target is a valid bird target or a interference target based on the verification result. If the target is determined to be valid... Upon detecting a bird target, the control layer immediately sends a wake-up signal to the communication module of the communication layer. If the target is determined to be an interference target, or if no valid target is detected after the camera is woken up, the control layer issues a sleep command to the camera in the perception layer. Furthermore, based on the invalid trigger data, the control layer automatically adjusts the pre-stored millimeter-wave radar trigger threshold according to preset quantization rules. Additionally, the control layer collects the remaining SOC (State of Charge) of the power supply battery in real time and performs three-level adaptive energy-saving control based on the SOC value. The algorithm layer is equipped with a customized lightweight image recognition model. This model is designed for a lightweight backbone network, using bird feature template vectors as the core benchmark. After four layers of quantitative optimization—bird feature contribution pruning, mixed precision quantization, directional knowledge distillation, and dual-template feature matching—it quantitatively distinguishes between birds and interference targets based on the target visual image data, obtaining a refined screening result for bird targets, which is then transmitted back to the control layer. The communication layer is equipped with a communication module used to perform lightweight compression processing on the target image data based on the wake-up signal from the control layer and upload it to the web monitoring platform of the application layer, triggering a remote alarm.

[0018] According to a third aspect of the present invention, an electronic device is provided, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the aforementioned low-power bird recognition method based on millimeter-wave radar and image recognition.

[0019] According to a fourth aspect of the invention, a readable storage medium is provided, which, when the instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to perform the aforementioned low-power bird recognition method based on millimeter-wave radar and image recognition.

[0020] The advantages of this invention compared to the prior art are as follows: (1) The present invention adopts an event-driven mechanism of normal monitoring of millimeter-wave radar and deep sleep of camera and communication module. The subsequent modules are only woken up when the radar initially screens and determines that it is a suspected target, which greatly reduces the power consumption from the source. Combined with the adaptive threshold calibration of the duration without target and the three-level low power adaptive control mechanism, it can achieve long-term stable monitoring in the off-grid power supply mode of solar energy and lithium battery, which effectively solves the technical defects of traditional bird monitoring equipment such as high power consumption, short battery life and inability to be deployed in the field.

[0021] (2) This invention uses a multi-level discrimination mechanism that combines N-frame anti-shake verification with quantitative determination of dual-template cosine similarity to accurately distinguish birds from wild interference targets such as plastic bags, drones, and dust, avoiding misjudgment problems caused by single sensor detection; by setting a similarity difference judgment threshold, the reliability of target classification is further improved, the false alarm rate of the system is greatly reduced, and the alarm results are real and effective.

[0022] (3) This invention proposes a full-link model optimization scheme for bird recognition scenarios, which includes customized pruning, hybrid precision quantization and directional knowledge distillation. While significantly reducing model parameters and computational load, it retains bird feature recognition capability through dynamic knowledge transfer weight redirection, thus compensating for the precision loss caused by lightweighting. The optimized model can be directly deployed on ARM Cortex-M series embedded chips, meeting the computing power and power consumption constraints of real-time inference on the edge, and breaking through the technical bottleneck that high-precision models cannot run on embedded devices.

[0023] (4) The present invention dynamically calibrates the radar trigger threshold based on the duration of no target. The longer the duration of no effective bird target, the higher the trigger threshold. It can adaptively adapt to the target density changes in different monitoring scenarios. At the same time, it configures image enhancement and clutter correction strategies for complex environments such as low light and backlight, so that the system can maintain stable recognition performance in all weather and multi-scenario field environments, and has strong environmental adaptability and robustness.

[0024] (5) The present invention adopts a dual-template feature matching and teacher-student model distillation learning mechanism to transfer the bird feature recognition experience of the high-precision model to the lightweight model, ensuring that the accuracy of bird target recognition meets the requirements of engineering applications; all judgment processes are implemented by quantitative formulas and numerical thresholds, with clear logic, traceability and debuggability, which facilitates parameter adaptation and system upgrade under different application scenarios. Attached Figure Description

[0025] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0026] Figure 1 This is a flowchart of a low-power bird recognition method based on millimeter-wave radar and image recognition provided in Embodiment 1 of the present invention.

[0027] Figure 2 This is a schematic diagram of a low-power bird recognition system based on millimeter-wave radar and image recognition provided in Embodiment 2 of the present invention.

[0028] Figure 3 This is a partial block diagram of the electronic device provided in Embodiment 3 of the present invention. Detailed Implementation

[0029] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0030] Example 1

[0031] like Figure 1 The diagram shows a low-power bird recognition method based on millimeter-wave radar and image recognition provided in this embodiment of the invention.

[0032] As an example, the method includes: Step S1, under normal conditions, the control layer sends a low-power monitoring command to the perception layer, regulating the millimeter-wave radar on the perception layer to enter a low-power acquisition state; Step S2, the millimeter-wave radar acquires a set of feature parameters in real time, including the target's flight speed, radar cross-section, azimuth change, and distance change, and synchronously transmits the feature parameter set back to the control layer cache; Step S3, the control layer calculates the target fusion value based on the feature parameter set using a pre-installed multi-feature fusion triggering model; Step S4, the control layer completes the initial screening of suspected bird targets by comparing the target fusion value with a pre-stored trigger threshold. After the initial screening is qualified, a wake-up command is issued to trigger the camera on the perception layer to start and acquire target visual image data; Step S5, the camera transmits the target visual image data to the algorithm layer, which is equipped with a customized lightweight image recognition model. The customized lightweight image recognition model is designed for a lightweight backbone network with bird feature template vectors as the core benchmark, and is modified by pruning bird feature contribution, mixed precision quantization, and directional knowledge. After four layers of quantitative optimization including distillation and dual-template feature matching, the target visual image data is used to quantitatively distinguish between birds and interference targets, obtaining the bird target screening result, which is then transmitted back to the control layer. Step S6: The control layer substitutes the target fusion value and the bird target screening result into a preset radar-image feature collaborative verification model for dual verification. Based on the verification result, the target is determined to be a valid bird target or an interference target. Step S7: If the target is determined to be a valid bird target, the control layer immediately wakes up the communication layer module, performs lightweight compression processing on the target image data, uploads it to the Web monitoring platform of the application layer, and triggers a remote alarm. Step S8: If the target is determined to be an interference target, or if no valid target is detected after the camera is woken up, the control layer sends a sleep command to the perception layer camera, and the camera immediately switches to a deep sleep state. Also, based on the invalid trigger data, the control layer automatically adjusts the pre-stored millimeter-wave radar trigger threshold according to the preset quantization rules. Step S9: The control layer collects the remaining SOC power of the power supply battery in real time and performs three-level adaptive energy-saving control based on the SOC value.

[0033] In some feasible implementations, the mathematical expression of the multi-feature fusion triggering model is: ; In the formula, The target fusion value; , , and These are the feature weights for target flight speed, radar cross-section, azimuth change, and range change, respectively. v represents the target's flight speed, in m / s; Here, is the average flight speed of birds, in m / s; s is the actual radar cross-section of the target, in m / s. ; This is a reference value for the average reflective area of ​​birds, in units of ; The change in azimuth angle of the target over three consecutive frames; This represents the maximum permissible variation in azimuth. The distance change of the target over three consecutive frames, in meters; This represents the maximum permissible variation in distance, expressed in meters (m).

[0034] Preferred, When in an airport setting, it increases Increase when in a power scenario ; m; ; ; It should be noted that the above... m, , The specific value is not limited here, and relevant technical personnel can change its value based on actual needs during practical application.

[0035] Preferably, step S4 further includes: the control layer retrieving its pre-stored consecutive confirmation frame number parameter, and calculating the average fusion value of multiple consecutive frames based on the target fusion value using the anti-shake frame confirmation formula. The formula for confirming the stabilized frame is: ; In the formula, N is the number of consecutive frames. The target fusion value corresponding to the i-th frame; Only when At that time, the control layer determines that the initial screening is qualified and issues a camera wake-up command. Pre-store trigger thresholds for the control layer, preferably Avoid invalid triggering caused by single clutter events.

[0036] In some feasible implementations, the bird feature contribution pruning involves sequentially traversing all convolutional kernels of the lightweight backbone network (such as the MobileNetV2 lightweight CNN model), calculating the contribution of each convolutional kernel to the extraction of bird features based on a preset bird feature contribution pruning formula, and directly discarding a convolutional kernel whose contribution is less than or equal to a preset contribution threshold; the bird feature contribution pruning formula is as follows: ; In the formula, The total number of convolution kernels, The number of parameters in a single convolution kernel. For the j-th parameter value of the k-th convolutional kernel, This is a bird feature template vector trained based on a dataset of common wild birds.

[0037] For a specific example, suppose the model has K=100 convolutional kernels, each of which is a 3×3 convolution (M=9). For a 256-dimensional bird feature template: calculate 9 parameters for the k=10th convolutional kernel. to and The dot product of the 100 convolution kernels is summed after taking their absolute values ​​to obtain the numerator; the contributions of all 100 convolution kernels are summed to obtain the denominator; the numerator and denominator are then calculated. =0.008< If the convolution kernel is removed, then the kernel is discarded. If , then retain it. To use bird feature templates trained on a bird dataset from the monitored scenarios as the sole criterion for judging the value of convolutional kernels before system deployment; this was achieved by deleting... Redundant convolutional kernels below a certain threshold are retained, with only those useful for bird recognition retained. The pruned model is smaller and has less computational load, resulting in faster inference speed and lower power consumption on embedded platforms, while bird recognition accuracy remains unaffected, thus reducing computational and power consumption from the source.

[0038] In some feasible implementations, the hybrid precision quantization optimization uses the bird feature template vector as a benchmark to distinguish the importance of network layers in bird feature extraction, thereby achieving hierarchical precision control. The hierarchical precision control includes the following steps: traversing all network layers of the pruned model and extracting the output feature vector of each layer. ,calculate Bird feature template vector The cosine similarity is used to select the network layer output feature vectors with the highest similarity. The normalized similarity of each layer is calculated using a mixed-precision quantization precision weight formula. The formula for the precision weighting of the mixed precision quantization is: ; in This is the function for calculating cosine similarity. Set feature layer decision threshold ,when When this layer is identified as a key feature layer for birds, 16-bit fixed-point quantization is used; when When the layer is determined to be a background-independent layer, 8-bit fixed-point quantization is used; after quantization, calibration is performed using real image data of the monitored scene to correct the quantization error.

[0039] Preferably, quantization is the core method for reducing embedded computing power. This step abandons the crude approach of "globally uniform precision" and quantitatively distinguishes between the "bird feature layer" and the "background layer" through a feature layer precision weight formula. High precision is used for core features, and low precision is used for irrelevant backgrounds, achieving a balance between precision and power consumption. Among these, the feature layer judgment threshold... The preferred value is 0.5.

[0040] In some feasible implementations, the directed knowledge distillation adopts a teacher-student model architecture, where the teacher model is a high-precision deep learning model trained on a large number of bird and interference samples, and the student model is a customized lightweight model optimized by pruning and mixed precision quantization. The knowledge distillation employs a customized loss function, focusing on the targeted transfer of bird feature recognition experience. The customized loss function is as follows: ; in The total training loss for the student model. For cross-entropy loss, For knowledge distillation loss, Weights for knowledge transfer related to bird features; The bird feature knowledge transfer weight The CIoU loss between birds and the background, calculated using the teacher model, is used to enable the student model to learn the bird feature distribution mastered by the teacher model, rather than the background feature distribution. The calculation formula is as follows: ; in For the teacher model, the CIoU loss for birds, The CIoU loss of the teacher model against the background; The distillation rule is as follows: the loss function of the student model is... Learn cross-entropy loss for the weights, The knowledge distillation loss is learned by weighting the model, focusing only on the learning of bird features. This avoids interference from background learning in student models, ensuring bird recognition accuracy while keeping the model lightweight, and adapting to the low-power operation requirements of embedded low-computing platforms.

[0041] Preferably, among which, The total training loss of the student model is the core basis for updating the backpropagation parameters of the student model. For cross-entropy loss, The loss is due to knowledge distillation; both are common knowledge in the field. It is a standard supervised learning loss function used in image classification tasks. It is a conventional distribution matching loss function in knowledge distillation technology (usually implemented using KL divergence), and its specific derivation process will not be elaborated here; The transfer weighting of bird feature knowledge is one of the core innovations of this invention.

[0042] The bird feature knowledge transfer weight The CIoU loss between birds and the background was dynamically calculated using a teacher model. The loss function of the teacher model is defined as the CIoU loss for bird targets, representing the difficulty of the teacher model in identifying bird targets and the importance of bird features; the loss function of the teacher model is defined as the CIoU loss for the background (including field interference), representing the intensity of background interference. The core design of this weight is to guide the student model to learn the bird feature distribution mastered by the teacher model, rather than the background feature distribution, thereby achieving specific transfer of bird features. The distillation rule is: the loss function of the student model is (1- The cross-entropy loss is used to learn the weights, solidifying the basic classification accuracy and ensuring the model closely matches the real labeled data; The knowledge distillation loss is learned by weighting the model to transfer the bird feature recognition experience of the teacher model. Through this dynamic weighting method, the student model focuses only on learning bird features, effectively avoiding the interference of background features in the student model. Finally, while making the model lightweight (after pruning and quantization optimization), the bird recognition accuracy is preserved to the maximum extent, perfectly adapting to the operation requirements of embedded low computing power and low power consumption.

[0043] Specifically, cross-entropy loss Cross-Entropy Loss is the fundamental loss function in supervised learning. It measures the difference between the student model's predictions and the true hard labels, serving as a basic anchor point for model training and ensuring that the student model's classification results match real-world scenarios. The specific calculation process includes: For the bird recognition task, each image in the training dataset is manually labeled to generate a 0 / 1 binary hard label: if the image contains a valid bird target, it is labeled as... (Positive sample); If the image is background / interference (plastic bag, tree, drone, etc.): labeled as (Negative samples); if it is a multi-class classification (identifying different bird species), then one-hot encoded hard labels are used, such as sparrow corresponding to... pigeon corresponds to .

[0044] Student model outputs predicted probabilities: The student model (lightweight CNN / YOLO) performs forward inference on the input image and outputs the predicted probability of birds. (range of values) ): The model estimates that the image has a 95% probability of being a bird. The model estimates that the image has an 8% probability of being a bird (most likely background / interference).

[0045] Formula for calculating binary cross-entropy loss: ; In the formula, : Realistic hard labeling (1=birds, 0=non-birds) The student model predicts the probability of birds. The logic is: when the prediction matches the label, the loss approaches 0; the greater the prediction deviation, the greater the loss. During backpropagation, the student model parameters are updated to make the prediction more closely match the true label.

[0046] Preferred, knowledge distillation loss It is a loss function for transferring the hidden knowledge of soft labels from the teacher model, measuring the difference between the predicted probability distribution of the student model and the predicted probability distribution of the teacher model. Its core function is to transfer the bird feature extraction experience and the subtle distinction logic between birds and background learned by the teacher model to the lightweight student model.

[0047] The calculation and acquisition process includes: Teacher Model: A high-precision bird-specific model (such as YOLOv5s), first trained on a large-scale bird / non-bird dataset. After training, all parameters are fixed and not updated further. Generating Soft Labels: Using the trained teacher model, forward inference is performed on all training samples to obtain the class probability distribution for each image. The soft labels are stored for use in training the student model. The student model performs forward inference on the same batch of training images and outputs its own class probability distribution. .

[0048] The Kullback-Leibler divergence is used to measure the difference between two probability distributions. The formula is as follows: ; In the formula, The predicted probability distribution of the student model. : Soft label probability distribution of the teacher model : Number of categories (2 for binary classification, and the number of bird species for multi-class classification); Logic: The smaller the KL divergence, the closer the two distributions are, and the more knowledge the student model learns from the teacher model; During backpropagation, the student model parameters are updated to align their distribution with the teacher model.

[0049] When bird targets are difficult to identify and their features are crucial Increase As the value increases, knowledge distillation loss occurs. With an increased proportion, the student model will focus on learning the high-level bird features from the teacher model to compensate for the accuracy loss caused by lightweight design; when background interference is strong and there is no need to focus on learning these features... The proportion increased. The value decreases, resulting in cross-entropy loss. With the increased proportion, the student model reverts to basic hard label supervision and rejects learning ineffective background interference features.

[0050] By using a customized loss function, the teacher's experience in bird feature recognition is accurately transferred, eliminating the shortcomings of traditional fixed-weight distillation. While ensuring the model is lightweight, it maximizes the retention of the core bird feature recognition capability, perfectly adapting to the low-power operation requirements of embedded low-computing platforms.

[0051] In some feasible implementations, the dual-template feature matching employs quantitative discrimination rules, using a bird-interference target matching formula to determine the target. This includes, after feature extraction is completed by the optimized model, quantitatively distinguishing bird targets from interference targets using a dual-template matching formula, outputting the determination result, and sending it back to the control layer. The bird-interference target matching formula is as follows: ; like If the target is identified as a bird, it is considered a dart; otherwise, it is considered a distracting target. In the formula, To detect the feature matching degree between the target and the bird template; To detect the feature matching degree between the target and the interfering target template; This is the image feature vector of the currently detected target; The feature template vector for the interference target is obtained by training on a dataset of common interference objects in the wild. The margin threshold is preferably 0.2.

[0052] in, The image feature vector of the current detection target is extracted by a lightweight model optimized through customized pruning, mixed-precision quantization, and directional knowledge distillation. It serves as the digital feature signature of the target to be determined, building upon the results of model optimization: the model retains only bird-related features, therefore... It focuses more on the essential characteristics of birds and weakens background interference, providing a foundation for subsequent accurate matching; The standard feature benchmarks for birds are generated in advance using a dataset of common wild birds (sparrows, pigeons, magpies, etc.), after model training, clustering, and averaging. These benchmarks are equivalent to standard identity cards for birds. The standard feature benchmark for interference objects is trained in advance using a dataset of common interference objects in the wild (plastic bags, drones, kites, insects, leaves, etc.), which is equivalent to a blacklist of interference objects. In the above implementation, the feature discrimination of the lightweight model is amplified by dual template matching, which makes up for the accuracy loss caused by pruning and quantization, and achieves a balance between lightweight model and high recognition accuracy.

[0053] In some feasible implementations, the mathematical expression of the preset radar-image feature co-verification model in step S6 includes: ; when When the target is in the correct position, it is considered a valid bird target; otherwise, it is considered a distracting target. In the formula, and These are radar feature weights and image feature weights, respectively. , , This is the preset verification threshold. Wherein, The preferred value is 0.7. The preferred value is 0.4. The preferred value is 0.6.

[0054] Preferably, the preset quantization rules in step S8 include: ; In the formula, The pre-stored trigger threshold before adjustment, The adjusted pre-stored trigger threshold, Here, t is a preset coefficient, and t is the duration of no effective target. This is the preset base time. Among them, The preferred value is 0.2. The optimal timeframe is 30 seconds. That is, when there is no valid target after the camera is activated, the radar trigger threshold is automatically quantized and expanded to reduce subsequent invalid triggers.

[0055] Preferably, when The process of adjusting the trigger threshold will only be triggered when the threshold is set to a certain value; otherwise, it will not be triggered.

[0056] For a specific example, if the camera is awake for 30 consecutive seconds but no birds are detected, then t = 30 seconds ≥ =30s, trigger calibration: ; The trigger threshold has been increased from 1.7 to 2.04. Subsequently, only targets with a fusion value ≥ 2.04 will trigger camera wake-up, reducing invalid triggers by about 60%.

[0057] In some feasible implementations, step S9 includes: when SOC > a first threshold, the system operates in normal mode, the radar maintains a preset normal acquisition frequency, and the camera maintains a preset wake-up delay; when a second threshold < SOC ≤ the first threshold, the system enters intermediate power-saving mode, the control layer reduces the radar acquisition frequency, and extends the camera wake-up delay to reduce the overall system power consumption; when SOC ≤ the second threshold, the system enters deep power-saving mode, retaining only the low-power acquisition function of the millimeter-wave radar, and prohibiting the wake-up of the camera, algorithm layer, and communication layer to achieve minimum power consumption operation. The first threshold is preferably 20%, and the second threshold is preferably 10%.

[0058] In some feasible implementations, the method further includes: real-time acquisition of the actual average brightness of the image; based on the actual average brightness of the image, when determining whether the scene is in low light or backlight, the control layer sends an image enhancement command to the camera of the perception layer; and adjusts the camera gain through a contrast correction formula to improve image recognition accuracy. The contrast correction formula includes: ; In the formula, For the corrected gain, Gain for the original image. This represents the actual average brightness of the image. This is a normal brightness reference value. This is the enhancement coefficient. Wherein, The value ranges from 1.5 to 2.0.

[0059] Example 2

[0060] Please see Figure 2 This invention provides a schematic diagram of a low-power bird recognition system based on millimeter-wave radar and image recognition.

[0061] As an example, the system includes five collaborative modules: a perception layer 1, a control layer 2, an algorithm layer 3, a communication layer 4, and an application layer 5. The perception layer 1 is equipped with a millimeter-wave radar 101 and a camera 102; the millimeter-wave radar 101 is used to collect a set of feature parameters in real time, including the target's flight speed, radar cross-section, azimuth angle change, and distance change; the camera 102 is used to collect target visual image data. The control layer 2 is equipped with an embedded main control module 201, which integrates a multi-feature fusion trigger model. This model calculates the target fusion value based on the feature parameter set, compares the target fusion value with a pre-stored trigger threshold to perform initial screening of suspected bird targets, and issues a wake-up command to activate the camera on the perception layer if the initial screening is successful. The control layer also performs dual verification by substituting the target fusion value and the bird target fine screening result into a preset radar-image feature collaborative verification model. Based on the verification result, the control layer determines whether the target is a valid bird target or a interference target. If the target is determined to be a valid bird target, the control layer immediately sends a wake-up signal to the communication module of the communication layer. If the target is determined to be a interference target, or if the camera does not detect a valid target after waking up, the control layer issues a sleep command to the camera on the perception layer. Furthermore, the control layer automatically adjusts the pre-stored millimeter-wave radar trigger threshold according to preset quantization rules based on the invalid trigger data. Finally, the control layer collects the remaining SOC (State of Charge) of the power supply battery in real time and performs three-level adaptive energy-saving control based on the SOC value. The algorithm layer 3 is equipped with a customized lightweight image recognition model. The customized lightweight image recognition model is designed for a lightweight backbone network with bird feature template vectors as the core benchmark. After four layers of quantitative optimization, including bird feature contribution pruning, mixed precision quantization, directional knowledge distillation, and dual template feature matching, it completes the quantitative differentiation between birds and interference targets based on the target visual image data, obtains the bird target screening result, and sends it back to the control layer. The communication layer 4 is equipped with a communication module, which is used to perform lightweight compression processing on the target image data based on the wake-up signal of the control layer and upload it to the Web monitoring platform of the application layer 5 and trigger a remote alarm.

[0062] It is not difficult to see that this embodiment is a system implementation corresponding to the first embodiment, and this embodiment can be implemented in conjunction with the first embodiment. The relevant technical details mentioned in the first embodiment are still valid in this embodiment, and will not be repeated here to reduce repetition. Accordingly, the relevant technical details mentioned in this embodiment can also be applied to the first embodiment.

[0063] It is worth mentioning that all modules involved in this embodiment are logical units. In practical applications, a logical unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. Furthermore, to highlight the innovative aspects of this invention, this embodiment does not introduce units that are not closely related to solving the technical problem proposed by this invention; however, this does not mean that other units are absent from this embodiment.

[0064] Example 3

[0065] Please see Figure 3The present invention also provides an electronic device, including: a memory and a processor; the memory stores at least one program instruction; the processor loads and executes the at least one program instruction to implement the low-power bird recognition method based on millimeter-wave radar and image recognition provided in Embodiment 1.

[0066] The memory 702 and processor 701 are connected via a bus, which may include any number of interconnecting buses and bridges, connecting various circuits of one or more processors 701 and memory 702 together. The bus may also connect various other circuits, such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. A bus interface provides an interface between the bus and the transceiver. The transceiver may be a single element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by processor 701 is transmitted over a wireless medium via an antenna, which further receives data and transmits it to processor 701.

[0067] Processor 701 is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory 702 can be used to store data used by processor 701 during operation.

[0068] Example 4

[0069] This invention also proposes a storage medium storing a low-power bird recognition method based on millimeter-wave radar and image recognition. When executed by a processor, the low-power bird recognition program based on millimeter-wave radar and image recognition implements the steps of the low-power bird recognition method based on millimeter-wave radar and image recognition as described above. Since this storage medium employs all the technical solutions of the above embodiments, it possesses at least all the beneficial effects brought about by the technical solutions of the above embodiments, which will not be elaborated upon further here.

[0070] The above descriptions are merely embodiments of the present invention. Commonly known structures and characteristics of the solutions are not described in detail here. Those skilled in the art are aware of all common technical knowledge in the field prior to the application date or priority date, are aware of all existing technologies in that field, and have the ability to apply conventional experimental methods prior to that date. Those skilled in the art can, under the guidance of this application, improve and implement this solution in combination with their own capabilities. Some typical known structures or methods should not be obstacles for those skilled in the art to implement this application. It should be noted that those skilled in the art can make several modifications and improvements without departing from the structure of the present invention. These should also be considered within the scope of protection of the present invention, and will not affect the effectiveness of the implementation of the present invention or the practicality of the patent. The scope of protection claimed in this application should be determined by the content of its claims, and the specific embodiments described in the specification can be used to interpret the content of the claims.

Claims

1. A low-power bird recognition method based on millimeter-wave radar and image recognition, characterized in that, The method includes: Step S1: Under normal conditions, the control layer sends a low-power monitoring command to the perception layer, which controls the millimeter-wave radar on the perception layer to enter a low-power acquisition state. Step S2: The millimeter-wave radar collects the feature parameter set in real time and synchronously transmits it back to the control layer cache; Step S3: The control layer calculates the target fusion value based on the feature parameter set using a pre-installed multi-feature fusion triggering model; Step S4: The control layer completes the initial screening of suspected bird targets by comparing the target fusion value with the pre-stored trigger threshold. After the initial screening is qualified, a wake-up command is issued to trigger the camera on the perception layer to start and collect target visual image data. Step S5: The camera transmits the target visual image data to the algorithm layer equipped with a customized lightweight image recognition model to complete the quantitative differentiation between birds and interference targets, obtain the bird target screening result, and transmit it back to the control layer. Step S6: The control layer substitutes the target fusion value and the bird target screening result into the preset radar-image feature collaborative verification model for double verification, and determines whether the target is a valid bird target or an interference target based on the verification result. Step S7: If the target is determined to be a valid bird, the control layer immediately wakes up the communication layer module, performs lightweight compression processing on the target image data, uploads it to the Web monitoring platform of the application layer, and triggers a remote alarm. Step S8: If the target is determined to be an interference target, or if no valid target is detected after the camera is woken up, the control layer sends a sleep command to the camera in the perception layer; and the control layer automatically adjusts the pre-stored millimeter-wave radar trigger threshold according to the preset quantization rules based on the invalid trigger data. Step S9: The control layer collects the remaining SOC power of the power supply battery in real time and executes three-level adaptive energy-saving control based on the SOC value.

2. The low-power bird recognition method based on millimeter-wave radar and image recognition according to claim 1, characterized in that, The set of characteristic parameters includes one or a combination of target flight speed, radar cross-section, azimuth change, and range change. The mathematical expression for the multi-feature fusion triggering model is: ； In the formula, The target fusion value; , , and These are the feature weights for target flight speed, radar cross-section, azimuth change, and range change, respectively. ; v represents the target's flight speed, measured in m / s; This is a reference value for the average flight speed of birds, in m / s; s represents the actual radar cross-section of the target, in units of ; This is a reference value for the average reflective area of ​​birds, in units of ; The change in azimuth angle of the target over three consecutive frames; This represents the maximum permissible variation in azimuth. The distance change of the target over three consecutive frames, in meters; This represents the maximum permissible variation in distance, expressed in meters (m). Step S4 further includes: the control layer retrieving its pre-stored consecutive confirmation frame count parameter, and calculating the average fusion value of multiple consecutive frames based on the target fusion value using the anti-shake frame confirmation formula. The formula for confirming the stabilized frame is: ； In the formula, N is the number of consecutive frames. The target fusion value corresponding to the i-th frame; Only when At that time, the control layer determines that the initial screening is qualified and issues a camera wake-up command. Pre-store trigger thresholds for the control layer.

3. The low-power bird recognition method based on millimeter-wave radar and image recognition according to claim 1, characterized in that, The customized lightweight image recognition model is a lightweight backbone network with bird feature template vectors as the core benchmark, and is quantitatively optimized through four layers: bird feature contribution pruning, mixed precision quantization, directional knowledge distillation, and dual template feature matching. The bird feature contribution pruning involves sequentially traversing all convolutional kernels of the lightweight backbone network, calculating the contribution of each convolutional kernel to the extraction of bird features based on a preset bird feature contribution pruning formula, and directly discarding convolutional kernels whose contribution is less than or equal to a preset contribution threshold. The bird feature contribution pruning formula is as follows: ； In the formula, The total number of convolution kernels, The number of parameters in a single convolution kernel. For the j-th parameter value of the k-th convolutional kernel, This is a bird feature template vector trained based on a dataset of common wild birds.

4. The low-power bird recognition method based on millimeter-wave radar and image recognition according to claim 3, characterized in that, The hybrid precision quantization optimization uses bird feature template vectors as a benchmark to distinguish the importance of network layers in bird feature extraction, thereby achieving hierarchical precision control. The layered precision control includes the following steps: traversing all network layers of the pruned model and extracting the output feature vector of each layer. ,calculate Bird feature template vector The cosine similarity is used to select the network layer output feature vectors with the highest similarity. The normalized similarity of each layer is calculated using a mixed-precision quantization precision weight formula. The formula for the precision weighting of the mixed precision quantization is: ； in This is the function for calculating cosine similarity. Set feature layer decision threshold ,when When this layer is identified as a key feature layer for birds, 16-bit fixed-point quantization is used; when When the layer is determined to be a background-independent layer, 8-bit fixed-point quantization is used; after quantization, calibration is performed using real image data of the monitored scene to correct the quantization error.

5. The low-power bird recognition method based on millimeter-wave radar and image recognition according to claim 4, characterized in that, The targeted knowledge distillation adopts a teacher-student model architecture. The teacher model is a high-precision deep learning model trained with a large number of bird and interference samples, and the student model is a customized lightweight model after pruning optimization and mixed precision quantization optimization. The knowledge distillation employs a customized loss function, focusing on the targeted transfer of bird feature recognition experience. The customized loss function is as follows: ； in The total training loss for the student model. For cross-entropy loss, For knowledge distillation loss, Weights for knowledge transfer related to bird features; The bird feature knowledge transfer weight The CIoU loss between birds and the background, calculated using the teacher model, is used to enable the student model to learn the bird feature distribution mastered by the teacher model, rather than the background feature distribution. The calculation formula is as follows: ； in For the teacher model, the CIoU loss for birds, The CIoU loss of the teacher model against the background; The distillation rule is as follows: the loss function of the student model is... Learn cross-entropy loss for the weights, The knowledge distillation loss is learned by weighting the model, focusing only on the learning of bird features. This avoids interference from background learning in student models, ensuring bird recognition accuracy while keeping the model lightweight, and adapting to the low-power operation requirements of embedded low-computing platforms.

6. The low-power bird recognition method based on millimeter-wave radar and image recognition according to claim 5, characterized in that, The dual-template feature matching employs quantitative discrimination rules, using a bird-interference target matching formula to determine the target. This includes, after feature extraction by the optimized model, quantitatively distinguishing bird targets from interference targets using the dual-template matching formula, outputting the determination result, and sending it back to the control layer. The bird-interference target matching formula is as follows: ； like If the target is identified as a bird, it is considered a dart; otherwise, it is considered a distracting target. In the formula, To detect the feature matching degree between the target and the bird template; To detect the feature matching degree between the target and the interfering target template; This is the image feature vector of the currently detected target; The feature template vector for the interference target is obtained by training on a dataset of common interference objects in the wild. This is the margin threshold.

7. The low-power bird recognition method based on millimeter-wave radar and image recognition according to claim 6, characterized in that, The mathematical expression of the preset radar-image feature co-verification model in step S6 includes: ； when When the target is in the correct position, it is considered a valid bird target; otherwise, it is considered a distracting target. In the formula, and These are radar feature weights and image feature weights, respectively. , , This is the preset verification threshold; The preset quantization rules in step S8 include: ； In the formula, The pre-stored trigger threshold before adjustment, The adjusted pre-stored trigger threshold, Here, t is a preset coefficient, and t is the duration of no effective target. This is the preset base time.

8. The low-power bird recognition method based on millimeter-wave radar and image recognition according to claim 1, characterized in that, Step S9 includes: When SOC > the first threshold, the system operates in normal mode, the radar maintains the preset normal acquisition frequency, and the camera maintains the preset wake-up delay; When the second threshold < SOC ≤ the first threshold, the system enters the intermediate energy-saving mode. The control layer reduces the radar acquisition frequency and extends the camera wake-up delay to reduce the overall power consumption of the system. When SOC ≤ the second threshold, the system enters deep power saving mode, retaining only the low-power acquisition function of the millimeter-wave radar, and prohibiting the wake-up of the camera, algorithm layer and communication layer to achieve the lowest power consumption operation.

9. The low-power bird recognition method based on millimeter-wave radar and image recognition according to claim 1, characterized in that, The method further includes: real-time acquisition of the actual average brightness of the image; based on the actual average brightness of the image, determining whether the scene is in low light or backlight, the control layer sends an image enhancement command to the camera of the perception layer; and adjusting the camera gain through a contrast correction formula to improve image recognition accuracy. The contrast correction formula includes: ； In the formula, For the corrected gain, Gain for the original image. This represents the actual average brightness of the image. This is a normal brightness reference value. This is the enhancement coefficient.

10. A low-power bird recognition system based on millimeter-wave radar and image recognition, implemented using the low-power bird recognition method based on millimeter-wave radar and image recognition as described in any one of claims 1-9, characterized in that, The system comprises five collaborative modules: a perception layer, a control layer, an algorithm layer, a communication layer, and an application layer. The perception layer is equipped with a millimeter-wave radar and a camera; the millimeter-wave radar is used to collect a set of characteristic parameters in real time, including the target's flight speed, radar cross-section, azimuth angle change, and distance change; the camera is used to collect visual image data of the target. The control layer is equipped with an embedded main control module, which integrates a multi-feature fusion trigger model. This model calculates the target fusion value based on the feature parameter set, compares the target fusion value with a pre-stored trigger threshold to perform initial screening of suspected bird targets, and issues a wake-up command to activate the camera on the perception layer if the initial screening is successful. The control layer also performs dual verification by substituting the target fusion value and the bird target fine screening result into a preset radar-image feature collaborative verification model. Based on the verification result, the control layer determines whether the target is a valid bird target or a interference target. If it is determined to be a valid bird target, the control layer immediately sends a wake-up signal to the communication module of the communication layer. If it is determined to be a interference target, or if the camera does not detect a valid target after wake-up, the control layer issues a sleep command to the camera in the perception layer. Furthermore, based on the invalid trigger data, the control layer automatically adjusts the pre-stored millimeter-wave radar trigger threshold according to preset quantization rules. Finally, the control layer collects the remaining SOC (State of Charge) of the power supply battery in real time and performs three-level adaptive energy-saving control based on the SOC value. The algorithm layer is equipped with a customized lightweight image recognition model. The customized lightweight image recognition model is designed for a lightweight backbone network with bird feature template vectors as the core benchmark. After four layers of quantitative optimization, including bird feature contribution pruning, mixed precision quantization, directional knowledge distillation, and dual template feature matching, it completes the quantitative differentiation between birds and interference targets based on the target visual image data, obtains the bird target screening result, and sends it back to the control layer. The communication layer is equipped with a communication module, which is used to perform lightweight compression processing on the target image data based on the wake-up signal of the control layer, upload it to the Web monitoring platform of the application layer, and trigger a remote alarm.

11. An electronic device, comprising: The memory, the processor, and the computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the low-power bird identification method based on millimeter-wave radar and image recognition as described in any one of claims 1-9.

12. A readable storage medium, wherein instructions in the storage medium, when executed by a processor of an electronic device, enable the electronic device to perform the low-power bird identification method based on millimeter-wave radar and image recognition as described in any one of claims 1-9.

Images