Video image recognition system based on computer informatization
By constructing a dual-channel deep neural network with multi-dimensional spatiotemporal features and a few-shot learning mechanism, the shortcomings of existing video image recognition systems in terms of multi-source heterogeneous data fusion, dynamic environment adaptability, and interpretability are solved, achieving high-precision, adaptive, and interpretable recognition results.
Patent Information
- Application Number
- CN202511216315.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-08-28
- Publication Date
- 2025-12-09
AI Technical Summary
Existing video image recognition systems have shortcomings in multi-source heterogeneous data fusion, dynamic environment adaptability, continuous model evolution capability, and decision-making process interpretability. In particular, the recognition accuracy is unstable in complex dynamic environments, it is difficult to adapt to changes in lighting and target occlusion, and there is a lack of multi-source data complementarity mechanism and interpretable analysis.
The system employs a data acquisition module, an environment-adaptive noise reduction module, a feature recognition module, a multi-dimensional spatiotemporal evolution map construction module, a confidence assessment and incremental learning module, and an interpretable ecological report generation module. Through a dual-channel deep neural network and a few-shot learning mechanism, it achieves simultaneous acquisition of multi-source data, environment-adaptive noise reduction, continuous model evolution, and interpretable recognition.
It improves the robustness of recognition in complex backgrounds, realizes the system's adaptability and continuous evolution in dynamic environments, and provides interpretable recognition results, thereby improving recognition accuracy and reliability.
Smart Images
Figure CN121095652A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of computer vision technology, specifically relating to a video image recognition system based on computer information technology. Background Technology
[0002] Video image recognition technology, as a core area of computer information applications, has been widely used in security monitoring, industrial quality inspection, and ecological monitoring. With the popularization of edge computing devices and the development of multimodal sensors, modern recognition systems need to achieve high-precision, interpretable real-time analysis in complex and dynamic environments. Current mainstream solutions are based on deep learning frameworks, extracting visual features and performing classification decisions through convolutional neural networks; however, limitations remain when facing the changing conditions in open environments. The shortcomings of existing technologies include:
[0003] 1. Defects in multi-source heterogeneous data fusion
[0004] Traditional methods struggle to effectively integrate video streams with data from other modal sensors (such as infrared thermograms, sound spectra, and environmental parameters). These data are typically processed in separate channels before undergoing simple decision-level fusion. This approach leads to a loss of spatiotemporal correlation, failing to capture deep semantic connections between cross-modal features. Furthermore, in complex scenarios with drastic lighting changes or partial target occlusion, the lack of a multi-source data complementarity mechanism results in recognition blind spots.
[0005] 2. Insufficient adaptability to dynamic environments
[0006] Existing recognition models generally employ fixed parameter configurations, making it difficult to adapt to continuously changing real-world environmental conditions. When there are significant fluctuations in light intensity, increased background complexity, or drastic changes in the target's motion speed, the system cannot automatically adjust its feature extraction strategy and noise suppression scheme, resulting in unstable fluctuations in recognition accuracy. In particular, when environmental parameters exceed the coverage of the training data, model performance may experience a precipitous drop.
[0007] 3. Lack of continuous model evolution capability
[0008] Most systems rely on static models trained offline, which cannot autonomously evolve their knowledge base during operation. When faced with new target categories or unknown scene patterns not encountered during training, service interruption is required for manual data annotation and model retraining. This mechanism not only results in a delayed response but also limits the long-term applicability of the system in open environments, making it difficult to meet the application requirements that need to continuously expand the recognition boundaries.
[0009] 4. The decision-making process lacks interpretability.
[0010] Mainstream deep neural networks typically operate as a "black box," only outputting the final identification result without providing traceability of the decision-making basis. When misidentification or controversial conclusions occur, they cannot provide crucial explanatory information such as feature contribution analysis and reasoning logic chains, severely impacting the reliability of the system in high-reliability scenarios such as medical diagnosis and judicial evidence collection.
[0011] Therefore, a video image recognition system based on computer information technology is proposed to address the above problems. Summary of the Invention
[0012] To address the shortcomings of existing technologies, this invention proposes a computer-based video image recognition system, which includes: a data acquisition module, an environment-adaptive noise reduction module, a feature recognition module, a multi-dimensional spatiotemporal evolution map construction module, a confidence assessment and incremental learning module, and an interpretable ecological report generation module.
[0013] The data acquisition module is used to collect field environment data;
[0014] The environmental adaptive noise reduction module preprocesses the collected field environmental data to obtain multimodal data;
[0015] The feature recognition module is used to process multimodal data and output species identification results;
[0016] The multidimensional spatiotemporal evolution map construction module constructs a multidimensional spatiotemporal evolution map based on collected field environmental data and species identification results;
[0017] The confidence assessment and incremental learning module is used to calculate the confidence of the species identification results based on the species identification results and field environment data, and to initiate few-sample learning based on the species identification results to update the model parameters in the feature identification module.
[0018] The interpretable ecological report generation module is used to generate interpretable ecological reports based on confidence level, multidimensional spatiotemporal evolution map and species identification results.
[0019] Preferably, the data acquisition module includes a starlight-level camera, a sound sensor, a vibration detector, and a weather station data acquisition unit;
[0020] The starlight-level camera is used to capture environmental videos;
[0021] The sound sensor is used to capture audio data in a specific frequency band;
[0022] The vibration detector is used to acquire ground vibration data;
[0023] The weather station data collector is used to collect meteorological data from the weather station.
[0024] Preferably, the processing steps of the multi-environment adaptive noise reduction module include:
[0025] Dynamic noise reduction based on light intensity is applied to environmental videos to balance bright and dark areas, resulting in balanced videos.
[0026] Spectral subtraction was used to remove environmental noise from the audio data and animal voiceprint features were extracted from it.
[0027] To differentiate between species footprints and geological activity in ground seismic data, and to extract the vibrational characteristics of species footprints;
[0028] The balanced video, animal voiceprint features, species footstep vibration features, and meteorological data were synchronized in time and space to obtain preprocessed multimodal data.
[0029] Preferably, the feature recognition module includes a feature extraction module, a cross-modal correlation analysis module, and an early warning module;
[0030] The feature extraction module includes a dual-channel deep neural network, which processes multimodal data and outputs fused features.
[0031] The cross-modal association analysis module processes the fused features to obtain species identification results, including species individual identification results, species behavior patterns, and species location;
[0032] The early warning module determines whether there is any abnormality in the visual target. If there is an abnormality, it issues an alarm.
[0033] Furthermore, the dual-channel deep neural network includes a main channel and an auxiliary channel. The main channel includes a deformable convolutional network and a multi-scale feature pyramid network; the auxiliary channel includes a dynamic graph neural network; the outputs of the two channels are processed by a cross-attention mechanism to output fused features.
[0034] Preferably, the confidence assessment and incremental learning module includes a confidence assessment module and an incremental learning module;
[0035] The confidence assessment module is used to calculate the confidence level of the species identification results;
[0036] The incremental learning module is used to initiate few-shot learning to train the model and update the model parameters when a new species is discovered.
[0037] Furthermore, the process of calculating the confidence level of the species identification results includes: calculating the feature matching degree, target visibility, environmental interference degree, sensor reliability, and historical identification accuracy based on the species identification results and field environment data; and weighting and summing the feature matching degree, target visibility, environmental interference degree, sensor reliability, and historical identification accuracy to obtain the confidence level of the species identification results.
[0038] Preferably, the process of generating an interpretable ecological report by the interpretable ecological report generation module includes: the interpretable ecological report includes an identification chain of evidence, a credibility assessment report, and a species ecological relationship report;
[0039] Generate a chain of evidence for behavior recognition based on feature matching degree;
[0040] A credibility assessment report is generated based on the species identification results and confidence levels.
[0041] Based on the analysis of multidimensional spatiotemporal evolution maps, ecological interactions and trends among species and between species and the environment are analyzed to generate a report on species ecological relationships.
[0042] The beneficial effects of this invention are as follows:
[0043] Compared with existing technologies, this invention constructs a dual-channel deep neural network model that integrates spatiotemporal features, combines it with a distributed heterogeneous sensor network to achieve synchronous acquisition of multi-source data, dynamically adjusts the noise reduction strategy based on environmental parameters, and enables continuous model evolution through a few-shot learning mechanism. The final output is an interpretable recognition result with a temporal decision path and a multi-dimensional dynamic confidence rating. Technically, this invention overcomes four major limitations of traditional solutions: its dual-channel collaborative architecture synchronously extracts geometrically invariant features of the target and scene topology, improving recognition robustness in complex backgrounds; the environmentally adaptive noise reduction mechanism enables the system to adapt to dynamic conditions such as illumination fluctuations and changes in motion speed; few-shot online transfer learning endows the model with continuous evolution capabilities in open scenes; and the visualized feature recognition evidence chain solves the black-box decision-making problem of deep learning models, improving recognition accuracy and demonstrating promising application prospects. Attached Figure Description
[0044] Figure 1 This is a block diagram of the computer-based video image recognition system of the present invention. Detailed Implementation
[0045] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0046] This invention proposes a video image recognition system based on computer information technology, such as... Figure 1 As shown, the system includes: a data acquisition module, an environment adaptive noise reduction module, a feature recognition module, a multi-dimensional spatiotemporal evolution map construction module, a confidence assessment and incremental learning module, and an interpretable ecological report generation module.
[0047] Data acquisition module:
[0048] The data acquisition module is used to collect field environmental data. Specifically, it involves deploying solar-powered intelligent monitoring stations (data acquisition modules) in different ecological areas of the protected area (such as forests, water bodies, and shrublands), equipped with various sensors.
[0049] Starlight-level cameras: used for all-weather video capture, employing a dynamic exposure strategy to adapt to dramatic changes in day and night lighting, ensuring clear capture of animal behavior even in low-light environments.
[0050] Sound sensors: These are specifically designed to capture audio streams in specific frequency bands (such as animal vocalizations) to provide audio data for species identification and behavioral analysis.
[0051] Vibration detectors: acquire ground vibration data; provide data for behavioral pattern analysis to identify the gait characteristics of large animals (such as elephant herds, deer herds, etc.).
[0052] Weather station data logger: Records environmental parameters such as temperature, humidity, and rainfall, providing real-time environmental background information.
[0053] All devices connect via LoRa wireless networking, employing an adaptive frequency hopping mechanism to cope with environmental interference (such as electromagnetic interference, climate change, etc.). Data is compressed and transmitted to the central server at minute intervals.
[0054] In key areas, redundant nodes with multiple sensors can be set up, automatically switching to backup links when the primary node fails, ensuring data continuity under harsh weather conditions (such as high humidity during the rainy season). Raw data is appended with geographic and meteorological tags to form a spatiotemporally correlated dataset, facilitating subsequent analysis.
[0055] Raw data (video, audio, vibration, etc.) collected by various sensors, along with geographic and meteorological tags, provide a foundation for subsequent data fusion and processing.
[0056] Environmental adaptive noise reduction module:
[0057] The collected heterogeneous data (field environment data) undergoes environmental adaptive processing to ensure high data quality and consistency, and to provide usable input for subsequent processing modules; specifically:
[0058] Environmental video: Dynamic noise reduction is implemented based on light intensity, especially in low light conditions (such as dusk and dawn). The HDR fusion algorithm balances the bright and dark areas of each frame to ensure the clarity and detail of the image under different lighting conditions.
[0059] Audio data: Environmental noise (such as wind and rain) is removed using spectral subtraction techniques, and animal voiceprint features, such as the frequency of specific animal calls, are extracted for subsequent species identification and behavior analysis.
[0060] Vibration data: Wavelet transform technology is applied to vibration signals to distinguish between animal footsteps and geological activities, ensuring that features related to animal behavior in the vibration signals can be effectively extracted and identified; the vibration characteristics of species footsteps are extracted.
[0061] Time and spatial synchronization is performed on balanced video, animal voiceprint features, species footstep vibration features, and meteorological data. Specifically, GPS second pulses are used to time-align the data from various sensors in step 1. This time synchronization mechanism ensures that the timestamps of each data point are consistent, avoiding data asynchrony caused by sensor clock deviations.
[0062] After processing by the multi-environment adaptive noise reduction module, multimodal data is output.
[0063] Feature recognition module:
[0064] The feature recognition module includes a feature extraction module, a cross-modal correlation analysis module, and an early warning module. The feature extraction module includes a dual-channel deep neural network, which processes multimodal data and outputs fused features.
[0065] The dual-channel deep neural network comprises a main channel and an auxiliary channel. The main channel includes a deformable convolutional network and a multi-scale feature pyramid network; the auxiliary channel includes a dynamic graph neural network. Specifically, the main channel processes the video stream, employing a deformable convolutional network to adapt to various animal postures (such as limb deformation during running), and using a multi-scale feature pyramid to identify multi-level biometrics from local features (stripes and textures) to global features (such as body shape and limb proportions). It combines appearance differences with the continuity of movement trajectories to distinguish individual species, thereby achieving animal detection and identification. Using identified animal individuals as nodes, topological connections are established based on species relationships (predator / symbiosis), spatial distance, and movement trajectory correlation. This topological information is input into the dynamic graph neural network constructed in the auxiliary channel for processing. The outputs of both channels undergo a cross-attention mechanism to output fused features.
[0066] The cross-modal association analysis module processes the fused features to obtain species identification results, including individual species identification results (including multiple feature matching degrees and specific species judgment results), species behavior patterns (such as predation and migration), and species location.
[0067] The early warning module determines whether there is any anomaly in the visual target. Specifically, when the voiceprint feature (such as the frequency feature of a tiger's roar) continues to deviate from the visual target (such as a tiger stripe pattern) and exceeds a reasonable range, a hidden target detection alarm is triggered.
[0068]
[0069] in, It is the maximum mean difference loss, , It is a kernel function used to measure the similarity of modal data.
[0070] Multidimensional spatiotemporal evolution map construction module:
[0071] A multidimensional spatiotemporal evolution map is constructed based on field environmental data and species identification results. Specifically, nodes are identified and constructed, which include ecological elements such as individual animals, plant communities, and water sources. Edge relationships cover ecological interactions such as "predation behavior," "water dependence," and "territorial overlap." This structured information is input into a graph neural network as adjacency relationships, and the feature representations of the nodes are iteratively propagated and updated. After passing through the graph neural network, the nodes not only contain the information of the node itself, but also integrate the context of neighboring nodes and relationships, achieving the effect of semantic enhancement, and finally obtaining the required multidimensional spatiotemporal evolution map.
[0072] Iteratively update node states using a graph neural network:
[0073]
[0074] in: It is a node In the Layer representation, It is a node The set of neighboring nodes, It is the first The weight matrix of the layer, It is an activation function, usually ReLU or Sigmoid.
[0075] Confidence assessment and incremental learning module:
[0076] The confidence assessment and incremental learning module is used to calculate the confidence level of species identification results based on the species identification results and field environment data, and to initiate few-shot learning based on the species identification results to update the model parameters in the feature identification module. The confidence assessment and incremental learning module includes a confidence assessment module and an incremental learning module.
[0077] The confidence assessment module is used to calculate the confidence level of the species identification results. Specifically, based on the species identification results and combined with the field environmental data collected by the sensors, a multi-dimensional confidence rating is implemented: feature matching degree, target visibility (such as the degree of foliage obstruction), environmental interference degree (such as the intensity of rain and fog), sensor reliability (such as the stability of camera focal length), and historical recognition accuracy are calculated; the feature matching degree, target visibility, environmental interference degree, sensor reliability, and historical recognition accuracy are weighted and summed to obtain the confidence level of the species identification results, thereby reflecting the reliability of the identification results.
[0078] The incremental learning module is used to initiate few-shot learning to train the model and update its parameters when a new species is discovered. Specifically, when a new species is discovered (e.g., feather patterns do not match those of known bird species), a few-shot learning process is initiated, generating pseudo-negative samples through feature reconstruction techniques to optimize the classification boundary. Knowledge distillation is performed monthly during periods of low activity, transferring species features updated by the central model to lightweight models at edge monitoring stations.
[0079] Interpretable Ecological Report Generation Module:
[0080] The interpretable ecological report generation module generates interpretable ecological reports based on confidence levels, multidimensional spatiotemporal evolution maps, and species identification results. These reports include an identification evidence chain, a credibility assessment report, and a species ecological relationship report. A behavioral identification evidence chain is generated based on feature matching scores (e.g., "Audio feature matching score 0.82 + footprint size matching score 0.75 → Siberian tiger identification conclusion"). A credibility assessment report is generated based on confidence levels and species identification results. A species ecological relationship report is generated by analyzing the ecological interactions and trends between species and between species and the environment based on the multidimensional spatiotemporal evolution map. By integrating the identification evidence chain, the credibility assessment report, and the species ecological relationship report, the corresponding interpretable ecological report is obtained.
[0081] Interpretable ecological reports provide decision support for managers and enhance the credibility and transparency of the system. Managers can use interpretable ecological reports to analyze the relationships between species and between species and the environment in the wild, thereby enabling the monitoring and protection of the wild environment.
[0082] The above-described embodiments further illustrate the purpose, technical solution, and advantages of the present invention. It should be understood that the above-described embodiments are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made to the present invention within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A video image recognition system based on computer information technology, characterized in that, include: The system includes a data acquisition module, an environmental adaptive noise reduction module, a feature recognition module, a multi-dimensional spatiotemporal evolution map construction module, a confidence assessment and incremental learning module, and an interpretable ecological report generation module. The data acquisition module is used to collect field environment data; The environment adaptive noise reduction module preprocesses the collected field environment data to obtain multimodal data; The feature recognition module is used to process multimodal data and output species identification results; The multidimensional spatiotemporal evolution map construction module constructs a multidimensional spatiotemporal evolution map based on field environmental data and species identification results; The confidence assessment and incremental learning module is used to calculate the confidence of the species identification results based on the species identification results and field environment data, and to initiate few-sample learning based on the species identification results to update the model parameters in the feature identification module. The interpretable ecological report generation module is used to generate interpretable ecological reports based on confidence level, multidimensional spatiotemporal evolution map and species identification results.
2. The video image recognition system based on computer information technology according to claim 1, characterized in that, The data acquisition module includes a starlight-level camera, a sound sensor, a vibration detector, and a weather station data acquisition unit; The starlight-level camera is used to capture environmental videos; The sound sensor is used to capture audio data in a specific frequency band; The vibration detector is used to acquire ground vibration data; The weather station data collector is used to collect meteorological data from the weather station.
3. The video image recognition system based on computer information technology according to claim 1, characterized in that, The processing steps of the multi-environment adaptive noise reduction module include: Dynamic noise reduction based on light intensity is applied to environmental videos to balance bright and dark areas, resulting in balanced videos. Spectral subtraction was used to remove environmental noise from the audio data and animal voiceprint features were extracted from it. To differentiate between species footprints and geological activity in ground motion data, and to extract the vibration characteristics of species footprints; The balanced video, animal voiceprint features, species footstep vibration features, and meteorological data were synchronized in time and space to obtain preprocessed multimodal data.
4. The video image recognition system based on computer information technology according to claim 1, characterized in that, The feature recognition module includes a feature extraction module, a cross-modal correlation analysis module, and an early warning module; The feature extraction module includes a dual-channel deep neural network, which processes multimodal data and outputs fused features. The cross-modal association analysis module processes the fused features to obtain species identification results, including species individual identification results, species behavior patterns, and species location; The early warning module determines whether there is any abnormality in the visual target. If there is an abnormality, it issues an alarm.
5. A video image recognition system based on computer information technology according to claim 4, characterized in that, A dual-channel deep neural network consists of a main channel and an auxiliary channel. The main channel includes a deformable convolutional network and a multi-scale feature pyramid network; the auxiliary channel includes a dynamic graph neural network; the outputs of the two channels are processed by a cross-attention mechanism to output fused features.
6. A video image recognition system based on computer information technology according to claim 1, characterized in that, The confidence assessment and incremental learning module includes a confidence assessment module and an incremental learning module; The confidence assessment module is used to calculate the confidence level of the species identification results; The incremental learning module is used to initiate few-shot learning to train the model and update the model parameters when a new species is discovered.
7. A video image recognition system based on computer information technology according to claim 6, characterized in that, The process of calculating the confidence level of species identification results includes: calculating feature matching degree, target visibility, environmental interference degree, sensor reliability, and historical identification accuracy based on species identification results and field environmental data; and weighting and summing the feature matching degree, target visibility, environmental interference degree, sensor reliability, and historical identification accuracy to obtain the confidence level of species identification results.
8. A video image recognition system based on computer information technology according to claim 1, characterized in that, The process of generating an interpretable ecological report by the interpretable ecological report generation module includes: the interpretable ecological report includes an identification chain of evidence, a credibility assessment report, and a species ecological relationship report; Generate a chain of evidence for behavior recognition based on feature matching degree; A credibility assessment report is generated based on the species identification results and confidence levels. Based on the analysis of multidimensional spatiotemporal evolution maps, ecological interactions and trends among species and between species and the environment are analyzed to generate a report on species ecological relationships.