Image recognition and analysis system based on AI

Abstract

The invention relates to the technical field of image processing, and discloses an image recognition and analysis system based on AI. The system comprises a data acquisition module, a feature extraction module, a model training module, a multi-modal fusion module, a dynamic optimization module and the like. The method comprises the steps of collecting real-time image data by a multi-source sensor, extracting features by a cascade convolutional neural network, generating an adversarial network training model, integrating multi-source data by multi-modal fusion, optimizing feature vectors by an improved genetic algorithm, and constructing a classification decision tree. In addition, an anomaly detection module, a real-time reasoning module, a data enhancement module and a visualization module are further arranged. The system can accurately identify and analyze images, improve the model performance and generalization ability, meet the real-time requirement of edge computing equipment, generate an interpretable report to assist decision making, and have wide application prospects in the fields of security, medical treatment, automatic driving and the like.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an AI-based image recognition and analysis system. Background Art

[0002] In today's digital age, image recognition and analysis technologies play a key role in numerous fields, such as security monitoring, autonomous driving, medical imaging diagnosis, and industrial inspection. However, traditional image recognition and analysis methods face numerous challenges. Early image recognition technologies were primarily based on handcrafted features and machine learning algorithms, such as the use of HOG (Histogram of Oriented Gradients) features combined with support vector machines (SVMs) for target detection. However, handcrafted features rely on manual design and are difficult to fully exploit the complex information in images. Consequently, recognition accuracy and generalization capabilities are poor when faced with complex scenes and diverse targets. For example, in security monitoring scenarios, varying lighting conditions, occlusions, and changes in the target's posture can significantly reduce the performance of recognition systems based on handcrafted features.

[0003] With the rise of deep learning, convolutional neural networks (CNNs) have achieved remarkable results in the field of image recognition. CNNs can automatically learn image features, eliminating the need for manual feature design, significantly improving recognition accuracy. However, single CNNs still have limitations when processing images in complex scenarios. For example, in autonomous driving scenarios, vehicles must not only identify other vehicles and pedestrians on the road but also simultaneously process various traffic signs, signal lights, and complex road conditions. CNNs alone cannot fully integrate information from multiple modalities, such as the vehicle's speed (time series data) and location information (spatial positioning data), resulting in an incomplete and inaccurate understanding of the scene.

[0004] Furthermore, existing image recognition systems also have shortcomings in model training and optimization. Traditional training methods are prone to falling into local optima, resulting in suboptimal model performance. Furthermore, in real-world applications, data size and quality vary widely, making it a pressing challenge to train efficient and accurate models using limited data. For example, in the field of medical imaging diagnosis, data volumes are often relatively small due to privacy concerns and the difficulty of obtaining them. This places higher demands on model training and generalization capabilities.

[0005] Regarding real-time performance, the widespread adoption of edge computing devices, such as smart cameras and in-vehicle terminals, necessitates deploying image recognition and analysis models on these resource-constrained devices. However, existing models are complex and computationally intensive, making them difficult to meet the real-time requirements of edge devices. For example, in intelligent security monitoring, abnormal behavior must be identified in real time on the local device. If the model calculation time is too long, alerts cannot be issued in a timely manner, compromising security effectiveness. Summary of the Invention

[0006] The purpose of the present invention is to provide an AI-based image recognition and analysis system to solve the problems raised in the above background technology.

[0007] To achieve the above objectives, the present invention provides the following technical solution: an AI-based image recognition and analysis system, the system comprising:

[0008] A data acquisition module is used to acquire real-time image data of the target scene through multi-source sensors, including a high-resolution camera, an infrared sensor, a depth sensor, a spectral sensor, and an ultrasonic sensor;

[0009] A feature extraction module performs multi-scale feature extraction on the real-time image data based on a cascaded convolutional neural network, and generates a semantic feature map of the target object through cross-layer feature fusion;

[0010] A model training module uses a generative adversarial network to perform adversarial training on the semantic feature map, wherein the generator network is constructed using residual dense blocks, and the discriminator network adopts a multi-branch structure and combines a self-attention mechanism to optimize feature discrimination capabilities;

[0011] A multimodal fusion module performs adaptive weighted fusion of the semantic feature map with time series data and spatial positioning data, constructs a dynamic feature selection mechanism based on a gated recurrent unit, and generates a fused multimodal feature vector;

[0012] The dynamic optimization module dynamically optimizes the multimodal feature vector based on an improved genetic algorithm, introduces an adaptive crossover probability and mutation probability mechanism, generates an optimal feature subset through an elite retention strategy, and constructs a classification decision tree based on the optimal feature subset.

[0013] Preferably, the feature extraction module performs multi-scale feature extraction on the real-time image data based on a cascaded convolutional neural network, and generates a semantic feature map of the target object by cross-layer feature fusion, including:

[0014] Constructing a cascaded convolutional neural network structure, wherein the cascaded convolutional neural network includes a shallow feature extraction branch, a middle feature extraction branch, and a deep feature extraction branch, each branch consisting of multiple convolution layers, normalization layers, and activation function layers;

[0015] Atrous convolution is used to expand the receptive field of shallow branches, grouped convolution is used to reduce the computational complexity of middle-layer branches, and a channel attention mechanism is introduced in deep branches to optimize feature channel weights.

[0016] Based on the feature pyramid structure, the output feature maps of the three branches are fused across layers. The feature map resolutions are aligned through upsampling and downsampling operations, and a multi-scale fused feature map is generated by element-by-element addition.

[0017] A semantic segmentation subnetwork is constructed, the multi-scale fusion feature map is input into the semantic segmentation subnetwork, multi-resolution context information is extracted through a spatial pyramid pooling module, the segmentation boundary is refined in combination with a conditional random field, and a semantic feature map of the target object is output.

[0018] Preferably, the model training module uses a generative adversarial network to perform adversarial training on the semantic feature map, including:

[0019] Construct a generator network, which includes an input layer, five residual dense blocks, and an output layer. Each residual dense block contains a dense connection layer and a skip connection layer. The skip connection layer optimizes feature transfer through a channel compression and expansion mechanism.

[0020] Construct a discriminator network consisting of three parallel branches, one for extracting local texture features, one for extracting global structural features, and one for extracting semantic consistency features. Each branch contains multiple convolutional layers and a self-attention module. The self-attention module calculates the spatial association weights of the feature maps through a query-key mechanism.

[0021] Design an adversarial loss function, which includes a generator reconstruction loss term, a discriminator classification loss term, and a feature matching loss term, where the feature matching loss term is obtained by calculating the mean squared error between the generator's intermediate layer features and the real sample features;

[0022] A progressive training strategy is adopted, starting from low-resolution images and gradually increasing the input resolution, and the network weights of the generator and discriminator are simultaneously optimized through a parameter sharing mechanism.

[0023] Preferably, the multimodal fusion module performs adaptive weighted fusion of the semantic feature map with the time series data and the spatial positioning data, including:

[0024] Construct a time series encoding network, use a bidirectional long short-term memory network to encode time series data, and capture long-distance temporal dependencies through a gating mechanism;

[0025] Construct a spatial positioning encoding network to encode spatial positioning data based on a graph convolutional network, and model spatial topological relationships through node feature aggregation and edge weight update mechanisms;

[0026] Design a dynamic feature selection gate, input the semantic feature map, time series encoding features, and spatial positioning encoding features into the dynamic feature selection gate, and generate channel attention weights and spatial attention weights through a fully connected layer;

[0027] The multimodal features are weighted and summed based on the channel attention weight and the spatial attention weight, and the weighted features are temporally fused through a gated recurrent unit to generate a dynamically updated multimodal feature vector.

[0028] Preferably, the dynamic optimization module dynamically optimizes the multimodal feature vector based on an improved genetic algorithm, including:

[0029] Constructing a chromosome encoding rule, taking the dimension of the multimodal feature vector as the chromosome length, and each gene corresponds to the selection state of the feature dimension;

[0030] Designing an adaptive crossover probability function, where the crossover probability decays exponentially with the number of population iterations, and dynamically adjusts the decay rate based on the population diversity index;

[0031] Design an adaptive mutation probability function, which dynamically adjusts the mutation probability according to the difference between the individual fitness value and the average fitness of the population, and adopts a high mutation probability for individuals with low fitness;

[0032] An elite retention strategy is introduced, where the top N individuals with the highest fitness are retained in each iteration and directly enter the next generation population, and the parent individuals are screened through a tournament selection mechanism;

[0033] A classification decision tree is constructed based on the optimal feature subset, the Gini coefficient is used as the node splitting criterion, and the maximum depth of the tree and the number of leaf nodes are limited by a pre-pruning strategy.

[0034] Preferably, the system further includes an anomaly detection module for performing credibility assessment and anomaly identification on the classification results, including:

[0035] Constructing a confidence assessment network based on a twin neural network structure to calculate confidence scores by comparing the classification results with the feature similarity of historical samples;

[0036] Design anomaly determination rules. When the confidence score is lower than the preset threshold, the anomaly detection process is triggered and the isolation forest algorithm is used to perform cluster analysis on abnormal samples.

[0037] A feedback correction mechanism is constructed to input abnormal samples into the incremental learning model and update the node splitting parameters of the classification decision tree through the online gradient descent algorithm.

[0038] Preferably, the system further includes a real-time inference module for deploying a lightweight inference engine on an edge computing device, including:

[0039] Using a network pruning algorithm to compress the classification decision tree and remove subtree nodes whose contribution is lower than a threshold;

[0040] Build a quantization-aware training framework to convert network weights and activation values from floating-point to 8-bit fixed-point numbers, and reduce quantization accuracy loss through an error compensation mechanism;

[0041] Design a dynamic loading strategy to dynamically adjust the parallelism and batch size of model inference based on device computing resources and real-time load conditions.

[0042] Preferably, the system further includes a data enhancement module for improving the generalization ability of the model during the training phase, including:

[0043] Design multimodal data augmentation strategies, apply random geometric transformations and color space perturbations to image data, and add Gaussian noise and time warping to time series data;

[0044] Construct an adversarial sample generator, generate adversarial samples through the fast gradient symbol attack method, and add them to the training dataset.

[0045] Preferably, the system further includes a visualization module for generating an interpretability analysis report including:

[0046] Construct a feature importance evaluation algorithm and calculate the contribution of each feature dimension to the classification result based on the permutation feature importance method.

[0047] Preferably, the present invention further includes an electronic device, comprising:

[0048] processor;

[0049] a memory for storing processor-executable instructions;

[0050] The processor is configured to call the instructions stored in the memory to execute the above-mentioned system functions of AI-based image recognition and analysis.

[0051] Compared with the prior art, the present invention has the following beneficial effects:

[0052] In terms of data collection and feature extraction, the system acquires rich data from multiple sensors, combined with multi-scale feature extraction and cross-layer fusion using a cascaded convolutional neural network. This allows the system to more comprehensively and accurately capture target scene information. For example, in security surveillance, high-resolution cameras provide clear image details, infrared sensors assist with imaging at night or in low light, depth sensors acquire object distances, spectral sensors analyze material properties, and ultrasonic sensors measure distances. This data, processed by the feature extraction module, can accurately identify target objects, from people hidden in shadows to tiny, unusual objects. Compared to traditional single-sensor acquisition and simple feature extraction methods, this significantly improves the ability to identify targets in complex scenes.

[0053] The model training module utilizes a generative adversarial network (GAN) and a unique structural design to enhance model performance. The generator's residual dense blocks and the discriminator's multiple branches work in tandem with a self-attention mechanism. Combined with a progressive training strategy, training is performed incrementally from low-resolution images, accelerating the training process while enabling the model to learn richer features. In industrial product defect detection scenarios, this technology can more accurately identify minute surface defects. Even when defect features are subtle, GAN adversarial training can effectively distinguish between normal and defective products, reducing false positives and improving the accuracy and reliability of product quality inspections.

[0054] The multimodal fusion module adaptively and weightedly fuses semantic feature maps with time series and spatial positioning data, enhancing the system's ability to understand complex scenarios. In the field of autonomous driving, combining vehicle speed (time series data) and position (spatial positioning data) with image semantic features enables vehicles to better plan their routes, accurately determine distances to surrounding vehicles and pedestrians, and their movement trends, improving driving safety. Compared to systems that don't integrate multimodal data, this system offers a more comprehensive understanding of the surrounding environment, avoiding decision-making errors caused by missing or inaccurate single data points.

[0055] The dynamic optimization module utilizes an improved genetic algorithm to dynamically optimize multimodal feature vectors and construct a classification decision tree. Adaptive crossover and mutation probability mechanisms, along with an elite retention strategy, enable the algorithm to quickly find the optimal feature subset, improving classification efficiency and accuracy. In image classification tasks, this module can more efficiently classify massive amounts of images. For example, in image search engines, it can quickly and accurately filter out the images desired by users from a large number of images, enhancing the user experience. A pre-pruning strategy limits the depth and number of leaf nodes in the decision tree, effectively preventing overfitting and enhancing the model's generalization capabilities, ensuring stable performance across diverse datasets and application scenarios.

[0056] The anomaly detection module assesses the credibility of classification results and identifies anomalies. The combination of a confidence assessment network and the isolation forest algorithm enables timely detection of anomalous samples. A feedback correction mechanism updates the classification decision tree through an incremental learning model, improving the system's ability to handle anomalies. In financial transaction risk monitoring, it can monitor transaction data images in real time, quickly identifying abnormal trading behavior, such as fraudulent transactions, and ensuring financial security.

[0057] The real-time inference module deploys a lightweight inference engine on edge computing devices. Through network pruning, quantization-aware training, and dynamic loading strategies, it improves inference speed and reduces resource consumption while ensuring accuracy. In smart cameras, it can analyze video images in real time, enabling real-time monitoring and early warning without experiencing lags or delays due to device resource constraints, thereby enhancing the application value of edge devices. The data enhancement module increases the diversity of training data and improves model generalization capabilities through multimodal data enhancement strategies and adversarial sample generators. In medical imaging diagnosis, it enhances small amounts of medical image data, enabling the model to learn more image variation features, improving the diagnostic accuracy of images from different cases, and reducing misdiagnoses caused by insufficient data.

[0058] The visualization module builds a feature importance assessment algorithm and generates interpretable analysis reports to help users understand the model's decision-making process. In image data analysis in scientific research, researchers can analyze feature importance to understand which factors have a key impact on experimental results, providing strong support for further research and model optimization. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 This is a diagram showing the working principle of the AI-based image recognition and analysis system of the present invention;

[0060] Figure 2 Workflow diagram for the model training module;

[0061] Figure 3 This is the workflow diagram of the dynamic optimization module;

[0062] Figure 4 This is the workflow diagram of the anomaly detection module. DETAILED DESCRIPTION

[0063] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0064] See also Figure 1-4The present invention provides an AI-based image recognition and analysis system that uses multi-source sensors to collect data and uses AI technology to perform image recognition and analysis. The overall implementation scheme is as follows:

[0065] The data acquisition module uses a variety of sensors to acquire real-time image data of the target scene. High-resolution cameras are used to obtain clear visual images, infrared sensors capture thermal radiation from targets at night or in low-light environments, depth sensors acquire depth information from objects in the scene, spectral sensors analyze the spectral characteristics of objects, and ultrasonic sensors measure distance and other information. These sensors work together to provide a rich source of data for subsequent image analysis.

[0066] Feature Extraction Module: This module extracts multi-scale features from real-time image data using a cascaded convolutional neural network. It then generates a semantic feature map of the target object through cross-layer feature fusion. Cascaded convolutional neural networks can extract image features from different layers and scales, while cross-layer feature fusion leverages the strengths of each layer to create a more representative semantic feature map.

[0067] Model training module: A generative adversarial network is used to perform adversarial training on semantic feature maps. The generator network is constructed using residual dense blocks, which effectively utilizes features and improves generation capabilities. The discriminator network uses a multi-branch structure and incorporates a self-attention mechanism to optimize feature discrimination, enabling the network to better distinguish between real and generated samples.

[0068] Multimodal Fusion Module: This module adaptively and weightedly fuses semantic feature maps with time series data and spatial positioning data. By building a dynamic feature selection mechanism based on gated recurrent units, it dynamically fuses different data according to their importance, generating a fused multimodal feature vector that fully utilizes information from multiple data types.

[0069] Dynamic Optimization Module: Dynamically optimizes multimodal feature vectors based on an improved genetic algorithm. Adaptive crossover and mutation probability mechanisms are introduced to dynamically adjust the probability of genetic operations based on the population's evolution, improving search efficiency. An elite retention strategy is used to generate an optimal feature subset, and a classification decision tree is constructed based on this subset to make classification decisions for the target object.

[0070] The implementation of the present invention will be further described below with reference to Examples 1 to 6.

[0071] Example 1:

[0072] In the feature extraction module, in order to more effectively extract the semantic feature map of the target object from the real-time image data, the following specific methods are adopted:

[0073] First, a cascaded convolutional neural network structure is constructed, consisting of shallow, mid-level, and deep-level feature extraction branches. Each branch consists of multiple convolutional layers, normalization layers, and activation layers. The convolutional layers extract image features, the normalization layers normalize the data, making network training more stable, and the activation layers introduce nonlinearity to the network, enhancing its expressive power.

[0074] In shallow branches, dilated convolution is used to expand the receptive field. Dilated convolution inserts holes into the convolution kernel, expanding its receptive field without increasing parameters or computational complexity, thereby capturing a wider range of image information. For example, a 3×3 convolution kernel can be made to have a receptive field equivalent to a 5×5 or even larger kernel through dilated convolution, thus capturing a wider range of features in the image.

[0075] For mid-level branches, grouped convolution is used to reduce computational complexity. Grouped convolution divides the input channels into multiple groups. Each convolution kernel performs convolution operations only within its corresponding group, and then concatenates the results. Assuming the number of input channels is C and the number of groups is G, each convolution kernel only needs to process data from C / G channels, significantly reducing the amount of computation while also improving the model's generalization capabilities.

[0076] In the deep branches, a channel attention mechanism is introduced to optimize feature channel weights. This mechanism performs global average pooling on the features of each channel to generate a global feature vector representing the channel's characteristics. This vector is then input into a network containing fully connected layers, where a series of calculations are performed to obtain per-channel weights. These weights are used to weight the original channel features, enhancing the features of important channels and suppressing unimportant ones, thereby optimizing feature representation.

[0077] Based on a feature pyramid structure, the output feature maps of the three branches are fused across layers. Upsampling and downsampling operations align feature map resolutions. Upsampling amplifies low-resolution feature maps, while downsampling reduces high-resolution feature maps, achieving consistent resolution for feature maps from different branches. Multi-scale fused feature maps are then generated using element-by-element addition. This simple and effective fusion method fully integrates feature information at different scales.

[0078] A semantic segmentation subnetwork is constructed, and the multi-scale fused feature map is input into it. Multi-resolution contextual information is extracted through the spatial pyramid pooling module, which pools feature maps at different scales and then concatenates these pooling results to obtain rich contextual information. Segmentation boundaries are refined using conditional random fields. Conditional random fields consider the spatial relationships and feature similarities between pixels, enabling better optimization of segmentation boundaries and ultimately outputting a semantic feature map of the target object. These specific steps enable more accurate extraction of the semantic feature map of the target object, providing high-quality feature data for subsequent model training and analysis.

[0079] Example 2:

[0080] The model training module uses a generative adversarial network to perform adversarial training on the semantic feature map. The specific implementation methods include:

[0081] First, the generator network is constructed. The generator network includes an input layer, five residual dense blocks, and an output layer. Each residual dense block contains a dense connection layer and a skip connection layer. The skip connection layer optimizes feature transfer through channel compression and expansion mechanisms. In the dense connection layer, the input of each layer contains the output of all previous layers. This connection method can fully utilize features and avoid information loss. The skip connection layer adjusts the number of feature channels through channel compression and expansion mechanisms. Assume that the number of input feature channels is C in , first compress the number of channels to C through a convolutional layer s (C s <C in ) to reduce the amount of computation, and then another convolutional layer is used to expand the number of channels back to C in , which not only ensures the effective transfer of features but also optimizes the computational efficiency.

[0082] Construct a discriminator network, which consists of three parallel branches, which are used to extract local texture features, global structural features, and semantic consistency features respectively. Each branch contains multiple convolutional layers and self-attention modules. The self-attention module calculates the spatial association weight of the feature map through the query-key-value mechanism. Specifically, for the input feature map F, it is mapped into the query vector Q, the key vector K, and the value vector V respectively. By calculating the dot product of Q and K and performing normalization, the spatial association weight W is obtained, that is, Among them QK T represents the dot product of the query vector and the transpose of the key vector, The dot product results at all positions are summed and exponentially calculated. The weight W is then multiplied by the value vector V to obtain a feature map processed by the self-attention mechanism. This method can adaptively adjust the weight of features based on the degree of correlation between different positions in the feature map, thereby extracting features more accurately.

[0083] Design an adversarial loss function, which includes the generator reconstruction loss term, the discriminator classification loss term, and the feature matching loss term. The feature matching loss term is obtained by calculating the mean square error between the generator intermediate layer features and the real sample features. Let the generator intermediate layer features be G mid , the true sample feature is T, the feature matching loss term is L fm The calculation formula is n is the feature dimension, G mid (i) and T(i) represent the values of the generator intermediate layer features and the real sample features in the i-th dimension, respectively. The generator reconstruction loss term is used to measure the difference between the generated samples and the real samples, prompting the generator to generate images that are closer to the real samples; the discriminator classification loss term is used to train the discriminator to accurately distinguish between real samples and generated samples.

[0084] A progressive training strategy is employed, starting with low-resolution images and gradually increasing the input resolution. Using low-resolution images in the early stages of training reduces computational effort, speeds up training, and helps the model learn the basic features of the image. As training progresses, the input resolution is gradually increased, enabling the model to learn more refined features. A parameter sharing mechanism is used to simultaneously optimize the network weights of the generator and discriminator, ensuring that the two networks mutually enhance and optimize each other during training, improving the model's training effectiveness and performance.

[0085] Example 3:

[0086] The multimodal fusion module performs adaptive weighted fusion of semantic feature maps with time series data and spatial positioning data. The specific implementation methods include:

[0087] First, a time series encoding network is constructed, using a bidirectional long short-term memory (Bi-LSTM) network to encode time series data. The Bi-LSTM consists of a forward LSTM and a backward LSTM. The forward LSTM processes data sequentially starting from the beginning of the time series, while the backward LSTM processes data backward from the end of the time series. Through this bidirectional structure, the Bi-LSTM can simultaneously capture past and future information, using a gating mechanism to capture long-range temporal dependencies. The gating mechanism includes an input gate, a forget gate, and an output gate. The input gate controls the input of new information, the forget gate determines which historical information to retain or discard, and the output gate determines the output content. For example, when processing a piece of temperature data that changes over time, the Bi-LSTM can use the gating mechanism to remember temperature trends over a period of time, as well as recent temperature fluctuations, thereby more accurately encoding the temperature characteristics of the current point in time.

[0088] Construct a spatial positioning encoding network to encode spatial positioning data based on a graph convolutional network. The graph convolutional network models spatial topological relationships through node feature aggregation and edge weight update mechanisms. Assume that spatial positioning data can be represented as a graph structure G = (V, E), where V is a set of nodes representing various locations in space, and E is a set of edges representing the connection relationships between nodes. In the graph convolutional network, the features of each node are updated by aggregating the features of its adjacent nodes, and the edge weights are updated based on factors such as the distance between nodes and the connection strength. For example, in an indoor positioning scenario, different rooms can be regarded as nodes in the graph, and the channels between rooms are edges. In this way, the graph convolutional network can learn the spatial relationship between rooms and the position features of each room in the entire space.

[0089] Design a dynamic feature selection gate and input the semantic feature map, time series encoding feature, and spatial positioning encoding feature into the gate. Generate channel attention weights and spatial attention weights through the fully connected layer. The fully connected layer can comprehensively process the input features and generate corresponding weights according to the importance of different features. Assuming that the input semantic feature map is S, the time series encoding feature is T, and the spatial positioning encoding feature is L, the output of the fully connected layer is the channel attention weight W c and spatial attention weight W s The channel attention weight is used to adjust the importance of different channel features, and the spatial attention weight is used to adjust the importance of features in spatial positions.

[0090] Based on the channel attention weight and spatial attention weight, the multimodal features are weighted and summed, and the weighted features are temporally fused through the gated recurrent unit to generate a dynamically updated multimodal feature vector. The weighted summation process is: F = W c ·S+W cT+W c L, where F represents the weighted multimodal features, W c is the channel attention weight, S is the semantic feature map, T is the time series encoding feature, and L is the spatial location encoding feature. The Gated Recurrent Unit (GRU) is similar to the LSTM, but with a simpler structure. It controls the flow of information through reset and update gates. During time series fusion, the GRU can dynamically update the multimodal feature vector based on feature changes at different moments, enabling it to better reflect dynamic changes in the data and provide more accurate information for subsequent analysis and decision-making.

[0091] Example 4:

[0092] The dynamic optimization module dynamically optimizes the multimodal feature vector based on an improved genetic algorithm. The implementation method includes:

[0093] A chromosome encoding rule is constructed, where the dimensions of the multimodal feature vector are used as the chromosome length, and each gene corresponds to the selection status of a feature dimension. For example, if the multimodal feature vector has n dimensions, then the chromosome is a binary string of length n, where each gene has a value of 0 or 1, with 0 indicating that the corresponding feature dimension is not selected and 1 indicating that it is selected. This encoding method can intuitively represent the selection of feature subsets.

[0094] Design an adaptive crossover probability function, the crossover probability decays exponentially with the increase of population iteration number, and the decay rate is dynamically adjusted based on the population diversity index. Let the crossover probability be P c , the number of population iterations is t, the population diversity index is D, and the initial crossover probability is P c0 , the attenuation coefficient is α, then the calculation formula of the crossover probability is P c =P c0 ×e -αt×D In the early stages of the algorithm, a higher crossover probability promotes gene exchange between individuals, increasing population diversity. As the number of iterations increases, the crossover probability gradually decreases to prevent over-searching and ensure algorithm stability. The population diversity index D can be calculated by calculating the degree of difference between individuals in the population, for example, by calculating the average Euclidean distance between individuals.

[0095] Design an adaptive mutation probability function, the mutation probability is dynamically adjusted with the difference between the individual fitness value and the population average fitness, and a high mutation probability is used for individuals with low fitness. Let the mutation probability be P m , the individual fitness value is f i , the average fitness of the population is The mutation probability adjustment coefficient is β, and the calculation formula of the mutation probability is: (when When When P m It can be set to a smaller fixed value. This will give individuals with low fitness more opportunities to mutate, which may produce better individuals and improve the overall fitness of the population.

[0096] An elite retention strategy is introduced. The top N individuals with the highest fitness are retained in each iteration and directly enter the next generation. Parent individuals are then selected through a tournament selection mechanism. The tournament selection mechanism randomly selects k individuals from the population (k is the tournament size) and then selects the individual with the highest fitness from these k individuals as the parent. This selection method ensures that outstanding individuals have a greater chance of participating in reproduction while also increasing the randomness of the selection process, preventing the algorithm from falling into local optima.

[0097] A classification decision tree is constructed based on the optimal feature subset, and the Gini coefficient is used as the node splitting criterion. The maximum depth of the tree and the number of leaf nodes are limited by the pre-pruning strategy. The Gini coefficient is used to measure the impurity of the data and is calculated as follows: Where m is the number of data categories, p i is the proportion of data in the i-th category in the sample. When building a classification decision tree, the feature with the smallest Gini coefficient is selected as the splitting feature to ensure that the data in the child nodes after the split is purer. The pre-pruning strategy prevents excessive growth of the decision tree and overfitting by setting an upper limit on the maximum depth of the tree and the number of leaf nodes, thereby improving the generalization ability of the model.

[0098] Example 5:

[0099] For the anomaly detection module, we first build a confidence assessment network based on the twin neural network structure. The twin neural network contains two sub-networks with the same structure, which extract features from the classification results and historical samples respectively. The confidence score is calculated by comparing the feature similarity between the classification results and historical samples. Assume that the feature vector of the classification result is F c , the characteristic vector of the historical sample is F h , the similarity between them can be calculated using methods such as cosine similarity, such as confidence score where · represents the dot product of the vectors, and ||·|| represents the norm of the vectors. A higher similarity means a higher confidence score, indicating a more reliable classification result.

[0100] Design anomaly determination rules. When the confidence score falls below a preset threshold, the anomaly detection process is triggered. The preset threshold can be adjusted based on the actual application scenario and data characteristics. Use the isolation forest algorithm to cluster anomaly samples. The isolation forest algorithm constructs a series of random binary trees and calculates the path length of each sample within the tree. Normal samples have short path lengths, while anomalous samples have long path lengths. By comparing a sample's path length with the average path length, it can be determined whether the sample is an anomaly.

[0101] A feedback correction mechanism is built to feed abnormal samples into an incremental learning model and update the node splitting parameters of the classification decision tree using an online gradient descent algorithm. This incremental learning model can update model parameters based on new sample data without retraining the entire model. The online gradient descent algorithm calculates the gradient for each new sample and gradually updates the model parameters, enabling the classification decision tree to better adapt to the new data distribution and improving the model's ability to identify abnormal samples.

[0102] For the data enhancement module, a multimodal data enhancement strategy is designed. Random geometric transformations and color space perturbations are applied to the image data. Random geometric transformations include rotation, translation, scaling and other operations. For example, random rotation of a certain angle can increase the diversity of the image, allowing the model to learn the target features at different angles; random translation can simulate the situation of the target in different positions. Color space perturbation can change the brightness, contrast, saturation, etc. of the image, making the model more adaptable to images of different color styles. Gaussian noise and time distortion are added to time series data. Adding Gaussian noise can simulate the noise interference in the actual environment and enhance the model's anti-noise ability; time distortion enables the model to learn different change patterns of the time series by disrupting or stretching the order of the time series to a certain extent.

[0103] Construct an adversarial sample generator and generate adversarial samples using the fast gradient sign attack method. The fast gradient sign attack method is based on the gradient information of the model. By adding a small perturbation to the input sample, the output of the model is wrong. Let the loss function of the model be L, the input sample be x, the label be y, and the calculation formula of perturbation ∈ is where ∈0 is the perturbation step size, sign(·) is the sign function, is the gradient of the loss function with respect to the input sample x. The generated adversarial samples are added to the training dataset so that the model can learn the characteristics of the adversarial samples during training and improve the robustness of the model.

[0104] Example 6:

[0105] In the real-time inference module, given the limited resources of edge computing devices, a series of optimization measures have been implemented to efficiently deploy lightweight inference engines on these devices. First, a network pruning algorithm is used to compress the classification decision tree. After constructing the classification decision tree, it is necessary to determine which subtree nodes contribute less to the final classification result. By analyzing the role of each node in the classification process, the node's contribution metric is calculated. For example, contribution can be measured by calculating the change in data purity before and after a node split. If the data purity does not improve significantly after a node split, the node's contribution to the classification result is relatively low. A contribution threshold is set, and subtree nodes with contributions below the threshold are removed. This significantly reduces the size of the decision tree and reduces computational complexity. This allows for faster traversal of the decision tree and classification results during inference on the edge computing device, effectively improving inference efficiency.

[0106] Next, a quantization-aware training framework is constructed to convert network weights and activation values from floating-point to 8-bit fixed-point numbers. In actual neural network calculations, while floating-point data can accurately represent numerical values, it occupies a large amount of storage space and is computationally expensive. 8-bit fixed-point data, on the other hand, can significantly reduce both data storage and computational complexity. Precision loss is inevitable during the conversion process. To mitigate this loss, an error compensation mechanism is introduced. The error compensation mechanism estimates and compensates for the quantization error during the quantization process. For example, this mechanism records the difference between the data before and after quantization and, in subsequent calculations, processes this difference according to specific rules to correct for the quantization error. Compensation nodes can be added after quantization in key layers of the network, such as convolutional or fully connected layers. These nodes adjust the quantized data according to a pre-defined compensation algorithm, ensuring that the overall calculation results are as close as possible to those using floating-point data. This improves computational speed while maintaining model accuracy.

[0107] At the same time, a dynamic loading strategy is designed to dynamically adjust the parallelism and batch size of model inference based on the device's computing resources and real-time load. During edge computing device operation, the computing resources and real-time load are constantly changing. When the device has sufficient computing resources and a low real-time load, the parallelism of model inference can be increased and the batch size can be increased. Increased parallelism allows for simultaneous processing of multiple data samples, while increased batch size fully utilizes the device's computing power and improves inference efficiency. Conversely, when device computing resources are limited or the real-time load is high, the parallelism and batch size can be reduced to avoid performance degradation or even crashes due to overload. By monitoring device metrics such as CPU usage and memory utilization in real time to assess the device's computing resources and real-time load, model inference parameters can be dynamically adjusted to ensure stable and efficient model operation on the edge computing device.

[0108] In the visualization module, a feature importance assessment algorithm is constructed, using the permutation feature importance method to calculate the contribution of each feature dimension to the classification result. Specifically, for a trained model and training dataset, the model's classification accuracy on the original dataset is first recorded. Then, the values of a feature dimension in the dataset are randomly permuted, disrupting the order of the data in that dimension. The model is then inferred using the permuted dataset, and the model's classification accuracy is recorded. The difference between the two accuracy scores is calculated, reflecting the influence of that feature dimension on the classification result. The larger the difference, the greater the contribution of that feature dimension to the classification result. This operation is repeated for each feature dimension in the dataset, resulting in an assessment of the importance of each feature dimension. Based on these results, intuitive visualization charts, such as bar charts, can be generated, with each feature dimension plotted on the horizontal axis and the corresponding contribution value on the vertical axis. This approach allows users to clearly understand which features play a key role in image recognition and analysis, providing valuable support for further model optimization, selecting appropriate features, and understanding the model's decision-making process.

[0109] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "includes," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus.

[0110] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. An AI-based image recognition and analysis system, characterized in that: include: A data acquisition module is used to acquire real-time image data of the target scene through multi-source sensors, including a high-resolution camera, an infrared sensor, a depth sensor, a spectral sensor, and an ultrasonic sensor; A feature extraction module performs multi-scale feature extraction on the real-time image data based on a cascaded convolutional neural network, and generates a semantic feature map of the target object through cross-layer feature fusion; A model training module uses a generative adversarial network to perform adversarial training on the semantic feature map, wherein the generator network is constructed using residual dense blocks, and the discriminator network adopts a multi-branch structure and combines a self-attention mechanism to optimize feature discrimination capabilities; A multimodal fusion module performs adaptive weighted fusion of the semantic feature map with time series data and spatial positioning data, constructs a dynamic feature selection mechanism based on a gated recurrent unit, and generates a fused multimodal feature vector; The dynamic optimization module dynamically optimizes the multimodal feature vector based on an improved genetic algorithm, introduces an adaptive crossover probability and mutation probability mechanism, generates an optimal feature subset through an elite retention strategy, and constructs a classification decision tree based on the optimal feature subset.

2. The system according to claim 1, wherein: The feature extraction module performs multi-scale feature extraction on the real-time image data based on a cascaded convolutional neural network and generates a semantic feature map of the target object by cross-layer feature fusion, including: Constructing a cascaded convolutional neural network structure, wherein the cascaded convolutional neural network includes a shallow feature extraction branch, a middle feature extraction branch, and a deep feature extraction branch, each branch consisting of multiple convolution layers, normalization layers, and activation function layers; Atrous convolution is used to expand the receptive field of shallow branches, grouped convolution is used to reduce the computational complexity of middle-layer branches, and a channel attention mechanism is introduced in deep branches to optimize feature channel weights. Based on the feature pyramid structure, the output feature maps of the three branches are fused across layers. The feature map resolutions are aligned through upsampling and downsampling operations, and a multi-scale fused feature map is generated by element-by-element addition. A semantic segmentation subnetwork is constructed, the multi-scale fusion feature map is input into the semantic segmentation subnetwork, multi-resolution context information is extracted through a spatial pyramid pooling module, the segmentation boundary is refined in combination with a conditional random field, and a semantic feature map of the target object is output.

3. The system according to claim 1, wherein: The model training module, which uses a generative adversarial network to perform adversarial training on the semantic feature map, includes: Construct a generator network, which includes an input layer, five residual dense blocks, and an output layer. Each residual dense block contains a dense connection layer and a skip connection layer. The skip connection layer optimizes feature transfer through a channel compression and expansion mechanism. Construct a discriminator network consisting of three parallel branches, one for extracting local texture features, one for extracting global structural features, and one for extracting semantic consistency features. Each branch contains multiple convolutional layers and a self-attention module. The self-attention module calculates the spatial association weights of the feature maps through a query-key mechanism. Design an adversarial loss function, which includes a generator reconstruction loss term, a discriminator classification loss term, and a feature matching loss term, where the feature matching loss term is obtained by calculating the mean squared error between the generator's intermediate layer features and the real sample features; A progressive training strategy is adopted, starting from low-resolution images and gradually increasing the input resolution, and the network weights of the generator and discriminator are simultaneously optimized through a parameter sharing mechanism.

4. The system according to claim 1, wherein: The multimodal fusion module performs adaptive weighted fusion of the semantic feature map with the time series data and the spatial positioning data, including: Construct a time series encoding network, use a bidirectional long short-term memory network to encode time series data, and capture long-distance temporal dependencies through a gating mechanism; Construct a spatial positioning encoding network to encode spatial positioning data based on a graph convolutional network, and model spatial topological relationships through node feature aggregation and edge weight update mechanisms; Design a dynamic feature selection gate, input the semantic feature map, time series encoding features, and spatial positioning encoding features into the dynamic feature selection gate, and generate channel attention weights and spatial attention weights through a fully connected layer; The multimodal features are weighted and summed based on the channel attention weight and the spatial attention weight, and the weighted features are temporally fused through a gated recurrent unit to generate a dynamically updated multimodal feature vector.

5. The system according to claim 1, wherein: The dynamic optimization module dynamically optimizes the multimodal feature vector based on an improved genetic algorithm, including: Constructing a chromosome encoding rule, taking the dimension of the multimodal feature vector as the chromosome length, and each gene corresponds to the selection state of the feature dimension; Designing an adaptive crossover probability function, where the crossover probability decays exponentially with the number of population iterations, and dynamically adjusts the decay rate based on the population diversity index; Design an adaptive mutation probability function, which dynamically adjusts the mutation probability according to the difference between the individual fitness value and the average fitness of the population, and adopts a high mutation probability for individuals with low fitness; An elite retention strategy is introduced, where the top N individuals with the highest fitness are retained in each iteration and directly enter the next generation population, and the parent individuals are screened through a tournament selection mechanism; A classification decision tree is constructed based on the optimal feature subset, the Gini coefficient is used as the node splitting criterion, and the maximum depth of the tree and the number of leaf nodes are limited by a pre-pruning strategy.

6. The system according to claim 1, wherein: It also includes an anomaly detection module, which is used to evaluate the credibility of classification results and identify anomalies, including: Constructing a confidence assessment network based on a twin neural network structure to calculate confidence scores by comparing the classification results with the feature similarity of historical samples; Design anomaly determination rules. When the confidence score is lower than the preset threshold, the anomaly detection process is triggered and the isolation forest algorithm is used to perform cluster analysis on abnormal samples. A feedback correction mechanism is constructed to input abnormal samples into the incremental learning model and update the node splitting parameters of the classification decision tree through the online gradient descent algorithm.

7. The system according to claim 1, wherein: It also includes a real-time inference module for deploying lightweight inference engines on edge computing devices, including: Using a network pruning algorithm to compress the classification decision tree and remove subtree nodes whose contribution is lower than a threshold; Build a quantization-aware training framework to convert network weights and activation values from floating-point to 8-bit fixed-point numbers, and reduce quantization accuracy loss through an error compensation mechanism; Design a dynamic loading strategy to dynamically adjust the parallelism and batch size of model inference based on device computing resources and real-time load conditions.

8. The system according to claim 1, wherein: It also includes a data augmentation module to improve the generalization ability of the model during the training phase, including: Design multimodal data augmentation strategies, apply random geometric transformations and color space perturbations to image data, and add Gaussian noise and time warping to time series data; Construct an adversarial sample generator, generate adversarial samples through the fast gradient symbol attack method, and add them to the training dataset.

9. The system according to claim 1, wherein: It also includes a visualization module for generating interpretable analysis reports including: Construct a feature importance evaluation algorithm and calculate the contribution of each feature dimension to the classification result based on the permutation feature importance method.

10. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the system function described in any one of claims 1 to 9.

Images