Zero-sample visual identification method supporting minute-level modeling and hour-level online

Abstract

The invention provides a zero-sample visual identification method supporting minute-level modeling and hour-level online, and relates to the technical field of computer vision, and the method comprises the steps: carrying out the semantic analysis fusion of a monitoring sample sequence set according to a multi-modal semantic analysis module; introducing a category distinguishing attention enhancement mechanism, and constructing a prototype generation channel; introducing a visual identification global loss function, and establishing a zero-sample visual identification channel; obtaining a real-time monitoring image, and inputting the real-time monitoring image to a multi-modal semantic analysis module; inputting the multi-modal semantic feature vector into a zero sample visual identification channel; and when the first identification result is unknown, activating a prototype driving optimization identification mechanism, and performing identification analysis in combination with a prototype generation channel. According to the method and the device, the technical problem that in the prior art, zero-sample visual identification is difficult to meet the requirements of rapid modeling and efficient identification, so that the bottleneck of zero-sample visual identification efficiency is prominent is solved, minute-level modeling and hour-level online are realized, and the visual identification efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a zero-sample visual recognition method that supports minute-level modeling and hour-level online deployment. Background Art

[0002] The field of computer vision has experienced rapid growth in recent years, with widespread application in scenarios such as intelligent security, industrial inspection, and e-commerce platforms. Currently, zero-shot visual recognition methods primarily focus on methods based on embedding space mapping and generative models. Both methods suffer from significant shortcomings in practical applications, making it difficult to meet the demands for rapid modeling and efficient recognition. Methods based on embedding space mapping rely on complex cross-modal mapping networks, with a single inference time of 200 to 300 milliseconds, making them difficult to meet real-time requirements. Generative model-based methods require multiple GPUs to train continuously for 6 to 8 hours, resulting in a lengthy training process from data preparation to model availability, making it impossible to achieve minute-level modeling and hour-level production. Methods based on embedding space mapping suffer from cross-modal alignment errors due to the semantic gap, which impacts recognition accuracy. Generative model-based methods generate virtual samples with distribution differences from real data, which can easily lead to model overfitting in complex scenarios, resulting in reduced recognition accuracy.

[0003] In summary, there is a technical problem in the existing technology that zero-shot visual recognition cannot meet the requirements of rapid modeling and efficient recognition, resulting in a prominent bottleneck in the efficiency of zero-shot visual recognition. Summary of the Invention

[0004] The purpose of this application is to provide a zero-sample visual recognition method that supports minute-level modeling and hour-level online launch, so as to solve the technical problem in the existing technology that zero-sample visual recognition efficiency bottleneck is prominent because zero-sample visual recognition cannot meet the needs of rapid modeling and efficient recognition.

[0005] In view of the above problems, the present application provides a zero-sample visual recognition method that supports minute-level modeling and hour-level online launch, wherein the zero-sample visual recognition method that supports minute-level modeling and hour-level online launch includes: performing semantic parsing fusion on the monitoring sample sequence set of the monitoring system according to the multimodal semantic parsing module to establish a semantic parsing vector space; introducing a category-differentiated attention enhancement mechanism to perform GAN learning on the semantic parsing vector space to construct a prototype generation channel; introducing a global visual recognition loss function, combining the semantic parsing vector space to perform knowledge distillation on the lightweight model to establish a zero-sample visual recognition channel; according to the monitoring system, obtaining a real-time monitoring image, and inputting the real-time monitoring image into the multimodal semantic parsing module to obtain a multimodal semantic feature vector; inputting the multimodal semantic feature vector into the zero-sample visual recognition channel to obtain a first recognition result; when the first recognition result is unknown, activating the prototype-driven optimization recognition mechanism, combining the prototype generation channel to perform recognition analysis on the multimodal semantic feature vector to obtain a second recognition result.

[0006] Optionally, the monitoring sample sequence set is input into the multimodal semantic parsing module to obtain a semantic parsing first space, which includes multiple sample feature vector sets; conflict detection is performed on the semantic parsing first space according to the monitoring sample sequence set to obtain a parsing conflict detection result, and the semantic parsing first space is corrected according to the parsing conflict detection result to obtain a semantic parsing second space; scale alignment mapping is performed on the semantic parsing second space to obtain a semantic parsing third space; multi-layer perceptron fusion is performed on the semantic parsing third space to generate a semantic parsing fourth space; and normalization output is performed on the semantic parsing fourth space to generate the semantic parsing vector space.

[0007] Optionally, according to the monitoring system, a known category image sample set is retrieved; text description configuration is performed based on the known category image sample set to obtain a known category text description sample set; inter-category relationship configuration is performed based on the known category image sample set to construct a relationship graph for each category; the known category image sample set, the known category text description sample set and the relationship graph for each category are aligned to generate the monitoring sample sequence set.

[0008] Optionally, a visual feature extraction module is constructed based on a convolutional neural network; a text feature extraction module is built based on a natural language processing model; a semantic relationship extraction module is constructed based on a graph neural network; the visual feature extraction module, the text feature extraction module and the semantic relationship extraction module are connected as parallel nodes to generate the multimodal semantic parsing module.

[0009] Optionally, attention enhancement is performed on the semantic parsing vector space according to the category-discriminating attention enhancement mechanism to obtain an enhanced semantic vector space; supervised training is performed on the generation network using the enhanced semantic vector space as input data and a known category image sample set as output data to obtain a prototype generator; multiple prototype generation images of the prototype generator during the training process are synchronously collected to obtain a prototype generation space; multi-dimensional loss evaluation is performed on the prototype generation space according to the known category image sample set to establish a generation loss space; discriminative learning is performed on the discriminative network using the prototype generation space as an input node and the generation loss space as an output node to generate a prototype discriminator; alternating iterative optimization training is performed on the prototype generator and the prototype discriminator to generate the prototype generation channel that meets the predetermined convergence conditions.

[0010] Optionally, feature decomposition is performed on each vector in the semantic parsing vector space to obtain each vector feature set; category differentiation contribution evaluation is performed on each vector feature set to obtain each feature contribution evaluation set; attention weights are allocated to each vector feature set based on each feature contribution evaluation set to obtain multiple attention weight vectors, and attention enhancement is performed on the semantic parsing vector space based on the multiple attention weight vectors.

[0011] Optionally, the prototype generation space is evaluated for pixel distribution difference loss based on the known category image sample set to obtain a first feature loss evaluation set; the prototype generation space is evaluated for image authenticity adversarial loss based on the known category image sample set to obtain a second feature loss evaluation set; the prototype generation space is evaluated for category semantic consistency loss based on the known category image sample set to obtain a third feature loss evaluation set; the first feature loss evaluation set, the second feature loss evaluation set and the third feature loss evaluation set are aligned to obtain the generation loss space.

[0012] Optionally, iterative training is performed according to the semantic parsing vector space to obtain a trained visual recognition teacher model; knowledge distillation is performed on the lightweight model according to the visual recognition teacher model to obtain a visual recognition student model; loss analysis is performed on the visual recognition student model according to the visual recognition teacher model to determine the first loss coefficient of visual recognition; loss analysis is performed on the visual recognition student model according to the semantic parsing vector space to determine the second loss coefficient of visual recognition; the first loss coefficient of visual recognition and the second loss coefficient of visual recognition are input into the visual recognition global loss function to obtain the global loss coefficient of visual recognition; if the global loss coefficient of visual recognition is greater than or equal to the global loss threshold of visual recognition, iterative distillation optimization is performed on the visual recognition student model according to the global loss threshold of visual recognition to generate the zero-sample visual recognition channel.

[0013] Optionally, the lightweight model includes but is not limited to MobileNet, ShuffleNet, SqueezeNet, EfficientNet-Lite and NanoViT.

[0014] Optionally, when the first recognition result is unknown, the multimodal semantic feature vector is input into the prototype generation channel to obtain multiple predicted prototype images; the multiple predicted prototype images are optimized and recognized according to the prototype-driven optimization recognition mechanism to generate the second recognition result; wherein, the prototype-driven optimization recognition mechanism includes: constructing multiple prototype feature vectors corresponding to the multiple predicted prototype images; performing similarity evaluation on each prototype feature vector according to the multimodal semantic feature vector to obtain a prototype matching evaluation set; performing prototype matching degree maximization optimization on the multiple predicted prototype images according to the prototype matching evaluation set to determine the prototype matching image, and recording the category corresponding to the prototype matching image as the second recognition result.

[0015] The technical solution provided in this application has at least the following beneficial effects:

[0016] A semantic parsing vector space is established by performing semantic parsing fusion on a monitoring sample sequence set of a monitoring system according to a multimodal semantic parsing module; a category-differentiated attention enhancement mechanism is introduced to perform GAN learning on the semantic parsing vector space to construct a prototype generation channel; a global loss function for visual recognition is introduced, and knowledge distillation is performed on a lightweight model in combination with the semantic parsing vector space to establish a zero-sample visual recognition channel; a real-time monitoring image is obtained according to the monitoring system, and the real-time monitoring image is input into the multimodal semantic parsing module to obtain a multimodal semantic feature vector; the multimodal semantic feature vector is input into the zero-sample visual recognition channel to obtain a first recognition result; when the first recognition result is unknown, the prototype-driven optimal recognition mechanism is activated, and the multimodal semantic feature vector is recognized and analyzed in combination with the prototype generation channel to obtain a second recognition result. That is to say, by establishing a semantic parsing vector space and using a generative adversarial network to construct a prototype generation channel, the semantic parsing vector space is used to perform knowledge distillation on the lightweight network, and the multimodal semantic feature vector of the image to be identified is input into the zero-sample visual recognition channel for identification. If the result is an unknown class, the prototype-driven optimal recognition mechanism is activated, and the recognition is optimized again through the prototype generation channel to output the second result, achieving modeling in minutes and online launch in hours, thereby improving the efficiency of visual recognition.

[0017] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, which can be implemented in accordance with the contents of the description, and to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are specifically listed below. It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easy to understand through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in this application or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely exemplary, and a person of ordinary skill in the art can obtain other drawings based on the provided drawings without creative work.

[0019] Figure 1 This is a flowchart of a zero-sample visual recognition method that supports minute-level modeling and hour-level online deployment for this application.

[0020] Figure 2 This is a flowchart for establishing a semantic parsing vector space in a zero-shot visual recognition method that supports minute-level modeling and hour-level online deployment. DETAILED DESCRIPTION

[0021] This application solves the technical problem in the prior art that zero-sample visual recognition efficiency bottlenecks are prominent because zero-sample visual recognition cannot meet the needs of rapid modeling and efficient recognition by providing a zero-sample visual recognition method that supports minute-level modeling and hour-level online production. By establishing a semantic parsing vector space and using a generative adversarial network to construct a prototype generation channel, the semantic parsing vector space is used to perform knowledge distillation on the lightweight network, and the multimodal semantic feature vector of the image to be recognized is input into the zero-sample visual recognition channel for recognition. If the result is an unknown class, the prototype-driven optimal recognition mechanism is activated, and the recognition is optimized again through the prototype generation channel to output a second result, achieving minute-level modeling and hour-level online production, and improving the efficiency of visual recognition.

[0022] Below, the technical solutions in this application will be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of this application, rather than all the embodiments of this application. It should be understood that this application is not limited to the example embodiments described herein. Based on the embodiments of this application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application. It should also be noted that, for the convenience of description, only the parts related to this application, rather than all of them, are shown in the accompanying drawings.

[0023] For examples, please see the attached Figure 1 The present application provides a zero-sample visual recognition method that supports minute-level modeling and hour-level online production. The zero-sample visual recognition method that supports minute-level modeling and hour-level online production specifically includes the following steps:

[0024] S100: Perform semantic parsing and fusion on the monitoring sample sequence set of the monitoring system according to the multimodal semantic parsing module to establish a semantic parsing vector space.

[0025] Further, as attached Figure 2 As shown, this application S100 includes:

[0026] S110: Input the monitoring sample sequence set into the multimodal semantic parsing module to obtain a semantic parsing first space, wherein the semantic parsing first space includes multiple sample feature vector sets; S120: Perform conflict detection on the semantic parsing first space according to the monitoring sample sequence set to obtain a parsing conflict detection result, and correct the semantic parsing first space according to the parsing conflict detection result to obtain a semantic parsing second space; S130: Perform scale alignment mapping on the semantic parsing second space to obtain a semantic parsing third space; S140: Perform multi-layer perceptron fusion on the semantic parsing third space to generate a semantic parsing fourth space; S150: Perform normalization output on the semantic parsing fourth space to generate the semantic parsing vector space.

[0027] Specifically, the monitoring sample sequence set is input into the multimodal semantic parsing module to obtain the first semantic parsing space, which includes multiple sample feature vector sets, that is, each sample feature vector set includes a sample visual feature vector, a sample text feature vector and a sample semantic relationship feature vector. The visual feature extraction module in the multimodal semantic parsing module extracts visual features from the monitoring sample sequence set to obtain the visual feature vector of the monitoring sample sequence set. The text feature extraction module in the multimodal semantic parsing module extracts text features, that is, text description information related to the image, from the monitoring sample sequence set, and uses a natural language processing model to encode the text description to obtain a text feature vector. The semantic relationship in the knowledge graph is converted into a feature vector by the semantic relationship extraction module in the multimodal semantic parsing module, such as obtaining relevant text introductions through network searches, and extracting semantic relationship information related to the category to which the image belongs from the knowledge graph.

[0028] Conflict detection is performed on the semantic parsing first space using a set of monitoring sample sequences to identify conflicting areas between samples, such as those with semantically similar but inconsistent labels or excessive overlap in feature space distribution. Cluster analysis is performed on all category feature vectors in the semantic parsing first space, and the angle or cosine similarity between the mean vectors of the categories is calculated. The mean vector is the category center vector, which is the average value of all feature vectors in a category, representing the center position of that category in the feature space. A conflict determination threshold is set, such as a cosine similarity > 0.87, which is considered a conflict. The parsing conflict detection result is generated. The parsing conflict detection result is a set of quantitative indicators (such as the inter-category similarity matrix, confusion score, misclassification rate, etc.) output during the detection process, indicating which categories need to be corrected. For example, assume that the monitoring samples involve the following five categories: courier (C1), security guard (C2), cleaner (C3), firefighter (C4), and stranger (C5), with approximately 200 image samples per category. After feature extraction, the following partial mean vectors (simplified to a 6-dimensional representation) are as follows: the mean vectors for C1 are 0.4, 0.2, 0.6, 0.1, 0.2, and 0.3; the mean vectors for C2 are 0.42, 0.21, 0.58, 0.12, 0.22, and 0.31; and the mean vectors for C3 are 0.2, 0.1, 0.3, 0.2, 0.3, and 0.5. The cosine similarity calculated based on the mean vectors shows that C1 and C2 are 0.9989, indicating a high-conflict classification. This indicates that the embedded features of the courier and the security guard are similar and difficult to distinguish, making them a high-conflict classification pair. The cosine similarity between C1 and C3 is 8452, indicating no conflict; and the cosine similarity between C2 and C3 is 0.8589, indicating no conflict.

[0029] Based on the results of parsing conflict detection, the first semantic parsing space is corrected to obtain the second semantic parsing space. Correction is the optimization of the conflicting areas in the first semantic parsing space, such as widening the distance between similar categories and enhancing discriminative features to improve classification performance. Feature space stretching (such as using Center Loss) pulls similar categories apart; feature remapping is used to retrain the mapping layer to enhance intra-class consistency and inter-class discrimination; class adversarial training is used to additionally enhance features for highly confused categories (such as adding interpretable factors: color, action, environment, etc.). The second semantic parsing space is a clearer embedding space with more distinct semantic boundaries and fewer conflicts.

[0030] The second space of semantic parsing is scale-aligned and mapped, that is, the feature vectors extracted from multiple modalities (such as images, text, and semantic maps) are uniformly mapped on the dimensional scale, distribution scale, or numerical scale, so that they can be compared and fused on the same semantic scale. Features from different sources (modalities) have inconsistent expression scales. For example, the vector extracted by image CNN may have a mean of 0.5 and a variance of 0.2, while the text BERT encoding result has a mean of 0.01 and a variance of 0.0003. If they are directly spliced ​​or fused without alignment, one modality will dominate the expression, affecting the overall semantic analysis. The mean and standard deviation of the feature set of each modality are calculated separately, and the Z-score normalization method is used to map all modal features to a mean of 0 and a standard deviation of 1, resulting in the third space of semantic parsing. The third space of semantic parsing is a new feature space after scale alignment. All modal features have been mapped to the same numerical range and statistical structure, which is more convenient for subsequent deep fusion. For example, to identify the categories of couriers and strangers, image features were extracted using ResNet50 and text features were output using BERT. After Z-score normalization of the extracted results, the mean of the image and text vectors was uniformly set to approximately 0, with a standard deviation of 1. The cosine similarity of the raw angle between categories decreased from 0.92 to 0.64, indicating that different modalities are no longer mistakenly perceived as similar due to scale issues. After normalization, multimodal fusion classification training using an MLP improved the validation set accuracy from 85.1% to 90.8%.

[0031] Multi-layer perceptron fusion is performed on the semantic parsing third space to generate the semantic parsing fourth space. This high-level semantic space is generated by deep fusion, dimensionality transformation, and semantic abstraction of the third space features using a multi-layer perceptron. This preserves the interactions between multimodal information and is suitable for downstream tasks such as classification and recognition. The multi-layer perceptron is a feedforward neural network structure consisting of multiple fully connected layers, each typically followed by an activation function (such as ReLU or GELU). It can be used to extract nonlinear high-order features and perform tasks such as classification or regression. Design the multi-layer perceptron structure, including the input layer, hidden layer 1, hidden layer 2, and output layer.

[0032] The fourth space of semantic parsing is normalized and output, and the feature vector is scaled to the unit norm or a fixed range (such as 0-1 or -1 to 1). Normalized output is a numerical normalization process performed on the vector so that the feature vector meets the unified scale requirement, such as L2 normalization, min-max normalization or z-score normalization. Normalization can reduce the deviation caused by different modalities and different dimensions in model training. The semantic parsing vector space is the final normalized feature space, which is used for zero-sample recognition tasks and is the basis for the model to perform operations such as distance calculation, similarity judgment, and sample comparison. Through conflict detection and correction, the inconsistency in the multimodal data parsing process is resolved and the accuracy of the parsing is improved. Through scale alignment mapping and multi-layer perceptron fusion, the effective fusion of multimodal features is achieved, and the accuracy of semantic parsing is improved. Through normalization, the data is kept within the same scale range, which improves the generalization effect.

[0033] Furthermore, the present application further comprises the following steps:

[0034] According to the monitoring system, a known category image sample set is retrieved; text description configuration is performed based on the known category image sample set to obtain a known category text description sample set; inter-category relationship configuration is performed based on the known category image sample set to construct a relationship map of each category; the known category image sample set, the known category text description sample set and the each category relationship map are aligned to generate the monitoring sample sequence set.

[0035] Specifically, a set of image samples of known categories is collected from the monitoring system, that is, a collection of image data that has been annotated with category labels. A monitoring system is a system installed in target locations such as industrial sites and security areas, and is used to capture video images, identify behaviors, and detect anomalies. It contains cameras, image acquisition modules, etc. Text descriptions are configured based on the set of image samples of known categories. That is, a set of natural language descriptions is generated for each category of image samples as semantic auxiliary information. For example, image data of known categories, such as common images of people, vehicles, and objects, is collected from security monitoring, and detailed text descriptions are provided for each category, such as descriptions of people's appearance characteristics and descriptions of vehicle brands and models.

[0036] Based on a sample set of known image categories, we configure inter-category relationships and, by establishing relationships between different categories, generate a relationship graph for each category. In each relationship graph, nodes represent different categories, while edges represent semantic or functional connections between categories. For example, a knowledge graph related to security monitoring can include semantic relationships between categories, such as the driving relationship between people and vehicles, or the association between objects and places.

[0037] Align the known category image sample set, the known category text description sample set, and the category relationship graph. This means integrating the known category image sample set, the known category text description sample set, and the category relationship graph according to certain rules. For each image sample, the corresponding text description is found, ensuring a one-to-one correspondence between the image and the text description. Using the category relationship graph, each category is associated with its related image and text description. In this way, each image sample, its corresponding text description, and the related category relationship are integrated into a monitoring sample sequence set.

[0038] By aligning images, text, and graphs, we can understand richer semantic information, avoiding the information deficit or semantic gap caused by relying solely on a single modality (such as images or text). By constructing a multimodal collaborative representation of monitoring sample sequences, we successfully achieve a unified semantic representation of images, text, and knowledge graphs, building a monitoring sample sequence that supports multimodal recognition and enhances semantic understanding capabilities.

[0039] Furthermore, the present application further comprises the following steps:

[0040] Based on the convolutional neural network, a visual feature extraction module is constructed; based on the natural language processing model, a text feature extraction module is built; based on the graph neural network, a semantic relationship extraction module is constructed; the visual feature extraction module, the text feature extraction module and the semantic relationship extraction module are connected as parallel nodes to generate the multimodal semantic parsing module.

[0041] Specifically, convolutional neural networks are the most commonly used deep learning models for processing image data. They effectively extract local spatial features and are commonly used in tasks such as visual object detection, image recognition, and image segmentation. Using a pretrained visual model, such as a convolutional neural network (CNN), image features are extracted to obtain the image's visual feature vector. Based on the task requirements and data characteristics, an appropriate CNN model, such as AlexNet, VGG, or ResNet, is selected as the base model. The input image data is preprocessed, such as by resizing and normalizing, to meet the input requirements of the CNN model. The preprocessed image is then fed into the CNN model, where convolutional and pooling layers extract local features, capturing visual information such as shape, color, and texture. The extracted features are aggregated and represented to form the image's visual feature vector. The visual feature extraction module, an image processing submodule based on the CNN network, converts the input image into a numerical feature vector, reflecting the core representation of the image's content information. The visual feature extraction module extracts high-dimensional semantic visual vectors from the input image, such as the posture and clothing characteristics of pedestrians.

[0042] A text feature extraction module is established using a natural language processing (NLP) model. A NLP model is a deep learning framework for processing text data. It converts natural language descriptions into vector embeddings, capturing grammatical structure and semantic information. Based on the task requirements and data characteristics, an appropriate NLP model is selected as the base model, such as the bag-of-words model, TF-IDF, Word2Vec, or BERT. The input text data is preprocessed, such as through word segmentation, stop word removal, and stemming, to meet the NLP model's input requirements. The preprocessed text is then fed into the NLP model, where the model's structure and algorithms extract features to capture lexical, grammatical, and semantic information. The extracted features are aggregated and represented to form a feature vector, which is then output as a vector-based semantic representation. Natural language processing techniques are used to encode the text description to generate a text feature vector. For example, BERT is used as the NLP model to extract features from a text description about cats. After preprocessing, the text is fed into BERT, where its structure and algorithms extract features. Finally, a 768-dimensional text feature vector is obtained, which contains information about the vocabulary, grammar, semantics, etc. of cats.

[0043] Knowledge graph information is processed using a graph neural network (GNN), converting semantic relationships within the knowledge graph into feature vectors. Graph neural networks are a type of deep learning model that models graph-structured data, such as knowledge graphs. They mine relational patterns between entities by aggregating and propagating information between node neighbors. For example, the BERT model, pre-trained on a large-scale corpus, is used to segment and encode category description text in the monitoring system, such as "man carrying a black backpack" and "red SUV model," ensuring that the text length meets the model input limits (e.g., a maximum of 128 words). The BERT model's Transformer layer converts the text into a 768-dimensional semantic vector, representing the text's semantic features, including inter-word relationships and contextual information. The semantic relationship extraction module utilizes a GNN to extract semantic relationships between entities from graph-structured data. It is capable of understanding complex relationships between entities, such as similarity, subordination, and causality.

[0044] The visual feature extraction module, text feature extraction module, and semantic relationship extraction module are connected as parallel nodes, all inputting the same sample set and uniformly outputting the fused multimodal semantic vectors to form a multimodal semantic parsing module. Parallel node connection means running multiple modules in parallel, extracting features from the input data simultaneously, and finally fusing the outputs of multiple modalities. The multimodal semantic parsing module is a unified encoding structure that integrates visual, textual, and graphical features. It is used to improve cross-modal recognition accuracy and generalization capabilities and is a key component of zero-shot recognition. By connecting the visual, textual, and semantic relationship feature extraction modules in parallel, multimodal data can be processed and parsed simultaneously, improving information processing capabilities.

[0045] S200: Introduce a category-differentiated attention enhancement mechanism to perform GAN learning on the semantic parsing vector space and construct a prototype generation channel.

[0046] Furthermore, the present application S200 includes:

[0047] The attention enhancement mechanism of the category differentiation is used to enhance the semantic parsing vector space to obtain an enhanced semantic vector space; the enhanced semantic vector space is used as input data and the known category image sample set is used as output data to perform supervised training on the generation network to obtain a prototype generator; multiple prototype generation images of the prototype generator during the training process are synchronously collected to obtain a prototype generation space; the prototype generation space is evaluated for multi-dimensional loss according to the known category image sample set to establish a generation loss space; the prototype generation space is used as an input node and the generation loss space is used as an output node to perform discriminative learning on the discriminant network to generate a prototype discriminator; alternating iterative optimization training is performed on the prototype generator and the prototype discriminator to generate the prototype generation channel that meets the predetermined convergence conditions.

[0048] Furthermore, the present application further comprises the following steps:

[0049] Feature decomposition is performed on each vector in the semantic parsing vector space to obtain each vector feature set; category differentiation contribution evaluation is performed on each vector feature set to obtain each feature contribution evaluation set; attention weights are allocated to each vector feature set based on each feature contribution evaluation set to obtain multiple attention weight vectors, and attention enhancement is performed on the semantic parsing vector space based on the multiple attention weight vectors.

[0050] Specifically, the category-discriminating attention enhancement mechanism is a mechanism used to improve the discriminative ability of semantic vectors. By analyzing the contribution of each feature dimension to category discrimination, the weight of the feature vector is adjusted to enhance discriminative features and suppress interfering features. Each vector in the semantic parsing vector space is decomposed to obtain the feature set corresponding to each sample. In other words, a vector is split into separate feature components by dimension (for example, a 256-dimensional vector is treated as 256 single eigenvalues).

[0051] Statistical analysis is performed on the decomposed feature sets to assess the contribution of each feature dimension in distinguishing different categories. Contribution evaluation is used to quantify the impact of a feature in distinguishing between categories. Common metrics include inter-class variance and entropy weighting. For example, the discriminatory power of each feature dimension is calculated using the inter-class variance ratio. The mean and variance of each feature dimension are input into the inter-class variance formula to obtain a contribution evaluation set for each feature. Attention weights are assigned to each feature set based on the contribution evaluation set, resulting in multiple attention weight vectors. The attention weight vector represents the importance of each feature dimension and typically ranges from a floating-point value between [0 and 1]. For example, if the evaluation indicates that the third feature is the most important, the third value in its weight vector is the largest. Attention is enhanced by multiplying the original vector dimension by the attention weights dimension by dimension, resulting in an enhanced semantic vector space. The enhanced semantic vector space is a vector space that has undergone attention weighting and has stronger semantic discriminatory power, making it more suitable for identifying new categories.

[0052] The enhanced semantic vector space is used as input data, and a sample set of known-category images retrieved from a surveillance system is used as output data. A generative network is supervised and trained to obtain a prototype generator. Using the concept of a generative adversarial network (GAN), a prototype generator is constructed based on the training data of the known-category image sample set. The prototype generator takes the fused multimodal semantic feature vector as input and generates prototype images of the corresponding category. During training, the network learns how to generate images similar to real images (i.e., the known-category image sample set) based on the input semantic vector (i.e., the enhanced semantic vector space). During training, the network continuously adjusts its parameters to make the generated images increasingly similar to the real images, typically by minimizing the difference between the generated images and the real images, such as by using the mean squared error as a loss function. The prototype generator, a trained generative network, is able to generate images similar to the known-category image sample set based on the input enhanced semantic vector.

[0053] During training, the generator output is sampled periodically (e.g., once every 100 training runs) to construct a set of prototype images, i.e., multiple prototype-generated images, thus forming the prototype-generated space. The prototype-generated space is a set of images output by the prototype generator that represent the typical visual features of different categories. A multidimensional loss evaluation is performed on the prototype-generated space based on a sample set of images of known categories. The original and generated images are compared, and the difference between them is calculated to establish the generated loss space.

[0054] Using the prototype generation space as input nodes and the generation loss space as output nodes, the discriminative network performs discriminative learning to generate a prototype discriminator. The prototype discriminator discriminates the generated prototype images and compares them with real training images, guiding the generator to continuously optimize the generated prototype images to more closely match the feature distribution of real images. The prototype discriminator learns to distinguish between real images and prototype-generated images and outputs loss feedback. The training goal is to improve both recognition rate and image quality. The prototype image output by the prototype generator is input to the prototype discriminator, which determines whether the input image is real or generated. The prototype generator and prototype discriminator are trained and optimized alternately, with each round improving recognition ability while the prototype generator improves image quality, ultimately reaching a balance. When predetermined convergence conditions are met, the prototype generation channel is obtained. Predetermined convergence conditions are the criteria for training termination, such as when the loss function is stable, the generated image FID reaches the target, or the perceptual difference between the generated image and the real sample is less than a threshold.

[0055] The prototype generator generates prototype images of known categories, and the prototype discriminator performs discrimination, continuously optimizing the generator to make the generated prototype images more representative. The prototype generation pipeline is capable of generating high-quality images that are indistinguishable from real images. The generated images retain the semantic information of the input semantic vector. Through alternating iterative optimization training, the model's generalization ability is improved, enabling it to adapt to different input data and scenarios.

[0056] Furthermore, the present application further comprises the following steps:

[0057] The prototype generation space is evaluated by pixel distribution difference loss according to the known category image sample set to obtain a first feature loss evaluation set; the prototype generation space is evaluated by image authenticity adversarial loss according to the known category image sample set to obtain a second feature loss evaluation set; the prototype generation space is evaluated by category semantic consistency loss according to the known category image sample set to obtain a third feature loss evaluation set; the first feature loss evaluation set, the second feature loss evaluation set and the third feature loss evaluation set are aligned to obtain the generation loss space.

[0058] Specifically, a pixel distribution difference loss is performed on the prototype generation space using a sample set of image samples of known categories to measure the difference between the generated image and the real image at the pixel level, such as the L1 distance. For example, the L1 loss is calculated based on the mean distribution (mean, standard deviation) of the pixels in the real image and the mean distribution (mean, standard deviation) of the pixels in the generated image, resulting in the first feature loss evaluation set. The image authenticity adversarial loss is performed on the prototype generation space using a sample set of image samples of known categories to obtain the second feature loss set. To determine whether an image is a real image, the smaller the loss value, the more realistic the generated image. The generative network attempts to generate images that can deceive the discriminative network, while the discriminative network attempts to distinguish between real and generated images. For example, in the first round of training, the probability of a generated image being identified as fake was 0.72, and the loss value was 0.328; in the 15th round of training, this value dropped to 0.11, and the loss was 0.094.

[0059] The prototype generation space is evaluated using a set of known category image samples using category semantic consistency loss. This evaluates the category semantic consistency between the generated image and the input semantic vector, and measures whether the generated image matches the target semantic description. A text-image alignment model (such as CLIP) is used to extract the semantic vectors of the text and image, and the cosine similarity loss is calculated to obtain the third feature loss evaluation set.

[0060] The first, second, and third feature loss evaluation sets are aligned. This involves normalizing the first, second, and third feature loss evaluation sets to a uniform scale (e.g., 0-1) and then weightedly fusing them to obtain a generative loss space. The generative loss space is formed by aligning multiple loss evaluation sets and is used to measure the quality of generated images in multiple aspects. Comprehensively evaluating images at the pixel level, image authenticity, and category semantic consistency through multi-dimensional loss (pixel, authenticity, and semantic) improves the performance of the prototype generator, enabling it to generate higher-quality, more compliant images.

[0061] S300: Introduce a global loss function for visual recognition, combine the semantic parsing vector space to perform knowledge distillation on the lightweight model, and establish a zero-shot visual recognition channel.

[0062] The lightweight models include but are not limited to MobileNet, ShuffleNet, SqueezeNet, EfficientNet-Lite and NanoViT.

[0063] Furthermore, the present application S300 includes:

[0064] Iterative training is performed according to the semantic parsing vector space to obtain a trained visual recognition teacher model; knowledge distillation is performed on the lightweight model according to the visual recognition teacher model to obtain a visual recognition student model; loss analysis is performed on the visual recognition student model according to the visual recognition teacher model to determine the first loss coefficient of visual recognition; loss analysis is performed on the visual recognition student model according to the semantic parsing vector space to determine the second loss coefficient of visual recognition; the first loss coefficient of visual recognition and the second loss coefficient of visual recognition are input into the visual recognition global loss function to obtain the global loss coefficient of visual recognition; if the global loss coefficient of visual recognition is greater than or equal to the global loss threshold of visual recognition, iterative distillation optimization is performed on the visual recognition student model according to the global loss threshold of visual recognition to generate the zero-sample visual recognition channel.

[0065] Specifically, the semantic parsing vector space is iteratively trained to obtain a visual recognition teacher model, which uses supervised learning, inputs semantic features, and outputs category labels. The visual recognition teacher model is a large, high-performance visual recognition network model that has been trained. It has strong recognition accuracy and generalization capabilities and serves as a teacher for knowledge distillation. The input monitoring samples undergo multimodal semantic parsing and are converted into a high-dimensional semantic parsing vector space. This is used as the input feature of the training samples, and paired with the corresponding labels to train a deep visual recognition model. The training process is repeated through iterations to adjust the network parameters so that the model can accurately identify the sample category. The trained teacher model is used as a teacher to guide the lightweight model (student model) to learn its output. The soft labels (predicted probability distribution) and intermediate features generated by the visual recognition teacher model for the input samples are used as targets for training the visual recognition student model. The visual recognition student model learns richer knowledge by minimizing the difference with the teacher model output, thereby improving performance.

[0066] To achieve model building within minutes and launch within hours while reducing computational costs, a fast inference module based on lightweight networks is adopted. Lightweight neural network architectures, such as MobileNet, ShuffleNet, SqueezeNet, EfficientNet-Lite, and NanoViT, are selected and optimized and customized for zero-shot visual recognition tasks. Lightweight models are characterized by compact structures, low computational overhead, and suitable for deployment on resource-constrained devices (such as mobile and embedded devices). MobileNet is a class of efficient and lightweight convolutional neural networks that uses depthwise separable convolutions to significantly reduce computational overhead and model parameters. ShuffleNet is a lightweight network that introduces a channel shuffle operation to address information flow issues in grouped convolutions, resulting in high computational efficiency. SqueezeNet achieves similar accuracy to AlexNet with an extremely small model size. It uses the flame module to fuse 1×1 and 3×3 convolutions to reduce parameter count. EfficientNet-Lite is a lightweight version of EfficientNet that uses a compound scaling strategy to balance width, depth, and resolution. NanoViT is a lightweight visual Transformer model that combines the expressive power of the Transformer with a lightweight structural design.

[0067] Loss analysis of the visual recognition student model based on the visual recognition teacher model yields the first visual recognition loss coefficient, which measures the feature difference between the teacher and student models. Loss analysis of the visual recognition student model based on the semantic parsing vector space yields the second visual recognition loss coefficient, which measures the error between the semantic parsing vector space and the output of the visual recognition student model, ensuring that the student model maintains its semantic understanding capabilities.

[0068] The first and second visual recognition loss coefficients are input into the global visual recognition loss function to obtain the global visual recognition loss coefficient. The global visual recognition loss function is: Global visual recognition loss coefficient = First visual recognition loss coefficient * First loss weight + Second visual recognition loss coefficient * Second loss weight. The first and second loss weights are hyperparameters used to adjust the contribution of the two loss coefficients to the global loss, helping to balance the importance of different loss objectives.

[0069] The global loss threshold for visual recognition is a preset loss value threshold used to determine whether the distillation process needs to continue to be optimized. If the global loss for visual recognition is greater than or equal to the global loss threshold for visual recognition, the weights of the visual recognition student model are updated by continuing distillation training until the loss meets the convergence requirements, completing the training of the zero-shot visual recognition model. Through knowledge distillation, the knowledge of the large and complex teacher model is transferred to the lightweight student model to achieve model compression and acceleration. Iterative distillation optimization is performed on the visual recognition student model, that is, knowledge distillation training is repeated to gradually fine-tune the parameters of the visual recognition student model so that its output is closer to the teacher model and its adaptability to semantic parsing is improved. After multiple rounds of iterative training, the performance of the visual recognition student model reaches a certain level, thereby forming a zero-shot visual recognition channel specifically optimized for zero-shot recognition scenarios, which can better identify new categories or scenes that have not been seen.

[0070] If the global visual recognition loss falls below the global visual recognition loss threshold, the model has achieved the expected performance and distillation optimization can be stopped. By building a lightweight, efficient, and scalable zero-shot visual recognition pipeline, the recognition capabilities of a large teacher model are successfully transferred to a student model deployable on edge devices. Using a global loss function as a control mechanism, we ensure both training efficiency and result stability.

[0071] S400: According to the monitoring system, a real-time monitoring image is obtained, and the real-time monitoring image is input into the multimodal semantic parsing module to obtain a multimodal semantic feature vector.

[0072] S500: Inputting the multimodal semantic feature vector into the zero-sample visual recognition channel to obtain a first recognition result.

[0073] Specifically, real-time surveillance images are acquired from the surveillance system and fed into the multimodal semantic parsing module for semantic parsing. For example, for an image to be identified, not only are its visual features extracted, but also textual descriptions related to the image are obtained simultaneously. This can include obtaining relevant text descriptions through online searches, as well as extracting semantic relationship information related to the category to which the image belongs from the knowledge graph. Multimodal semantic feature vectors are processed vectorized semantic information that integrates the image's visual features and semantic information from other modalities. For example, when a surveillance camera captures an image, the image is fed into the online model, and the multimodal semantic parsing module extracts the image's multimodal semantic feature vector. This allows for rapid conversion of captured surveillance images into multimodal semantic information to meet the real-time needs of surveillance scenarios.

[0074] The multimodal semantic feature vector is input into the zero-shot visual recognition pipeline, which determines the category to which the input belongs by calculating the similarity between the semantic vector and the category prototype (the semantic description of the category). The first recognition result is the preliminary recognition category or label output by the zero-shot recognition pipeline based on the input semantic features. This includes information such as the target category label and confidence level. The zero-shot visual recognition pipeline not only recognizes categories seen during training, but also accurately identifies new categories, improving the adaptability of the surveillance system.

[0075] In other words, the image to be recognized is fed into a trained lightweight network (i.e., the zero-shot visual recognition pipeline). The lightweight network rapidly processes the image's multimodal semantic features and outputs a predicted category. This lightweight network design significantly reduces the model's computational workload, enabling it to complete recognition and inference for large numbers of images in a short period of time, meeting the real-time requirements of minute-by-minute modeling and hour-by-hour production deployment. This reduces computational costs and improves model efficiency.

[0076] S600: When the first recognition result is unknown, activate the prototype-driven optimization recognition mechanism, combine the prototype generation channel to perform recognition analysis on the multimodal semantic feature vector, and obtain a second recognition result.

[0077] Furthermore, the present application S600 includes:

[0078] When the first recognition result is unknown, the multimodal semantic feature vector is input into the prototype generation channel to obtain multiple predicted prototype images; the multiple predicted prototype images are optimized and recognized according to the prototype-driven optimization recognition mechanism to generate the second recognition result; wherein, the prototype-driven optimization recognition mechanism includes: constructing multiple prototype feature vectors corresponding to the multiple predicted prototype images; performing similarity evaluation on each prototype feature vector according to the multimodal semantic feature vector to obtain a prototype matching evaluation set; performing prototype matching degree maximization optimization on the multiple predicted prototype images according to the prototype matching evaluation set to determine the prototype matching image, and recording the category corresponding to the prototype matching image as the second recognition result.

[0079] Specifically, when the first recognition result is unknown, that is, for images of unseen categories, the prototype-driven optimal recognition mechanism is activated. This is a mechanism that optimizes the recognition process by evaluating the similarity between the prototype image features and the target semantic features. It includes three steps: feature vector construction, similarity evaluation, and maximum matching optimization. The prototype generator of the prototype generation channel generates multiple predicted prototype images based on their text descriptions. The predicted prototype image is an image generated by the prototype generation channel based on the input multimodal semantic feature vector, representing a possible category prototype. Then, the multiple predicted prototype images are optimally recognized according to the prototype-driven optimal recognition mechanism to obtain the second recognition result.

[0080] For each predicted prototype image, a prototype feature vector is extracted through a convolutional neural network, that is, visual features are extracted for each predicted prototype image, and the generated high-dimensional representation vector is used to match the target semantic vector. The similarity of each prototype feature vector is evaluated according to the multimodal semantic feature vector to obtain a prototype matching evaluation set, that is, the result set after calculating the similarity of each prototype feature vector with the input multimodal semantic feature vector. According to the prototype matching evaluation set, multiple predicted prototype images are optimized to maximize the prototype matching degree, and the prototype matching image with the highest score is found, and its corresponding category is used as the second recognition result. In other words, the multimodal semantic feature vector of the image to be identified is matched with the feature vector of the generated prototype image, and the similarity between the two is calculated using a measurement method such as cosine similarity. The category corresponding to the prototype image with high similarity is the predicted category of the image to be identified. By quickly generating prototype images, virtual training samples are provided for unseen categories, effectively solving the problem of data scarcity.

[0081] To enable the model to continuously adapt to emerging categories and changing scenarios, an online update and adaptive optimization module was designed. When new category data emerges, the model automatically detects the new data and incorporates its multimodal semantic features into the existing model. For example, through incremental learning algorithms, the parameters of the lightweight network are updated online, enabling the model to learn the characteristics of new categories.

[0082] At the same time, the model's parameters and hyperparameters are automatically adjusted based on performance feedback from the model during actual operation, such as recognition accuracy and recall. For example, if the model's recognition accuracy for a certain category is found to be low, the optimization algorithm can be used to adjust the generation parameters of the rapid prototype generator or the weights of different modal information in the multimodal semantic fusion module, allowing the model to adaptively optimize its own performance and maintain efficient recognition capabilities for various categories of objects.

[0083] For example, if a new category, such as a new type of safety equipment, emerges during security monitoring system operation, the system automatically collects image data and text descriptions of the new category. Using an incremental learning algorithm, the model is updated online, adjusting the parameters of the lightweight network to enable the model to learn the characteristics of the new category. Based on performance feedback from the model during monitoring, such as recognition accuracy and false alarm rate, the system automatically adjusts the weights of different modal information in the multimodal semantic fusion module and the generation parameters of the rapid prototype generator to optimize model performance and improve the ability to recognize various targets.

[0084] In summary, the zero-shot visual recognition method provided by this application, which supports minute-level modeling and hour-level online production, has the following beneficial effects:

[0085] A semantic parsing vector space is established by performing semantic parsing fusion on a monitoring sample sequence set of a monitoring system according to a multimodal semantic parsing module; a category-differentiated attention enhancement mechanism is introduced to perform GAN learning on the semantic parsing vector space to construct a prototype generation channel; a global loss function for visual recognition is introduced, and knowledge distillation is performed on a lightweight model in combination with the semantic parsing vector space to establish a zero-sample visual recognition channel; a real-time monitoring image is obtained according to the monitoring system, and the real-time monitoring image is input into the multimodal semantic parsing module to obtain a multimodal semantic feature vector; the multimodal semantic feature vector is input into the zero-sample visual recognition channel to obtain a first recognition result; when the first recognition result is unknown, the prototype-driven optimal recognition mechanism is activated, and the multimodal semantic feature vector is recognized and analyzed in combination with the prototype generation channel to obtain a second recognition result. That is to say, by establishing a semantic parsing vector space and using a generative adversarial network to construct a prototype generation channel, the semantic parsing vector space is used to perform knowledge distillation on the lightweight network, and the multimodal semantic feature vector of the image to be identified is input into the zero-sample visual recognition channel for identification. If the result is an unknown class, the prototype-driven optimal recognition mechanism is activated, and the recognition is optimized again through the prototype generation channel to output the second result, achieving modeling in minutes and online launch in hours, thereby improving the efficiency of visual recognition.

[0086] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present application. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application is not limited to the embodiments shown herein, but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

[0087] Obviously, for those skilled in the art, several improvements and modifications can be made to the present application without departing from the principles of the present application, and these improvements and modifications also fall within the scope of protection of the present application.

Claims

1. A zero-shot visual recognition method that supports minute-level modeling and hour-level online production, characterized by: include: According to the multimodal semantic parsing module, the monitoring sample sequence set of the monitoring system is semantically parsed and fused to establish a semantic parsing vector space; Introducing a category-differentiated attention enhancement mechanism to perform GAN learning on the semantic parsing vector space and construct a prototype generation channel; A global loss function for visual recognition is introduced, and knowledge distillation of the lightweight model is performed in combination with the semantic parsing vector space to establish a zero-shot visual recognition channel. According to the monitoring system, a real-time monitoring image is obtained, and the real-time monitoring image is input into the multimodal semantic parsing module to obtain a multimodal semantic feature vector; Inputting the multimodal semantic feature vector into the zero-sample visual recognition channel to obtain a first recognition result; When the first recognition result is unknown, the prototype-driven optimization recognition mechanism is activated, and the multimodal semantic feature vector is recognized and analyzed in combination with the prototype generation channel to obtain a second recognition result.

2. The zero-sample visual recognition method supporting minute-level modeling and hour-level online production according to claim 1 is characterized in that: The multimodal semantic parsing module performs semantic parsing fusion on the monitoring sample sequence set of the monitoring system and establishes a semantic parsing vector space, including: Inputting the monitoring sample sequence set into the multimodal semantic parsing module to obtain a first semantic parsing space, wherein the first semantic parsing space includes a plurality of sample feature vector sets; performing conflict detection on the first semantic parsing space according to the monitoring sample sequence set to obtain a parsing conflict detection result, and correcting the first semantic parsing space according to the parsing conflict detection result to obtain a second semantic parsing space; Performing scale alignment mapping on the semantic parsing second space to obtain a semantic parsing third space; Performing multi-layer perceptron fusion on the semantic parsing third space to generate a semantic parsing fourth space; The semantic parsing fourth space is normalized and outputted to generate the semantic parsing vector space.

3. The zero-sample visual recognition method supporting minute-level modeling and hour-level online production according to claim 1 is characterized in that: A category-differentiated attention enhancement mechanism is introduced to perform GAN learning on the semantic parsing vector space and construct a prototype generation channel, including: Performing attention enhancement on the semantic parsing vector space according to the category-differentiated attention enhancement mechanism to obtain an enhanced semantic vector space; Using the enhanced semantic vector space as input data and a sample set of known category images as output data, the generative network is supervisedly trained to obtain a prototype generator; Synchronously collecting multiple prototype generation images of the prototype generator during the training process to obtain a prototype generation space; Performing a multi-dimensional loss evaluation on the prototype generation space according to the known category image sample set to establish a generation loss space; Using the prototype generation space as an input node and the generation loss space as an output node, performing discriminative learning on the discriminant network to generate a prototype discriminator; Alternating iterative optimization training is performed on the prototype generator and the prototype discriminator to generate the prototype generation channel that meets a predetermined convergence condition.

4. The zero-sample visual recognition method supporting minute-level modeling and hour-level online production according to claim 1 is characterized in that: The category-differentiated attention enhancement mechanism includes: Decomposing each vector in the semantic parsing vector space to obtain a feature set of each vector; Performing category distinction contribution evaluation based on each vector feature set to obtain each feature contribution evaluation set; Attention weights are allocated to each vector feature set according to each feature contribution evaluation set to obtain a plurality of attention weight vectors, and attention enhancement is performed on the semantic parsing vector space according to the plurality of attention weight vectors.

5. The zero-sample visual recognition method supporting minute-level modeling and hour-level online production according to claim 3 is characterized in that: Performing a multi-dimensional loss evaluation on the prototype generation space according to the known category image sample set to establish a generation loss space, including: Performing pixel distribution difference loss evaluation on the prototype generation space according to the known category image sample set to obtain a first feature loss evaluation set; Performing an image authenticity adversarial loss evaluation on the prototype generation space according to the known category image sample set to obtain a second feature loss evaluation set; Performing a category semantic consistency loss evaluation on the prototype generation space according to the known category image sample set to obtain a third feature loss evaluation set; The first feature loss evaluation set, the second feature loss evaluation set, and the third feature loss evaluation set are aligned to obtain the generated loss space.

6. The zero-sample visual recognition method supporting minute-level modeling and hour-level online production according to claim 1 is characterized in that: A global loss function for visual recognition is introduced, and knowledge distillation is performed on the lightweight model in combination with the semantic parsing vector space to establish a zero-shot visual recognition channel, including: Iterative training is performed according to the semantic parsing vector space to obtain a trained visual recognition teacher model; Performing knowledge distillation on the lightweight model according to the visual recognition teacher model to obtain a visual recognition student model; Performing a loss analysis on the visual recognition student model according to the visual recognition teacher model to determine a first loss coefficient of visual recognition; Performing loss analysis on the visual recognition student model according to the semantic parsing vector space to determine a second loss coefficient for visual recognition; Inputting the first visual recognition loss coefficient and the second visual recognition loss coefficient into the visual recognition global loss function to obtain a visual recognition global loss coefficient; If the visual recognition global loss coefficient is greater than or equal to the visual recognition global loss threshold, the visual recognition student model is iteratively distilled and optimized according to the visual recognition global loss threshold to generate the zero-sample visual recognition channel.

7. The zero-sample visual recognition method supporting minute-level modeling and hour-level online production according to claim 1 is characterized in that: When the first recognition result is unknown, activating the prototype-driven optimization recognition mechanism, combining the prototype generation channel to perform recognition analysis on the multimodal semantic feature vector to obtain a second recognition result, including: When the first recognition result is unknown, inputting the multimodal semantic feature vector into the prototype generation channel to obtain a plurality of predicted prototype images; performing optimal recognition on the plurality of predicted prototype images according to the prototype-driven optimal recognition mechanism to generate the second recognition result; The prototype-driven optimization and identification mechanism includes: constructing a plurality of prototype feature vectors corresponding to the plurality of predicted prototype images; Performing similarity evaluation on each prototype feature vector according to the multimodal semantic feature vector to obtain a prototype matching evaluation set; The prototype matching degree of the plurality of predicted prototype images is maximized according to the prototype matching evaluation set to determine a prototype matching image, and the category corresponding to the prototype matching image is recorded as the second recognition result.

8. The zero-sample visual recognition method supporting minute-level modeling and hour-level online production according to claim 1 is characterized in that: The method comprises: According to the monitoring system, a sample set of images of known categories is retrieved; Performing text description configuration according to the known category image sample set to obtain a known category text description sample set; Performing inter-category relationship configuration based on the known category image sample set to construct a relationship map for each category; The known category image sample set, the known category text description sample set, and the category relationship graph are aligned to generate the monitoring sample sequence set.

9. The zero-sample visual recognition method supporting minute-level modeling and hour-level online production according to claim 1, characterized in that: The method comprises: Based on the convolutional neural network, a visual feature extraction module is constructed; Build a text feature extraction module based on the natural language processing model; Based on the graph neural network, a semantic relationship extraction module is constructed; The visual feature extraction module, the text feature extraction module and the semantic relationship extraction module are connected as parallel nodes to generate the multimodal semantic parsing module.

10. The zero-sample visual recognition method supporting minute-level modeling and hour-level online production according to claim 1, characterized in that: The lightweight models include but are not limited to MobileNet, ShuffleNet, SqueezeNet, EfficientNet-Lite and NanoViT.

Images