226 results about "Relevant feature" patented technology

A method is provided for improving image classification accuracy in few-shot learning scenarios, where only a limited number of training examples are available. The method combines the use of Gabor filters and convolutional neural networks (CNNs) to extract detailed texture and orientation features from images. These features are then enhanced through global average pooling, aggregated into comprehensive feature vectors, and refined using an attention mechanism that identifies and emphasizes the most relevant features for classification. Masks generated from this attention process selectively enhance critical features, which, after optional re-encoding, are used to train a classifier via a metric learning approach. This method aims to increase feature separability and classification performance, facilitating more accurate classification of new images with minimal training data.

The invention discloses a multi-modal target detection method based on feature enhancement and alignment fusion. The method mainly comprises the following steps: (1) extracting features of a visible light image and an infrared image by using a backbone network, and better guiding the backbone network to extract related features from two modes and enhance feature representation by using an FEM in multiple stages of extracting the image features by the backbone network; (2) performing global-to-local alignment on the feature maps of the two modalities after feature enhancement by using FAM (Frequency Administration and Maintenance); (3) utilizing FFM to fuse the aligned feature maps from the two modalities; (4) using SFFM to further fuse the deeper feature map after cross-modal fusion in the step (3); and (5) inputting the fused feature maps under different scales into a detection head to obtain a target detection result. The method is mainly applied to the technical field of target detection in computer vision, and has wide application prospects in the fields of video monitoring, automatic driving and the like.

The invention relates to the technical field of audio and video identity recognition, in particular to an audio and video identity recognition system based on multimode clue driving. The audio and video identity recognition system comprises an audio feature extraction module, a video feature extraction module, a multimode clue fusion module, a living body detection module and an identity recognition and verification module. The audio feature extraction module is used for capturing a voice signal of a user and extracting key voiceprint features, the video feature extraction module is used for acquiring facial features or limb features of the user and extracting related features, and the multi-mode clue fusion module is used for performing intelligent weighted fusion on the features of audio and video modes. The method comprises the following steps: firstly, using a living body detection module to comprehensively analyze the dynamic characteristics of audio and video, judging whether a user is a real individual, and finally, completing the final verification of the identity of the user through an identity recognition and verification module. The problem of feature extraction and modal fusion in the prior art is effectively solved.

According to the movie and television play table book extraction method and device, the storage medium and the computer equipment provided by the invention, after an audio and video file of a movie and television play is split into a video file and an audio file, feature recognition is performed on the video file to obtain a subtitle text, speaker face information and a video understanding text; performing voice understanding on the audio file to obtain a voice transcription text and a voice understanding text; wherein the voice transcription text can be corrected into the standard transcription text with high accuracy through the subtitle text. Therefore, based on the face information of the speaker, the line segment of each speaker in the standard transcriptional text and the audio and video file is aligned, so that speaker information with accurate segmentation and semantic coherence can be obtained; and then, through combination with a character side-writing text generated by side-writing analysis on the speaker based on the video, the voice understanding text and the speaker information, table book information is constructed, and related feature description of the character can be covered on the basis of containing the line content, so that the content and depth of the table book are enriched.

The invention discloses a zero sample relation extraction-oriented semantic distillation method. The method comprises the following steps of S1, generating entity and context representation vectors from input sentences; s2, generating entity and context representation vectors of the input relation description; s3, weakening irrelevant features in the context representation through a bidirectional semantic distiller, and strengthening relevant features; s4, performing comparative learning by utilizing enhanced representations of different context representations in the same batch; and S5, performing fine-grained matching on the entity and the context representation of the input sentence and the entity and the context representation in the relation description through a fine-grained semantic matching module. Complex context interference can be effectively dealt with, and generalization ability to different semantic boundaries is improved.

A system and method of landmark detection using emotion recognition in multimedia videos using multi-modal fusion based deep neural network is provided. The system includes circuitry and a memory configured to store a multimodal fusion network which includes one or more feature extractors, a network of transformer encoders, a fusion attention network, and an output network coupled to the fusion attention network. The system inputs a multimodal input to the one or more feature extractors. The multimodal input is associated with an utterance depicted in one or more videos. The system generates input embeddings as an output of the one or more feature extractors for the input and further generates a set of emotion-relevant features based on the input embeddings. The system further generates a fused-feature representation of the set of emotion-relevant features and predicts an emotion label for the utterance based on fused-feature representation.

The invention provides a metallurgical furnace working condition intelligent identification method fusing multi-modal data. A double-branch multi-level residual structure feature encoder is designed, and metallurgical furnace process variable data features and image data features are fully extracted. Aggregation and separation among modal features are carried out through a comparative learning method, feature semantic alignment is realized, and feature distribution is optimized. Under the constraint of orthogonal loss, a feature decomposer is utilized to effectively separate out inter-modal correlation features and modal private features, and an information bottleneck theory is utilized to promote redundancy removal of key correlation features of process variable data and image data. A multi-level and multi-dimensional feature fusion module is constructed, single-level feature interaction fusion and inter-level feature fusion are sequentially realized under the action of bidirectional cross attention, and fusion features rich in information content are obtained. And finally, the information is sent to the classifier, so that intelligent and accurate identification of the metallurgical furnace working condition under multi-modal data input is realized.

The invention discloses a positive and negative unmarked image classification method and system based on depth metric learning, and relates to the technical field of sample classification, and the method comprises the steps: obtaining a marked positive sample image and an unmarked image, inputting the marked positive sample image and the unmarked image into a depth metric learning model, adjusting the marked positive sample image in a depth feature space, and obtaining the unmarked image; minimizing the distance between the marked positive sample image and the center of the positive sample image, aligning the unmarked image and the corresponding enhanced view in a representation space through self-consistent metric learning, and outputting related feature representation; using related feature representation to select sample images which are in multiple proportion to the positive sample images as reliable negative samples; and taking the related feature representation as input, using a positive sample image and a reliable negative sample as supervision signals to train a binary classifier, and classifying positive and negative unmarked images to be detected. According to the method, dependence on heuristic negative sample selection is avoided, and high-quality feature representation with discrimination on positive and negative samples can be learned.

The invention discloses a hyperspectral and laser radar data fusion classification method based on feature confidence, and belongs to the field of remote sensing image classification. The method solves the problem that the traditional method does not consider the importance difference of the characteristics of different data sources, so that the classification precision is low. The method comprises the following steps: capturing and learning hyperspectral space-spectrum joint features from hyperspectral image data, and obtaining elevation features from LiDAR-DSM data; the multi-modal features are fused through a fusion feature confidence distribution module, and different confidence weights are distributed for different features, so that high-quality and high-correlation feature information can be ensured to get more attention, and the negative influence of low-confidence features on a result is inhibited in a fusion process; the designed multi-stage feature interactive fusion encoder can fuse multi-modal semantic features for multiple times, enhance the ability of the model to capture the relevance of different modal data, avoid information loss or insufficient fusion, and significantly improve the classification precision. The method can be applied to remote sensing image classification.

The invention relates to a dynamic concept cognitive anomaly detection method and system under an unbalanced condition, and the method comprises the steps: collecting the related feature data of a sample set in a scene facing a specific extreme event, and dividing an initial data set and a test set; constructing an initial learning model; performing dynamic event type prediction based on the test set, and classifying the event as an extreme event or a non-extreme event; constructing an active three-way concept learning module to form a positive concept space and a negative concept space based on a three-way decision idea and prediction probability values of extreme events and non-extreme events; according to the positive concept space and the negative concept space, introducing a concept center domain strategy for dynamic updating; and adopting a loss function minimization strategy to obtain an optimal learning model. The effectiveness of dynamic detection is improved by utilizing an active three-way concept learning method, a concept center field and a concept dynamic optimization strategy, and the capability of identifying extreme events under the extremely unbalanced scene condition is realized.

The invention discloses a feature selection method based on a differential evolution algorithm, a medium and equipment, and the method utilizes mutual information for population initialization, which allows high-correlation features to be included in an evolution process, and at the same time preserves low-correlation features which may still be beneficial to a model. A two-stage variation strategy and a single-bit variation restoration strategy based on ecological niche are introduced, and population evolution is divided into two stages, namely an exploration stage and a development stage. In the exploration stage, the algorithm is used for widely exploring the whole search space; in the development stage, finer local search is carried out by adopting an ecological niche strategy, so that the probability of finding a globally optimal solution is improved. The problems that a traditional differential evolution algorithm is prone to falling into local optimum, premature convergence and the like in the feature selection process are solved, the classification accuracy is improved, and the scale of feature subsets is reduced.

The invention proposes a visible light and infrared image fusion method based on scene-semantic dissociation, and the method employs a multistage coding-fusion network, achieves a scene-semantic dissociation and fusion strategy through a fusion module, and specifically comprises the steps: respectively extracting and fusing scene and semantic related features, injecting the fusion semantics into the scene features, and achieving the fusion of the scene and the semantic related features. According to the method, context information in fusion features is enriched, the fidelity of a fusion image is kept, meta-feature embedding is further introduced, a coding-fusion network and a downstream application network are connected in the training process, the fusion effect is optimized by enhancing the semantic extraction capability, and a semantic segmentation task is served; according to the infrared and visible light fusion image enhanced by the method provided by the invention, abundant scene information can be reserved in fusion features, context information in the fusion features is enhanced, and the performance of downstream tasks is optimized.

The invention relates to the technical field of speech recognition, in particular to a speech deception detection method and system based on multistage mixed feature fusion. The invention provides a voice deception detection model with a new architecture, and the method comprises the steps: firstly extracting a mixed feature expression X1 mixed with a context-related feature XSSL and a fundamental frequency information feature XF0 through a multi-stage mixed feature extraction part, thereby enabling the model to be more focused on a non-silence segment while recognizing a deep voice feature; the X1 is subjected to information fusion in a channel, a time domain and a frequency domain through a mixed feature residual network part to obtain a fusion feature expression X2, so that the model can be guided to recognize forged clues in the voice more accurately; and finally, X2 is classified through a feature classifier to obtain a discrimination result. Through experimental comparison, even if a mute section is removed, the method still keeps excellent detection performance, and is obviously superior to a baseline method and other frontier models.

The invention discloses a mild cognitive impairment recognition method and system based on fNIRS data under multi-task normal form fusion. The method comprises the steps that fNIRS data, obtained based on a resting state task and an n-back work memory task, of a testee are obtained; based on the obtained fNIRS data, extracting fNIRS related features; performing feature selection on the extracted fNIRS related features based on a feature selection algorithm to form a feature set; and inputting the formed feature set into the trained deep learning model, and outputting a mild cognitive impairment recognition result. According to the method, the data of the fNIRS under the resting state and the n-back work memory cognition task are obtained, the multi-dimensional fNIRS related features are calculated, feature selection is carried out based on a Gradient Boosting algorithm, and finally accurate MCI recognition is carried out in combination with a deep learning model. The method not only can effectively extract high-dimensional features, but also can realize automatic MCI diagnosis through model training and reasoning, and has relatively high accuracy and relatively strong popularization and application potential.

The invention provides a multi-modal document understanding method and device and electronic equipment, and relates to the technical field of document understanding. The method comprises the following steps: extracting an initial feature map from a document image, and performing down-sampling on the initial feature map to obtain a visual data unit; features of a text area are extracted from the initial feature map, the features of the text area and a text data unit of the document are processed based on a Q-Former structure model, a query vector is obtained, and the query vector is used for extracting features most relevant to the current task from the features of the text area and the text data unit; and inputting the visual data unit, the text data unit and the query vector into a large language model for processing, and understanding the document. The technical problems that an existing multi-mode large language model document understanding technology is poor in visual perception ability and high in computing resource requirement are solved.

The invention relates to a three-dimensional semantic scene completion method and device based on geometric and time sequence modeling hierarchical context alignment, and a medium. The method comprises the following steps: obtaining depth features and context features, sensing a cross attention mechanism through depth confidence, supplementing information of a low depth confidence region by using the context features, and generating related features of a current frame; historical frame related features are obtained from the attitude network, cross-frame feature affinity is calculated, the historical frame related features are dynamically optimized, and multi-layer historical frame aggregation features are obtained; and in the unified space, the depth hypothesis of the time feature voxels is adopted as a distance axis, the volume feature voxels are projected to the unified space, global alignment and combination are performed on the geometric feature voxels and the time feature voxels, and final aggregation features are obtained. Compared with the prior art, by introducing a hierarchical context alignment mechanism based on geometry and time sequence modeling, a complex 3D scene can be more accurately understood, and the method is especially suitable for environmental perception in an automatic driving system.

The invention relates to the technical field of man-machine interaction and voice processing, in particular to a gesture operation voice playing output method based on end-cloud collaboration, which comprises the following steps: S1, collecting gesture motion data of a user in real time; s2, extracting a bone point coordinate change rate, and generating a compression feature sequence; s3, segmenting the compressed feature sequence into a privacy related feature layer and a behavior representation feature layer; s4, dynamically matching the behavior representation feature layer with a preset semantic template; s5, selecting the target semantics with the highest weighted score and the associated voice template; and S6, encoding the voice template into streaming media data, and distributing the streaming media data to local equipment for voice broadcast. According to the method, multi-mode perception, dynamic semantic matching and edge streaming media coding are fused, high precision of gesture recognition and semantic analysis is achieved, meanwhile, user privacy protection and real-time voice interaction are ensured, and system response efficiency and application safety are remarkably improved.

The invention discloses an industrial equipment fault real-time monitoring method and system based on a neural network, and the method comprises the steps: carrying out the segmentation and classification of a monitoring video frame of target industrial equipment, obtaining recognition frames, selecting doubt recognition image blocks which are strongly correlated with other recognition frames, and reference recognition image blocks which are weakly correlated with other recognition frames, and carrying out the recognition of the doubt recognition image blocks. And performing brightness and texture feature analysis and processing on the doubtful recognition image block to enlarge the features of the doubtful recognition image block which do not belong to the equipment device, enhancing the reference image block to enhance the features of the reference image block which belong to the equipment device, then performing fault recognition based on the doubtful fusion feature map and the reference enhancement feature map, and outputting a fault monitoring result. In the scheme, the doubtful image block corresponding to the obstacle is weakened, and meanwhile, the related characteristics of the reference image block belonging to the equipment device are enhanced, so that the influence of obstacle shielding and the like on the fault monitoring result can be effectively avoided, and the fault monitoring accuracy of the industrial equipment is improved. The system also has the above technical effects.

A navigation path can be determined for an object using one or more neural networks. In various embodiments, image data is obtained that is representative of an environment in which the object is to be navigated. Relevant features are identified from the image, and a curve fit to those features. Loss values for the potential paths are scaled based at least in part upon the distance of those features in the real world. This can include, in at least some embodiments, performing the scaling as a function of the curvature of the curve fit to the features. Temporal smoothing can be performed with respect to prior path predictions in order to prevent sudden changes in the predicted path. The paths are analyzed to select a path with a highest confidence value that also at least satisfies a minimum confidence criterion. The path can be converted into three-dimensional navigation information.

The invention discloses an improved PointPill three-dimensional target detection method based on fusion sparse enhancement and geometric attention, and belongs to the field of deep learning and three-dimensional target detection. According to the method, an SP-Point Pill improved framework fusing a sparse Pill enhancement mechanism (SEPA) and a point awareness space attention mechanism (PASA) is provided. A sparse Pilllar enhancement mechanism is fused, and geometric information of sparse Pilllar is efficiently filled through neighborhood real point borrowing and a virtual point interpolation strategy; the point sensing space attention mechanism generates a dynamic weight based on the point cloud density and the average distance, and preferentially enhances target related features; the pedestrian detection method and the pedestrian detection device cooperate with each other to form a link closed loop of information completion and feature focusing, so that the pedestrian detection precision is remarkably improved on a KITTI data set, and meanwhile, the efficient reasoning speed is maintained.

The invention belongs to the technical field of computer vision and intelligent video analysis, and particularly relates to a multi-target tracking method based on trajectory recovery. The method comprises the following steps: firstly, performing target detection on an input video frame to obtain a bounding box, confidence and related feature information of a candidate target; then, a current detection result is matched with a historical track through Kalman filtering and appearance features, and track updating of the first stage is achieved; when an unmatched target exists, the position of the unmatched detection frame is dynamically corrected by calculating the average displacement of the center points of the front and back frame detection target, and whether the distance between the target and the camera is lower than the preset track recovery threshold value is judged by combining the distance information between the target and the camera. And when the conditions are met, the target is directly activated and recovered to be an effective tracking trajectory of the current frame, so that quick re-association of the lost target is realized.

The invention discloses an enterprise risk clue generation method and device, and the method comprises the steps: obtaining related feature data of an enterprise information system, inputting the related feature data into a trained risk clue prediction model, and outputting a risk clue prediction result of an enterprise; explaining the output risk clue prediction result through an SHAP method, and determining the contribution degree; and utilizing a causal forest model to predict the risk clues of the enterprise according to the related feature data and the corresponding contribution degrees, and obtaining an average value of output results of all the causal trees in the causal forest model as a final enterprise risk clue prediction result. Retrieving in an RAG knowledge database to obtain a corresponding text information retrieval result, and generating corresponding prompt word information; and inputting the prompt word information into the large language model to generate an answer to the enterprise risk clue. According to the invention, the accuracy and interpretability of risk clue prediction can be improved.

The invention discloses a depth image classification model evaluation method and system based on background pseudo-correlation measurement, and the method comprises the steps: obtaining a foreground target mask through a pre-training semantic segmentation model, and separating a foreground image; and generating various background images by using a text-to-image generation model in combination with random noise and semantic guidance weight. Then, background controllability constraint is adopted to adjust background change, meanwhile, the foreground is kept unchanged, and foreground and background images are fused to construct a composite image set; and then, inputting the synthesized images into a to-be-evaluated model, calculating category prediction probability difference, semantic representation of a feature extraction layer and an uncertainty metric value, and finally obtaining a correlation analysis error value to evaluate the dependency degree of the deep learning classification model on the pseudo-correlation features. Through combination of semantic segmentation, text-to-image generation and background controllability constraint technologies, quantification of the background pseudo-correlation dependency degree of the deep learning classification model is realized.

The invention relates to a food multi-modal detection data fusion analysis method and device, equipment and a medium. The method comprises the following steps: acquiring original multi-source heterogeneous data of a food spectrum, an image and a smell, and performing normalization and noise reduction preprocessing to obtain a standardized feature set; classifying and screening high-correlation features by using a support vector machine, and generating a structured description; dynamic parameters are extracted, a quality trend vector is generated through time sequence analysis, a safety score is calculated, and a qualified mark is output if the safety score reaches the standard; constructing an extension index set based on the score, inputting a dynamic model to generate an authentication update link, and combining the block chain time to obtain new-version authentication; and associating the historical records to generate an initial report, and verifying and correcting to obtain an optimized report. By adopting the method, the consistency and availability of food safety data can be improved, the credibility of safety certification is enhanced, and a systematic and efficient solution is provided for food quality safety detection.

The invention discloses a 3D target detection method based on instance-scene fusion enhancement, and aims to improve 3D target detection performance of multi-modal sensor fusion. Firstly, instance information is effectively transmitted from an image view angle to a bird's-eye view (BEV) through a cross-view-angle correlation mechanism, and preliminary fusion of instance features and scene features is achieved. Secondly, through a geometric and semantic double-attention mechanism, instance related features in the instance-scene BEV features are further enhanced, and deep enhancement of instance-scene fusion is realized. Experimental results show that the precision and robustness of 3D target detection in a 4D millimeter wave radar and image fusion scene are remarkably improved.

The invention discloses an artificial intelligence model-based image classification robustness test enhancement method and system, and the method comprises the steps: carrying out the deep semantic analysis of the image content based on a multi-modal artificial intelligence model, automatically recognizing and extracting the class related features and non-class related features in an image, and generating a structured feature analysis report; the method comprises the following steps: generating a Keep Strategy and a Replace Strategy which are complementary to each other, and generating a Keep Strategy and a Replace Strategy which are complementary to each other; performing image editing operation based on the generated strategy to generate a corresponding image sample; performing automatic quality verification on the edited image sample, filtering out low-quality samples which do not conform to expectation, and generating a test sample; and comprehensively evaluating the target classification model based on the generated test samples, calculating performance indexes of the model on different types of test samples, generating a detailed robustness analysis report, and identifying weak links and improvement directions of the model. According to the scheme, a complete'generation-test-evaluation 'closed-loop system can be established, a standardized and quantifiable robustness evaluation index system is formed, and the model robustness can be scientifically and quantitatively evaluated.

The embodiment of the invention provides an action matching method and device for a digital human. The method comprises the following steps: acquiring a target audio to be subjected to action matching; determining a target feature corresponding to the target audio based on the target audio; on the basis of the target features, action videos matched with the target audio are determined from a specified database, the specified database comprises a plurality of action videos and action related features corresponding to the action videos, and the target features and the action related features are located in the same feature space, so that more appropriate digital human actions are obtained through matching.

A method for training an interpretable model trained to deliver, as an output, a prediction regarding a physiological state or a biologically or a clinically relevant characteristic of a patient, the method comprising: > the training comprising: ∘ A. Determining a feature characterizing the prediction of a black-box model, ∘ B. Injecting said plurality of images in a first interpretable submodel, ∘ C. Injecting said at least one feature together with said plurality of images in said first interpretable submodel, ∘ D. Obtaining, as an output, said prediction, ∘ E. Determining a residual between the predictions, ∘ F. Injecting said residual, said plurality of medical images and at least one feature characterizing the prediction of said black-box model into another interpretable submodel to obtain another residual, ∘ G: aggregating said first and other interpretable submodels, and ∘ H: iteratively performing steps F and G until a criterion is met.

This application provides a method and apparatus for auxiliary assessment of student quality based on a multimodal large model, belonging to the field of large model assessment technology. The method includes: acquiring student data of target personnel; using a multimodal large model to extract features from the student data in the following manner to obtain student quality features of the target personnel in multiple dimensions: converting structured data in the student data into corresponding student quality features according to preset mapping rules; extracting student quality features corresponding to unstructured text data in the student data using an attention mechanism based on preset prompts; and extracting student quality features corresponding to unstructured image data in the student data using a visual encoder. These multiple-dimensional student quality features are used to assist in assessing the student quality of the target personnel. This application can simultaneously process structured and unstructured data in the student data, extracting relevant features, facilitating the work of relevant personnel, and improving assessment efficiency.

The application provides a target tracking method and system fusing co-occurrence statistics and fhog gradient features, and the method comprises the following steps: determining an initial position according to a target region; calculating a target position p t and a scale factor s t‑1 of a previous frame of a current frame I t‑1 , so as to determine a target region and extract relevant features of the current frame; co-occurrence filter is used to obtain co-occurrence statistical information between pixel pairs in a tracked target image, fDSST algorithm is used to obtain fhog features of the tracking target, and the co-occurrence statistical information and the fhog features of the tracking target are fused, so as to be used as target image features; when the model is updated, the target region is determined according to a target center p t and a target scale s t of the current frame I t , and the target region is extracted; the target region extraction features are sent into a position filter and a scale filter, so as to obtain a target tracking result through iterative updating. The application solves the technical problems that the prior art cannot completely and effectively use feature expression of a target, has low robustness, and has poor tracking effect in a specific scene.