14 results about "Modal method" patented technology

The invention relates to the technical field of data processing, in particular to a classroom teacher teaching performance description method, model and system based on multi-modal data fusion and a storage medium. By introducing a multi-modal data fusion strategy, cooperative processing of classroom teacher visual information, audio information and text information is realized, and the accuracy and time sequence continuity of teacher target perception are significantly improved. Compared with a traditional single-mode method, the method not only can extract the posture, expression and action characteristics of the teacher from the visual mode, but also can recognize the voice emotion and the side language signal from the audio mode, and achieves the full-dimensional description of the teaching behavior of the teacher in combination with the text semantics. The objective of the invention is to solve the problem of how to perform multi-dimensional evaluation on teaching performance of a classroom teacher based on data of multiple modalities.

The invention discloses a multimodal transport method and system suitable for a salt mist environment, and the method comprises the steps: building a multimodal transport salt mist adaptive transport capacity grid model with city / hub nodes as vertexes and connected road sections as edges; configuring a salt mist adaptation device at the node, and marking the node protection capability, the availability time window and the supply capability; meanwhile, a model hypothesis is set, and cargo transportation integrity, facility availability, transportation rules and a parameter acquisition mode are determined; defining a multi-dimensional decision variable, a parameter and a state quantity; incorporating the salt mist environment influence into cost composition, and constructing a salt mist correlation total cost model; constructing a mixed integer programming model containing path connection and flow conservation, protection capability constraint, energy / endurance constraint and corrosion / exposure constraint by taking the minimum total cost as a target; and designing an improved hybrid immune-particle swarm dual optimization algorithm, and carrying out optimization solution, optimal path output, mode selection and protection parameter configuration through hybrid coding, security repair operators and a memory mechanism.

The invention discloses a scene identification method and system based on vision-point cloud multi-modal fusion, and the method comprises the steps: collecting a visual image and laser radar point cloud data in a target scene, carrying out the preprocessing, and extracting corresponding visual image features and point cloud features; generating a global feature, a visual modal local feature and a point cloud modal local feature in the target scene; performing global retrieval in a preset scene database by utilizing the global features to obtain a plurality of candidate positions and corresponding global matching scores; performing local matching and geometric verification on the candidate positions based on the visual modal local features and the point cloud modal local features to obtain a local matching score of each candidate position; and reordering the candidate positions according to the global matching score and the local matching score, and outputting an optimal scene recognition result. The method solves the problems of poor feature alignment, insufficient modal fusion, weak complex scene adaptability and data redundancy in the existing multi-modal method.

This invention discloses a dual-modal fusion wavelet-enhanced six-image hyperspectral classification method and system. The method includes: acquiring spectral and RGB image data of the six images; extracting features from the spectral and RGB image data respectively; enhancing the feature-extracted data using a learnable wavelet enhancement module; flattening and stitching the enhanced data; fusing the stitched feature sequences to obtain a fused feature sequence; embedding location information and category labels into the fused feature sequence, and feeding it into a Transformer encoder for global context modeling and classification. This invention's dual-modal fusion wavelet-enhanced six-image hyperspectral classification method and system achieves the fusion and utilization of hyperspectral imaging, near-infrared spectroscopy, and digital image features through feature-level and data-level fusion, fully utilizing the spatial-spectral information of multiple data sources, leveraging the advantages of various data, and effectively overcoming the limitations of single-modal methods.

The invention relates to the technical field of image classification, in particular to a visual text prompt-based multi-modal network few-sample image classification method, which comprises the following steps of: acquiring an image, a learnable prompt and a manual prompt; obtaining a frequency domain prompt fusion feature map; obtaining an original image feature vector; inputting the original image and the frequency domain visual fusion feature map into an image encoder to obtain a map frequency domain prompt fusion feature vector; respectively inputting the learnable prompt and the manual prompt into a text encoder to obtain a learnable prompt feature vector and a manual prompt feature vector; calculating a similarity score between the original image feature vector and the manual prompt feature vector; calculating a similarity score of the graph frequency domain prompt fusion feature vector and the learnable prompt feature vector; and constructing the regularization of the minimum divergence to the consistency among the vision, the text prompt and the distribution. According to the method, the problem of coordination and integration of different modal information and consistency of cross-modal semantics in an existing multi-modal method is solved.

The invention relates to an industrial video anomaly recognition method and device, electronic equipment and a storage medium, and the method comprises the steps: inputting a multi-view video, sensor data and equipment operation data into a visual language model, obtaining a depth feature representation, carrying out the multi-modal depth fusion of the depth feature representation through an industrial equipment knowledge graph, and obtaining an industrial equipment anomaly recognition result. A multi-modal fusion sequence is obtained; performing global context information enhancement on the features of the multi-modal fusion sequence through a space-time diagram to obtain final feature representation; performing video anomaly detection on the final feature representation, and outputting a video anomaly detection result; multi-view videos, sensor data and equipment operation data are integrated, multi-modal feature extraction is realized by using a visual language model, deep fusion is performed in combination with an industrial equipment knowledge graph, global context information is enhanced by means of a space-time diagram, the problem of incomplete feature representation of a traditional single-modal method is effectively solved, and the method is suitable for large-scale popularization and application. The method has the advantage that the industrial video anomaly detection accuracy and the system robustness are improved.

This invention discloses a frequency-domain modal method for determining the stability of vehicle-to-grid oscillations in new energy systems, applicable to converter impedance measurement. Specifically, it involves: acquiring the topology and component parameters of an electrified railway traction power supply system; performing frequency scanning on the single-input single-output frequency-domain impedance obtained through measurement or mathematical modeling in the frequency band to be analyzed, forming the system frequency-domain node admittance matrix at each frequency; performing eigenvalue decomposition to obtain the amplitude-frequency curve of the modal admittance or modal impedance; determining the oscillation frequency based on the extreme values ​​of the curve; determining the system stability based on the real and imaginary parts of the modal admittance or impedance at the oscillation frequency; and calculating the participation factor of each node in the unstable oscillation mode. This invention is applicable to low-frequency oscillation stability analysis of traction power supply systems with arbitrary topologies that do not contain three-phase converters. The SISO frequency-domain impedance of the single-phase voltage source converter interface power supply and locomotive load can be directly measured through impedance scanning without the need for dq-domain conversion.

The invention relates to the technical field of engineering machinery, and provides a pipeline management method based on modal displacement and electronic equipment, a multi-body dynamics-fluid pressure coupling simulation model is constructed based on a hydraulic excavator working device pipeline system, and target position parameters of a lug are obtained through a free modal method and a modal displacement algorithm. The multi-mode superposition effect principle is applied to layout optimization of the engineering machinery pipeline lug, accurate optimization of the position of the pipeline lug is achieved by modeling and analyzing the coupling vibration problem of a multi-order mode, and the problem of stress concentration caused by resonance is effectively avoided. Furthermore, actually-measured acquisition data are acquired through load spectrum acquisition, dynamic response analysis is performed to execute dynamic monitoring and early warning, and a dynamically-optimized and continuously-monitored closed-loop system is established through comparative analysis of simulation and actually-measured data, so that the simulation precision is improved, dynamic adjustment and early warning of actual working conditions are realized, and the working efficiency is improved. And the applicability and the reliability of the system are obviously improved.

The application relates to a near-infrared spectrum feature selection method and device, electronic equipment and a storage medium, wherein the method comprises: purifying an original spectrum to obtain spectrum data; performing parallel calculation of multiple heterogeneous strategies based on the spectrum data to generate an importance curve meeting a preset diversification condition; extracting a consistency index based on the importance curve, determining a basic weight according to the consistency index, and correcting the basic weight to generate a final importance curve for selecting near-infrared spectrum features according to the corrected basic weight. Thus, the problems in the related art that the single modal method has inherent bias, is sensitive to data disturbance and cannot balance resolution and smoothness, leading to incomplete feature selection, poor model stability, performance fluctuation under limited sample size or data disturbance, and lack of systematic multi-modal fusion strategy, so that the feature selection accuracy cannot be guaranteed while the stability and interpretability of the results are considered.

The application discloses a kind of spatio-temporal frequency decoupling and frequency domain enhancement cross-modal video confrontation disturbance generation method, belong to computer vision technical field, including steps: obtaining the video frame sequence containing clean image, initialize each frame disturbance;According to clean image and disturbance, generate confrontation frame and generate enhanced confrontation frame;The intermediate features of clean image and enhanced confrontation frame are extracted respectively;Constitute spatial high-frequency component and time low-frequency component;Constitute joint loss;With video frame sequence to minimize joint loss update disturbance to generate the optimal disturbance of each frame;With optimal disturbance generation optimal confrontation frame constructs confrontation video.The application is through the multi-component feature constraint of spatio-temporal frequency decoupling and optimization stage frequency domain-scale joint enhancement, effectively overcome the deficiency of existing frame-by-frame cross-modal method in migration stability and representation destruction comprehensiveness, can better meet the actual demand in video model security and robustness evaluation.

The invention discloses a bearing fault diagnosis method based on multiple modes, and the method comprises the steps: taking an original vibration signal of a collected bearing as a query sample, and extracting a one-dimensional vibration signal and a two-dimensional time-frequency signal corresponding to the one-dimensional vibration signal from the original vibration signal; extracting a one-dimensional time sequence feature of the one-dimensional vibration signal and a two-dimensional image feature of the two-dimensional time frequency signal, and fusing the one-dimensional time sequence feature and the two-dimensional image feature into a joint feature of the query sample; calculating the distance between the query sample and the joint feature corresponding to each support sample in the support set; and inputting the distance between the query sample and each support sample in the support set into a fault classifier, and outputting a fault category corresponding to the current original vibration signal and a prediction probability thereof by the classifier. A multi-branch AEDE structure is designed in image branches, local and global modes can be extracted at the same time by using multi-scale convolution, and different frequency characteristics are adapted. Key features are highlighted in combination with an attention mechanism, and redundant or noise features are suppressed.

The present disclosure concerns a methodology that allows a user to “orbit” around a model on a specific axis of rotation and view an orthographic floor plan of the model. A user may view and “walk through” the model while staying at a specific height above the ground with smooth transitions between orbiting, floor plan, and walking modes.

The invention discloses a motion function multi-dimensional intelligent evaluation method and system based on a multi-modal hierarchical fusion network. The method comprises the following steps: firstly, inputting multi-modal data, and receiving original data from a sensor; then data preprocessing and feature extraction are executed, and signal filtering, segmentation, alignment and feature extraction are completed; an MMHF-Net core model is adopted, and efficient fusion of multi-modal data is achieved; and finally, multi-scale evaluation result output is realized and is used for generating macroscopic, mesoscopic and microscopic three-level evaluation results. According to the method, kinematics, myoelectricity, electroencephalogram and visual information are deeply fused, the motion control process is comprehensively and accurately restored, and the evaluation precision is remarkably superior to that of a single-mode method. A multi-modal hierarchical fusion network MMHF-Net is provided, multi-modal space-time interaction can be modeled, key information can be dynamically focused, scale scores, sub-project indexes and recognition results are output, fine-grained clinical insight is provided, and excellent generalization performance is achieved based on multi-task learning and data enhancement.

The invention relates to the technical field of man-machine interaction, and particularly discloses a robot intention recognition and understanding method based on multi-modal fusion, comprising the following steps: S1, acquiring multi-modal input data; s2, performing layered feature extraction on the multi-modal input data; s3, carrying out adaptive gating fusion on the extracted feature vectors; s4, performing hierarchical semantic reasoning on the fused features, and outputting intention probability distribution; and S5, outputting an intention category based on the intention probability distribution, and activating an execution module corresponding to the robot according to the intention category. According to the method, the problems of single-mode dependence and rough semantic understanding are solved through multi-mode fusion and hierarchical reasoning, a lightweight model is used for voice and visual feature extraction, the calculation complexity is reduced, edge equipment is adapted, feature complementarity is improved through adaptive gating fusion, and the understanding depth is enhanced from fine granularity to coarse granularity in hierarchical semantic reasoning; and the intention recognition accuracy is obviously improved compared with that of a single mode.