194 results about "Global modeling" patented technology

Disclosed are a complex device fault diagnosis method and system based on multi-dimensional features, and the method comprises the following steps of: acquiring fault data of a target complex device, and carrying out input coding mapping and position coding; utilizing a multi-head attention mechanism and multi-head dilated convolution series connection, and acquiring fault features covering global and local deep information in combination with a feedforward neural network; fusing the extracted fault features to acquire a multi-dimensional feature vector; and acquiring a fault type probability score matrix by a classifier, and outputting a fault diagnosis result of the target complex device. The global modeling advantage of multi-head self-attention and the local multi-scale feature perception advantage of multi-head dilated convolution are fused, and a device fault diagnosis model in which a Transform model and the multi-head dilated convolution are mutually fused is established.

The invention belongs to the technical field of artificial intelligence and big data, and discloses a customer data processing and insight system based on a big language model. The system is composed of a multi-source data access module, a data preprocessing and label fusion module, a large language model semantic understanding module, a knowledge enhancement and semantic linkage module, an insight generation and visualization module, an intelligent strategy output module and a feedback learning and self-optimization module. According to the method, multi-source heterogeneous data such as texts, voices and structured behaviors are integrated, and the deep semantic analysis capability of a large language model is combined, so that global modeling of customer behaviors and intentions is realized; a multi-modal synchronous acquisition and standardization mechanism eliminates data format barriers, and a dynamic label mechanism adapts to context changes, so that the system can capture deep semantic association in customer expression, and compared with a traditional keyword matching method, the semantic understanding accuracy is improved by more than 40%, and a more complete data base is provided for insight generation.

The invention provides a medical image segmentation method fusing SAM global modeling and U-Net local optimization, and belongs to the technical field of medical image segmentation of image segmentation. The technical problems that the segmentation precision is insufficient and edge details are lost due to the fact that global semantic features and local detail features cannot be considered at the same time in the segmentation process of a traditional method are solved. According to the technical scheme, the method comprises the following steps of 1, preprocessing a medical image; step 2, constructing a segmentation model fusing a visual Transform and a convolutional network; 3, training the segmentation model by using a supervised learning method with a binary cross entropy loss function; and step 4, inputting an image to be segmented into the trained model, and generating a segmentation result of the medical image. The medical image segmentation method has the beneficial effects that the accuracy and robustness of medical image segmentation are effectively improved by combining the global modeling capability of SAM and the local optimization capability of U-Net.

The invention discloses an interactive action detection method based on multi-level features, and the method comprises the steps: S1, fusing the information of a low-level feature map, a middle-level feature map and a high-level feature map through a cascading fusion mode, and obtaining a fused feature map; s2, acquiring local detail features from the fused feature map, performing global modeling to obtain global features, fusing the global features with the local detail features to generate global context features, performing character detection, object detection and interaction detection in parallel through multi-task branches, outputting character features, object features and interaction features, and outputting the character features, the object features and the interaction features. S3, gradually fusing multi-level contexts through attention interaction of a unitary relationship, a pairwise relationship and a ternary relationship, generating text embedding for an interaction category by utilizing a pre-training text encoder, and aligning the text embedding with visual features to improve the accuracy of fine-grained interaction classification; and S4, explicitly modeling a complex interaction relationship, and finally predicting and outputting a final prediction result by using FFN.

The invention discloses a complex target recognition method based on a deep twin self-attention network, and relates to the technical field of target recognition methods. According to the method, a twin network based on a sliding window multi-head self-attention transformation mechanism is applied, the global modeling capability of a feature extraction network and the acquisition efficiency of long-distance semantic information are enhanced, and the bottom-layer feature extraction capability and the abstract effect of advanced semantic information are improved; the twin network personalized modeling improves the feature segmentation effect; the whole network forms a U-shaped structure, a local symmetric encoder and decoder structure is adopted, global context information is supplemented, and semantic information space association is enhanced; a residual link is added, so that the deep network is trained efficiently, and the recognition precision is enhanced; and adaptive image feature scale input processing is added, and adaptive scale transformation is performed on the input image, so that an accurate end-to-end semantic segmentation result is realized, and multi-scale input is realized to enhance the generalization ability and accuracy of a network model.

The invention discloses a three-dimensional attitude estimation method combining global modeling and local refinement, which comprises the following steps of: firstly, extracting a two-dimensional attitude sequence by using a human body video data set; secondly, inputting the two-dimensional attitude sequence into a structural modeling main branch, modeling a spatial topological relation and a time sequence dynamic state between joints, and outputting a global three-dimensional attitude sequence; and inputting the two-dimensional attitude sequence into a local refining branch, modeling dynamic change and detail information of a local area, and outputting a local three-dimensional attitude sequence. And finally, fusing the global three-dimensional attitude sequence and the local three-dimensional attitude sequence, generating a three-dimensional attitude sequence output, and completing three-dimensional attitude estimation. According to the method, the problem of insufficient cross-frame information transmission in a traditional method is relieved, and the accuracy and robustness of attitude estimation in a dynamic complex scene are remarkably improved.

The invention relates to a small focus segmentation and feature learning method based on Transform and CNN fusion. The method comprises the following steps: preprocessing an input three-dimensional image to obtain original input features, and inputting the original input features into a segmentation network model of an encoder-bottleneck-decoder structure; after passing through an encoder based on channel-Transform fusion, obtaining fusion features through a bottleneck used for expanding a receptive field; wherein the encoder comprises a channel convolution module for local feature capture and a local attention mechanism module for global modeling; the preprocessed input features pass through a channel convolution module and a local attention mechanism module in sequence, and then pass through a bottleneck containing cavity convolution to carry out feature fusion; and inputting the fused features into a decoder based on a jump connection mechanism to gradually restore the low-resolution features to the same size as the input image, and generating a final segmentation result or classification result. According to the method, efficient feature extraction and fine prediction of three-dimensional images such as medical images can be realized.

The invention discloses a unified framework for solving automatic driving track prediction and planning consistency based on a world model. According to the method, through cooperative work of the automatic driving domain controller and the vehicle-mounted sensing system, end-to-end joint optimization of track prediction and planning in a complex traffic scene is realized, time sequence dependence and interaction dynamics among intelligent agents are accurately captured, and the prediction capability and robustness of a model are remarkably improved. The method comprises the following specific steps: firstly, constructing a generative world model, and generating potential future state representation by utilizing a behavior conditional and backtracking expansion technology; secondly, in combination with global modeling and a local convolutional network, multi-scale features are extracted, adaptive fusion is carried out, and a multi-modal prediction trajectory is generated; then, a multi-target planning model is adopted to integrate various driving indexes, and a track with the minimum loss function is generated; finally, path planning parameters are dynamically optimized through real-time environment perception and decision feedback, and the problems of prediction uncertainty and planning consistency of the automatic driving track are effectively solved.

The invention discloses a hyperspectral image and LiDAR data collaborative classification method based on double-domain mask and multi-scale local reconstruction, and belongs to the field of remote sensing image classification. According to the method, the problems of scarcity of annotation data and insufficient multi-source feature fusion precision in cross-modal classification of a traditional method are solved. According to the invention, feature learning is carried out through mask random image blocks and channels; a hierarchical multi-scale reconstruction architecture is designed, a lower-layer encoder learns fine-grained features, an upper-layer encoder recovers macroscopic semantic information, and multi-level feature space alignment is realized in combination with deconvolution up-sampling and adaptive pooling. According to the method, a multi-modal feature interaction mechanism and a cross-modal attention module are constructed by fusing the local feature extraction advantages of a convolutional neural network (CNN) and the global modeling capability of Transform, and the complementarity of heterogeneous data is enhanced. According to the method, through multi-level feature dynamic fusion and adaptive weight distribution, the collaborative classification precision of the hyperspectral image and the LiDAR data is improved. The method can be applied to remote sensing image classification.

The invention belongs to the technical field of image processing, and particularly relates to an infrared-visible light image fusion method based on mask image modeling pre-training. The method comprises a pre-training stage and a fine tuning stage. In the pre-training stage, a mask auto-encoder is adopted, and a model is guided to learn global structure information and local detail features by randomly masking an input image; a convolutional attention mechanism is integrated into a self-attention layer of the Transform to form a mixed attention module, and the local sensing ability of the CNN and the global modeling ability of the Transform are combined while the compactness of the network structure is maintained; in the fine tuning stage, a progressive feature fusion module is introduced to refine feature consistency between modals and extract complementary information between features of different levels. Experimental results show that the method achieves an excellent fusion effect on a plurality of public data sets, and is remarkably improved in the aspects of visual quality, evaluation indexes and downstream task performance.

The invention discloses a semi-supervised seismic wave impedance inversion method and system based on Swin Transform iterative gradient correction, and belongs to the technical field of earthquakes in geophysical exploration. According to the method, the impedance gradient is iteratively corrected in the model space through the deep neural network, and complex nonlinear prior information is captured; and each anisotropic structure direction regularization item is established, and constraint is carried out along the direction of the stratum structure, so that the space consistency of the prediction result is ensured. Meanwhile, the USTNet network designed by the invention combines the advantages of the local feature capture capability of the convolutional neural network and the global modeling capability of Transform, and reduces the non-uniqueness of solutions. The network is used for inverting synthetic data and actual data, and the effectiveness of the method is verified.

A method and an apparatus for separating speeches based on local and global modeling network (LAGNet) are provided. A speech separation apparatus includes an encoder configured to progressively compress a sequence of mixed speeches by using a one-dimensional convolution-based local block to generate local information, a bottleneck configured to generate global information by using a multiple self attention-based global block, and a decoder configured to progressively reconstruct the sequence by using the local block. A speech separation apparatus also utilizes a skip connection that includes gates configured to filter the local information by using the global information.

The invention discloses an adaptive evaluation method and system for transient stability of a power system based on dynamic adversarial migration, and the method comprises the steps: building a transient stability evaluation model based on Swin Transform, performing self-attention calculation on the electric power data characteristics in the fixed window and the movable window to realize global modeling between the electric power data characteristics and the transient stability of the electric power system; on the basis of dynamic adversarial adaptive transfer learning, self-optimization-approaching adjustment is carried out on a transient stability evaluation model based on Swin Transformer, and optimization-approaching evolution of an original model is realized after the operation condition of a power grid is changed; and evaluating the transient stability evaluation model of the power system based on dynamic adversarial adaptation. A dynamic adversarial adaptive transfer learning method is adopted, an implicit metric function of inter-domain differences is learned in an adversarial learning mode, and the relative importance of inter-domain edge distribution and conditional distribution is dynamically measured, so that self-optimization-approaching adjustment of a transient stability evaluation model is realized.

The invention relates to the technical field of computer vision and fire monitoring, and provides a fire image detection and segmentation method based on an end-to-end unified framework and physical knowledge embedding. Aiming at the problems of computation redundancy, feature segmentation and strong dependence on visible light caused by traditional staged processing, the invention provides the following technical scheme: constructing an end-to-end network comprising an Officient Hybrid Ender encoder and a mask-dino decoder, and realizing global modeling and cross-scale feature fusion through a single-layer Transform; thermal imaging physical knowledge embedding is innovatively introduced, and three fusion modes of pixel-level addition, feature-level Embedding and interactive learning are adopted; a lightweight single-layer Transform architecture and a multi-task loss function are designed, and GIOU, Dice and a joint detection segmentation hybrid matching strategy are combined. According to the method, a bounding box and mask prediction are synchronously generated through a unified query mechanism, the adaptability of a low-illumination scene is enhanced by using thermal imaging data, and the training efficiency is improved by decoupling bounding box loss. According to the invention, the real-time performance, robustness and precision of fire monitoring are significantly improved in a complex scene.

The invention discloses a pathological image classification system and method based on Mama and a hybrid expert model, the pathological image classification system proposes a challenging task for WSI classification by combining the advantages of the Mama and the hybrid expert model, and develops a novel Mama-based hierarchical network called Mama-WSI. In order to promote feature representation while ensuring efficiency, the Mama-WSI introduces bidirectional Mama blocks in a pseudo packet and across the pseudo packet to explore correlation between instances in the pseudo packet and global pseudo packet representation. Meanwhile, a dynamic memory library is provided to store historical information related to label prediction, and a cross-pseudo packet fusion module is designed to fuse effective information from a dynamically updated dictionary module so as to enhance current pseudo packet representation and global modeling. Furthermore, the expert hybrid module is capable of adaptively coordinating global and local packet representations to achieve robust packet label prediction.

The invention discloses a multi-degraded image restoration method based on semantic guidance. According to the method, for degraded images captured by a vehicle-mounted camera under severe weather conditions, potential features of the images are extracted by adopting a trunk network based on Transform, multi-modal semantic information is extracted in combination with a CLIP visual language model, and semantic guidance is provided for different degradation types through a dynamic text prompt generation mechanism. A self-adaptive feature fusion module is designed, channel attention and space attention mechanisms are combined to realize effective integration of multi-modal features, and a degradation feature extraction and fusion module is introduced to enhance the generalization ability of the model. According to the semantic guidance multi-type image recovery network SGIRN provided by the invention, the global modeling capability of the Transform and the cross-modal representation capability of the CLIP visual language model are combined, so that high-quality recovery of various weather degradation types is realized.

The invention relates to an event camera target detection method based on sparse Mama, and the method comprises the steps: adaptively discarding no-event and noise marks based on time-space continuity evaluation, capturing a global dependency relationship in space and channel dimensions, and showing the optimal balance between accuracy and efficiency; an information-first local scanning strategy (IPL-Scan) is provided and is used for guiding the model to pay attention to marks with high information content in the scanning process, so that the space context modeling capability is improved; and a global channel interaction module (GCI) is designed to aggregate channel information from the perspective of global space, and global interaction is expanded to a 3D feature space, so that the global modeling capability is further improved.

The invention belongs to the field of computer vision based on a deep neural network, and discloses a lightweight human body posture estimation method. According to the method, human body postures are recognized in real time through a designed lightweight method. A lightweight global modeling network LGM-Pose which is approximate to a single branch is provided. In the network, a global feature extraction module is designed, and lightweight extraction of global information is realized through a proposed parameter-free dimension transformation sub-module and MLP Block. And a convolution feature fusion module is designed to fuse multi-scale information through shuffling operation and grouping convolution, so that the problem of performance reduction caused by lack of multi-scale combination of a single-branch structure is relieved. Experimental results on a COCO data set show that compared with a current lightweight method in the field of human body posture estimation, the method provided by the invention has fewer parameters and obtains more excellent performance at the same time.

The invention discloses a short-term power load prediction method and system based on a CNN-Transform hybrid model, and the method comprises the steps: carrying out the data collection of historical load data and meteorological data of a power system, carrying out the data preprocessing of the collected data, and carrying out the coding of a periodic time feature, and obtaining periodic time coding information; local space-time features of the load data and the meteorological data are extracted by using a convolutional neural network, and hierarchical expression of the features is realized through a multi-layer convolutional structure during extraction; the local spatiotemporal features and the periodic time coding information are fused to obtain fusion features containing a load sequence, the long-period dependency relationship of the load sequence is modeled through a Transform network, global modeling of the fusion features is achieved through a multi-head self-attention mechanism, and a CNN-Transform hybrid model is obtained; a CNN-Transform hybrid model is used for prediction, and a load prediction result is output; according to the invention, the precision of load prediction and the generalization ability of the model are significantly improved.

The invention relates to a hyperspectral classification method and system based on wavelet spatial spectrum Mamba, and belongs to the technical field of hyperspectral image processing and deep learning. According to the method, a hyperspectral image cube is used as input, an image is divided into overlapped 3D small blocks, spatial and spectral characteristic decomposition is carried out on each small block at the same time, multi-resolution analysis is carried out on reconstructed characteristics by using classical Haar wavelets, long-range dependence is captured through a Mama network of a state space model, and finally the image is input into a classifier for classification. The method effectively balances the relationship between the network model performance and the calculation complexity, has the multi-scale feature extraction capability and the global modeling capability at the same time, can reduce the calculation resource consumption while remarkably improving the classification precision, and has excellent robustness and generalization capability.

The invention relates to the technical field of computer vision, in particular to a target tracking method based on adaptive template updating and lightweight Transform. The invention provides an efficient and robust target tracking algorithm for solving the problems that a traditional twin network is insufficient in robustness in a complex scene and a Transform architecture is highly dependent on computing resources. Firstly, a supervision feedback module is designed, supervision information related to a task is introduced in a feature extraction stage, and a network is guided to be more focused on a target area, so that the feature discrimination capability is improved, and background interference is effectively suppressed; secondly, a lightweight Transform structure is constructed, the calculation complexity and the parameter scale are remarkably reduced while the global modeling capability is maintained, and the balance between the model performance and the calculation efficiency is achieved; and finally, designing a self-adaptive template updating mechanism, and dynamically updating the template content in combination with the state information of the current frame and the environment change, thereby enhancing the adaptability of the model to the target appearance change and reducing the tracking drift risk.

The invention discloses an employee performance evaluation method and system based on multi-dimensional data, and belongs to the technical field of enterprise talent intelligent management, and the method comprises the steps of multi-dimensional employee data preparation, employee behavior modeling, employee performance fluctuation prediction, employee performance attribution analysis and employee performance evaluation. According to the method, employee behavior modeling is carried out by adopting a graph construction method combining multi-dimensional cooperation characteristics and behavior characteristics, and overall modeling and dynamic sensing of real positions, interaction strength and multi-dimensional behavior states of individual employees in an organization cooperation network are realized; employee performance fluctuation prediction is carried out by using a graph convolution bidirectional long-short term network model optimized by a joint loss function, and employee individual performance and dynamic evolution of mutual influence in an organization cooperation network are comprehensively modeled; the employee performance attribution analysis method based on anti-fact simulation is adopted to perform attribution analysis, and the influence of the key cooperation relation on employee performance change is quantitatively evaluated by simulating the hypothesis situation.

The invention relates to the technical field of computers, and discloses a voice separation method and device and a storage medium. The method comprises the steps of obtaining a to-be-separated mixed signal; converting the mixed signal into time-frequency characteristic representation through an encoder; performing feature separation on the time-frequency feature representation through a separation model to obtain feature sequences corresponding to different output channels; the separation model comprises a cache unit and at least one separation unit connected in sequence; each separation unit comprises a frequency domain modeling subunit, a time domain modeling subunit and a causal attention refining subunit which are connected in sequence, and the time domain modeling subunit in each separation unit is connected with the cache unit; decoding the feature sequences corresponding to different output channels through a decoder to obtain a time domain voice signal corresponding to each output channel; the additionally arranged cache unit can store historical information during global modeling, so that the separation model can perform feature processing based on the historical information, and the overall voice separation performance of the model can be improved.

The invention belongs to the technical field of mechanical health monitoring and predictive maintenance, and discloses a rolling bearing residual life prediction method and system, and the method comprises the steps: carrying out the adaptive feature extraction, transforming a TimesFM large model structure to adapt to the performance degradation modeling of a rolling bearing, and carrying out the field fine tuning based on the bearing data; and carrying out online deployment on the trained time sequence large model, inputting online monitoring data into an adaptive feature extraction module, and finally realizing visual early warning. According to the method, a transfer learning strategy of freezing a Transform trunk and fine tuning a small sample is introduced, and only task related parameters are adjusted, so that the dependence on new data is greatly reduced; a 1D convolution + MLP parallel structure is introduced, and complementary advantages of local convolution and global modeling are combined. According to the method, starting from an actual application scene, a complete end-to-end industrial grade prediction system is constructed, a matched visual component supports real-time prediction result display and early warning feedback, and the man-machine interaction capability is enhanced.

The invention discloses a multi-level feature enhancement and cooperative calibration multi-person attitude estimation method, which comprises an MLCpose model, and is characterized in that the MLCpose model comprises a backbone network, a PAFPN and a Head; the backbone network takes a multi-layer CSPLayer and a convolution module as a core, fuses a BSCA module, and improves global modeling and local detail capture capability through coordinate attention and bidirectional space attention; the PAFPN combines the high-level semantic features and the low-level space detail features through remodeling and connection operations; in the Head, an X-axis coordinate classifier and a Y-axis coordinate classifier are adopted to accurately position the coordinates of the key points, meanwhile, a dynamic MLTCC mechanism is introduced, and accurate positioning of the key points of each individual is predicted. According to the method, the feature reduction and boundary response capability of the occlusion area is improved through local detail reconstruction, noise interference is suppressed by dynamically recognizing the occlusion and labeling error and other outliers, and the positioning precision and stability of the model in a multi-target scene are effectively improved.

The invention discloses a deep learning method for realizing mechanical fault diagnosis, and belongs to the technical field of intelligent manufacturing fault prediction and diagnosis. Aiming at the problem that the diagnosis precision is sharply reduced along with the improvement of noise due to insufficient front-end feature extraction and mutual superposition of time domain limitation of a self-attention mechanism, a fault diagnosis classification model composed of three levels of feature processing layers is constructed; each stage comprises a wavelet-guided adaptive multi-scale convolution module and a frequency domain enhanced self-attention module; the wavelet-guided adaptive multi-scale convolution module can extract abundant multi-scale features under high noise; the frequency domain enhanced self-attention module carries out global modeling in the frequency domain, and the influence of noise on the overall recognition precision is reduced. The method has strong multi-scale feature extraction capability and anti-noise interference capability, effectively solves the problem of inaccurate diagnosis precision caused by a high-noise environment under an actual industrial condition, and is suitable for fault diagnosis of rotating mechanical equipment such as bearings and gears.

The present invention discloses an improved remote sensing image road extraction method based on a dual encoder, comprising the following steps: preparing a data set; building a remote sensing image road extraction network; training the remote sensing image road extraction network; and testing the remote sensing image road extraction network. The present invention uses a dual encoder combining Swin Transformer and pre-trained Resnet‑34 in the encoder to perform feature extraction to improve the connectivity of road extraction, and combines a cascaded parallel hole convolution block to realize multi-scale feature representation and reduce the fragmentation of the extracted roads. The present invention designs a spatial pixel module in the encoder to capture the spatial information of pixel-level features, thereby alleviating the inaccurate semantic information caused by occlusion. And enhances the global modeling capability of Swin Transformer due to the limitation of the window mechanism. The present invention not only maintains the complete structure of the road network, but also enhances the connectivity and integrity of the road.

The invention discloses a point cloud classification method based on a state space model, relates to the field of deep learning and three-dimensional point cloud target classification, and solves the problem of insufficient extraction of local geometric features and global features of an existing point cloud deep learning model. According to the method, point cloud targets are classified through geometric feature coupling and a cross path feature coordination enhancer. Local feature discrimination capability is improved by utilizing local geometric pooling with a geometric feature coupling mechanism through coordinated propagation and dynamic aggregation of geometric information between a local center point and a neighborhood of the local center point; according to the method, a designed collaborative feature intensifier is used for adopting double-path hybrid interaction, local mutation and sparse key signals are effectively processed, the limitation of traditional SSM global modeling is broken through, and high-precision classification of point cloud targets is achieved.

The invention discloses a transform-based video behavior recognition algorithm. The algorithm comprises the following steps: S1, carrying out skeleton key point feature extraction, 3D convolution and down-sampling on video frames in an input video clip; s2, performing image pre-training vit block construction on a feature layer obtained by down-sampling, and performing local and global modeling through a corresponding space unification module and a time sequence unification module; s3, performing multi-level fusion on space and time sequence information features, and then sending the space and time sequence information features and skeleton key point feature information extracted by the pose formerV2 into a multi-feature fusion module for feature information fusion; and S4, the fused feature information passes through a non-training pruning module. According to the training-free pruning unified convolution and self-attention Transformer model framework, Vits space interaction capability is pre-trained, so that the Vits space interaction capability can be fully utilized and reserved in a space unified module, and the space modeling capability of PSUformer is enhanced; the hierarchical time sequence interaction in the efficient time sequence unification module can be flexibly used for enhancing the time sequence modeling capability.

The invention provides a time sequence state data multi-step prediction method, device and equipment. A time sequence state data multi-step prediction model comprises a time sequence state data encoder structure and a multi-step prediction decoder structure; the time sequence state data multi-step prediction model is constructed based on a Transform model architecture; the structure of the time sequence state data encoder is based on a Transform encoder model architecture, and a long short-term memory network model is inserted between a multi-head attention sub-layer and a feed-forward network sub-layer of the Transform encoder for construction. The multi-step prediction decoder structure is constructed on the basis of a Transform decoder model architecture, and comprises a plurality of Transform decoders. According to the invention, a long-short-term memory network model is introduced, and the prediction performance is enhanced by combining global modeling and local time sequence dependence capability. A multi-decoder structure is designed to process a multi-step prediction task, and the problem of error accumulation is effectively solved.