3081 results about "Recognition system" patented technology

The invention discloses a large-model real-time voice text intention recognition method and system based on text prediction, and the method comprises the steps: obtaining the real-time voice data of a user, carrying out the real-time voice recognition processing through a streaming voice recognition interface, and obtaining a part of transcriptional text; inputting the partial transcription text into a mask language model for text prediction, and generating a plurality of high-credibility complete sentence candidates; based on the complete sentence candidates, the complete sentence candidates are input into a large language model in parallel for intention recognition, a corresponding intention result is obtained, and a mapping relation between the candidate sentences and the intention recognition result is established; and obtaining a sentence completely expressed by the user, calculating the similarity between the complete actual sentence and a plurality of high-credibility complete sentence candidates through a multi-level text similarity algorithm, selecting the candidate sentence with the highest similarity score, and directly obtaining a corresponding final intention recognition result based on the mapping relationship. The objective of the invention is to solve the technical problem of high response delay of an existing voice intention recognition system.

The invention relates to the technical field of smart power grids and artificial intelligence, in particular to a substation autonomous intelligent inspection method and system based on multi-modal data, and the method comprises the steps: obtaining inspection data of multiple modals, and generating fusion features; identifying system alarm information; performing intention recognition and task classification to generate an executable task sequence; generating a multi-device cooperative scheduling scheme; executing the multi-device cooperative scheduling scheme; iterative optimization is carried out; according to the intelligent inspection method provided by the invention, more accurate and more robust multi-mode perception and diagnosis are realized, and deep understanding of complex instructions and safe and efficient cooperation of multiple devices are also realized; and meanwhile, through dynamic re-planning and a verification type feedback learning mechanism, high real-time performance and robustness are ensured, and meanwhile, the system is endowed with the capability of iterative optimization.

A system for material detection and identification includes an interface configured to access a. material database associating each of a plurality of materials with one or more corresponding resonance frequencies: an RF transmitter configured to, for each material of at least a subset of the plurality of materials in the material database, transmit into an environment an RF signal at a first resonance frequency for the material; an RF receiver configured, to receive a response signal from the environment for each RF signal; and a processor configured to analyze each response signal for resonance characteristics that indicate a presence of the material and identifying the material to a user if the presence of the material is indicated by the resonance characteristics.

The invention relates to the technical field of traditional Chinese medicinal material identification, in particular to a traditional Chinese medicinal material intelligent identification and grading system based on deep learning, which integrates image acquisition, feature extraction, expression optimization, identification evaluation and origin traceability into a whole. Curvature, structure tensor and spectral features are extracted in combination with a differential geometry theory; constructing a Riemannian manifold representation space and performing isometric embedding dimension reduction optimization; identifying the types of the medicinal materials by using a deep convolutional neural network, and comparing with a standard model to evaluate the quality grade; the origin discrimination is realized based on the multi-scale feature comparison of geodesic distance, the category, quality and traceability information of the medicinal materials are comprehensively output, the surface visual features and internal component information of the traditional Chinese medicinal materials are comprehensively utilized through a multi-source data fusion technology, and the feature expression ability and discrimination precision of the recognition system are comprehensively improved.

The invention relates to the technical field of multi-modal posture recognition, and relates to a multi-modal posture recognition system based on Transform fusion, which fully excavates the complementary advantages of visual information in spatial detail representation and inertial information in time sequence dynamic capture through space-time alignment of multi-modal data and deep fusion based on an attention mechanism. The accuracy and the stability of attitude estimation under the conditions of visual occlusion, rapid movement and complex illumination are obviously improved; posture optimization is carried out by introducing physical constraints such as bone length constancy and joint movement range limitation, and time domain smoothing and contact state correction are applied, so that the precision of the generated three-dimensional posture sequence is ensured, a solid technical foundation is laid for improving the naturalness, safety and intelligent level of man-machine interaction, and the method is suitable for popularization and application. And meanwhile, reliable application and deep development of the intelligent robot in key fields of service robots, virtual reality, remote cooperation and the like are powerfully promoted.

The invention relates to the technical field of power equipment monitoring, and discloses a hydropower station high-altitude equipment fault intelligent identification system based on multi-sensor fusion, which collects multi-modal data in real time and evaluates data quality by deploying multi-source sensors at key parts of high-altitude equipment. Extracting multi-scale features of each modal, performing normalization processing, calculating a fusion weight based on feature saliency and data credibility, and performing weighted fusion and dimension reduction on the multi-modal features; based on the fusion feature vector, intelligent matching analysis of fault features and intelligent identification of fault types are carried out; a fault identification result is obtained; in addition, the system also comprises safety monitoring of overhead working personnel, and realizes closed-loop management from fault identification to safety maintenance. According to the invention, early weak faults can be accurately identified, and the safe operation level of equipment and the intelligent degree of operation safety management are improved.

The invention discloses a dangerous driving critical state identification method, and relates to the technical field of intelligent driving safety. According to the method, multi-mode signals of eye movement, electrocardio, skin electricity, vehicle operation and the like are collected and converted into a unified phase field, and the synchronous coherence of the unified phase field is analyzed; the individual phase dynamics manifold of the driver is learned on line by using a Shenchang differential equation, and a system instability precursor is identified by detecting the behavior that a state point escapes from a steady state attractor; further, multi-dimensional indexes such as synchronous collapse and topological fracture are fused, collapse time is estimated in combination with a Lyapunov index, and an advanced early warning instruction is generated; and finally, based on the model predictive control and the personalized phase response curve, generating and executing targeted multi-mode phase reset intervention, and forming a sensing-early warning-intervention active safety closed loop. According to the invention, normal form transformation from post-event alarm to beforehand regulation and control is realized, and early warning advancement and intervention accuracy are improved.

The invention discloses a machine learning-based weld defect intelligent identification system, relates to the technical field of weld defect intelligent identification, solves the technical problems of multi-modal data fusion precision and robustness optimization and defect shielding or overlapping feature deficiency, and provides a machine learning-based weld defect intelligent identification method based on PSNR dynamic parameter adjustment and gradient weight optimization. The limitation of existing fixed parameter denoising is solved, the edge feature retention rate of cracks, air holes and other defects is improved, the omission ratio is reduced, improved DeepLabv3 + segmentation semantic masks are introduced and mapped to point cloud voxels, geometric + semantic double-attribute enhanced point clouds are formed, the defect area positioning accuracy is improved, and through a cross-modal attention module, the defect area positioning accuracy is improved. Weights are dynamically distributed according to illumination intensity and workpiece materials, feature waste caused by fixed weights is avoided, depth mutation and a shielding area with semantic defects are positioned by utilizing depth information of enhanced point cloud, real overlapping and projection overlapping can be effectively distinguished by combining an improved Poisson fusion algorithm, and the overlapping defect recognition accuracy is improved.

The invention relates to a tunnel three-dimensional disease intelligent identification system and method based on a large model, the system comprises a point cloud data acquisition module and a processor, and the processor comprises a data processing module, a three-dimensional tile optimization module and a disease identification module. The data processing module carries out standardization, noise reduction and registration processing on the received point cloud data; the three-dimensional tile optimization module constructs a multi-level tile pyramid structure based on the registered point cloud data, establishes a mapping relation between a space coordinate and a tile index, compresses tile data based on a curvature point cloud simplification algorithm and adjusts texture quality to form a three-dimensional tile image; calculating a comprehensive score of the tile quality to verify the quality of the three-dimensional tile image; a disease identification module extracts multi-modal fusion features of the preprocessed three-dimensional tile image and geometric features corresponding to disease types; fusing the multi-modal fusion feature and the geometric feature to obtain a joint fusion feature; performing field fine tuning on the joint fusion features; and obtaining a disease identification result.

The invention discloses an image recognition system for industrial part defect detection, and particularly relates to the field of part defect detection, which comprises a multi-axis controllable light source array module, a high-speed polarization camera module, an edge computing node module, a double-branch semantic segmentation network module and a physical constraint post-processing module, according to the invention, through combination of time-sharing stroboscopic illumination and polarization image sequence acquisition, multi-dimensional perception of surface topography and material differences is realized; generating an elevation map and a normal map by using photometric stereo solution, constructing a differential rendering layer reverse matching CAD model, and extracting flash sensitive features; a double-branch U-Net network is adopted to fuse geometric and polarization characteristics, the characterization capability is enhanced through a trans-attention mechanism, and a pixel-level mask is output; and finally, mapping a two-dimensional result to a three-dimensional coordinate system by means of calibration parameters, carrying out geometric verification in combination with a tolerance zone and a height threshold value, and automatically generating a structured defect report containing position, size, grade and visual information.

The invention discloses a wide-frequency-domain weak signal data acquisition method, system, equipment and medium, and relates to the technical field of power system monitoring and fault diagnosis, and the method comprises the steps: collecting an input signal, analyzing the signal characteristics in real time, obtaining the frequency composition and amplitude change information of the signal, and dynamically adjusting the sampling rate according to the signal characteristics. Performing noise reduction processing on the collected signals, eliminating noise interference and retaining effective signal components, performing time alignment processing on the data to form a data set with a unified time reference, extracting multi-category features based on the data set, performing fusion judgment, identifying whether the system is in a fault state, and when it is judged that a fault occurs, judging whether the system is in a fault state or not; if yes, fault analysis and positioning are executed, and an analysis result is generated and output. According to the method, fast Fourier transform analysis is carried out on the signals, the dominant frequency and harmonic components can be accurately recognized, the energy ratio can be calculated, and a reliable data basis is provided for signal feature extraction and fault diagnosis.

The invention discloses an autoclaved aerated concrete member surface defect intelligent identification system based on image processing, and particularly relates to the field of defect identification, comprising an image acquisition module, an image preprocessing module, a defect candidate region extraction module, a defect identification and classification module, and a result output and alarm module; according to the method, a high-definition industrial camera is used for collecting a component surface image, and adaptive median filtering and a Retinex algorithm are adopted for image denoising and enhancement, so that the influence of noise and uneven illumination is eliminated; utilizing an improved multi-threshold segmentation and Canny edge detection algorithm to accurately extract a defect candidate region; the method comprises the following steps: extracting three types of feature parameters of shape, texture and gray scale, and inputting the three types of feature parameters into a deep learning model taking ResNet50 as a basic network to realize automatic identification and classification of four types of typical defects of cracks, holes, unfilled corners and surface peeling; and finally, the system divides severity levels according to the defect size, and triggers differentiated visual alarm and linkage control.

The invention provides a deep learning-based oral cavity image recognition method, a storage medium and a deep learning-based oral cavity image recognition system. The method comprises the steps of deploying a federated learning framework and collecting a multi-modal oral cavity image data set; extracting local features to obtain image features, and generating a modal adaptive weight map; a multi-head self-attention mechanism is used for fusing the cross-modal features to generate a fused feature map, and deconvolution up-sampling is carried out to form high-resolution multi-modal feature representation. A tooth segmentation mask is generated based on this representation, and an initial diagnostic report is generated. And aggregating the attention weight of each client through an encryption protocol, and generating interpretable decision support data. And finally, generating a structured clinical report by using a natural language. According to the method, the Grad-CAM thermodynamic diagram is combined with the encrypted and aggregated attention weight, so that the privacy security is guaranteed, the model interpretability is enhanced, the clinical credibility and the diagnosis decision efficiency are improved, and the problems of insufficient diagnosis precision of complex lesions and insufficient utilization of multi-modal information in the prior art are solved.

The invention relates to the technical field of intelligent operation and maintenance of photovoltaic power stations, in particular to an AI video recognition system for photovoltaic power station equipment inspection, which comprises a data acquisition unit, a data preprocessing unit and a defect recognition unit, and is characterized in that characteristics from one mode are used as query, evidence information serving as keys and values is searched from corresponding area characteristics of other modes, and the data acquisition unit is used for acquiring data; the method is used for synergistically diagnosing compound and early defects with weak or invisible characteristics in a single mode. According to the method, cross-modal collaborative reasoning can be realized: when a suspicious feature is found in one modal, whether evidence features capable of mutually verifying exist in the same position in other modals or not can be inquired, the diagnostic logic of field experts is simulated, different physical phenomena can be associated, and the probability of mutual verification is reduced. Therefore, early-stage or composite defects which are extremely difficult to find in any single mode can be accurately identified. Therefore, the problem that the recognition performance is reduced due to environmental interference such as illumination and shadow can be fundamentally solved.

The invention discloses a comprehensive remote sensing recognition system for hidden danger of ice lake outburst. The system comprises a data acquisition and preprocessing module, an ice lake extraction module, an ice dam stability evaluation module, a risk calculation module and a visualization and early warning module. According to the system, optical images, SAR images, LiDAR point cloud and meteorological data are utilized, firstly, a glacial lake boundary is extracted through a water body index, the area and the volume are calculated, then the water level rising rate is monitored in combination with multi-temporal data, the dam body stability is evaluated by utilizing three-dimensional terrain and deformation parameters, and the dam body stability is evaluated by combining the temperature melting corrosion rate and triggering factors such as rainfall, earthquakes and inflow. And constructing a multi-factor outburst risk index model, and finally carrying out visualization and graded early warning on a result in a GIS platform. The method can achieve the quick, comprehensive and intelligent recognition of the hidden danger of ice lake outburst, has the advantages of being high in monitoring precision, high in automation degree, high in early warning timeliness and the like compared with a traditional single data or experience judgment method, and can be widely applied to the disaster prevention and control work of dense ice lake areas such as the Qinghai-Tibet Plateau and the Himalaya mountain.

The invention discloses a driver fatigue state real-time identification system and method based on multi-modal deep learning, and relates to the technical field of fatigue driving detection. Firstly, feature extraction is performed on brain wave shapes and eye movement coordinates, and respective weights are calculated by using an attention mechanism, so that dynamic distribution of different modal features is realized. And then, in-vehicle illumination data is introduced to establish a credibility mapping function so as to carry out adaptive correction on an eye movement weight, thereby effectively reducing interference of a complex illumination environment on an identification result. And carrying out weighted splicing on the corrected multi-modal features, mapping the multi-modal features into a brain-eye collaborative fatigue value, and carrying out judgment in combination with the duration, so as to finally realize stable and reliable early warning control. The method has the advantages of high fusion precision, high environmental adaptability and low false alarm rate while ensuring the real-time performance, and the driving safety guarantee capability can be remarkably improved.

The invention relates to the technical field of intelligent traffic and road maintenance, and provides a road disease intelligent identification system and method based on artificial intelligence and Beidou positioning, and the method comprises the steps: carrying out the edge calculation preprocessing of noise reduction, image enhancement and region-of-interest segmentation of collected road image data, and extracting initial image features; inputting the initial image features into a deep learning disease recognition model based on transfer learning optimization, outputting a disease type and a disease grading result, and forming multi-source fusion data; performing spatio-temporal data association analysis on the multi-source fusion data to realize disease environmental impact assessment; the method comprises the following steps: predicting future development conditions of road diseases, sending out early warning information, dynamically evaluating road health condition grades according to road disease data, making an optimal maintenance plan according to positions, types and grades of the diseases and environmental influence evaluation results, and optimizing traffic dispersion and route recommendation according to the road disease conditions.

The present disclosure relates to a method of enrolling an individual at a secure server and subsequently authenticating and identifying the individual at an authenticating party using an authentication token created during the enrolment and a secure server performing the method. The method comprises engaging, via a user device, in an enrolment process with the individual, registering the user device by receiving a public key, the public key being created by the user device along with a private key corresponding to the public key, acquiring a trusted identifier of the individual, associating the acquired trusted identifier of the individual with at least one database index to create an authentication token, the database index being utilized for look-up at the secure server, signing the authentication token, and sending the signed authentication token to the user device, while deleting the acquired trusted identifier at the secure server.

A system for material detection and identification includes an interface configured to access a material database associating each of a plurality of materials with one or more corresponding resonance frequencies; an RF transmitter configured to, for each material of at least a subset of the plurality of materials in the material database, transmit into an environment an RF signal at a first resonance frequency for the material; an RF receiver configured to receive a response signal from the environment for each RF signal; and a processor configured to analyze each response signal for resonance characteristics that indicate a presence of the material and identifying the material to a user if the presence of the material is indicated by the resonance characteristics.

The invention discloses a data migration method and device, computer equipment and a storage medium, and realizes a data migration processing mode oriented to an unstable network environment by combining neural network prediction, decision tree anomaly recognition and a distributed cache mechanism. Therefore, the data transmission stability and the migration task completion rate in the cross-network environment are obviously improved. Firstly, a pre-trained neural network is utilized to carry out deep prediction on a real-time network state, and a potential performance reduction trend is identified in advance, so that a migration process can actively adjust a path or a strategy before a risk occurs. And secondly, by performing decision tree identification on a data packet sending sequence, the system can position a potential interruption point based on fine-grained time sequence characteristics, and quantify the risk in an interpretable manner, so that migration control can be accurate to a specific data packet level. Finally, under the condition that the risk is unacceptable, the system automatically activates a distributed cache mechanism, and the durability and the restorability of the data are guaranteed in a redundant copy storage mode.

The invention provides a through-the-wall radar human body behavior recognition method and recognition system based on spatial-temporal characteristics, and the method comprises the steps: obtaining and processing through-the-wall radar human body behavior echo sampling signals, and obtaining a corresponding time-Doppler spectrogram as a sample set; and training a constructed TWRMama network by using the sample set, in which the network adopts a CA-Mama block as a core feature extraction module, can efficiently model and input a time sequence dynamic feature and a spatial dependency relationship of a time Doppler spectrogram, and effectively enhance the feature expression ability in a shielding scene, thereby realizing accurate recognition of human behaviors of the through-the-wall radar. Besides, information interaction between patches is introduced into a patch embedding module of the TWRMama network, so that the expression capability of the network can be further improved, and the calculation complexity of the network is effectively reduced.

The invention discloses a medical image automatic identification system based on a neural network, and relates to the technical field of medical image identification. The method is used for solving the problem that early recognition of neurodegenerative diseases is difficult due to medical image and genome data splitting and poor model interpretability in the prior art. The method comprises the following steps: firstly, extracting multi-scale features of a brain structure through a three-dimensional convolutional neural network and a self-attention mechanism, calculating a multi-gene risk score based on a risk site, and encoding the score into a feature vector; secondly, using a cross attention mechanism to take gene features as query vectors, fusing the gene features with image features, and generating brain structure anomaly features under gene regulation; then, gradient weighting class activation mapping is applied to generate a visual thermodynamic diagram, and gene-image association weight weighting is combined to construct a brain region risk distribution diagram; and finally, a high-risk brain region space coordinate set is extracted through threshold segmentation, and an accurate quantification basis is provided for early recognition.

The invention discloses a semi-supervised LPI radar signal modulation identification system and method based on entropy perception pseudo labels, and relates to the technical field of radar signal processing and mode identification. The system comprises a preprocessing module, a multi-scale reconstruction enhancer, a classification backbone network and a semi-supervised training module. The multi-scale reconstruction intensifier is used for reconstructing dual-channel separation through high-frequency detail enhancement and a low-frequency structure and enhancing discriminative characteristics in a noise environment; the classification backbone network introduces an adaptive contraction unit to realize channel-level noise suppression; and the semi-supervised training module dynamically evaluates the uncertainty of the unlabeled samples by adopting an entropy sensing mechanism, and generates weighted pseudo labels to carry out consistency regularization training. The method realizes signal modulation identification based on the system. According to the method, the problem of feature shielding under the condition of low signal-to-noise ratio is solved, the dependence of the model on labeled data is reduced through a reliable pseudo label generation mechanism, and stable and efficient modulation identification can still be realized in a severe channel environment with scarce labeled data.

The invention relates to the technical field of safety engineering, in particular to a tunnel unfavorable geologic body recognition system and method based on fused geological information, and the method comprises the steps: data collection and preprocessing, geological information quantification, sample set construction and enhancement, Faster-RCNN model construction of fused geological information, model training and optimization, and unfavorable geologic body intelligent recognition. The intelligent recognition of the unfavorable geologic body of the tunnel is realized by fusing the geological information and the seismic wave method imaging data and utilizing the convolutional neural network model, the recognition precision and robustness are effectively improved, and effective technical support is provided for tunnel construction safety early warning.

The invention relates to the field of endoscope image processing, and particularly discloses an automatic lesion recognition system for endoscopy of the digestive system department, which is characterized in that after a preprocessed original endoscope image is acquired, a double-branch parallel processing architecture is used to acquire characteristics with low resolution and rich semantic information through a deep context branch, and the characteristics of the original endoscope image are acquired. The potential area of the focus is accurately deduced; meanwhile, the fine texture of the mucous membrane is captured in a lossless manner through shallow detail branches which keep high resolution in the whole process. Furthermore, through a context-guided asymmetric enhancement mechanism, a global view of a deep branch is utilized to generate an uncertainty perception attention map as a reference, and weak detail features corresponding to a potential focus area in a shallow branch are accurately irradiated and adaptively enhanced. Thus, a conservative enhancement strategy is adopted in an uncertain focus area, background noise is effectively inhibited, and therefore the detection sensitivity and robustness of low-contrast and flat focuses are fundamentally improved.

The invention discloses a simulation platform for autonomous tracking of a subsea pipeline by a UUV. The simulation platform comprises a mother ship command and control node system used for issuing a task instruction to a UUV motion control and situation display system; the UUV motion control and situation display system is used for sending a course adjustment instruction to the UUV underwater unmanned vehicle and carrying out real-time graphical dynamic display on the navigation information of the UUV underwater unmanned vehicle; the UUV side-scan sonar model system is used for receiving the motion attitude information of the UUV underwater unmanned vehicle and carrying out detection result data production calculation on a target area in combination with the motion attitude information; the UUV subsea pipeline detection and identification system is used for carrying out pipeline identification according to side scanning detection result data and sending pipeline information to the UUV pipeline tracking strategy system; the UUV pipeline tracking strategy system is used for calculating the track information of the UUV underwater unmanned vehicle according to the pipeline information and the navigation information of the UUV underwater unmanned vehicle, and sending the track information to the mother ship command control node system.

The invention discloses an action recognition method based on a virtual inertial measurement signal generation model, and relates to the technical field of action recognition, and the method comprises the steps: collecting a surface electromyogram signal and an inertial measurement signal, and carrying out the preprocessing of the collected signals; constructing and training a generator to convert the surface electromyogram signals into virtual inertial measurement signals; inputting the generated virtual inertial measurement signal and the original surface electromyogram signal into an action recognition model, and training to obtain a classification module; the performance of an action recognition model is evaluated by using a test data set, the accuracy rate, the recall rate and the F1 score index are calculated, the recognition effect of the model is measured, a classification model and a generator are optimized according to the recognition effect, and the advantages of two modal signals can be fully utilized by fusing surface electromyogram signals and virtual inertial measurement signals, so that the accuracy of the action recognition model is improved. And the motion information of the fingers, the wrist, the forearm and other parts is analyzed, so that the complex limb motion is identified more accurately, and the performance of the limb motion identification system in practical application is improved.

A ship sub-assembly weld seam identification method, comprising: creating a point cloud weld seam identification training dataset by means of point cloud discretization, virtual sampling, weld-zone semi-automatic interactive annotation, and style transfer-based augmentation of ship sub-assembly CAD data; training a point cloud weld seam identification model, and exporting a weld seam identification model data file; using a 3D point cloud camera array as a sampling camera to collect workpiece 3D point cloud data of a sub-assembly workpiece to be welded at the production site; by means of the weld seam identification model data file, obtaining weld seam annotation position data and positioning a weld seam; and issuing a welding process plan and instruction to control a welding robotic arm and a control device therefor to complete sub-assembly workpiece welding. Also provided are a ship sub-assembly weld seam identification system, a computer device, and a computer-readable storage medium. The method overcomes the shortcomings in conventional 3D weld seam identification methods such as point cloud incompleteness and identification failures caused by steel types and the imaging angles of 3D cameras, thereby greatly improving the adaptability of automatic weld seam identification methods to complex workpieces.

The invention discloses a special gas cylinder state real-time monitoring and abnormal behavior recognition system, which belongs to the technical field of gas cylinder safety monitoring, and comprises a gas cylinder state data acquisition module for continuously acquiring the pressure, the temperature, the position and the gas concentration of a gas cylinder in real time within one second; the gas cylinder digital twinning module constructs a gas cylinder virtual three-dimensional digital model based on gas thermodynamics physical constraints, realizes double-source fusion of physical constraints and actually measured data by establishing an equivalent mapping relation between the virtual model and a gas cylinder real-time state, and performs adaptive correction on the equivalent mapping relation based on a deviation decomposition attribution mechanism; the deviation decomposition attribution mechanism divides deviations into random errors, systematic accumulated deviations and abnormal deviations, and differential correction strategies are adopted for different types; the equivalent mapping dynamic updating unit continuously optimizes a mapping function through an online learning algorithm, so that the model has a self-evolution capability; and conversion from passive monitoring to active prediction is realized.

The invention provides a material detection operation behavior identification method based on multi-modal perception, and belongs to the field of artificial intelligence and electric power operation and maintenance detection, and the method comprises the steps: carrying out the feature extraction of the preprocessed visual, motion and voice modal data, and obtaining visual, motion and voice modal features; the visual, action and voice modal features are input into the trained fine-grained behavior segmentation network, the multi-modal input layer is used for carrying out weighted fusion on the visual modal features, the action modal features and the voice modal features to obtain multi-modal global feature representation, and the time sequence modeling layer is used for carrying out frame-by-frame analysis and labeling on the multi-modal global feature representation to obtain the multi-modal behavior segmentation network. The hidden state sequence is output, and the classification output layer is used for classifying the hidden state sequence and recognizing behavior categories; matching and aligning the behavior type with a preset standard operation instruction, and judging whether the operation behavior is abnormal or not; the invention further provides an identification system. And accurate identification and compliance evaluation of the operation process are realized.