1882 results about "Action recognition" patented technology

An action localization method, device, electronic equipment, and computer-readable storage medium are provided. The action localization method includes: identifying at least one target video segment containing a target object in a video; acquiring a first action recognition result of at least one image frame in the at least one target video segment and a second action recognition result of the target video segment; and acquiring an action localization result of the video based on the first action recognition result and the second action recognition result.

The invention relates to the technical field of computer vision and artificial intelligence, and discloses a classroom teaching-oriented student dynamic expression and action recognition evaluation system and method, and the system comprises a video collection module, an image processing module, an expression and action recognition module, an evaluation analysis module, and a feedback output module. The video acquisition module is used for acquiring video data of a classroom site, the image processing module is used for preprocessing the video data, the expression and action recognition module is used for recognizing expressions and actions of students by using a deep learning algorithm, and the evaluation analysis module is used for evaluating learning states of the students. And the feedback output module outputs the evaluation result to a teacher in a chart or report form. By integrating video acquisition, image processing, deep learning recognition, data analysis and other technologies, expressions and actions of students are monitored and analyzed in real time, so that more accurate and timely learning state feedback is provided for teachers, the teachers can adjust the teaching method in time according to the results, and the classroom teaching effect can be effectively improved.

The invention relates to the technical field of artificial intelligence, in particular to a family service method and system based on artificial intelligence, and the method comprises the following steps: obtaining a statement analysis action sequence and a task direction, collecting the feedback of a device to generate a state structure, judging an instruction trend, recombining a statement to generate a behavior chain, and disassembling an action recognition conflict to extract a main control path. Analyzing a behavior time sequence generation prominent trend, and adjusting a display sequence to generate a function scheme. According to the method, through semantic component extraction and equipment state mapping, the accuracy of instruction target recognition is improved, word order rearrangement and logic connection enhance the continuity of a behavior chain, a control path is optimized and sorted according to verb density and object association, the task scheduling precision is improved, and behavior time sequence analysis recognizes a high-frequency operation forward trend; according to the method, priority dynamic adjustment and rearrangement of recommended content in combination with instruction frequency and time sequence characteristics are realized, the response initiative and the service matching degree are enhanced, and collaborative optimization of semantic understanding, behavior prediction and content recommendation is integrally realized.

The invention discloses a video analysis-based multi-scene operator violation behavior identification method and system, and belongs to the technical field of intelligent operation safety monitoring and artificial intelligence identification, and the method comprises the steps: collecting a real-time video stream of a multi-scene operation site; recognizing a continuous action time sequence in the real-time video stream by using an action recognition depth model; constructing the continuous action time sequence into an action behavior sequence; the action behavior sequence is constructed into a directed behavior graph with time, space and action labels, the directed behavior graph is compared with a directed behavior graph corresponding to the standard action behavior sequence, and illegal behaviors are recognized; and carrying out multi-mode early warning on the identified illegal behaviors. According to the method, the bottleneck that the traditional image recognition technology is weak in action sequence semantic understanding and poor in environmental adaptability is broken through, and accurate recognition and real-time early warning of illegal behaviors in multi-scene operation are achieved.

The invention relates to the technical field of physical education, and discloses an artificial intelligence-based physical education evaluation system, which comprises a data acquisition module, a multi-target detection module, a key point modeling module, an action trajectory analysis module, a multi-modal fusion module and a dynamic feedback module. The data acquisition module receives a video stream of the camera device and motion physiological data of the wearable sensor; the multi-target detection module locates a target based on a YOLOv8 algorithm; the key point modeling module constructs a three-dimensional action attitude model by using an OpenPose algorithm; the action trajectory analysis module evaluates action standard; the multi-modal fusion module integrates the data to construct a multi-dimensional feature matrix; the dynamic feedback module generates a real-time score and a personalized correction instruction through the optimized LSTM network. In addition, the system further comprises a model self-adaption module, an abnormal action recognition module and a distributed calculation module. According to the system, precise evaluation of physical education is realized, the sports safety is guaranteed, personalized teaching is promoted, and the physical education quality is effectively improved.

The invention discloses a remote monitoring and guiding system for rehabilitation training in the orthopedics department, particularly relates to the technical field of medical rehabilitation intelligent monitoring, and is used for solving the problems that in an existing remote rehabilitation system, timing sequence dislocation exists in action recognition and regulation and control instruction execution, and joint movement safety early warning is delayed. According to the method, localized time sequence alignment processing is performed on joint movement data and electromyographic signals based on edge computing nodes, a joint linkage action time sequence difference is generated, and an abnormal action risk level is judged in real time by combining an electromyographic signal pre-activation feature and a conflict detection mechanism of a standard action intention; historical abnormal records are matched through the cloud platform to generate a hierarchical regulation and control instruction set, and progressive safety intervention is triggered by the patient end equipment according to the dynamic deviation of the joint activity; through a collaborative mechanism of edge side action stage accurate analysis, electromyographic signal feed-forward verification and multi-stage instruction dynamic binding, millisecond-level response of joint linkage abnormity in staged rehabilitation training is realized, and the risk of joint over-limit activity is effectively avoided.

The invention discloses a physical fitness evaluation method based on human skeleton trajectory tracking, which comprises the following steps: S1, acquiring whole-process video data of a physical fitness test through video acquisition equipment, and generating a video frame according to an acquisition frequency; s2, processing the video frame by using a skeleton dynamic analysis algorithm, and extracting human skeleton key points; s3, extracting identity features of the testee through a motion map identity recognition network, and tracking and confirming the identity features; s4, performing action recognition, fragment segmentation and compliance judgment on the skeleton key point time sequence track; s5, aiming at the abnormal skeleton key points, performing complementation and time sequence smoothing by adopting an inverse kinematics inference method; s6, inputting the complete skeleton key point time sequence data into the human body action evaluation model for processing, and quantizing and outputting a physical ability evaluation index; and S7, generating a physical ability evaluation report and giving action feedback and optimization suggestions. According to the invention, objectivity, accuracy and intelligent level of physical ability evaluation are effectively improved.

The invention relates to the technical field of action recognition, in particular to a somatosensory action interaction recognition method and system based on skeleton coordinate points. The method comprises the following steps of collecting real-time skeleton coordinate data of a human body and performing multi-modal feature extraction to obtain a real-time skeleton coordinate sequence; obtaining a standard skeleton posture corresponding to the target interaction action, performing pre-recording and feature coding, and generating a target posture skeleton feature template library; performing skeleton time sequence filtering and joint mapping and joint included angle calculation on the real-time skeleton coordinate sequence, performing similarity measurement and dynamic binding tracking at the same time, and starting a binding recovery mechanism when binding loss is detected so as to guide the user to execute a preset binding posture and re-establish a binding relationship; and mapping the joint included angle time sequence data to a corresponding joint of the virtual human shape interaction model in real time, outputting a somatosensory interaction instruction, and driving to repeat a human body action so as to trigger a somatosensory action interaction event. According to the invention, the stability of somatosensory action interaction recognition can be improved.

The invention relates to an ICU patient dysphoria video recognition method based on space-time motion detection. Compared with the prior art, the defect that dysphoria video recognition is difficult to carry out on an ICU patient is overcome. The method comprises the following steps: acquiring and preprocessing a data set; constructing an ICU patient dysphoria identification model; training an ICU patient dysphoria identification model; obtaining video stream data to be identified; iCU patient dysphoria atom action recognition; and performing dysphoria quantitative classification based on an RASS score table. According to the method, the Slowfast basic model is improved, the detection precision and speed of the model are remarkably improved, the dysphoria behavior of the patient is automatically and accurately detected and understood from the perspective of space-time motion detection, the dysphoria condition of the patient is automatically scored according to the RASS score table, and accurate recognition of the dysphoria video of the ICU patient is achieved.

The invention discloses a micro posture recognition method based on multi-modal feature fusion and fine adjustment, and relates to the field of computer vision and action recognition. The method is characterized in that a universal cross-modal knowledge fusion framework is provided, and multi-modal features are respectively extracted through a fine tuning network by using three kinds of modal information of video, skeleton and text. Meanwhile, a video-skeleton and text-skeleton fusion module is introduced to enhance the interactivity between modals. Compared learning is adopted to align feature distribution, and model training is optimized in combination with a freezing-fine tuning strategy, so that the calculation complexity is reduced, and the recognition efficiency is improved. According to the method provided by the invention, the problem of single-mode information loss can be solved, the perception capability of tiny attitude change is enhanced, the recognition precision and robustness are improved, and the method is suitable for multiple application scenes such as behavior monitoring, human-computer interaction and safety protection.

The invention relates to the technical field of computer vision data analysis, in particular to an agent training analysis algorithm based on a large model. The computer vision technology is used for extracting bone key points of officers and soldiers from videos, and accurate action analysis is carried out. And deep learning, especially a convolutional neural network CNN and a generative adversarial network GAN, is used for improving the accuracy of skeleton key point recognition and action recognition. The physical training technology comprises an officer and soldier training method and effect evaluation, a traditional method depends on observation of a coach, and a modern technology can improve the training efficiency and quality through video analysis, especially in a training scene needing accurate action analysis. The video analysis technology serves as a core link and is responsible for extracting action features from video data. Through advanced algorithms and models, the feature data are processed and analyzed, and detailed feedback and improvement suggestions are provided, so that coaches and officers and soldiers are helped to optimize training effects.

The invention belongs to the technical field of vehicle control, and particularly relates to a vehicle-mounted image recognition and target detection system based on deep learning, and the system comprises a distributed monitoring module which collects the operation, obstacle and traffic signal information of a target vehicle through multi-modal classification and scene matching, completes the marking of a shielding region and the matching of information through the combination of shared data, and achieves the recognition of the target vehicle. Forming an enhanced monitoring set; the label planning module constructs an enhanced topological space based on the enhanced monitoring set, and adjusts moving tracks in different scenes by combining with vehicle and pedestrian track probability distribution fed back by dynamic intention recognition; the action recognition module predicts trajectory parameters and collision probabilities of non-target vehicles and pedestrians by using Bayesian and multi-modal algorithms; the decision-making module generates a real-time control instruction through particle swarm optimization and fuzzy control, and optimal control parameters are fed back through simulation; according to the invention, intelligent track planning and real-time control in a complex scene are realized, and the detection precision and control robustness of the shielded and label-free area are improved.

The invention discloses a multi-source federal cross-domain and source-domain enhanced millimeter wave action recognition method and system, and the method comprises the steps: generating a micro-Doppler spectrogram through a millimeter wave radar, and extracting the motion features of human body actions through a signal processing module; in the federal multi-source domain adaptation module, dynamically evaluating and fusing knowledge of a plurality of source domains by adopting a voting-based pseudo-tag method and a weighted knowledge aggregation mechanism, and optimizing the generalization ability of a target model; and through a generalization gap optimization method, the performance of the source domain model is improved, and the robustness of the system in different environments is ensured. Through combination of a federated learning framework and a multi-source domain adaptation technology, unsupervised learning under the condition that a target domain has no annotated data is realized, only a single set of millimeter wave equipment is needed, a millimeter wave communication protocol is compatible, and the method has the characteristics of privacy protection, unsupervised learning, multi-source knowledge fusion and strong generalization ability. The method is suitable for application scenes of smart home, health monitoring, man-machine interaction and the like, and has wide practical application value and research prospect.

The invention discloses a generative question and answer method and device based on classification guidance and a readable medium, and the method comprises the steps: constructing an original dialogue data set, and training to obtain a trained theme classification model, a trained intention classification model and a trained action recognition model; extracting partially processed dialogue corpora from the original dialogue data set, constructing a classification guide instruction dialogue data set by utilizing the trained subject classification model, the trained intention classification model and the trained action recognition model, and mixing the original dialogue data set and the classification guide instruction dialogue data set to obtain a fine tuning data set; performing fine tuning on the pre-trained generative question and answer model by adopting the fine tuning data set to obtain a fine-tuned generative question and answer model; and obtaining a historical dialogue to be replied, and inputting the historical dialogue to the fine-tuned generative question and answer model to obtain a corresponding output statement. According to the invention, the guidance of the dialogue can be enhanced, and the answer is controllable.

The invention discloses a human motion posture recognition method and system based on multi-modal data fusion, and relates to the technical field of posture recognition, and the method comprises the steps: firstly, synchronously obtaining a human motion posture video frame and an IMU segment, then carrying out the feature extraction of the two kinds of heterogeneous data in a shunt parallel mode, and carrying out the feature extraction of the two kinds of heterogeneous data; and respectively capturing visual space attitude information and dynamic inertial characteristics of the IMU. Afterwards, specific features of the modals are uniformly expressed through a mixed Token and embedding mechanism, and a space-inertia collaborative attention fusion mechanism is further introduced to realize dynamic association and deep fusion of cross-modal information; and finally, classifying the multi-modal fusion representation vector obtained by fusion so as to realize accurate recognition of the human motion posture. In this way, the defect that traditional fusion is insufficient in capturing subtle action differences can be overcome, and the accuracy and stability of action recognition in a complex scene are improved.

The invention discloses a sports motion recognition method based on 3D space-time attention and a slowfast network, and the method comprises the steps: obtaining an original human body motion video, sampling the original human body motion video through different frame rates, generating Slow branch and Fast branch input images, and taking the Slow branch and Fast branch input images as training samples; constructing a dual-branch network based on spatio-temporal feature extraction; designing a loss function, and training the dual-branch network based on spatial-temporal feature extraction by using the training sample; and sampling a to-be-identified human body behavior video through different frame rates to obtain corresponding video frame images, and inputting the video frame images into the trained dual-branch network based on spatial-temporal feature extraction to obtain an identified human body behavior category. According to the sports action recognition method based on the 3D space-time attention and slowfast network, the problems that in the prior art, the locality of the dependency relationship between the features in the space-time dimension is captured, and the model complexity is high are solved.

The invention discloses a Pick ball athlete action recognition and analysis system based on multi-modal data fusion, and the system comprises the steps: collecting the video image data and sensor data of a Pick ball athlete in the motion process, and carrying out the feature extraction of the body posture of the athlete in the video image data through a convolutional neural network; according to the method, a recurrent neural network is combined to perform action classification on a feature sequence, meanwhile, sensor data are fused, accurate recognition of actions of an athlete is achieved, meanwhile, detailed technical parameters of the athlete in the ball hitting action can be obtained by deeply analyzing the recognized actions, accurate guidance is provided for training of the athlete, and the training efficiency is improved. And the technical level and the competitive ability of athletes can be improved.

The invention relates to the field of computer vision technology and the like, and discloses a video action recognition method and system based on space-time interaction Transform and object interactivity prediction.The method constructs the video action recognition system by integrating the space-time interaction Transform and the object interactivity prediction.The action in a video can be effectively recognized and analyzed, and the video action recognition efficiency is improved. The method has a good effect especially in a multi-user interaction scene, and comprises the following specific steps: preprocessing an original video, and obtaining necessary video data comprising a video frame sequence and multi-modal features; encoding the preprocessed video data into a discrete token sequence; the token sequence is input into Transform, M-layer self-attention calculation is carried out, and spatial-temporal features are extracted; an object interactivity prediction module is used for judging and modeling the interaction relationship between objects in the video data; and carrying out classification by utilizing the fused multi-modal features, and outputting action categories in the video.

The invention belongs to the technical field of action recognition, and discloses a small sample human body action recognition method based on a cross attention mechanism. An end-to-end small sample action recognition model is provided, the end-to-end small sample action recognition model is based on a prototype network, on a feature extraction-prototype construction-distance measurement framework of the model, a spatial-temporal feature extraction network is adopted for feature extraction, a prototype-like construction module is adopted for prototype construction, and an end-to-end small sample action recognition model is obtained. And adopting a classification and prediction module for the measurement distance. According to the method, the skeleton position image is introduced to effectively reduce the interference of irrelevant factors such as illumination and view angle change, and the robustness of the model to human body action recognition is enhanced; through the adaptive spatio-temporal feature fusion method, the model can accurately capture fine-grained features, so that the accuracy of action recognition is improved. The design of the multi-branch feature classification loss function optimizes the feature extraction and coding capability, so that the model has better performance in a small sample learning scene.

The invention relates to the technical field of shot action recognition, and discloses a shot throwing action recognition method based on a single attitude sensor. The method comprises the following steps: acquiring a single attitude sensor original data stream in a shot throwing process, and extracting a three-axis acceleration sequence and a three-axis angular velocity sequence from the single attitude sensor original data stream; performing motion event segmentation processing on the two sequences, and generating a key action time period mark set comprising a preparation stage, a sliding step stage, a power generation stage and a release stage; based on the mark set, calling an attitude feature solution algorithm to perform spatial trajectory reconstruction on the original data stream to obtain a key attitude frame set containing a joint angle change curve and a centroid displacement trajectory; and inputting the key attitude frame set into an action recognition model for attitude mode matching, and outputting a shot throwing action classification result which comprises an action deviation parameter generated based on a standard action template library and a key frame correction identifier. According to the method, key attitude information is reconstructed, and support is provided for shot throwing action analysis and improvement.

The invention discloses an intelligent digital human training method and system based on multi-modal interaction, and relates to the technical field of digital human training, and the method comprises the steps: designing a touch feedback strategy, and defining the output corresponding to a digital human according to different touch information; converting the voice input into a text; generating a natural language reply; converting into digital human voice output by using a voice synthesis technology; capturing dynamic characteristics of the motion and converting the dynamic characteristics into actions of a digital human; identifying and extracting facial feature points, and constructing a facial model of the digital human; analyzing relative positions and motions of the facial feature points, and mapping the relative positions and motions to a facial model of the digital human; aligning the time sequence of the audio with the lip movement track of the digital human; constructing a basic character model; analyzing intention and emotion input by the user in real time; a comprehensive corresponding strategy is generated, and actions, facial expressions, voices and appearance feedback which should be taken by the digital person in the current situation are determined.

The invention relates to the technical field of spine rehabilitation training, and discloses a computer vision recognition and guidance system for spine rehabilitation training actions. The system comprises a data acquisition module for acquiring user spine motion three-dimensional dynamic image data by using a multi-camera array; the action recognition module tracks bone joint points based on an improved algorithm to generate a real-time action sequence; the posture evaluation module is used for extracting spine curvature and joint angle features through a multi-scale convolutional neural network; the guidance instruction generation module inputs the features into a reinforcement learning model to generate a personalized instruction; and the feedback execution module constructs a fuzzy self-adaptive control model to correct the motion trail of the spine. According to the system, training actions can be accurately recognized, personalized guidance is provided, real-time correction is achieved, the spine rehabilitation training effect is effectively improved, subjectivity and errors of manual evaluation are reduced, and rehabilitation requirements of different patients are met.

The invention discloses a control method of a vehicle-mounted mosquito repelling device, the vehicle-mounted mosquito repelling device and a storage medium, relates to the field of vehicle-mounted technologies, and discloses the control method of the vehicle-mounted mosquito repelling device, the control method is applied to the vehicle-mounted mosquito repelling device, and the vehicle-mounted mosquito repelling device comprises an image acquisition device and a sound wave mosquito repelling device. The method comprises the following steps: determining a mosquito identification result based on an image in a cabin acquired by an image acquisition device; determining an action recognition result according to the image in the cabin; determining a mosquito repelling mode according to the mosquito recognition result and the action recognition result; and controlling the sound wave mosquito repelling device to operate according to the operation parameters corresponding to the mosquito repelling mode. On the basis, the mosquito information can be automatically detected, the mosquito repelling device is automatically started after the mosquito information is detected, and the mosquito repelling effect is improved.

The invention belongs to the technical field of computer application, and relates to a limb rehabilitation action recognition and evaluation method based on deep learning, and the method comprises the steps: collecting video data and skeleton data of limb function rehabilitation actions, carrying out the preprocessing, and constructing a training set and a verification set of the video data and the skeleton data; a hybrid model of SE-ResBlock3D and LSTM is constructed, and recognition and classification of limb function rehabilitation actions are achieved on input video data through an execution convolution layer, four residual block sequences and an LSTM module in sequence by the hybrid model; and a deep learning network SGP-Net is constructed, and the deep learning network SGP-Net extracts and evaluates space and time sequence characteristics of the limb function rehabilitation actions by executing an STGCN module, a GAT module and a PMTC module on the input preprocessed skeleton data in sequence, so that quality evaluation of the limb function rehabilitation actions is realized.

The invention relates to the technical field of data processing, in particular to an integrated multi-mode culture resource intelligent data governance and management system. The system comprises a data acquisition module, an intelligent processing module, a data management module, a knowledge organization module and an interactive display module. Synchronous acquisition of videos, audios, images and three-dimensional point clouds is realized through camera equipment, audio acquisition equipment and a laser scanner, and cultural elements are extracted through action recognition, speech recognition model processing dialect transcription, image semantic segmentation and named entity recognition by adopting a convolutional neural network. The system realizes structured description and semantic mapping of data on the basis of non-abandoned domain ontology, and constructs a knowledge graph by means of a graph database and a graph neural network. And finally, realizing three-dimensional visual display of the cultural resources by combining a virtual reality technology. According to the method, the problems of non-abandoned multi-modal data processing splitting, insufficient semantic organization and weak display interactivity in the prior art are solved, and the digital governance level and the propagation capability of cultural resources are improved.

The invention discloses a sign language recognition method and device based on multi-modal deep learning, and the method comprises the steps: multi-modal data input: capturing hand motions, gesture tracks and facial expressions at the same time through a camera, a motion capture sensor and other devices, and forming multi-modal data input; sign language action recognition: precise recognition of sign language actions is realized through a combined model of a deep convolutional neural network and a long-short-term memory network; facial expression and gesture track combined recognition: realizing understanding and translation of complex sign language sentences by combining the captured facial expressions and gesture tracks; and context natural language processing: generating a target statement in combination with context semantic understanding, and outputting a translation result. Through hand motion capture, facial expression analysis, gesture trajectory tracking and context natural language processing, complex sign language motions can be recognized more accurately and translated into characters or voices in real time, and low delay and high accuracy are achieved.

A method includes obtaining video datasets each including pairs of a training video and a ground-truth action classification of the training video. The method also includes generating an action recognition model that includes a shared encoder model and action classification heads. A number of the action classifications heads may be equal to a number of the video datasets, and each action classification head may be configured to, based on an output of the shared encoder model, classify training videos sampled from a corresponding video dataset. The method also includes determining, by the action recognition model and for each training video sampled from the video datasets, an inferred action classification. The method further includes determining a loss value based on the inferred action classifications and the ground-truth action classifications, and adjusting parameters of the action recognition model based on the loss value.

The invention discloses a method, a device and equipment for identifying illegal actions in a video and a storage medium, and relates to the technical field of computers. The method comprises the following steps: carrying out joint point detection on a target video stream to determine a joint point coordinate of each video frame, and generating a joint point coordinate sequence based on the joint point coordinates; inputting the articulation point coordinate sequence and the target video stream into a double-flow feature modeling network, and extracting articulation point spatial-temporal features of the articulation point coordinate sequence and image semantic features of the target video stream; and performing cross attention processing on the joint point spatial-temporal features and the image semantic features, and performing illegal action recognition according to the obtained fused features. It can be seen that the double-flow feature modeling network is utilized to extract the joint point spatio-temporal features and the image semantic features at the same time, the fused features rich in the context relation are generated through cross attention processing, illegal action recognition is conducted according to the fused features, and the accuracy of illegal action recognition in the video is improved through the multi-modal features.

The invention discloses an aircraft basic action recognition method based on a selectable feature fusion model, and the method comprises the steps: inputting original time sequence data into an original time sequence feature construction module, obtaining features related to aircraft actions, amplifying the features into the original time sequence data, obtaining new time sequence data, and carrying out the recognition of the aircraft basic actions according to the new time sequence data. Input features of the selectable feature fusion model and corresponding aircraft basic action category labels are obtained; constructing a selective feature fusion model; training the selectable feature fusion model through the total loss function to obtain a trained selectable feature fusion model and parameters thereof; the method comprises the following steps: inputting original data into an original time sequence feature construction module, converting the original data into an input feature format required by a selectable feature fusion model, loading trained selectable feature fusion model parameters, and carrying out end-to-end reasoning to obtain an identification result of basic actions of an aircraft; according to the invention, the performance of aircraft basic action recognition is improved.

The invention relates to the field of robot technology, machine vision, deep learning, computer graphics and digital twinning, in particular to a mechanical arm action recognition and control method based on the machine vision and the digital twinning technology. The motion state and environment information of the mechanical arm are captured in real time through the machine vision technology, high-precision recognition of the motion of the mechanical arm can be achieved in combination with an image processing algorithm, and a control strategy is adjusted in real time according to the recognition result; based on the digital twin model, the state change of the mechanical arm can be monitored in real time, and the mechanical arm and the virtual model are synchronously driven, so that the action of the mechanical arm is ensured to be consistent with the expected action; through intelligent identification and automatic adjustment, manual operation errors are reduced, the working efficiency and the production precision are improved, and the complexity of system maintenance is reduced; real-time optimization of the operation process of the mechanical arm can be achieved through combination of digital twinning and machine vision, potential faults are avoided through simulation and prediction, and stable operation of a production line is ensured.