Virtual maintenance training assessment management method

Abstract

The invention discloses a virtual maintenance training assessment management method, and the method comprises the steps: collecting a virtual maintenance monitoring video of an assessed person through a camera, and introducing a data processing and image analysis algorithm based on artificial intelligence and machine vision at a rear end to carry out the analysis of the virtual maintenance monitoring video. According to the method, the virtual maintenance action semantics and the time sequence relevance relation of the examined person are captured, so that the non-standard operation or wrong action of the maintenance action is recognized, and when the abnormal operation behavior or wrong action is detected, prompt information is sent out, and an examination report is generated. Therefore, more intelligent virtual maintenance training assessment management can be realized, and the maintenance action time sequence semantics of an assessed person can be understood by performing global analysis based on the time sequence context on the action semantics of the assessed person so as to perform exception identification and error detection of maintenance training behaviors; the application range and the reliability of the virtual maintenance training assessment scheme can be improved, and a more objective and comprehensive assessment result can be provided.

Description

Technical Field

[0001] The present application relates to the field of intelligent management, and more specifically, to a virtual maintenance training assessment management method. Background Art

[0002] Vehicle maintenance training assessment is an important part of evaluating maintenance personnel's operating skills and maintenance capabilities. Traditional maintenance training assessment mainly uses actual vehicle or equipment operation drills to assess maintenance personnel's ability to handle faults, repairs, and use tools. However, this traditional method has some problems, such as the assessment process is easily affected by human subjective factors, high cost, poor flexibility and safety, etc.

[0003] Virtual technology simulates the real-world maintenance environment and allows maintenance personnel to train without physical equipment, which not only reduces training costs, but also avoids risks that may occur in actual operations and improves the flexibility of maintenance training assessment. However, in traditional virtual maintenance training assessment management solutions, vehicle maintenance training assessment is usually evaluated based on standardized models and single rules. Although standardized assessment methods are convenient for evaluation and comparison, they may limit the creativity and flexibility of maintenance personnel. In other words, in actual maintenance, the same fault may have multiple causes and troubleshooting methods. Traditional virtual training assessments often find it difficult to simulate this diversity, which may lead to overly single training content and operation methods. Summary of the invention

[0004] In order to solve the above technical problems, the present application is proposed. The embodiment of the present application provides a virtual maintenance training assessment management method, which collects virtual maintenance monitoring videos of the assessed personnel through a camera, and introduces data processing and image analysis algorithms based on artificial intelligence and machine vision at the back end to analyze the virtual maintenance monitoring videos, so as to capture the semantics and temporal correlation relationship of the virtual maintenance actions of the assessed personnel, thereby identifying irregular operations or erroneous actions of maintenance actions, etc., and when abnormal operation behaviors or erroneous actions are detected, prompt information is issued and an assessment report is generated. In this way, more intelligent virtual maintenance training assessment management can be achieved, and the timing semantics of the maintenance actions of the assessor can be understood by performing a global analysis of the action semantics of the assessor based on the temporal context, so as to perform abnormal identification and error detection of maintenance training behaviors, which helps to improve the scope of application and reliability of the virtual maintenance training assessment scheme and provide more objective and comprehensive assessment results.

[0005] According to one aspect of the present application, a virtual maintenance training assessment management method is provided, which includes: Obtain virtual maintenance monitoring video of the person being assessed; Performing key frame sampling on the virtual maintenance monitoring video to obtain a time sequence of virtual maintenance monitoring key frames; Performing multi-scale feature extraction and fusion processing based on virtual maintenance action semantics on each virtual maintenance monitoring key frame in the time series of the virtual maintenance monitoring key frames to obtain a time series of virtual maintenance action semantic feature maps; Performing channel pruning optimization processing on each virtual maintenance action semantic feature graph in the time series of the virtual maintenance action semantic feature graph to obtain a time series of optimized virtual maintenance action semantic feature graphs; The optimized virtual maintenance action semantic feature graphs in the time series of the optimized virtual maintenance action semantic feature graphs are subjected to feature aggregation along the channel dimension and then input into the global feature perception fusion module to obtain the virtual maintenance action semantic full-time domain correlation feature graph; Based on the virtual maintenance action semantic full-time domain correlation feature map, it is determined whether there are errors in the maintenance training operation actions of the assessed personnel, whether to issue a prompt message, and generate an assessment report.

[0006] Compared with the prior art, the virtual maintenance training assessment management method provided by the present application collects virtual maintenance monitoring videos of the assessed personnel through a camera, and introduces data processing and image analysis algorithms based on artificial intelligence and machine vision at the back end to analyze the virtual maintenance monitoring videos, so as to capture the semantics and temporal correlation of the virtual maintenance actions of the assessed personnel, thereby identifying irregular operations or erroneous actions of maintenance actions, etc., and when abnormal operation behaviors or erroneous actions are detected, prompt information is issued and an assessment report is generated. In this way, more intelligent virtual maintenance training assessment management can be achieved, and the timing semantics of the maintenance actions of the assessor can be understood by performing a global analysis of the action semantics of the assessor based on the temporal context, so as to perform abnormal identification and error detection of maintenance training behaviors, which helps to improve the scope of application and reliability of the virtual maintenance training assessment scheme and provide more objective and comprehensive assessment results. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0008] Figure 1 is a flowchart of a virtual maintenance training assessment management method according to an embodiment of the present application; Figure 2It is a data flow diagram of the virtual maintenance training assessment management method according to an embodiment of the present application. DETAILED DESCRIPTION

[0009] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0010] As shown in this application and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0011] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, any number of different modules can be used and run on the user terminal and / or server. The modules are only illustrative, and different aspects of the system and method can use different modules.

[0012] Flowcharts are used in the present application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, various steps may be processed in reverse order or simultaneously as required. Meanwhile, other operations may also be added to these processes, or a certain step or several steps of operations may be removed from these processes.

[0013] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0014] In traditional virtual maintenance training assessment management solutions, vehicle maintenance training assessment is usually evaluated by relying on standardized models and single rules. Although standardized assessment methods are convenient for evaluation and comparison, they may limit the creativity and flexibility of maintenance personnel. In other words, in actual maintenance, the same fault may have multiple causes and troubleshooting methods. Traditional virtual training assessments often find it difficult to simulate this diversity, which may lead to overly simple training content and operation methods. Therefore, in the actual vehicle maintenance training assessment, it is very important to flexibly use multiple methods to troubleshoot faults, which puts higher requirements and intelligence on the virtual maintenance training assessment process. Therefore, an optimized virtual maintenance training assessment management solution is desired.

[0015] In the technical solution of the present application, a virtual maintenance training assessment management method is proposed. Figure 1 Flow chart of a virtual maintenance training assessment management method according to an embodiment of the present application. Figure 2 : is a data flow diagram of the virtual maintenance training assessment management method according to an embodiment of the present application. Figure 1 and Figure 2 As shown, the virtual maintenance training assessment management method according to the embodiment of the present application includes the steps of: S1, obtaining the virtual maintenance monitoring video of the person being assessed; S2, sampling the key frames of the virtual maintenance monitoring video to obtain a time series of virtual maintenance monitoring key frames; S3, performing multi-scale feature extraction and fusion processing based on virtual maintenance action semantics on each virtual maintenance monitoring key frame in the time series of virtual maintenance monitoring key frames to obtain a time series of virtual maintenance action semantic feature maps; S4, performing channel pruning optimization processing on each virtual maintenance action semantic feature graph in the time series of the virtual maintenance action semantic feature graph to obtain a time series of optimized virtual maintenance action semantic feature graphs; S5, perform feature aggregation along the channel dimension on each optimized virtual maintenance action semantic feature map in the time series of the optimized virtual maintenance action semantic feature map, and then obtain the virtual maintenance action semantic full-time domain correlation feature map through the global feature perception fusion module; S6, based on the virtual maintenance action semantic full-time domain correlation feature map, determine whether there are errors in the maintenance training operation actions of the assessed personnel, and determine whether to issue a prompt message and generate an assessment report.

[0016] In particular, in S1 and S2, a virtual maintenance monitoring video of the person being assessed is obtained; and key frames of the virtual maintenance monitoring video are sampled to obtain a time series of virtual maintenance monitoring key frames. It should be understood that in the actual process of conducting a virtual vehicle maintenance training assessment, it is necessary to monitor and evaluate the correctness of the operating maintenance steps of the assessor, the clarity of the fault cause analysis ideas, the standardization of the use of tools and equipment, the rationality of the use of fault troubleshooting methods, and the safety of equipment operation. The difficulty lies in the non-uniqueness of the troubleshooting of the fault point, and the troubleshooting methods used by different personnel are different, which cannot be standardized uniformly. In addition, repair personnel should be encouraged to flexibly use various methods to troubleshoot fault points and repair equipment. Therefore, it is necessary to effectively analyze the virtual maintenance monitoring video to capture the semantics of the virtual maintenance actions of the person being assessed and the temporal context correlation relationship. However, since the virtual maintenance monitoring video contains a large amount of redundant information and repeated semantics, these redundancies will affect the efficiency and computational complexity of the subsequent virtual maintenance semantic analysis of the person being assessed. Based on this, in the technical solution of the present application, it is necessary to sample the key frames of the virtual maintenance monitoring video to obtain a time series of virtual maintenance monitoring key frames.

[0017] In particular, in S3, multi-scale feature extraction and fusion processing based on virtual maintenance action semantics are performed on each virtual maintenance monitoring key frame in the time series of virtual maintenance monitoring key frames to obtain a time series of virtual maintenance action semantic feature maps. Considering that in the actual process of virtual maintenance time series semantic feature extraction and operation action error detection, since each virtual maintenance monitoring key frame contains semantic information about the virtual maintenance action of the personnel at a specific time point, these action semantics will be presented at different levels and scales of the image frame, that is, in the maintenance action, the features of different levels and scales in the key frame may correspond to different operation details, such as gestures, tool usage, etc. Therefore, it is necessary to extract action features at different levels and scales in the key frame. This is crucial for understanding complex actions and details. In the technical solution of the present application, each virtual maintenance monitoring key frame in the time series of virtual maintenance monitoring key frames is respectively passed through a virtual maintenance action semantic feature extractor based on a pyramid information transfer network to obtain a time series of virtual maintenance action semantic feature maps.

[0018] It is worth mentioning that in the pyramid information transfer network, the bottom network is mainly responsible for extracting low-level features and spatial detail information of the image, while the high-level network is responsible for extracting more abstract high-level features. However, in the traditional network structure, the position information extracted by the bottom network is difficult to directly affect the high-level network, resulting in distortion of the position information in the feature extraction at different levels. In addition, in the pyramid information transfer network, the transmission of features usually requires a large amount of computation, especially when processing large-scale image key frame data. With the increase of the number of network layers, the amount of computation increases exponentially, which may cause the training and reasoning process to become very time-consuming. Based on this, in order to be able to cope with the problem that the shallow position information in the pyramid information transfer network is difficult to affect the deep features and will cause a large amount of computation required when transmitting feature information, in the technical solution of the present application, each virtual maintenance monitoring key frame in the time series of virtual maintenance monitoring key frames is further passed through a virtual maintenance action semantic feature extractor based on the pyramid information transfer network to obtain a time series of virtual maintenance action semantic feature maps. The virtual maintenance action semantic feature extractor can extract semantic information related to maintenance actions from the video. These features include not only the form of the action, but also the intention of the action, which is crucial to understanding the accuracy and rationality of the maintenance action. In particular, in the technical solution of the present application, the virtual maintenance action semantic feature extractor based on the pyramid information transfer network includes a feature extraction network and a feature fusion network. In the feature fusion network, information transfer is achieved through residual fusion and jump connection to effectively fuse the shallow features and deep features in the virtual maintenance monitoring key frame, which can effectively transfer the position information extracted from the bottom layer to the upper layer, so that the deep features can better fuse the bottom layer position information in the key frame, thereby improving the expressiveness and accuracy of the features. In addition, the residual feature extraction and information transfer of virtual maintenance action features at different levels are performed through the pyramid structure to reduce the amount of calculation in the feature transfer process. By extracting the virtual maintenance semantic features of maintenance personnel at different scales and aggregating information, efficient transmission and utilization of feature information is achieved, while reducing unnecessary calculation overhead and improving the calculation efficiency and speed of the network.

[0019] In an embodiment of the present application, each virtual maintenance monitoring key frame in the time series of virtual maintenance monitoring key frames is respectively passed through a virtual maintenance action semantic feature extractor based on a pyramid information transfer network to obtain a time series of virtual maintenance action semantic feature maps, including: passing the virtual maintenance monitoring key frames through a convolution layer with a convolution kernel of 3×3 to obtain a shallow feature map of the virtual maintenance monitoring key frames; passing the shallow feature map of the virtual maintenance monitoring key frames through a virtual maintenance action shallow residual information fusion enhancement module to obtain a shallow residual fusion feature map of the virtual maintenance monitoring key frames; passing the shallow residual fusion feature map of the virtual maintenance monitoring key frames through a convolution layer with a convolution kernel of 3×3 to obtain a deep feature map of the virtual maintenance monitoring key frames; passing the deep feature map of the virtual maintenance monitoring key frames through a convolution layer with a convolution kernel of 3×3 to obtain a deep feature map of the virtual maintenance monitoring key frames. The deep residual information fusion enhancement module of the virtual maintenance action is used to obtain the deep residual fusion feature map of the virtual maintenance monitoring key frame; the deep residual fusion feature map of the virtual maintenance monitoring key frame is input into the scale modulation module based on the SPPF layer to obtain the deep modulation feature map of the virtual maintenance monitoring key frame; the deep modulation feature map of the virtual maintenance monitoring key frame is upsampled to obtain the upsampled deep modulation feature map of the virtual maintenance monitoring key frame; the upsampled deep modulation feature map of the virtual maintenance monitoring key frame is fused with the shallow residual fusion feature map of the virtual maintenance monitoring key frame to obtain the multi-scale feature map of the virtual maintenance monitoring key frame; the multi-scale feature map of the virtual maintenance monitoring key frame is passed through the multi-scale residual information fusion enhancement module of the virtual maintenance action to obtain the semantic feature map of the virtual maintenance action.

[0020] Among them, the shallow feature map of the virtual maintenance monitoring key frame is passed through the virtual maintenance action shallow residual information fusion enhancement module to obtain the shallow residual fusion feature map of the virtual maintenance monitoring key frame, including: the shallow feature map of the virtual maintenance monitoring key frame is modulated by channel dimension reduction through a point convolution layer to obtain the shallow feature map of the virtual maintenance monitoring key frame modulated by channel dimension reduction; the shallow feature map of the virtual maintenance monitoring key frame modulated by channel dimension reduction is extracted through a convolution layer with a convolution kernel of 3×3 to obtain the shallow implicit information feature map of the virtual maintenance monitoring key frame; the shallow implicit information feature map of the virtual maintenance monitoring key frame is modulated by channel dimension increase through a point convolution layer to obtain the shallow feature map of the virtual maintenance monitoring key frame after channel modulation; the shallow feature map of the virtual maintenance monitoring key frame after channel modulation and the shallow feature map of the virtual maintenance monitoring key frame modulated by channel dimension reduction are fused to obtain the shallow residual fusion feature map of the virtual maintenance monitoring key frame.

[0021] More specifically, the deep residual fusion feature map of the virtual maintenance monitoring key frame is input into the scale modulation module based on the SPPF layer to obtain the deep modulation feature map of the virtual maintenance monitoring key frame, including: passing the deep residual fusion feature map of the virtual maintenance monitoring key frame through the scale modulation module based on the point convolution layer to obtain the scale-adjusted deep residual fusion feature map of the virtual maintenance monitoring key frame; performing maximum pooling processing on each feature matrix along the channel dimension in the scale-adjusted deep residual fusion feature map of the virtual maintenance monitoring key frame to obtain a deep residual fusion feature vector of the virtual maintenance monitoring key frame; and performing channel weighted enhancement on the scale-adjusted deep residual fusion feature map of the virtual maintenance monitoring key frame based on the deep residual fusion feature vector of the virtual maintenance monitoring key frame to obtain a deep modulation feature map of the virtual maintenance monitoring key frame.

[0022] In particular, in S4, each virtual maintenance action semantic feature map in the time series of the virtual maintenance action semantic feature map is subjected to channel pruning optimization processing to obtain the time series of optimized virtual maintenance action semantic feature maps. Considering that each virtual maintenance action semantic feature map contains the virtual maintenance action semantic multi-level fusion features of the assessed personnel at a specific time point, some of these features are of great significance for the subsequent abnormal operation action and error detection tasks, while some features are some background interference or redundant information, and the contribution to the subsequent detection tasks is relatively low. Based on this, in the technical solution of the present application, each virtual maintenance action semantic feature map in the time series of the virtual maintenance action semantic feature map is further subjected to channel pruning optimization processing to obtain the time series of optimized virtual maintenance action semantic feature maps. Through channel pruning, those channels that contribute less or are redundant to the model performance and the subsequent maintenance training operation action error detection tasks can be removed, thereby reducing the amount of calculation and speeding up the processing speed. This is particularly important for real-time or near real-time virtual maintenance training assessments. In addition, the virtual maintenance action semantic feature map after pruning optimization is more concise, which helps to avoid overfitting and improves the generalization ability of the model on unknown data. This means that the model can more stably identify and evaluate new maintenance scenarios or different maintenance actions.

[0023] In an embodiment of the present application, each virtual maintenance action semantic feature graph in the time series of the virtual maintenance action semantic feature graph is subjected to channel pruning optimization processing to obtain the time series of the optimized virtual maintenance action semantic feature graph, including: performing global mean pooling processing on each feature matrix along the channel dimension in the virtual maintenance action semantic feature graph to obtain a virtual maintenance action semantic channel feature vector; first multiplying the virtual maintenance action semantic channel feature vector with a weight matrix and then adding it to a bias vector by position to obtain a virtual maintenance action semantic channel weight vector; performing masking processing on the virtual maintenance action semantic channel weight vector to obtain a masked virtual maintenance action semantic feature. The channel weight vector of the masked virtual maintenance action semantic feature map is obtained by performing a masking process on the virtual maintenance action semantic channel weight vector to obtain the channel weight vector of the masked virtual maintenance action semantic feature map; taking each position feature value in the channel weight vector of the masked virtual maintenance action semantic feature map as a weighting coefficient, and performing weighted optimization on each feature matrix along the channel dimension in the virtual maintenance action semantic feature map to obtain an optimized virtual maintenance action semantic feature map; wherein, the virtual maintenance action semantic channel weight vector is masked to obtain the channel weight vector of the masked virtual maintenance action semantic feature map, comprising: in response to each position feature value in the virtual maintenance action semantic channel weight vector being greater than or equal to a predetermined threshold, taking the original value of the position feature value, otherwise setting it to zero to obtain the channel weight vector of the masked virtual maintenance action semantic feature map.

[0024] In summary, in the above embodiment, each virtual maintenance action semantic feature graph in the time series of the virtual maintenance action semantic feature graph is subjected to channel pruning optimization processing to obtain the time series of optimized virtual maintenance action semantic feature graphs, including: performing channel pruning optimization processing on each virtual maintenance action semantic feature graph in the time series of the virtual maintenance action semantic feature graph using the following channel pruning optimization formula to obtain the time series of optimized virtual maintenance action semantic feature graphs; wherein the formula is:

[0025]

[0026]

[0027]

[0028] in, is the semantic feature graph of each virtual maintenance action, Indicates that global mean pooling is performed on each feature matrix along the channel dimension in the feature map. is the semantic channel feature vector of the virtual maintenance action, is the weight matrix, is the bias vector, is the channel weight vector of the virtual maintenance action semantic feature map, For masking operation, is the masked virtual maintenance action semantic channel weight vector, In order to use the feature values ​​at each position in the vector to perform positional point multiplication on the feature map along the channel dimension, It is the semantic feature map of the optimized virtual maintenance action.

[0029] In particular, in S5, each optimized virtual maintenance action semantic feature map in the time series of the optimized virtual maintenance action semantic feature map is subjected to feature aggregation along the channel dimension and then passed through a global feature perception fusion module to obtain a virtual maintenance action semantic full-time domain correlation feature map. It should be understood that since each optimized virtual maintenance action semantic feature map contains virtual maintenance action semantic features optimized by significant features at a specific time point, and these action semantics have temporal correlation relationships and mutual influences, therefore, in order to be able to more comprehensively and accurately understand the temporal semantics and intentions of the virtual maintenance actions of the assessed personnel, and thus utilize the temporal contextual correlation relationship and mutual influence of these action semantics to perform subsequent maintenance training operation action error detection, in the technical solution of the present application, each optimized virtual maintenance action semantic feature map in the time series of the optimized virtual maintenance action semantic feature map is further subjected to feature aggregation along the channel dimension and then passed through a global feature perception fusion module to obtain a virtual maintenance action semantic full-time domain correlation feature map. By aggregating the optimized virtual maintenance action semantic feature graphs at each time point along the channel dimension, these maintenance action semantics and features can be integrated to enhance the temporal relevance and expressiveness of the features. This helps capture the overall trend and pattern of maintenance actions, rather than just local details, which is very useful for evaluating the fluency and logic of maintenance actions.

[0030] In particular, after performing feature aggregation along the channel dimension for each optimized virtual maintenance action semantic feature graph to form a full-time optimized virtual maintenance action semantic aggregation feature graph, the feature graph is further subjected to global feature perception and time-series correlation feature extraction, so as to capture the global time-series correlation semantic feature information of the virtual maintenance action of the person being assessed, and improve the model's ability to understand the full-time context semantics of its action. It is worth mentioning that through the processing of the global feature perception fusion module, the importance of the virtual maintenance action semantics at different time points can be automatically learned and captured, that is, the contribution to the subsequent maintenance training operation action detection and error recognition tasks, and the virtual maintenance action semantics of these different time sequences are adaptively weighted and enhanced to optimize the overall virtual maintenance action semantics full-time domain correlation feature expression, providing a more accurate basis for the subsequent maintenance training operation action error recognition.

[0031] In an embodiment of the present application, each optimized virtual maintenance action semantic feature map in the time series of the optimized virtual maintenance action semantic feature map is subjected to feature aggregation along the channel dimension and then passed through a global feature perception fusion module to obtain a virtual maintenance action semantic full-time domain associated feature map, including: each optimized virtual maintenance action semantic feature map in the time series of the optimized virtual maintenance action semantic feature map is subjected to feature aggregation along the channel dimension to obtain an optimized virtual maintenance action semantic aggregation feature map; in the global feature perception fusion module, the optimized virtual maintenance action semantic aggregation feature map is subjected to linear transformation to obtain a virtual maintenance action semantic linear transformation matrix; in the global feature perception fusion module, the virtual maintenance action semantic linear transformation matrix is ​​passed through The soft maximum normalization process is performed in the function to obtain the virtual maintenance action semantic linear transformation weight matrix; in the global feature perception fusion module, each feature matrix along the channel dimension in the optimized virtual maintenance action semantic aggregation feature map is multiplied by the virtual maintenance action semantic linear transformation weight matrix by position to obtain the virtual maintenance action semantic global channel feature map; in the global feature perception fusion module, the virtual maintenance action semantic global channel feature map is multiplied by After activation processing in the activation function, it is multiplied by the trainable scaling factor and added to the optimized virtual maintenance action semantic aggregation feature map by position to obtain the virtual maintenance action semantic full-time domain correlation feature map.

[0032] Among them, in the global feature perception fusion module, the optimized virtual maintenance action semantic aggregation feature map is linearly transformed to obtain the virtual maintenance action semantic linear transformation matrix, including: processing the optimized virtual maintenance action semantic aggregation feature map through a convolutional layer to obtain the virtual maintenance action semantic feature matrix; after multiplying the virtual maintenance action semantic feature matrix with the global perception weight matrix, the obtained weight modulated virtual maintenance action semantic feature matrix and the global perception bias matrix are summed by position to obtain the virtual maintenance action semantic linear transformation matrix.

[0033] In summary, in the above embodiment, each optimized virtual maintenance action semantic feature map in the time series of the optimized virtual maintenance action semantic feature map is subjected to feature aggregation along the channel dimension and then passed through a global feature perception fusion module to obtain a virtual maintenance action semantic full-time domain associated feature map, including: the optimized virtual maintenance action semantic aggregation feature map is processed by a global feature perception fusion module with the following global feature perception fusion formula to obtain a virtual maintenance action semantic full-time domain associated feature map; wherein the global feature perception fusion formula is:

[0034]

[0035] in, To optimize the semantic aggregation feature map of virtual maintenance actions, To perform convolution operation on the feature map, is the global perception weight matrix, is the global perception bias matrix, express function, To multiply by position, is the semantic global channel feature map of virtual maintenance action, express Activation function, is the trainable scaling factor, To add by position, It is the semantic full-time domain correlation feature map of virtual maintenance actions.

[0036] In particular, in S6, based on the virtual maintenance action semantic full-time domain correlation feature map, it is determined whether there is an error in the maintenance training operation action of the examinee, and it is determined whether to issue a prompt message and generate an assessment report. In a specific example of the present application, the virtual maintenance action semantic full-time domain correlation feature map is passed through a maintenance training operation action error identifier based on a classifier to obtain a recognition result, and the recognition result is used to indicate whether there is an error in the maintenance training operation action of the examinee. That is, the full-time domain correlation feature of the virtual maintenance action semantics is used for classification processing to identify irregular operations or erroneous operations of the maintenance training operation action. In response to the recognition result that there is an error in the maintenance training operation action of the examinee, a prompt message is issued and an assessment report is generated. In this way, a more intelligent virtual maintenance training assessment management can be achieved, and by performing a global analysis of the action semantics of the examinee based on the temporal context, the maintenance action temporal semantics of the examinee can be understood, so as to perform abnormal identification and error detection of the maintenance training behavior, avoid the limitation of the single regularization and standardized mode on the maintenance flexibility and diversity of the examinee, help to improve the scope of application and reliability of the virtual maintenance training assessment scheme, and provide a more objective and comprehensive assessment result.

[0037] Considering that the time series of virtual maintenance action semantic feature maps expresses the temporal distribution of image semantic features of virtual maintenance monitoring key frames, when performing global feature perception fusion after channel pruning optimization and feature aggregation, the image semantic differences of each virtual maintenance action semantic feature map in the time series of virtual maintenance action semantic feature maps will cause an imbalance in attention weight distribution when performing global feature perception fusion. Therefore, it is expected to improve the detail image semantic aggregation expression effect based on the fusion significant difference of the virtual maintenance action semantic full-time domain correlation feature map.

[0038] Preferably, the virtual maintenance action semantic full-time domain correlation feature graph is passed through a maintenance training operation action error identifier based on a classifier to obtain a recognition result, including: Calculate the sum of the absolute values ​​of each eigenvalue of the virtual maintenance action semantic full-time domain association feature map to obtain a first virtual maintenance action semantic full-time domain association and modulation value, and calculate the square root of the sum of the squares of each eigenvalue of the virtual maintenance action semantic full-time domain association feature map to obtain a second virtual maintenance action semantic full-time domain association and modulation value; After point-wise subtraction of the virtual maintenance action semantic full-time domain association feature map and the second virtual maintenance action semantic full-time domain association and modulation value, point-wise multiplication is performed with the number of eigenvalues ​​of the virtual maintenance action semantic full-time domain association feature map and the reciprocal of the first virtual maintenance action semantic full-time domain association and modulation value, and the reciprocal of each eigenvalue is taken to obtain the first virtual maintenance action semantic full-time domain association phase conversion feature map; After point-wise subtraction of the virtual maintenance action semantic full-time domain association feature map from the first virtual maintenance action semantic full-time domain association and modulation value, point-wise multiplication is performed with the square root of the number of eigenvalues ​​of the virtual maintenance action semantic full-time domain association feature map and the inverse of the second virtual maintenance action semantic full-time domain association and modulation value, and the inverse of each eigenvalue is taken to obtain the second virtual maintenance action semantic full-time domain association phase conversion feature map; The first virtual maintenance action semantics full time domain associated phase conversion feature map is point-subtracted from the second virtual maintenance action semantics full time domain associated phase conversion feature map and the dot product feature map of the weighted hyperparameter to obtain an optimized virtual maintenance action semantics full time domain associated feature map; The optimized virtual maintenance action semantic full-time domain correlation feature map is input into the classifier-based maintenance training operation action error identifier to obtain the recognition result.

[0039] Here, the virtual maintenance action semantic full-time domain correlation feature map The optimization expression is:

[0040]

[0041]

[0042]

[0043]

[0044]

[0045]

[0046] in, It is the semantic full-time domain correlation feature map of virtual maintenance action. represents the set of real numbers, , , They represent the width, length and height of the semantic full-time domain correlation feature map of virtual maintenance action, The first one represents the semantic full-time domain correlation feature graph of virtual maintenance action The eigenvalues ​​at the positions, The number of eigenvalues ​​representing the semantic full-time domain correlation feature graph of virtual maintenance action, Represents the semantic full time domain association and modulation value of the first virtual maintenance action, Represents the second virtual maintenance action semantics full time domain association and modulation value, It means point multiplication by position. is matrix multiplication, It means subtracting by position. Indicates the calculation of the inverse of each eigenvalue of the feature map, The first virtual maintenance action semantic full time domain associated phase transition feature map, The second virtual maintenance action semantic full time domain associated phase transition feature map, represents the weighted hyperparameter, Represents the optimized semantic full-time correlation feature map of virtual maintenance actions.

[0047] In a preferred example, the difference of the feature value of the virtual maintenance action semantic full-time domain association feature map relative to the overall feature set of the virtual maintenance action semantic full-time domain association feature map is used as the semantic change intensity information, and the phase-like conversion corresponding to the position-based intensity modulation is performed through different and modulation representation forms, so that the aggregation enhancement of the semantic change phase perception can improve the axial aggregation receptive field along the feature aggregation direction by performing a spatial translation operation based on alternating stacking under the set scale balance of the virtual maintenance action semantic full-time domain association feature map, thereby improving the perception effect of the aggregated semantics of the virtual maintenance action semantic full-time domain association feature map on the detail semantic change, so as to improve the expression effect of the virtual maintenance action semantic full-time domain association feature map, and improve the accuracy of the recognition result obtained by inputting it into the maintenance training operation action error recognizer based on the classifier. In this way, the irregular operation or wrong action of the virtual maintenance training operation action can be more accurately identified, and when abnormal operation behavior or wrong action is detected, prompt information is issued and an assessment report is generated, thereby realizing more intelligent virtual maintenance training assessment management.

[0048] In summary, the virtual maintenance training assessment management method according to the embodiment of the present application is explained, which collects the virtual maintenance monitoring video of the person being assessed through a camera, and introduces data processing and image analysis algorithms based on artificial intelligence and machine vision at the back end to analyze the virtual maintenance monitoring video, so as to capture the semantics and temporal correlation relationship of the virtual maintenance action of the person being assessed, thereby identifying the irregular operation or wrong action of the maintenance action, and when abnormal operation behavior or wrong action is detected, a prompt message is issued and an assessment report is generated. In this way, a more intelligent virtual maintenance training assessment management can be achieved, and the time sequence semantics of the maintenance action of the person being assessed can be understood by performing a global analysis of the action semantics of the assessor based on the temporal context, so as to perform abnormal identification and error detection of the maintenance training behavior, which helps to improve the scope of application and reliability of the virtual maintenance training assessment scheme and provide more objective and comprehensive assessment results.

[0049] In one example, for the assessment of operational subjects, it is sufficient to conduct conditional testing on a certain operation step. That is, if the condition is found to be incorrect through conditional testing, the operation is judged to be wrong, and no points will be given for the wrong step. However, incorrect operations that affect the safety and combat performance of the vehicle will be directly deemed as failing. At the same time, in terms of the scoring of the assessment time, 1 point will be added for every 5% of the standard time ahead of time, with a maximum of 5 points; 1 point will be deducted for every 1 minute of overtime. If the overtime is 5 minutes, the system will automatically require to stop the operation, and the assessment result will be directly deemed as failing. The assessment and evaluation strategy is as follows: (1) Search the operating steps of the assessor. If i=0, go to "(6)"; (2) Number each operation step in chronological order, denoted as ni; steps that can be performed simultaneously are denoted as mi; (3) Determine whether each specified step has been performed, whether the operation timing is correct, and whether there are related operations. If so, check whether the related operations are completed; (4) If the specified step has been performed, points will be deducted according to the repeated operation error; if the operation timing is incorrect (early or late), points will be deducted according to the incorrect operation timing; (5) If the associated operation of the operation is not completed, the operation cannot be continued and the score of the step will be deducted; if it is completed, the operation can be continued and the score will be calculated based on whether the operation is correct; if the associated operation is a sequential error, the score will be deducted according to the sequential error; if the associated operation is out of order, the score will be deducted according to the omission error; (6) The assessment results are calculated based on the comprehensive operation safety and operation time.

[0050] For repair subject training, it is organized and implemented in the following stages: material preparation, fault discovery, fault cause analysis, troubleshooting, repair, verification, and vehicle recovery. Since the focus of each stage is different, different scoring methods should be adopted.

[0051] 1. Scoring method for material preparation In the material preparation phase, the main assessment is whether the personnel have prepared the tools and equipment sufficiently, whether the tools and equipment are selected appropriately, and whether the methods of use are appropriate. Therefore, the scoring should be based on the importance of the tools and equipment.

[0052] 2. Scoring method for fault detection In the fault detection phase, the main assessment is whether the personnel are operating the equipment correctly and whether they can accurately interpret the fault phenomenon. Therefore, the operation assessment of a certain type of vehicle can be scored using the above method. If no fault phenomenon is found, it will be directly determined as a failure.

[0053] 3. Scoring method for fault cause analysis Fault cause analysis is the premise for the implementation of subsequent fault diagnosis and repair links, and directly determines the effect of troubleshooting. It mainly assesses whether the repair personnel have clear ideas in fault analysis and whether the order of troubleshooting is reasonable. Therefore, a higher score weight should be given. At the same time, the determination of fault points and the order of troubleshooting varies from person to person, and there is no order of priority. The order of fault points selected by the assessor should be scored. For example: the monitor shows the fault phenomenon of "XXX fault", and its possible fault points include cables, push button switches, and electronic boxes. Among them, the electronic box has the highest probability of failure, followed by the push button switch. The optimal troubleshooting order is electronic box, push button switch, and cable. Choosing this troubleshooting order should get the highest score. The specific scoring algorithm is as follows: Assume that there are n possible fault points of a certain fault phenomenon, denoted as F1, ..., Fn respectively. The probability of each fault occurrence is determined based on experience and denoted as p1, p2, ..., pn respectively. The sorting positions of the fault points are denoted as S1, ..., Sn from the front to the back. Starting from position S1, the probability difference Dij between the current position Si and the Sj fault point in the subsequent position is calculated in sequence.

[0054] Dij=1 pSi>=pSj Dij=0 pSi <pSj

[0055] Let Rmax represent the maximum value of R, and the full score of this part is Cmax. Then the final score of this ranking is C=R*Cmax / Rmax, and the final score is rounded up to the nearest 4.

[0056] 4. Scoring method for troubleshooting Troubleshooting mainly involves troubleshooting item by item based on the fault points identified by fault analysis, and assessing the repair personnel's ability to use tools, instruments, and testing equipment to locate faults. Since there is no time sequence requirement for troubleshooting of fault points, and repair personnel are also encouraged to quickly locate the fault points using the least amount of testing means, there is no order for troubleshooting of each fault point, and the scoring is mainly based on the method and steps of troubleshooting each fault point. However, when assigning points, the difficulty, complexity, and probability of troubleshooting each fault point should be considered, and different score weights should be given. Taking the fault of "XXX fault" displayed on the shooter display as an example, the possible fault points include cables, push buttons, and electronic boxes. In this fault subject assessment, the score for troubleshooting is 25, among which the probability of electronic box failure is the highest, followed by push buttons and cables. Therefore, the score for troubleshooting of the electronic box is 15 points, including 10 points for using the detection device to detect the electronic box, and 5 points for using the detector to detect the electronic box and determine the fault to the circuit board; the score for cable and push button switch troubleshooting is 5 points each. When troubleshooting each fault point, scoring is required according to the operating steps, using the same method as the operational assessment subjects.

[0057] 5. Scoring method for repair phase Repair mainly adopts the method of replacing parts to repair the fault points found through troubleshooting, and assesses the repair personnel's ability to use maintenance tools to disassemble components, replace spare parts, assemble and debug. Repair should be carried out in the order required by the regulations. The overall repair process has time sequence requirements, and some operations have no sequence requirements. Therefore, a conditional test is carried out on a certain operation step. If the condition is found to be incorrect through the conditional test, the operation is judged to be wrong, and no points will be given for the wrong steps. Take the "XXX fault" displayed on the shooter display as an example. The fault point is the AP02 board of the electronic box. The repair steps are: first, use a cross screwdriver to remove the top cover of the electronic box, then use a flat screwdriver to loosen the AP02 locking mechanism, unplug AP02, and replace the spare AP02 circuit board. This operation step should be strictly executed in order. When performing a certain step, check whether the previous step is completed. If it is completed, you will score. If it is not completed, you must complete the previous step before continuing the operation. At this time, the previous step will not be scored.

[0058] 6. Assessment method for team collaborative operation The assessment of team collaboration is mainly aimed at the repair subjects that require multiple members of the team to conduct troubleshooting and repairs. In addition to assessing individual operation and repair capabilities, it is also necessary to assess the degree of tacit cooperation between members of the team, which is mainly scored by time. In the repair subjects, for possible fault points, multiple people can simultaneously troubleshoot and repair multiple fault points, or multiple people can simultaneously troubleshoot and repair one fault point. This requires the team leader in the team to assign tasks and decide which collaboration method to adopt. The troubleshooting and repair of each fault point are independent of each other and have no time sequence. Therefore, the scoring method adopted is still based on the troubleshooting and repair method steps of each fault point, and points are assigned according to the operating steps. The software performs conditional detection on a certain operating step. If the condition detection finds that the state is incorrect, it is judged that the operation is wrong. No points will be given for wrong steps, and there is no need to consider the identity of the repair personnel. The assessment software only needs to solve the problem of multiple people operating the vehicle at the same time. The solution of the project is to use multi-threaded parallel operation for two unrelated fault points, and score them independently. For fault points with a correlation, multiple people cannot troubleshoot and repair the fault points at the same time. Taking the shooter display showing "XXX fault" as an example, the possible fault points include cables, push button switches, and electronic boxes. The cables are connected to the electronic box, and the two are related. The cables must be removed first before the electronic box and cables can be troubleshooted and repaired separately. If the repairman removes the ignition and selection box without removing the cables, the software will determine that this operation step is an operation timing error and points should be deducted.

[0059] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A virtual maintenance training assessment management method, characterized in that: include: Obtain virtual maintenance monitoring video of the person being assessed; Performing key frame sampling on the virtual maintenance monitoring video to obtain a time sequence of virtual maintenance monitoring key frames; Performing multi-scale feature extraction and fusion processing based on virtual maintenance action semantics on each virtual maintenance monitoring key frame in the time series of the virtual maintenance monitoring key frames, respectively, to obtain a time series of virtual maintenance action semantic feature maps; Performing channel pruning optimization processing on each virtual maintenance action semantic feature graph in the time series of the virtual maintenance action semantic feature graph to obtain a time series of optimized virtual maintenance action semantic feature graphs; The optimized virtual maintenance action semantic feature graphs in the time series of the optimized virtual maintenance action semantic feature graphs are subjected to feature aggregation along the channel dimension and then input into the global feature perception fusion module to obtain the virtual maintenance action semantic full-time domain correlation feature graph; Based on the virtual maintenance action semantic full-time domain correlation feature map, it is determined whether there are errors in the maintenance training operation actions of the assessed personnel, whether to issue a prompt message, and generate an assessment report.

2. The virtual maintenance training assessment management method according to claim 1 is characterized in that: The step of performing multi-scale feature extraction and fusion processing based on virtual maintenance action semantics on each virtual maintenance monitoring key frame in the time series of the virtual maintenance monitoring key frames to obtain a time series of virtual maintenance action semantic feature graphs includes: Each virtual maintenance monitoring key frame in the time series of the virtual maintenance monitoring key frames is respectively passed through a virtual maintenance action semantic feature extractor based on a pyramid information transfer network to obtain a time series of the virtual maintenance action semantic feature graph.

3. The virtual maintenance training assessment management method according to claim 2 is characterized in that: The step of inputting each virtual maintenance monitoring key frame in the time sequence of the virtual maintenance monitoring key frame into a virtual maintenance action semantic feature extractor based on a pyramid information transfer network to obtain the time sequence of the virtual maintenance action semantic feature graph comprises: Inputting the virtual maintenance monitoring key frame into a convolution layer with a convolution kernel of 3×3 to obtain a shallow feature map of the virtual maintenance monitoring key frame; Inputting the shallow feature map of the virtual maintenance monitoring key frame into the shallow residual information fusion enhancement module of the virtual maintenance action to obtain the shallow residual fusion feature map of the virtual maintenance monitoring key frame; Inputting the shallow residual fusion feature map of the virtual maintenance monitoring key frame into a convolution layer with a convolution kernel of 3×3 to obtain a deep feature map of the virtual maintenance monitoring key frame; Inputting the virtual maintenance monitoring key frame deep feature map into the virtual maintenance action deep residual information fusion enhancement module to obtain the virtual maintenance monitoring key frame deep residual fusion feature map; Inputting the virtual maintenance monitoring key frame deep residual fusion feature map into the scale modulation module based on the SPPF layer to obtain the virtual maintenance monitoring key frame deep modulation feature map; Upsampling the virtual maintenance monitoring key frame deep modulation feature map to obtain an upsampled virtual maintenance monitoring key frame deep modulation feature map; The upsampled virtual maintenance monitoring key frame deep modulation feature map is fused with the virtual maintenance monitoring key frame shallow residual fusion feature map to obtain a virtual maintenance monitoring key frame multi-scale feature map; The virtual maintenance monitoring key frame multi-scale feature map is input into a virtual maintenance action multi-scale residual information fusion enhancement module to obtain the virtual maintenance action semantic feature map.

4. The virtual maintenance training assessment management method according to claim 3 is characterized in that: The step of inputting the shallow feature map of the virtual maintenance monitoring key frame into the shallow residual information fusion enhancement module of the virtual maintenance action to obtain the shallow residual fusion feature map of the virtual maintenance monitoring key frame includes: Inputting the shallow feature map of the virtual maintenance monitoring key frame into the point convolution layer for channel dimension reduction modulation to obtain the channel dimension reduction modulated shallow feature map of the virtual maintenance monitoring key frame; Inputting the shallow features of the virtual maintenance monitoring key frame modulated by the channel dimension reduction into a convolution layer with a convolution kernel of 3×3 to extract features, thereby obtaining a shallow implicit information feature map of the virtual maintenance monitoring key frame; Inputting the shallow implicit information feature map of the virtual maintenance monitoring key frame into the point convolution layer for channel dimensionality modulation to obtain the shallow feature map of the virtual maintenance monitoring key frame after channel modulation; The shallow feature map of the virtual maintenance monitoring key frame after channel modulation and the shallow feature map of the virtual maintenance monitoring key frame after channel dimension reduction modulation are fused to obtain the shallow residual fusion feature map of the virtual maintenance monitoring key frame.

5. The virtual maintenance training assessment management method according to claim 4 is characterized in that: The step of inputting the virtual maintenance monitoring key frame deep residual fusion feature map into a scale modulation module based on an SPPF layer to obtain a virtual maintenance monitoring key frame deep modulation feature map comprises: Inputting the virtual maintenance monitoring key frame deep residual fusion feature map into a scale modulation module based on a point convolution layer to obtain a scale-adjusted virtual maintenance monitoring key frame deep residual fusion feature map; Performing maximum pooling processing on each feature matrix along the channel dimension in the scale-adjusted virtual maintenance monitoring key frame deep residual fusion feature map to obtain a virtual maintenance monitoring key frame deep residual fusion feature vector; Based on the deep residual fusion feature vector of the virtual maintenance monitoring key frame, channel weighted enhancement is performed on the scale-adjusted virtual maintenance monitoring key frame deep residual fusion feature map to obtain the virtual maintenance monitoring key frame deep modulation feature map.

6. The virtual maintenance training assessment management method according to claim 5 is characterized in that: The step of performing channel pruning optimization processing on each virtual maintenance action semantic feature graph in the time series of the virtual maintenance action semantic feature graph to obtain the time series of optimized virtual maintenance action semantic feature graphs includes: Performing global mean pooling processing on each feature matrix along the channel dimension in the virtual maintenance action semantic feature map to obtain a virtual maintenance action semantic channel feature vector; The virtual maintenance action semantic channel feature vector is first multiplied by the weight matrix and then added to the bias vector by position to obtain the virtual maintenance action semantic channel weight vector; Masking the virtual maintenance action semantic channel weight vector to obtain a channel weight vector of a masked virtual maintenance action semantic feature map; Taking each position feature value in the channel weight vector of the masked virtual maintenance action semantic feature map as a weighting coefficient, weighted optimization is performed on each feature matrix along the channel dimension in the virtual maintenance action semantic feature map to obtain the optimized virtual maintenance action semantic feature map; Among them, the virtual maintenance action semantic channel weight vector is masked to obtain the channel weight vector of the masked virtual maintenance action semantic feature map, including: in response to each position feature value in the virtual maintenance action semantic channel weight vector being greater than or equal to the predetermined threshold, taking the original value of the position feature value corresponding to the predetermined threshold that is greater than or equal to the predetermined threshold; when each position feature value in the virtual maintenance action semantic channel weight vector is less than the predetermined threshold, setting the position feature value to zero to obtain the channel weight vector of the masked virtual maintenance action semantic feature map.

7. The virtual maintenance training assessment management method according to claim 6 is characterized in that: Each optimized virtual maintenance action semantic feature map in the time series of the optimized virtual maintenance action semantic feature map is subjected to feature aggregation along the channel dimension and then input into the global feature perception fusion module to obtain a virtual maintenance action semantic full-time domain correlation feature map, including: Performing feature aggregation along the channel dimension on each optimized virtual maintenance action semantic feature graph in the time series of the optimized virtual maintenance action semantic feature graph to obtain an optimized virtual maintenance action semantic aggregation feature graph; In the global feature perception fusion module, the optimized virtual maintenance action semantic aggregation feature map is linearly transformed to obtain a virtual maintenance action semantic linear transformation matrix; In the global feature perception fusion module, the virtual maintenance action semantic linear transformation matrix is ​​subjected to soft maximum normalization processing in a Softmax function to obtain a virtual maintenance action semantic linear transformation weight matrix; In the global feature perception fusion module, each feature matrix along the channel dimension in the optimized virtual maintenance action semantic aggregation feature map is multiplied by position with the virtual maintenance action semantic linear transformation weight matrix to obtain a virtual maintenance action semantic global channel feature map; In the global feature perception fusion module, the virtual maintenance action semantic global channel feature map is activated through the Relu activation function and then multiplied by the trainable scaling factor, and is added to the optimized virtual maintenance action semantic aggregation feature map by position to obtain the virtual maintenance action semantic full-time domain association feature map.

8. The virtual maintenance training assessment management method according to claim 7 is characterized in that: In the global feature perception fusion module, the optimized virtual maintenance action semantic aggregation feature map is linearly transformed to obtain a virtual maintenance action semantic linear transformation matrix, including: Processing the optimized virtual maintenance action semantic aggregation feature map through a convolution layer to obtain a virtual maintenance action semantic feature matrix; After multiplying the virtual maintenance action semantic feature matrix with the global perception weight matrix, the obtained weight-modulated virtual maintenance action semantic feature matrix and the global perception bias matrix are summed by position to obtain the virtual maintenance action semantic linear transformation matrix.

9. The virtual maintenance training assessment management method according to claim 8 is characterized in that: The method of determining whether there is an error in the maintenance training operation action of the assessed person based on the semantic full-time domain association feature graph of the virtual maintenance action, determining whether to issue a prompt message, and generating an assessment report includes: Inputting the virtual maintenance action semantic full-time domain correlation feature map into a maintenance training operation action error identifier based on a classifier to obtain a recognition result, wherein the recognition result is used to indicate whether there is an error in the maintenance training operation action of the person being assessed; In response to the recognition result, a prompt message is issued to the person being assessed if there are errors in the maintenance training operation actions, and an assessment report is generated.

Images