A training system for operation and maintenance personnel based on image processing and behavior recognition

Abstract

The present invention relates to the technical field of image processing, and discloses an operation and maintenance personnel training system based on image processing and behavior recognition, comprising: a modeling module, which is used to extract modeling information and generate a simulation training model based on the modeling information; a training data acquisition unit, which is used to acquire training data in the operation and maintenance personnel training process; a training data preprocessing unit, which is used to process the acquired training data to obtain a current training model; a model preprocessing module, which is used to preprocess the current training model and a reference training model; a scoring module, which is used to score the training of the operation and maintenance personnel; the present invention performs operation and maintenance training simulation based on a virtual reality environment, and acquires physiological data, motion data and image data as training data, processes heterogeneous training data through a comprehensive data processing method, and evaluates the level of the operation and maintenance personnel from multiple dimensions of physiology, psychology and knowledge.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and more specifically, to an operation and maintenance personnel training system based on image processing and behavior recognition. Background Art

[0002] Due to the complex situation on the substation site and the uneven quality of personnel, substation operation and maintenance operations still have great safety risks caused by human factors. Therefore, it is extremely important to evaluate the comprehensive level of substation operation and maintenance personnel, which can effectively reduce the safety risks caused by human factors. The conventional method of using mechanical test papers to evaluate the level of substation operation and maintenance personnel is relatively simple in content and deviates greatly from actual operations, and cannot evaluate the operation and maintenance level of substation operation and maintenance personnel in real situations. Summary of the invention

[0003] The present invention provides an operation and maintenance personnel training system based on image processing and behavior recognition, which solves the technical problem in the related art that the operation and maintenance level of substation operation and maintenance personnel in real situations cannot be evaluated.

[0004] The present invention provides an operation and maintenance personnel training system based on image processing and behavior recognition, comprising:

[0005] A modeling module, which is used to extract modeling information and generate a simulation training model based on the modeling information;

[0006] Personnel information management module, which is used to manage the personnel information of operation and maintenance personnel;

[0007] A training start module, which is used to start training;

[0008] A training data collection unit is used to collect training data during the training of operation and maintenance personnel; the training data includes physiological data, motion data, and image data;

[0009] A training data preprocessing unit, which is used to process the collected training data to obtain a current training model;

[0010] A training database, which is used to store a reference training model, the reference training model includes a plurality of action nodes and a data slice set corresponding to the action nodes;

[0011] A model preprocessing module, which is used to preprocess the current training model and the reference training model;

[0012] A reference model selection module is used to calculate the similarity between the first node set of the current training model and the first node set of the reference training model. , select the similarity with the current training model The largest reference training model served as the final reference training model;

[0013] Scoring module, which is used to score the training of operation and maintenance personnel, and the score is related to the similarity between the current training model and the final reference training model. is proportional to;

[0014] The teaching module recommends teaching content to operation and maintenance personnel based on their training scores.

[0015] Furthermore, the modeling module includes a modeling data acquisition module and a model generation module. The modeling data acquisition module is used to collect modeling data, and the model generation module generates a simulation training model based on the modeling data.

[0016] Furthermore, the personnel information management module includes:

[0017] A personnel information storage module, which is used to store the personnel information of operation and maintenance personnel;

[0018] A personnel information updating module, which is used to update the personnel information stored in the personnel information storage module;

[0019] The personnel identification and verification module matches the corresponding personnel information based on the output biometric information. If the corresponding personnel information cannot be matched, the verification cannot be passed and the access is denied.

[0020] Further, the training initiation module includes a training item storage module and a training item selection module, wherein the training item storage module is used to store training items;

[0021] The training project selection module is used to select training projects;

[0022] The training project selection module may select a training project based on a request input by an operation and maintenance personnel or a system administrator through a human-computer interaction interface.

[0023] The training environment generation module generates a training environment based on the scene model, device model, and tool model corresponding to the training project selected by the training project selection module.

[0024] Furthermore, each training project matches one or more scene models, one or more device models, and one or more tool models, and the training environment includes scene models, device models, and personnel models.

[0025] Furthermore, the training data preprocessing unit determines the time point of the operation and maintenance personnel's action based on the motion data as the action node, and extracts the image data and physiological data at the same time as the data slice set based on the time of the action node; extracts all the data slice sets of the operation and maintenance personnel as the current training model;

[0026] The training data preprocessing unit determines the action nodes including:

[0027] The action data when the operation and maintenance personnel start training is recorded as the initial action data, and the action data is traversed backwards from the initial action data time frame by time frame until the traversal termination condition is reached, and the time point when the traversal is terminated is recorded as the new action node;

[0028] Then iterate the following process, starting from the action data corresponding to the latest action node that has been recorded, traverse the action data backwards time frame by time frame until the traversal termination condition is reached, and record the time point when the traversal terminates as the new action node; the traversal termination condition is the distance between the traversed action data and the motion data corresponding to the previous action node Exceeding a set first distance threshold;

[0029] The termination condition of iterative execution is that all action data have been traversed once.

[0030] Furthermore, the step of preprocessing the current training model and the reference training model includes:

[0031] Step 101, generating a corresponding slice object for the data slice of the training model, wherein the attributes of the slice object are generated based on the motion data and the physiological data;

[0032] Step 102, generating N first nodes, each of which maps a two-dimensional coordinate point, the two-dimensional coordinate points of all first nodes are distributed in a matrix array, the number of two-dimensional coordinate points in each column is the same as the number of two-dimensional coordinate points in each row, and the minimum distance between two adjacent two-dimensional coordinate points is 1;

[0033] Step 103, randomly selecting N slice set objects of the training model as first slice set objects, and mapping the first slice set objects one by one to the first node;

[0034] Step 104: randomly select a slice object from the training model as the second slice object, and calculate the similarity between the second slice object and the first slice object. , select the similarity with the second set of objects The largest first slice set object, and updating the attribute values ​​of the first slice set object and the first slice set objects associated with it;

[0035] The formula for updating the attribute values ​​of the first slice set object and the first slice set object associated with it is as follows:

[0036]

[0037] in Indicates the value of the j-th attribute of the first slice object after the update. Indicates the value of the j-th attribute of the first slice set object before the update. represents the value of the jth attribute of the second slice set object, and t represents the number of times all the first slice set objects are updated;

[0038] Step 105, iteratively executing step 104 until all slice set objects are selected as second slice set objects.

[0039] Furthermore, the similarity The calculation methods include:

[0040] Given a weighted bipartite graph , where the set is the first node set of the current training model, set Y is the first node set of the reference training model, and the maximum weighted perfect matching of the weighted bipartite graph G is solved by the Kuhn-Munkres algorithm;

[0041] Solve the maximum weight perfect matching of the weighted bipartite graph G with the initial weights being the similarity of the slice objects mapped by the two first nodes ;

[0042] Based on the maximum weight perfect matching, the weight and similarity after matching are obtained .

[0043] Furthermore, the similarity The calculation methods include:

[0044] Generating a second node for each data slice set of the current training model and the final reference training model;

[0045] Given a weighted bipartite graph , where the set is the second node set of the current training model, set B is the second node set of the final reference training model, and the maximum weighted perfect matching of the weighted bipartite graph U is solved by the Kuhn-Munkres algorithm;

[0046] When solving the maximum weight perfect matching of the weighted bipartite graph U, the initial weight is the similarity of the two second nodes. ;

[0047] Based on the maximum weight perfect matching, the weight and similarity after matching are obtained ;

[0048] The similarity of the two second nodes The calculation formula is as follows:

[0049]

[0050] in, represents the distance of the physiological data corresponding to the second node, Indicates the similarity of the image data corresponding to the second node; and The first weight and the second weight are defined respectively, and the similarity between the physiological data and the image data can be adjusted by adjusting the first weight and the second weight. The weight of .

[0051] Furthermore, the calculation formula of the score H is as follows:

[0052] in, is the similarity between the current training model and the final reference training model.

[0053] The beneficial effects of the present invention are:

[0054] The present invention conducts operation and maintenance training simulation based on a virtual reality environment, collects physiological data, motion data and image data as training data, processes heterogeneous training data through a comprehensive data processing method, and evaluates the level of operation and maintenance personnel from multiple dimensions of physiology, psychology and knowledge. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 It is a module schematic diagram of an operation and maintenance personnel training system based on image processing and behavior recognition of the present invention;

[0056] Figure 2 is a module schematic diagram of the modeling module of the present invention;

[0057] Figure 3 is a module schematic diagram of a personnel information management module of the present invention;

[0058] Figure 4 is a module schematic diagram of the training start module of the present invention;

[0059] Figure 5 It is a flow chart of the method of preprocessing the current training model and the reference training model of the present invention.

[0060] In the figure: modeling module 101, modeling data acquisition module 1011, model generation module 1012, personnel information management module 102, personnel information storage module 1021, personnel information update module 1022, personnel identification and verification module 1023, training start module 103, training project storage module 1031, training project selection module 1032, training environment generation module 104, training data acquisition unit 105, training data preprocessing unit 106, training database 107, model preprocessing module 108, reference model selection module 109, scoring module 110, teaching module 111. DETAILED DESCRIPTION

[0061] The subject matter described herein will now be discussed with reference to example embodiments. It should be understood that the discussion of these embodiments is only to enable those skilled in the art to better understand and implement the subject matter described herein, and the functions and arrangements of the elements discussed may be changed without departing from the scope of protection of the contents of this specification. Each example may omit, replace or add various processes or components as needed. In addition, the features described relative to some examples may also be combined in other examples.

[0062] Embodiment 1

[0063] like Figure 1-Figure 5 As shown, an operation and maintenance personnel training system based on image processing and behavior recognition includes:

[0064] A modeling module 101, which is used to extract modeling information and generate a simulation training model based on the modeling information;

[0065] In one embodiment of the present invention, the simulation training model includes a scenario model, a device model, a tool model, and a personnel model, wherein the device model includes a normal device model and a faulty device model.

[0066] In one embodiment of the present invention, the modeling module 101 includes a modeling data acquisition module 1011 and a model generation module 1012, wherein the modeling data acquisition module 1011 is used to collect modeling data, and the model generation module 1012 generates a simulation training model based on the modeling data;

[0067] A personnel information management module 102, which is used to manage the personnel information of operation and maintenance personnel;

[0068] In one embodiment of the present invention, the personnel information of the operation and maintenance personnel includes personnel ID, personnel gender, personnel age, personnel length of service, personnel height, personnel weight, personnel biometric information and other personnel information.

[0069] In one embodiment of the present invention, the personnel information management module 102 includes:

[0070] Personnel information storage module 1021, which is used to store personnel information of operation and maintenance personnel;

[0071] A personnel information updating module 1022, which is used to update the personnel information stored in the personnel information storage module 1021;

[0072] The personnel identification and verification module 1023 matches the corresponding personnel information based on the output biometric information. If the corresponding personnel information cannot be matched, the verification cannot be passed and the access is denied.

[0073] The biometric information may be input through an external biometric information collection device.

[0074] The training start module 103 is used to start training;

[0075] In one embodiment of the present invention, the training initiation module 103 includes a training project storage module 1031 and a training project selection module 1032, wherein the training project storage module 1031 is used to store training projects, each training project matches more than one scene model, more than one device model, and more than one tool model;

[0076] The training item selection module 1032 is used to select a training item;

[0077] The training project selection module 1032 may select the training project based on a request input by an operation and maintenance personnel or a system administrator through a human-computer interaction interface.

[0078] The training environment generation module 104 generates a training environment based on the scene model, device model, and tool model corresponding to the training project selected by the training project selection module 1032.

[0079] The training environment includes scene models, equipment models, and personnel models, where the number of personnel models is the same as the number of people participating in the training project.

[0080] For example, the same training environment can accommodate three persons, ABC, to train together. In this case, three personnel models corresponding to the three persons, ABC, are set up in the training environment. If one person is allowed to train, only one personnel model needs to be generated in the training environment.

[0081] A training data collection unit 105, which is used to collect training data during the training process of operation and maintenance personnel;

[0082] The training data includes physiological data, motion data, and image data;

[0083] In one embodiment of the present invention, the physiological data represents physical characteristic indicators of the operation and maintenance personnel, such as heart rate, blood pressure, body temperature, etc.

[0084] In one embodiment of the present invention, the image data is a virtual image collected in a virtual training environment, and at least one image collection point is set in a training environment;

[0085] In one embodiment of the present invention, the motion data is collected by a wearable motion detection device, and the wearable motion detection device includes a sensor worn by an operation and maintenance personnel.

[0086] A training data preprocessing unit 106, which is used to process the collected training data to obtain a current training model;

[0087] The time point of the operation and maintenance personnel's action is determined based on the motion data as the action node, and the image data and physiological data at the same time are extracted as the data slice set based on the time of the action node; all the data slice sets of the operation and maintenance personnel are extracted as the current training model;

[0088] The data slice set is essentially a multi-source heterogeneous data set extracted at a specific time node. In one embodiment of the present invention, a method for determining an action node is provided, including:

[0089] The distance between the motion data corresponding to two adjacent action nodes Exceeding a set first distance threshold;

[0090] The action data is traversed backwards from the motion data corresponding to the previous action node, time frame by time frame. The traversal is terminated when the distance between the traversed action data and the motion data corresponding to the previous action node is When the set first distance threshold is exceeded, the time point at which the traversal terminates the corresponding action data is taken as the next action node;

[0091] The action data when the operation and maintenance personnel start training is taken as the first action node, and traversal starts from this action node.

[0092] Specifically, the action data of the operation and maintenance personnel when they start training is recorded as the initial action data, and the action data is traversed backwards time frame by time frame from the initial action data until the traversal termination condition is reached, and the time point of traversal termination is recorded as the new action node;

[0093] Then iteratively execute the following process, starting from the action data corresponding to the latest action node that has been recorded, traverse the action data backwards time frame by time frame until the traversal termination condition is reached, and record the time point when the traversal is terminated as the new action node;

[0094] The termination condition of iterative execution is that all action data have been traversed once.

[0095] In one embodiment of the present invention, the motion data is taken as an object to calculate the distance between the motion data by using distance calculation methods such as Euclidean distance and cosine distance;

[0096] Specifically, the motion data includes the wrist joint angle, knee joint angle, elbow joint angle, and head angle. The distance between the motion data can be calculated by the following formula: :

[0097]

[0098] in, and Respectively represent the i-th body angle of the two action data;

[0099] In one embodiment of the present invention, the length of the time frame is equal to the sampling interval of the motion data. For example, if the sampling interval of the motion data is 0.5 s, the time frame is equivalently set to 0.5 s.

[0100] A training database 107, which is used to store a reference training model, the reference training model includes a plurality of action nodes and a data piece set corresponding to the action nodes;

[0101] Similar to the current training model, the data slice set of the reference training model also includes image data and physiological data;

[0102] In one embodiment of the present invention, the reference training model can be generated by the training records of the test personnel performing operation and maintenance operations according to the standard operating procedures, which is used as a comparison reference; for a training project, multiple reference training models can be generated by different test personnel according to different training records, so it is necessary to select a suitable reference training model as a reference during comparison.

[0103] A model preprocessing module 108, which is used to preprocess the current training model and the reference training model;

[0104] The preprocessing steps include:

[0105] Step 101, generating a corresponding slice object for the data slice of the training model, wherein the attributes of the slice object are generated based on the motion data and the physiological data;

[0106] For example, the attributes of a slice object include wrist joint angle, knee joint angle, elbow joint angle, head angle, heart rate, blood pressure, and body temperature.

[0107] Step 102, generating N first nodes, each of which maps a two-dimensional coordinate point, the two-dimensional coordinate points of all first nodes are distributed in a matrix array, the number of two-dimensional coordinate points in each column is the same as the number of two-dimensional coordinate points in each row, and the minimum distance between two adjacent two-dimensional coordinate points is 1;

[0108] N is proportional to the total number of action nodes of the training model. In one embodiment of the present invention, the calculation formula of N is as follows:

[0109] N= , where S is the total number of action nodes of the training model.

[0110] Step 103, randomly selecting N slice set objects of the training model as first slice set objects, and mapping the first slice set objects one by one to the first node;

[0111] Step 104: randomly select a slice object from the training model as the second slice object, and calculate the similarity between the second slice object and the first slice object. , select the similarity with the second set of objects The largest first slice set object, and updating the attribute values ​​of the first slice set object and the first slice set objects associated with it;

[0112] The formula for updating the attribute values ​​of the first slice set object and the first slice set object associated with it is as follows:

[0113]

[0114] in Indicates the value of the j-th attribute of the first slice object after the update. Indicates the value of the j-th attribute of the first slice set object before the update. represents the value of the jth attribute of the second slice set object, and t represents the number of times all the first slice set objects are updated.

[0115] In one embodiment of the present invention, the similarity between two episode objects The calculation formula is as follows:

[0116] in, and are the values ​​of the zth attributes of the two slice objects respectively, and p is the total number of attributes of the slice objects.

[0117] Of course, cosine similarity can also be used to calculate the similarity of the set objects .

[0118] Step 105, iteratively executing step 104 until all slice set objects are selected as second slice set objects;

[0119] During the iteration process, no duplicate slice set object will be selected as the second slice set object.

[0120] Reference model selection module 109, which is used to calculate the similarity between the current training model and the reference training model , select the similarity with the current training model The largest reference training model served as the final reference training model;

[0121] In one embodiment of the present invention, the similarity The calculation methods include:

[0122] Given a weighted bipartite graph , where the set is the first node set of the current training model, set Y is the first node set of the reference training model, and the maximum weighted perfect matching of the weighted bipartite graph G is solved by the Kuhn-Munkres algorithm;

[0123] Solve the maximum weight perfect matching of the weighted bipartite graph G with the initial weights being the similarity of the slice objects mapped by the two first nodes ;

[0124] Based on the maximum weight perfect matching, the weight and similarity after matching are obtained .

[0125] The present invention comprehensively selects the most appropriate reference training model to participate in scoring the operation and maintenance personnel through multiple aspects such as the operation and maintenance personnel's physical data, motion data, and the number of motion nodes.

[0126] Scoring module 110 is used to score the training of operation and maintenance personnel, and the score is related to the similarity between the current training model and the final reference training model. Directly proportional.

[0127] Similarity The calculation methods include:

[0128] Generating a second node for each data slice set of the current training model and the final reference training model;

[0129] Given a weighted bipartite graph , where the set is the second node set of the current training model, set B is the second node set of the final reference training model, and the maximum weighted perfect matching of the weighted bipartite graph U is solved by the Kuhn-Munkres algorithm;

[0130] When solving the maximum weight perfect matching of the weighted bipartite graph U, the initial weight is the similarity of the two second nodes. ;

[0131] Based on the maximum weight perfect matching, the weight and similarity after matching are obtained .

[0132] The similarity of the two second nodes The calculation formula is as follows:

[0133]

[0134] in, represents the distance of the physiological data corresponding to the second node, Indicates the similarity of the image data corresponding to the second node; and The first weight and the second weight are defined respectively, and the similarity between the physiological data and the image data can be adjusted by adjusting the first weight and the second weight. The weight of .

[0135] The similarity calculation method of image data can adopt methods such as histogram matching and image similarity calculation based on feature points.

[0136] Distance of physiological data The distance can be calculated using distance calculation methods such as Euclidean distance and cosine distance.

[0137] The teaching module 111 recommends teaching content to the operation and maintenance personnel based on their training scores.

[0138] If the training score of the operation and maintenance personnel is lower than the score threshold of the current training project, teaching content is recommended to them.

[0139] The teaching content includes teaching videos generated based on the image data of the final reference training model.

[0140] The above describes an embodiment of the present embodiment, but the present embodiment is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present embodiment, ordinary technicians in this field can also make many forms, all of which are protected by the present embodiment.

Claims

1. An operation and maintenance personnel training system based on image processing and behavior recognition, characterized in that: include: A modeling module, which is used to extract modeling information and generate a simulation training model based on the modeling information; Personnel information management module, which is used to manage the personnel information of operation and maintenance personnel; A training start module, which is used to start training; A training data collection unit, which is used to collect training data during the training process of operation and maintenance personnel; The training data includes physiological data, motion data, and image data; A training data preprocessing unit, which is used to process the collected training data to obtain a current training model; The training data preprocessing unit determines the time point of the operation and maintenance personnel's action based on the motion data as the action node, and extracts the image data and physiological data at the same time as the data slice set based on the time of the action node; extracts all the data slice sets of the operation and maintenance personnel as the current training model; The training data preprocessing unit determines the action nodes including: The action data when the operation and maintenance personnel start training is recorded as the initial action data, and the action data is traversed backwards from the initial action data time frame by time frame until the traversal termination condition is reached, and the time point when the traversal is terminated is recorded as the new action node; Then iterate the following process, starting from the action data corresponding to the latest action node that has been recorded, traverse the action data backwards time frame by time frame until the traversal termination condition is reached, and record the time point when the traversal terminates as the new action node; the traversal termination condition is the distance between the traversed action data and the motion data corresponding to the previous action node Exceeding a set first distance threshold; The termination condition of iterative execution is: all action data have been traversed once; A training database, which is used to store a reference training model, the reference training model includes a plurality of action nodes and a data slice set corresponding to the action nodes; A model preprocessing module, which is used to preprocess the current training model and the reference training model; The steps of preprocessing the current training model and the reference training model include: Step 101, generating a corresponding slice object for the data slice of the training model, wherein the attributes of the slice object are generated based on the motion data and the physiological data; Step 102, generating N first nodes, each of which maps a two-dimensional coordinate point, the two-dimensional coordinate points of all first nodes are distributed in a matrix array, the number of two-dimensional coordinate points in each column is the same as the number of two-dimensional coordinate points in each row, and the minimum distance between two adjacent two-dimensional coordinate points is 1; Step 103, randomly selecting N slice set objects of the training model as first slice set objects, and mapping the first slice set objects one by one to the first node; Step 104: randomly select a slice object from the training model as the second slice object, and calculate the similarity between the second slice object and the first slice object. , select the similarity with the second set of objects The largest first slice set object, and updating the attribute values ​​of the first slice set object and the first slice set objects associated with it; The formula for updating the attribute values ​​of the first slice set object and the first slice set object associated with it is as follows: in Indicates the value of the j-th attribute of the first slice object after the update. Indicates the value of the j-th attribute of the first slice set object before the update. represents the value of the jth attribute of the second slice set object, and t represents the number of times all the first slice set objects are updated; Step 105, iteratively executing step 104 until all slice set objects are selected as second slice set objects; A reference model selection module is used to calculate the similarity between the first node set of the current training model and the first node set of the reference training model. , select the similarity with the current training model The largest reference training model served as the final reference training model; Scoring module, which is used to score the training of operation and maintenance personnel, and the score is related to the similarity between the current training model and the final reference training model. is proportional to; The teaching module recommends teaching content to operation and maintenance personnel based on their training scores.

2. According to claim 1, the operation and maintenance personnel training system based on image processing and behavior recognition is characterized in that: The modeling module includes a modeling data acquisition module and a model generation module. The modeling data acquisition module is used to collect modeling data, and the model generation module generates a simulation training model based on the modeling data.

3. The operation and maintenance personnel training system based on image processing and behavior recognition according to claim 1 is characterized in that: The personnel information management module includes: A personnel information storage module, which is used to store the personnel information of operation and maintenance personnel; A personnel information updating module, which is used to update the personnel information stored in the personnel information storage module; The personnel identification and verification module matches the corresponding personnel information based on the output biometric information. If the corresponding personnel information cannot be matched, the verification cannot be passed and the access is denied.

4. The operation and maintenance personnel training system based on image processing and behavior recognition according to claim 1 is characterized in that: The training start module includes a training project storage module and a training project selection module, wherein the training project storage module is used to store training projects; The training project selection module is used to select training projects; The training project selection module selects training projects based on requests input by operation and maintenance personnel or system administrators through the human-computer interaction interface; The training environment generation module generates a training environment based on the scene model, device model, and tool model corresponding to the training project selected by the training project selection module.

5. The operation and maintenance personnel training system based on image processing and behavior recognition according to claim 4 is characterized in that: Each training project matches one or more scene models, one or more equipment models, and one or more tool models. The training environment includes scene models, equipment models, and personnel models.

6. The operation and maintenance personnel training system based on image processing and behavior recognition according to claim 1 is characterized in that: Similarity The calculation methods include: Given a weighted bipartite graph: , where set X is the first node set of the current training model, and set Y is the first node set of the reference training model. The maximum weighted perfect matching of the weighted bipartite graph G is solved by the Kuhn-Munkres algorithm; Solve the maximum weight perfect matching of the weighted bipartite graph G with the initial weights being the similarity of the slice objects mapped by the two first nodes ; Based on the maximum weight perfect matching, the weight and similarity after matching are obtained .

7. The operation and maintenance personnel training system based on image processing and behavior recognition according to claim 1 is characterized in that: Similarity The calculation methods include: Generating a second node for each data slice set of the current training model and the final reference training model; Given a weighted bipartite graph: , where set V is the second node set of the current training model, set B is the second node set of the final reference training model, and the maximum weighted perfect matching of the weighted bipartite graph U is solved by the Kuhn-Munkres algorithm; When solving the maximum weight perfect matching of the weighted bipartite graph U, the initial weight is the similarity of the two second nodes. ; Based on the maximum weight perfect matching, the weight and similarity after matching are obtained ; The similarity of the two second nodes The calculation formula is as follows: in, represents the distance of the physiological data corresponding to the second node, Indicates the similarity of the image data corresponding to the second node; and The first weight and the second weight are defined respectively, and the similarity between the physiological data and the image data can be adjusted by adjusting the first weight and the second weight. The weight of .

8. The operation and maintenance personnel training system based on image processing and behavior recognition according to claim 1 is characterized in that: The calculation formula of score H is as follows: in, is the similarity between the current training model and the final reference training model.

Images