An equipment operation training assistance method and system

Abstract

The application provides an equipment operation training auxiliary method and system, relates to the field of equipment operation training, and the method comprises the following steps: arranging positioning points for data acquisition and projection calibration on equipment to be assisted in operation; cleaning, preprocessing and classifying data; automatically enhancing labeling of the classified data; realizing migration and adaptation of cross-equipment scenes by using small sample learning technology; completing model training and learning; generating an auxiliary operation model; when a user operates equipment, identifying and reasoning equipment scenes and operation intentions by the auxiliary operation model, filtering potential risks, generating operation suggestions, and controlling a projection device to project the operation suggestions to corresponding areas of the equipment according to the positioning points; and the application realizes structured knowledge precipitation and real-time accurate guidance, breaks through the scene adaptation bottleneck by means of small sample learning, efficiently enables operation personnel training and talent team construction, and comprehensively improves operation safety and accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of equipment operation training, in particular to an equipment operation training auxiliary method and system. BACKGROUND

[0002] With the wide application of high-end equipment and precision instruments in the fields of industry, medical treatment, scientific research, etc., the operation complexity and professionalism thereof are increasingly improved. However, the current training and real-time assistance means for the operators are relatively lagging behind.

[0003] Firstly, the traditional training mode is inefficient and costly. New operators usually rely on the experience transmission of "apprenticeship" or refer to lengthy paper / electronic manuals, and the learning curve is long. When the personnel flow, the valuable operation experience and fault handling knowledge are easily lost, which leads to unstable training quality and even causes the risk of misoperation. Secondly, the utilization efficiency of the existing technical documents is low. The equipment is updated at a fast speed, and the operation manual and fault library are often content-rich and inconvenient to search. The operators are difficult to quickly locate the required information in the tense working environment, especially in emergency disposal, and the time delay may cause serious consequences. Thirdly, there is a lack of intelligent real-time operation guidance. Although there are currently some auxiliary systems based on screen prompts or voice guidance, they cannot be deeply integrated with the physical operation environment. The operator needs to repeatedly switch the line of sight between the device and the auxiliary screen, which is easy to cause distraction, and the prompt information is not intuitive enough to achieve the precise guidance of "pointing to do". SUMMARY

[0004] Therefore, the purpose of the present application is to provide an equipment operation training auxiliary method and system to solve at least one problem in the background art.

[0005] The first aspect of the present application provides an equipment operation training auxiliary method, which comprises:

[0006] Laying the positioning points for data acquisition and projection calibration on the equipment to be assisted in operation to perform data acquisition;

[0007] Cleaning, preprocessing and classifying the collected data, wherein the data comprises image data and text data;

[0008] Automatically enhancing the labeled data after classification, establishing the structured association of the image data and the text data, using small sample learning technology to realize the migration and adaptation of cross-equipment scenes, completing the model training and learning, and generating an auxiliary operation model with operation recognition, logical reasoning and operation suggestion;

[0009] When the user operates the equipment, the auxiliary operation model is used to identify and infer the equipment scene and operation intention, filter potential risks, generate operation suggestions, and control the projection device to project the operation suggestions to the corresponding area of the equipment according to the positioning points.

[0010] According to an aspect of the above technical solution, the steps of cleaning, preprocessing and classifying the collected data specifically include:

[0011] The image data and the text data are cleaned, preprocessed and classified respectively;

[0012] The image data and the text data are cleaned by an abnormality detection algorithm based on reinforcement learning;

[0013] The preprocessing and classification steps of the image data include:

[0014] Based on the position coordinates of the positioning points, the image data is effectively extracted by taking the positioning points as the boundary box anchor points to obtain effective image data;

[0015] According to the distribution positions of the positioning points, the effective image data is regionally divided to obtain image data;

[0016] According to the user operation characteristics, the image data is classified into action images and state images.

[0017] According to an aspect of the above technical solution, the preprocessing and classification steps of the text data include:

[0018] The text data includes operation manuals, fault libraries and dangerous operation warning data sources, and the operation manuals, the fault libraries and the dangerous operation warning data sources are parsed by a semantic recognition algorithm;

[0019] Based on the operation manuals and the fault libraries, operation entities, step sequences, fault phenomena, solving actions and their logical relationships are extracted to construct a preliminary knowledge base;

[0020] Features of the dangerous operation warning data sources are extracted to establish a high-risk operation data set;

[0021] According to four dimensions of operation logic description, fixed step description, fault elimination logic and fault elimination step, the contents in the preliminary knowledge base are finely classified and labeled, and are associated with the high-risk operation data set.

[0022] According to an aspect of the above technical solution, the steps of automatically enhancing the labeling of the classified data and establishing the structured association of the image data and the text data specifically include:

[0023] The classified data is preliminarily labeled by using a pre-assisted operation model combined with an expert rule engine.

[0024] The structured association between the image data and the text data is established by deep semantic coding of the text data through the knowledge graph and the large language model, and in combination with the preliminary labeling result.

[0025] According to an aspect of the above technical solution, the small sample learning technology is used to realize the migration and adaptation of cross-equipment scenes, complete model training and learning, and generate an auxiliary operation model with operation recognition, logical reasoning, and operation suggestion providing functions. The steps include:

[0026] When a new equipment scene that is not covered by the data is encountered during the training of the auxiliary operation model, analogy reasoning and knowledge retrieval are performed based on the existing equipment scenes, so as to adapt the new equipment scene through small sample learning, and obtain an equipment scene library.

[0027] Based on the equipment scene library, a fusion architecture of the Transformer is used to deeply integrate the image data and the text data.

[0028] During the training of the auxiliary operation model, the explainable AI technology is used to provide suggestion attribution analysis, identify the key steps in the operation process, record the basis for the suggestions, apply reinforcement learning and evolutionary algorithms to automatically explore and optimize the operation suggestions, and through a continuous learning mechanism, the real expert operation experience on the line is silently absorbed to iteratively update the parameters of the auxiliary operation model in a non-destructive manner.

[0029] According to an aspect of the above technical solution, when the user operates the equipment, the steps of identifying and reasoning the equipment scene and the operation intention through the auxiliary operation model include:

[0030] When the user operates the equipment, the image data is captured in real time, and the operation instructions input by the user through natural language are parsed. The auxiliary operation model is used to identify and reason the equipment scene and the operation intention based on the image data and the operation instructions, and in combination with the knowledge base, preliminary operation suggestions are generated.

[0031] According to an aspect of the above technical solution, the steps of generating operation suggestions by filtering potential risks include:

[0032] A safety check constraint is constructed based on a high-risk operation data set.

[0033] The preliminary operation suggestions are compared and checked based on the safety check constraint, potential risk behaviors are filtered or corrected, and the operation suggestions are obtained.

[0034] The second aspect of the present application provides an equipment operation training auxiliary system, which is used to execute the above-mentioned equipment operation training auxiliary method. The system includes:

[0035] A data collection module is configured to arrange positioning points for data collection and projection calibration on equipment to be assisted in operation, and collect data;

[0036] A data processing module is configured to clean, preprocess and classify the collected data, wherein the data includes image data and text data;

[0037] A model learning module is configured to automatically enhance the labeled data after classification, establish a structured association between the image data and the text data, realize migration and adaptation across equipment scenes by using small sample learning technology, complete model training and learning, and generate an auxiliary operation model capable of operation recognition, logical reasoning and operation suggestion.

[0038] An inference assistance module is configured to identify and reason the equipment scene and operation intention by using the auxiliary operation model when a user operates the equipment, filter potential risks, generate operation suggestions, and control a projection device to project the operation suggestions to a corresponding area of the equipment according to the positioning points.

[0039] A third aspect of the present application provides a readable storage medium having a computer program stored thereon, wherein the program is executed by a processor to implement the steps of the equipment operation training assistance method.

[0040] A fourth aspect of the present application provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the equipment operation training assistance method when executing the program.

[0041] Compared with the prior art, the present application has the following advantages:

[0042] By means of structured knowledge base construction, real-time projection guidance and structured transformation of expert rule engine, a new operator can quickly master the core operation skills of the equipment without relying on experienced foremen for long-term teaching, thereby effectively avoiding experience loss caused by personnel flow; meanwhile, a safety checking mechanism established based on a high-risk operation data set is combined with the accurate projection of operation suggestions realized by the positioning points to filter illegal operations and warn potential risks in real time, thereby avoiding safety accidents and invalid operations from the source and improving the safety and accuracy of operations.

[0043] 3. Through the deep empowerment of the continuous learning mechanism, excellent operation experience in practice can be silently absorbed, the auxiliary operation model is continuously iteratively optimized, and continuous adaptation to equipment upgrading and operation demand changes. BRIEF DESCRIPTION OF DRAWINGS

[0044] Figure 1 The figure is a flow chart of an equipment operation training assistance method in embodiment 1 of the application.

[0045] Figure 2 The figure is a structural block diagram of an equipment operation training assistance system in embodiment 2 of the application.

[0046] Explanation of the symbols of the components in the drawings:

[0047] Data acquisition module 100, data processing module 200, model learning module 300, and inference assistance module 400.

[0048] The following specific embodiments will further illustrate the application in conjunction with the above-mentioned drawings. DETAILED DESCRIPTION

[0049] In order to facilitate the understanding of the application, the application will be described more fully below with reference to the relevant drawings. The drawings show several embodiments of the application. However, the application can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the application more thorough and comprehensive.

[0050] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the application belongs. The terms used in the specification of the application herein are only for the purpose of describing the specific embodiments and are not intended to limit the application. The term "and / or" used herein includes any and all combinations of one or more related listed items.

[0051] Embodiment 1

[0052] Please refer to Figure 1 , which shows an equipment operation training assistance method provided by embodiment 1 of the application, comprising steps S1-S4.

[0053] Step S1, laying out positioning points for data acquisition and projection calibration on the equipment to be assisted in operation;

[0054] For example, high-contrast, waterproof, and wear-resistant adhesive or installed positioning stickers (such as red bottom + white two-dimensional code encoding) are selected to lay out positioning points in the core area of the equipment to be assisted in operation, including but not limited to: four corners of the control panel, periphery of key operation components (buttons / knobs), and edge of the interface area.

[0055] Step S2, cleaning, preprocessing and classifying the collected data, including image data and text data;

[0056] Specifically, the image data and the text data are cleaned, preprocessed and classified respectively.

[0057] The image data and the text data are cleaned by an anomaly detection algorithm based on reinforcement learning.

[0058] By way of example but not limitation, the image data is captured by a high-definition industrial camera with a resolution of ≥1080P and a frame rate of ≥30fps, and the image data can include: standard operation process images of manufacturer technicians; operation images for new users to learn and practice; equipment running status (indicator light, parameter display) and fault scene images; and real-time operation logs (such as parameter adjustment records, fault alarm logs) of the equipment can also be selected to supplement the data dimensions. The text data includes operation manuals (including function descriptions, operation processes, parameter specifications), fault libraries (including fault phenomena, cause analysis, troubleshooting plans), and dangerous operation warning data sources (including prohibited behaviors, risk consequences, and avoidance requirements).

[0059] The preprocessing and classification steps of the image data include:

[0060] Based on the position coordinates of the positioning points, the effective image data is obtained by extracting the effective area of the image data with the positioning points as the bounding box anchor points.

[0061] For example, the image processing algorithm (such as OpenCV contour detection) is used to filter out only the pictures containing the equipment operation area, and filter out the environment background, irrelevant equipment and other invalid areas.

[0062] According to the distribution position of the positioning points, the image data is divided into regions.

[0063] According to the user operation characteristics, the image data is classified into action images and state images.

[0064] For example, according to the user operation characteristics, the action images (with human operation) and the state images (without human operation) are distinguished.

[0065] Further, the preprocessing and classification steps of the text data include:

[0066] The operation manual, the fault library, and the dangerous operation warning data source are parsed by a semantic recognition algorithm.

[0067] Based on the operation manual and the fault library, the operation entities, step sequences, fault phenomena, and solving actions and their logical relationships are extracted to construct a preliminary knowledge base.

[0068] Extract the features of the dangerous operation warning data source, and establish a high-risk operation data set; for example, ungrounded operation and other features.

[0069] According to the operation logic description, the fixed step description, the troubleshooting logic, and the troubleshooting step, the contents in the preliminary knowledge base are classified and labeled in detail, and are associated with the high-risk operation data set.

[0070] Among them, the operation logic description: explains the operation purpose / principle (such as connecting the gas source before starting, to avoid damaging the pneumatic components under no load); the fixed step description: clearly defines the specific operation process (such as 1. Connect the gas source→2. Turn on the main power→3. Set the working parameters); the troubleshooting logic: analyzes the fault causes and troubleshooting ideas (such as ineffective parameter adjustment, prioritize troubleshooting mechanical jamming, and then check the circuit connection); the troubleshooting step: clearly defines the fault resolution process (such as 1. Turn off the power→2. Remove the knob housing→3. Clean the jamming foreign matter).

[0071] Step S3, automatically enhance the labeling of the classified data, establish the structured association between image data and text data, use small sample learning technology to realize the migration and adaptation of cross-equipment scenes, complete model training and learning, and generate an auxiliary operation model with operation recognition, logical reasoning, and operation suggestion providing functions;

[0072] Specifically, using the pre-auxiliary operation model combined with the expert rule engine, the classified data is preliminarily labeled;

[0073] Through the knowledge graph and the large language model, the text data is deeply semantically encoded, and the structured association between the image data and the text data is established based on the preliminary labeling results.

[0074] By way of example and not limitation, the text data is deeply semantically encoded by the knowledge graph, the key information of the four-dimensional classified text (such as step 1 of starting: press the start button) is associated with the labeling results of the image data (such as the action image of pressing the start button), and additional associated information is supplemented, operation parameters (press for 3 seconds), safety labels (no risk / high risk), corresponding operation areas (panel 1) are added to the associated data, forming a structured data pair of image picture-text description-parameter requirement-safety attribute-operation area.

[0075] Further, when the auxiliary operation model encounters a new equipment scene that is not covered by the data during training, analogy reasoning and knowledge retrieval are carried out based on the existing equipment scenes, and small sample learning is carried out to adapt to the new equipment scene, and an equipment scene library is obtained.

[0076] That is, when encountering a new equipment scene (data is not covered), the core features of the new equipment (operation logic framework, fault type characteristics, key operation components) are extracted, and analogical reasoning is performed with existing equipment scenes; through knowledge retrieval, the structured annotation data of existing equipment scenes are called, only a small amount of samples of the new equipment are collected, small sample learning (such as Few-Shot Learning based on ProtoNet adaptation algorithm) is carried out, the new equipment scene is quickly adapted, and the equipment scene library is updated.

[0077] Based on the equipment scene library, a fusion architecture of Transformer is used to deeply intermingle image data and text data; wherein, the fusion architecture of Transformer, such as a ViT+BERT cross-modal fusion model, deeply intermingles image data (operation action / equipment state) and text data (four-dimensional classification / high-risk data), and realizes collaborative modeling of multi-source information.

[0078] In the process of training the auxiliary operation model, the explainable AI technology is used to provide suggestion attribution analysis, the key steps in the operation process are identified and the suggested basis is recorded, the reinforcement learning and evolutionary algorithm are applied to automatically explore and optimize the operation suggestions, and the continuous learning mechanism is used to silently absorb the real expert operation experience online, so that the parameters of the auxiliary operation model are iteratively updated in a non-destructive manner.

[0079] For example, the LIME algorithm is used to provide suggestion attribution analysis, the key steps in the operation process (such as the core step of the start-up process is to connect the air source) are identified, and the suggested basis of the auxiliary operation model is recorded; the reinforcement learning (such as PPO algorithm) and evolutionary algorithm are applied to automatically explore the optimal operation suggestion path (such as the shortest effective process of fault diagnosis); the real expert operation experience (such as the expert optimized fault elimination steps) is silently absorbed, and the parameters of the auxiliary operation model are iteratively updated through the non-destructive updating mechanism (such as incremental training), so as to avoid covering the historical effective data.

[0080] In addition, the trained auxiliary operation model is pruned, quantized and distilled, the inference delay is reduced, and the real-time response speed is improved.

[0081] Step S4, when the user operates the equipment, the auxiliary operation model is used to identify and infer the equipment scene and operation intention, filter potential risks, generate operation suggestions, and control the projection device to project the operation suggestions to the corresponding area of the equipment according to the positioning point.

[0082] Specifically, when the user operates the equipment, image data is captured in real time, and the operation instructions input by the user through natural language are analyzed, the auxiliary operation model is used to identify and infer the equipment scene and operation intention based on the image data and operation instructions, and the knowledge base is combined to generate preliminary operation suggestions.

[0083] In addition, the operation intention can be obtained based on the operation purpose explicitly informed by the user in natural language.

[0084] Then, the safety check constraint is constructed based on the high-risk operation data set; for example, ungrounded operation → prohibition of starting the device, and hyperparameter adjustment → triggering of early warning.

[0085] The preliminary operation suggestion is subjected to comparison and checking of the safety check constraint, and potential risk behaviors are filtered or corrected to obtain the operation suggestion.

[0086] For example, the suggestion containing the risk behavior is filtered (such as deleting the step of pressing the start button without grounding), and the suggestion with safety hazards is corrected (such as correcting the direct adjustment to 380V to the adjustment to the standard 220V).

[0087] Then, the auxiliary operation model calls the position coordinate code of the positioning point according to the operation object (such as panel 1-red start button) corresponding to the operation suggestion, calculates the optimal projection angle and range of the projection device, and adjusts the attitude of the cloud platform.

[0088] The control instruction is sent to the projection cloud platform to adjust the attitude of the cloud platform, so that the projection area of the operation suggestion and the actual operation area of the equipment coincide by ≥98%.

[0089] The highlight box and the text prompt are adopted, the highlight box accurately selects the target operation component, and the text prompt contains the step number, operation action, parameter requirement and safety prompt (such as step 1 / 3: press this button (3 seconds), start the device | safety prompt: it has been confirmed that the air source connection is normal).

[0090] The user operation is tracked in real time, if it is detected that the user has completed the current step (such as having pressed the start button), the auxiliary operation model immediately generates the next operation suggestion, and the projection is updated synchronously; if the user operation deviates from the operation suggestion, the auxiliary operation model re-reasons and adjusts the guidance (such as prompting operation error, please press the red start button of panel 1).

[0091] Embodiment 2

[0092] Please refer to Figure 2 , which is an equipment operation training auxiliary system provided by the embodiment 2 of the application, and the system comprises:

[0093] The data acquisition module 100 is used for arranging the positioning points for data acquisition and projection calibration on the equipment to be assisted in operation, and performing data acquisition.

[0094] The data processing module 200 is used for cleaning, preprocessing and classifying the collected data, and the data includes image data and text data.

[0095] The model learning module 300 is configured to automatically enhance the labeling of the classified data, establish the structured association between the image data and the text data, realize the migration and adaptation between different equipment scenes by using a small sample learning technology, complete the model training and learning, and generate an auxiliary operation model capable of operation recognition, logical reasoning and operation suggestion.

[0096] The reasoning assistance module 400 is configured to identify and reason the equipment scene and operation intention by using the auxiliary operation model when the user operates the equipment, filter potential risks, generate operation suggestions, and control a projection device to project the operation suggestions to the corresponding area of the equipment according to the positioning point.

[0097] Embodiment 3

[0098] Embodiment 3 of the present application provides a storage medium having computer instructions stored thereon, which, when executed by a processor, implement the steps of the method described in Embodiment 1.

[0099] Embodiment 4

[0100] Embodiment 4 of the present application provides a device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps of the method described in Embodiment 1 when executing the program.

[0101] The technical features of each of the above embodiments can be combined in any manner. To make the description concise, not all possible combinations of the technical features in the above embodiments are described, but as long as the combinations of the technical features do not contradict, they should be considered within the scope of the present application.

[0102] Those skilled in the art can understand that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a list of executable instructions for implementing the logic function, which can be specifically implemented in any computer readable storage medium for use by or in conjunction with an instruction execution system, device or apparatus, such as a computer-based system, a system including a processor or other system that can fetch and execute instructions from an instruction execution system, device or apparatus. For the present application, the "computer readable storage medium" can be any device that can contain, store, communicate, propagate or transport programs for use by or in conjunction with an instruction execution system, device or apparatus, or in conjunction with these instruction execution systems, devices or apparatus.

[0103] More specific examples (a non-exhaustive list) of the computer-readable storage medium include the following: an electrical connection having one or more wires (electrical, optical, and the like) a portable computer diskette (magnetic, or optical, e.g., Blu-ray® disk, etc.) a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, and a portable compact disc read-only memory (CDROM). Additionally, the computer readable storage medium can even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, for instance via an optical scanner, then compiled, interpreted, or otherwise processed, using an appropriate medium, into a computer program in a suitable language.

[0104] It should be understood that aspects of the application can be implemented in hardware, software, firmware or combinations thereof. In the embodiments described above, various steps or methods can be implemented, for example, by software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, and in another embodiment, any of the following techniques, which are well known in the art, can be used to implement the application: a hybrid of the techniques mentioned above; a combination of one or more of the techniques mentioned above; or one or more other techniques suitable for use in the computer-based systems described above.

[0105] In the description of the specification, the description of the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" and the like means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the application. In the specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any appropriate manner in one or more embodiments or examples.

[0106] The above-described embodiments only express several implementation manners of the present application, which are described in a more specific and detailed manner, but cannot be understood as a limitation on the patent scope of the present application. It should be noted that, for those skilled in the art, several modifications and improvements can be made without departing from the concept of the present application, which are all within the protection scope of the present application. Therefore, the patent protection scope of the present application should be subject to the appended claims.

Claims

1. An equipment operation training assistance method characterized by comprising: The method comprises: deploying positioning points of data acquisition and projection calibration on equipment to be assisted in operation to perform data acquisition; cleaning, preprocessing and classifying the collected data, including image data and text data, comprising: cleaning, preprocessing and classifying the image data and the text data respectively, cleaning the image data and the text data respectively based on an abnormality detection algorithm of reinforcement learning, the preprocessing and classification steps of the image data comprising: based on the position coordinates of the positioning points, taking the positioning points as the anchor points of the bounding box, performing effective region extraction on the image data to obtain effective image data, based on the distribution positions of the positioning points, performing region division on the effective image data to obtain image data, classifying the image data into action images and state images according to user operation characteristics, the preprocessing and classification steps of the text data comprising: the text data including operation manuals, fault libraries and dangerous operation warning data sources, and the operation manuals, the fault libraries and the dangerous operation warning data sources are parsed through a semantic recognition algorithm, based on the operation manuals and the fault libraries, extracting operation entities, step sequences, fault phenomena, solving actions and their logical relationships to construct a preliminary knowledge base, extracting features of the dangerous operation warning data sources to establish a high-risk operation data set, according to four dimensions of operation logic description, fixed step description, fault elimination logic and fault elimination step, classifying and labeling the contents in the preliminary knowledge base in detail, and associating with the high-risk operation data set; automatically enhancing the labeled data to establish a structured association between the image data and the text data, using small sample learning technology to realize migration and adaptation across equipment scenes, completing model training and learning, and generating an auxiliary operation model with operation recognition, logical reasoning and operation suggestion, comprising: using a pre-assisted operation model combined with an expert rule engine to preliminarily label the classified data, deeply encoding the text data through a knowledge graph and a large language model, combining the preliminary labeling results to establish a structured association between the image data and the text data, when a new equipment scene not covered by the data is encountered during training of the auxiliary operation model, analogical reasoning and knowledge retrieval are performed based on the existing equipment scenes to adapt the new equipment scene through small sample learning, obtaining an equipment scene library, based on the equipment scene library, a fusion architecture of Transformer is used to deeply integrate the image data and the text data, in the training process of the auxiliary operation model, an explainable AI technology is used to provide suggestion attribution analysis, identify the key steps in the operation process and record the basis for the suggestions, apply reinforcement learning and evolutionary algorithms to automatically explore and optimize the operation suggestions, and through a continuous learning mechanism, the experience of real expert operations is silently absorbed to iteratively update the parameters of the auxiliary operation model in a non-destructive manner; when a user operates the equipment, the auxiliary operation model identifies and reasons the equipment scene and the operation intention, filters potential risks, generates operation suggestions, and controls a projection device to project the operation suggestions to the corresponding area of the equipment according to the positioning points.

2. The equipment operation training assistance method according to claim 1, characterized by, The step of identifying and reasoning the equipment scene and operation intention through the auxiliary operation model when the user operates the equipment, specifically includes: Real-time capture of image data and parsing of operation instructions input by the user through natural language, identification and reasoning of the equipment scene and operation intention through the auxiliary operation model, generation of preliminary operation suggestions in combination with the knowledge base.

3. The equipment operation training assistance method according to claim 2, characterized by, And the step of filtering potential risks and generating operation suggestions, specifically includes: Constructing safety check constraints with high-risk operation data sets; Comparative checking of preliminary operation suggestions against safety check constraints, filtering or correcting potential risky behaviors, and obtaining operation suggestions.

4. An equipment operation training assistance system characterized by comprising: The system is used to perform the equipment operation training auxiliary method of any one of claims 1 to 3, and the system includes: A data acquisition module for laying out data acquisition and projection calibration positioning points on the equipment to be assisted in operation to perform data acquisition; A data processing module for cleaning, preprocessing and classifying the collected data, including image data and text data, including: Cleaning, preprocessing and classifying the image data and text data respectively, Cleaning the image data and text data respectively based on the reinforcement learning-based anomaly detection algorithm, The preprocessing and classification steps of the image data include: Based on the position coordinates of the positioning points, taking the positioning points as the boundary box anchor points, the effective area of the image data is extracted to obtain effective image data, According to the distribution position of the positioning points, the effective image data is regionally divided to obtain image data, According to the user operation characteristics, the image data is classified into action images and state images, The preprocessing and classification steps of the text data include: The text data includes operation manuals, fault libraries and dangerous operation warning data sources, and the operation manuals, fault libraries and dangerous operation warning data sources are parsed through a semantic recognition algorithm, Based on the operation manuals and fault libraries, operation entities, step sequences, fault phenomena, solving actions and their logical relationships are extracted to construct a preliminary knowledge base, Features of the dangerous operation warning data source are extracted to establish a high-risk operation data set, According to four dimensions of operation logic description, fixed step description, fault elimination logic and fault elimination steps, the contents in the preliminary knowledge base are finely classified and labeled, and are associated with the high-risk operation data set; A model learning module for automatically enhancing the labeled data after classification, establishing a structured association between image data and text data, realizing cross-equipment scene migration and adaptation using small sample learning technology, completing model training and learning, and generating an auxiliary operation model with operation recognition, logical reasoning and operation suggestion providing, including: Preliminary labeling of the classified data using the pre-assisted operation model in combination with an expert rule engine, Through the knowledge graph and the large language model, the text data is deeply semantically encoded, the structured association between the image data and the text data is established in combination with the preliminary labeling results, and the auxiliary operation model is generated. When the auxiliary operation model encounters a new equipment scene that is not covered by the data in the training, analogical reasoning and knowledge retrieval are performed based on the existing equipment scenes to adapt to the new equipment scene through small sample learning, and an equipment scene library is obtained, Based on the equipment scene library, a fusion architecture of Transformer is used to deeply integrate image data and text data, In the auxiliary operation model training process, an explainable AI technology is used to provide a suggestion attribution analysis, key steps in the operation process are identified and the basis for the suggestions is recorded, reinforcement learning and evolutionary algorithms are applied to automatically explore and optimize the operation suggestions, and a continuous learning mechanism is used to silently absorb real expert operation experience online to iteratively update the parameters of the auxiliary operation model in a non-destructive manner; The reasoning assistance module is configured to, when a user operates the equipment, identify and reason the equipment scene and operation intention through the auxiliary operation model, filter potential risks, generate operation suggestions, and control a projection device to project the operation suggestions to the corresponding area of the equipment according to the positioning points.

5. A readable storage medium, having stored thereon a computer program, characterized in that, The program, when executed by a processor, implements the steps of the equipment operation training assistance method according to any one of claims 1 to 3.

6. An electronic device, comprising: A computer program product includes a memory, a processor, and a computer program stored on the memory and executable on the processor, and the processor implements the steps of the equipment operation training assistance method according to any one of claims 1 to 3 when executing the program. A computer program product includes a memory, a processor, and a computer program stored on the memory and executable on the processor, and the processor implements the steps of the equipment operation training assistance method according to any one of claims 1 to 3 when executing the program.

Images