Device maintenance assistance method and apparatus, electronic device, and storage medium

Abstract

The application provides a device maintenance assistance method and device, an electronic device and a storage medium, and belongs to the technical field of device maintenance. The method comprises the following steps: determining current maintenance node information of a target device; performing maintenance object matching according to the current maintenance node information to obtain maintenance object information of the target device; collecting maintenance object operation video data corresponding to the maintenance object information by using a target virtual reality device; performing operation behavior analysis according to the maintenance object operation video data to obtain operation behavior information corresponding to the maintenance object information; performing maintenance compliance judgment according to the operation behavior information to obtain a maintenance compliance judgment result; and in response to the maintenance compliance judgment result not satisfying a preset maintenance compliance condition, outputting maintenance assistance guide information corresponding to the current maintenance node information by using the target virtual reality device. The application can effectively assist the device maintenance process in a real operation scene and reduce the maintenance assistance cost.

Description

Technical Field

[0001] This application relates to the field of equipment maintenance technology, and in particular to an auxiliary method and apparatus for equipment maintenance, electronic equipment and storage medium. Background Technology

[0002] In the field of equipment maintenance technology, new operators typically require on-site assistance from experienced mentors through observation, judgment, and verbal guidance to complete the maintenance of target equipment. For example, in the maintenance of replacing seals on a vacuum pump, which involves multiple maintenance points and the use of maintenance objects such as tool retrieval and part installation, new operators are highly prone to making mistakes in part installation or missing tools without the assistance of a mentor. However, this method of equipment maintenance assistance relies on the individual experience of the mentor, resulting in high maintenance assistance costs, low efficiency, and difficulty in scaling up.

[0003] Currently, some methods have been developed to improve upon these manual equipment maintenance assistance methods. These include using virtual reality (VR) devices to create virtual environments and provide maintenance guidance information to assist new operators in simulated equipment maintenance; or using augmented reality (AR) devices to overlay digital information onto real equipment to provide on-site guidance for new operators. However, these VR-based methods suffer from a disconnect between the virtual and real worlds, making it difficult to transfer operational skills to real maintenance scenarios and reducing the effectiveness of maintenance assistance. Augmented reality (AR) methods, on the other hand, rely on high-precision spatial positioning technology and expensive dedicated display hardware, increasing maintenance assistance costs. Therefore, how to effectively assist the equipment maintenance process in real-world operating scenarios while reducing maintenance assistance costs is a pressing technical problem that the industry needs to solve. Summary of the Invention

[0004] The main objective of this application is to provide a method, apparatus, electronic device, and storage medium for assisting equipment maintenance, aiming to effectively assist the equipment maintenance process in real-world operating scenarios and reduce maintenance assistance costs.

[0005] To achieve the above objectives, a first aspect of this application provides an auxiliary method for equipment maintenance, the method comprising: Determine the current maintenance status of the target equipment; Based on the current maintenance node information, the maintenance items are matched to obtain the maintenance item information of the target equipment; The target virtual reality device is used to collect video data of the repair work corresponding to the repair work information; Based on the video data of the repaired object, the operation behavior is analyzed to obtain the operation behavior information corresponding to the repaired object information; Based on the work behavior information, a maintenance compliance judgment is made to obtain the maintenance compliance judgment result; In response to the fact that the maintenance compliance judgment result does not meet the preset maintenance compliance conditions, the target virtual reality device outputs maintenance assistance guidance information corresponding to the current maintenance node information.

[0006] In some embodiments, the repair object information includes repair tool information, and the repair object operation video data includes tool operation video data corresponding to the repair tool information. The step of performing operation behavior analysis based on the repair object operation video data to obtain operation behavior information corresponding to the repair object information includes: The tool operation sequence is analyzed based on the tool operation video data to obtain tool operation sequence information; Based on the tool operation video data, the tool operation duration is analyzed to obtain tool operation duration information; The operation behavior information corresponding to the repaired object information is determined based on the tool operation sequence information and the tool operation duration information.

[0007] In some embodiments, determining the work behavior information corresponding to the repair object information based on the tool operation sequence information and the tool operation duration information includes: Based on the tool operation video data, analyze the tool operation actions to obtain tool operation action information; The work behavior information corresponding to the repair object information is determined based on the tool operation action information, the tool operation sequence information, and the tool operation duration information.

[0008] In some embodiments, the repair object information includes repair part information, and the repair object operation video data includes part installation video data corresponding to the repair part information. The step of performing operation behavior analysis based on the repair object operation video data to obtain operation behavior information corresponding to the repair object information includes: Based on the component installation video data, the component installation sequence is analyzed to obtain component installation sequence information; Based on the video data of the component installation, the component installation position is analyzed to obtain the component installation position information; The operation behavior information corresponding to the maintenance object information is determined based on the component installation sequence information and the component installation location information.

[0009] In some embodiments, determining the current maintenance node information of the target device includes: The target virtual reality device is used to acquire images of the maintenance scene corresponding to the target device; Based on the maintenance scene image, maintenance node detection is performed to obtain the current maintenance node information of the target device.

[0010] In some embodiments, the step of detecting maintenance nodes based on the maintenance scene image to obtain the current maintenance node information of the target device includes: Based on the maintenance scene image, the main body status of the equipment is detected to obtain the main body status information of the equipment; Based on the maintenance scene image, the equipment component status is detected to obtain the equipment component status information; The current maintenance node information of the target device is determined based on the device body status information and the device component status information.

[0011] In some embodiments, after outputting maintenance assistance guidance information corresponding to the current maintenance node information using the target virtual reality device in response to the maintenance compliance judgment result not meeting the preset maintenance compliance conditions, the method further includes: Obtain a repair operation query instruction for the information of the repaired item; Based on the maintenance operation query command and the maintenance auxiliary guidance information, the query intent is parsed to obtain maintenance operation guidance information; The maintenance operation guidance information is output using the target virtual reality device.

[0012] To achieve the above objectives, a second aspect of this application provides an equipment maintenance assistance device applied to a target virtual reality device, the device comprising: The maintenance node determination unit is used to determine the current maintenance node information of the target equipment; A maintenance part matching unit is used to match maintenance parts according to the current maintenance node information to obtain maintenance part information of the target equipment. A maintenance video acquisition unit is used to acquire maintenance operation video data corresponding to the maintenance object information using the target virtual reality device; The maintenance behavior analysis unit is used to perform operation behavior analysis based on the operation video data of the maintenance object to obtain the operation behavior information corresponding to the maintenance object information. The maintenance compliance judgment unit is used to judge the maintenance compliance based on the work behavior information and obtain the maintenance compliance judgment result; The maintenance assistance unit, in response to the maintenance compliance judgment result not meeting the preset maintenance compliance conditions, uses the target virtual reality device to output maintenance assistance guidance information corresponding to the current maintenance node information.

[0013] To achieve the above objectives, a third aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method described in the first aspect.

[0014] To achieve the above objectives, a fourth aspect of the present application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in the first aspect.

[0015] This embodiment of the application determines the current maintenance node information of the target equipment, then matches the maintenance objects based on the current maintenance node information to obtain the maintenance object information of the target equipment. Next, it uses the target virtual reality device to collect video data of the maintenance object operation corresponding to the maintenance object information. Then, it analyzes the operation behavior based on the maintenance object operation video data to obtain the operation behavior information corresponding to the maintenance object information. Further, it judges the maintenance compliance based on the operation behavior information to obtain the maintenance compliance judgment result. Finally, in response to the maintenance compliance judgment result not meeting the preset maintenance compliance conditions, it uses the target virtual reality device to output maintenance assistance guidance information corresponding to the current maintenance node information. Thus, this embodiment of the application analyzes the operation behavior of maintenance object video data to perceive the operator's operation behavior with maintenance tools and / or parts in real time, and judges the compliance of the maintenance process based on the operation behavior information. When the operation is non-compliant, it actively outputs maintenance assistance guidance information, thereby providing effective equipment maintenance assistance to operators in real operation scenarios. At the same time, by using low-cost virtual reality devices to replace expensive dedicated augmented reality hardware, it effectively reduces maintenance assistance costs. In other words, this embodiment of the application can effectively assist the equipment maintenance process in real operation scenarios and reduce maintenance assistance costs. Attached Figure Description

[0016] Figure 1 This is a flowchart of the equipment maintenance auxiliary method provided in the embodiments of this application; Figure 2 yes Figure 1 The flowchart of step S104 in the process; Figure 3 yes Figure 2 The flowchart of step S203 in the process; Figure 4 yes Figure 1 Another flowchart of step S104 in the process; Figure 5 yes Figure 1 The flowchart of step S101 in the text; Figure 6 yes Figure 5The flowchart of step S502 in the document; Figure 7 yes Figure 1 The flowchart of the steps following step S106; Figure 8 This is a schematic diagram of the equipment maintenance auxiliary device provided in the embodiments of this application; Figure 9 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0018] It should be noted that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, and the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0019] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0020] First, let's analyze some of the terms used in this application: Virtual Reality (VR) devices refer to devices that use computer technology to construct a completely virtual three-dimensional environment, allowing operators to immerse themselves in and interact with the virtual environment. For example, a VR device can be a head-mounted display that combines spatial positioning and controller interaction to output maintenance assistance and guidance information in the virtual environment, assisting operators in simulating equipment maintenance operations.

[0021] Maintenance assistance guidance information refers to information output by virtual reality devices during equipment maintenance to assist operators in completing corresponding maintenance steps. For example, maintenance assistance guidance information can be visually highlighted prompts, prominently displaying the maintenance tools or parts involved in the corresponding maintenance step and the corresponding operating sequence within the operator's field of vision; alternatively, it can be 3D animation demonstrations, dynamically showcasing the actual operating procedures for the corresponding maintenance step; or it can be voice prompts, broadcasting the operating instructions or precautions for the corresponding maintenance step. Understandably, the specific form of maintenance assistance guidance information can be adjusted according to the corresponding maintenance step and actual needs.

[0022] Maintenance nodes: These refer to specific operational steps within the equipment maintenance process. For example, in replacing seals on a vacuum pump, maintenance nodes could include "removing the equipment housing," "removing the elbow pipe," "removing the pump housing," "removing the claw rotor," "wiping and polishing the pump chamber and rotor," "removing the bushing and sealing ring," "installing the sealing ring," "installing the pump housing," "installing the bushing," "installing the metal ring," "installing the claw rotor," "testing the gap," and "installing the equipment housing." It's understood that equipment maintenance processes typically consist of multiple sequentially executed maintenance nodes, covering the entire process from disassembling and replacing maintenance parts to installing them. Each maintenance node corresponds to a specific operational stage in the maintenance process; after a maintenance part is disassembled at one maintenance node, it will be reinstalled at a subsequent maintenance node.

[0023] Augmented Reality (AR) devices refer to devices that overlay computer-generated virtual information (such as images, text, and 3D models) onto the real physical environment in real time, allowing operators to obtain additional information while maintaining their awareness of the real world. For example, AR devices can be head-mounted augmented reality glasses or handheld tablets used to assist operators in providing on-site guidance for equipment maintenance in real-world operating scenarios.

[0024] In the field of equipment maintenance technology, the quality of equipment maintenance directly affects production safety, equipment lifespan, and operational efficiency. The standardization, accuracy, and timeliness of maintenance operations are crucial to ensuring maintenance quality. During equipment maintenance, the proper use of maintenance tools and the correct installation of repair parts directly impact the maintenance outcome. Therefore, ensuring that operators can complete equipment maintenance in a standardized and accurate manner is a key issue that requires focused attention in equipment operation and maintenance management.

[0025] Currently, new operators typically require on-site assistance from experienced mentors through observation, judgment, and verbal guidance to complete the maintenance of target equipment. For example, in the maintenance of replacing seals on a vacuum pump, which involves multiple maintenance steps and the use of various tools and components, such as tool retrieval and part installation, new operators are highly prone to making mistakes in part installation or missing tools without the assistance of a mentor. However, this method of equipment maintenance assistance relies on the individual experience of the mentor, resulting in high maintenance costs, low efficiency, and difficulty in scaling up.

[0026] Several methods have been developed to improve upon traditional manual equipment maintenance assistance. These methods utilize virtual reality (VR) devices to create virtual environments and provide maintenance guidance, enabling new operators to learn operational procedures and precautions in a virtual environment, thus reducing error rates and safety hazards in real-world operations. Alternatively, augmented reality (AR) devices can overlay digital information onto real equipment to provide on-site guidance, offering real-time operational instructions and location prompts within a real-world scenario, reducing reliance on human intervention. However, these VR-based methods keep new operators in a completely virtual environment, unable to access real equipment, tools, and parts. This makes it difficult to transfer skills learned in the virtual environment to real-world scenarios, creating a disconnect between the virtual and real worlds and reducing the effectiveness of maintenance assistance. Furthermore, these AR-based methods rely on high-precision spatial positioning technology and dedicated display hardware, resulting in high hardware costs and increased maintenance assistance costs. Therefore, this application provides an equipment maintenance assistance method, apparatus, electronic device, and storage medium, aiming to effectively assist the equipment maintenance process in real-world scenarios and reduce maintenance assistance costs.

[0027] The equipment maintenance assistance method provided in this application relates to the field of equipment maintenance technology. This method can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, etc.; the server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application implementing the equipment maintenance assistance method, but is not limited to the above forms.

[0028] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0029] Figure 1 This is an optional flowchart of the equipment maintenance auxiliary method provided in the embodiments of this application. Figure 1 The method described above, when applied to a target virtual reality device, may include, but is not limited to, steps S101 to S106: Step S101: Determine the current maintenance node information of the target equipment; Step S102: Match the repair items according to the current repair node information to obtain the repair item information of the target equipment; Step S103: Use the target virtual reality device to collect video data of the repair work corresponding to the repair work information; Step S104: Perform work behavior analysis based on the video data of the repaired object to obtain the work behavior information corresponding to the repaired object information; Step S105: Based on the work behavior information, determine the maintenance compliance and obtain the maintenance compliance determination result; Step S106: In response to the fact that the maintenance compliance judgment result does not meet the preset maintenance compliance conditions, maintenance assistance guidance information corresponding to the current maintenance node information is output using the target virtual reality device.

[0030] Steps S101 to S106 as shown in this embodiment of the application involve determining the current maintenance node information of the target device, matching maintenance objects based on the current maintenance node information to obtain maintenance object information for the target device, then using the target virtual reality device to collect maintenance object operation video data corresponding to the maintenance object information, then performing operation behavior analysis based on the maintenance object operation video data to obtain operation behavior information corresponding to the maintenance object information, further performing maintenance compliance judgment based on the operation behavior information to obtain maintenance compliance judgment result, and finally, in response to the maintenance compliance judgment result not meeting the preset maintenance compliance conditions, using the target virtual reality device to output maintenance auxiliary guidance information corresponding to the current maintenance node information. Thus, this embodiment of the application analyzes the operational behavior of video data of repaired objects, perceives the operator's operational behavior with repair tools and / or repair parts in real time, and judges the compliance of the repair process based on the operational behavior information. When the operation is non-compliant, it actively outputs auxiliary guidance information, thereby providing effective equipment repair assistance to operators in real operation scenarios. At the same time, by using low-cost virtual reality equipment to replace expensive dedicated augmented reality hardware, the repair assistance cost is effectively reduced. That is, this embodiment of the application can effectively assist the equipment repair process in real operation scenarios and reduce the repair assistance cost.

[0031] In step S101 of some embodiments, the target virtual reality device may refer to a virtual reality device that has image acquisition and spatial positioning functions and is capable of outputting maintenance assistance guidance information. It should be noted that, in order not to affect the operator's maintenance operations in the real environment, when the maintenance assistance guidance information is a visually highlighted prompt or a 3D animation demonstration, the maintenance assistance guidance information can be set to the side or edge of the operator's field of vision, so that the operator can view the guidance content by turning their head when needed, while their field of vision remains uninterrupted during normal operation.

[0032] The target equipment can refer to physical equipment that requires maintenance. For example, the target equipment could be a vacuum pump that needs seal replacement, or a centrifugal compressor that needs impeller replacement. It is understood that the specific type of target equipment is not limited, but the maintenance of the target equipment typically follows a preset standard operating procedure. The current maintenance node information can refer to the identification information corresponding to the specific operation stage currently being performed during equipment maintenance. For example, if the target equipment is a vacuum pump that needs seal replacement, the current maintenance node information could be "removing the claw-type rotor"; or, if the target equipment is a centrifugal compressor that needs impeller replacement, the current maintenance node information could be "disassembling the impeller". It should be noted that in this application embodiment, the current maintenance node information can be directly determined by the operator or instructor based on maintenance manuals or experience and input into the target virtual reality device; alternatively, the target virtual reality device can be used to capture images corresponding to the target equipment and analyze the images to determine the current maintenance node information. The specific method for determining the current maintenance node information can be adjusted according to actual needs.

[0033] In step S102 of some embodiments, repair object matching can refer to the process of determining the repair objects involved in the current repair node based on the pre-established mapping relationship between repair node information and repair objects. Repair object information can refer to information obtained after repair object matching, used to indicate the repair tools and / or repair parts involved in the current repair node. For example, in the repair of replacing seals on a vacuum pump, if the current repair node information is "cleaning the chamber," the repair object information may include information on repair tools such as lint-free cloths and sponge sandpaper; if the current repair node information is "installing a sealing ring," the repair object information may include information on repair parts such as sealing rings, and information on repair tools such as rubber hammers and special sockets; if the current repair node information is "removing a claw-type rotor," the repair object information may include information on repair parts such as claw-type rotors, and information on repair tools such as claw-type rotor disassembly fixtures, Allen screwdrivers, double-ended wrenches, and markers; or, if the current repair node information is "testing gaps," the repair object information may include information on repair tools such as feeler gauges. It is understandable that if the current maintenance node involves the disassembly of maintenance parts, the maintenance item information also includes information on the maintenance parts to be disassembled.

[0034] In step S103 of some embodiments, the repair object operation video data can refer to video data obtained by the target virtual reality device through continuous image acquisition of the entire process from picking up, operating to placing the repair object involved in the current repair node. For example, if the current repair node information is "installing a sealing ring", then the repair object operation video data can be video data obtained by continuous image acquisition of the complete process from picking up, operating to placing the repair object such as a sealing ring, rubber hammer, and special sleeve; if the current repair node information is "cleaning the cavity", then the repair object operation video data can be video data obtained by continuous image acquisition of the complete process from picking up, operating to placing the repair tool such as a lint-free cloth and sponge sandpaper. It should be noted that, in this embodiment of the application, the tool placement area and part placement area pre-defined by the built-in camera of the target virtual reality device, combined with spatial positioning technology, can be used to perform regional positioning and dynamic tracking of the repair object, thereby realizing video data collection of the entire process of the repair object from picking up, operating to placing. The tool placement area and part placement area can be specific location ranges of repair tools and repair parts in real space that are pre-set in the target virtual reality device using spatial positioning technology; or, the built-in camera of the target virtual reality device can be used in combination with target detection and tracking algorithms to continuously identify and track the repair object itself, thereby realizing video data collection of the entire process of the repair object from picking up, operating to placing. The specific method of collecting video data of the repair object operation can be adjusted according to actual needs.

[0035] In step S104 of some embodiments, job behavior analysis can refer to the process of extracting and analyzing features from video data of repair work to detect the behavioral characteristics of the repair workpiece during handling, operation, or placement. Job behavior information can refer to information obtained after job behavior analysis that indicates the behavioral characteristics of the repair workpiece. For example, in the repair of replacing seals on a vacuum pump, if the current repair node only involves the operation of repair tools, job behavior analysis can detect the use of repair tools to obtain job behavior information indicating the use of tools; if the current repair node only involves the installation of repair parts, job behavior analysis can detect the installation of repair parts to obtain job behavior information indicating the installation of repair parts; if the current repair node involves both repair tools and repair parts, job behavior analysis can simultaneously detect the use of repair tools and the installation of repair parts to obtain comprehensive job behavior information. It is understood that the specific content of job behavior analysis can be adjusted according to the type of repair workpiece involved in the current repair node.

[0036] In step S105 of some embodiments, the maintenance compliance judgment can refer to the process of determining whether the operation of the current maintenance node is compliant based on the work behavior information. The maintenance compliance judgment result can refer to the result information obtained after the maintenance compliance judgment, used to characterize whether the operation of the current maintenance node is compliant. For example, the maintenance compliance judgment result can be "compliant" or "non-compliant" to characterize whether the operation is compliant; or the maintenance compliance judgment result can be "1" or "0" to characterize whether the operation is compliant. The specific expression of the maintenance compliance judgment result can be adjusted according to actual needs.

[0037] In step S106 of some embodiments, the preset maintenance compliance condition may refer to a pre-set judgment condition used to determine whether the maintenance compliance judgment result meets the compliance requirements. For example, the preset maintenance compliance condition may be that the maintenance compliance judgment result is "compliant," which represents operational compliance. If the maintenance compliance judgment result is also "compliant," then it is determined that the maintenance compliance judgment result meets the preset maintenance compliance condition. Alternatively, the preset maintenance compliance condition may be that the maintenance compliance judgment result is "1," which represents operational compliance. If the maintenance compliance judgment result is also "1," then it is determined that the maintenance compliance judgment result meets the preset maintenance compliance condition. In response to the maintenance compliance judgment result not meeting the preset maintenance compliance condition, it indicates that there is an error or omission in the operation of the current maintenance node, and it is necessary to guide the operator to correct the operation by outputting auxiliary information. This can be achieved by using the target virtual reality device to output maintenance auxiliary guidance information corresponding to the current maintenance node information. For example, when the maintenance compliance assessment indicates that the order in which maintenance tools are retrieved is incorrect at the current maintenance node, the maintenance assistance guidance information can be a visual highlight, prominently displaying the correct maintenance tool in the operator's field of vision. When the maintenance compliance assessment indicates that the installation position of a maintenance part is incorrect at the current maintenance node, the maintenance assistance guidance information can be a 3D animation demonstrating the correct installation method for the maintenance part. If the maintenance compliance assessment result meets the preset maintenance compliance conditions, it means that the operation at the current maintenance node is compliant, and no maintenance assistance guidance information needs to be output.

[0038] Please see Figure 2 In some embodiments, the repair object information includes repair tool information, and the repair object operation video data includes tool operation video data corresponding to the repair tool information. Step S104 may include, but is not limited to, steps S201 to S203: Step S201: Analyze the tool operation sequence based on the tool operation video data to obtain tool operation sequence information; Step S202: Analyze the tool operation duration based on the tool operation video data to obtain tool operation duration information; Step S203: Determine the work behavior information corresponding to the repair object information based on the tool operation sequence information and tool operation duration information.

[0039] In step S201 of some embodiments, the maintenance tool information may refer to the information of the maintenance tools required for the current maintenance node. Tool operation video data may refer to video data obtained by continuously capturing images of the entire process of picking up, operating, and placing the maintenance tool. For example, if the current maintenance node information is "cleaning the cavity," the maintenance tool information may include information about tools such as lint-free cloths and sandpaper, and the tool operation video data may be video data obtained by continuously capturing images of the entire process of picking up, operating, and placing the lint-free cloths and sandpaper; or, if the current maintenance node information is "testing the gap," the maintenance tool information may include information about tools such as feeler gauges, and the tool operation video data may be video data obtained by continuously capturing images of the entire process of picking up, operating, and placing the feeler gauge. It should be noted that in the embodiments of this application, the current maintenance node information only involves maintenance tool operation.

[0040] Tool operation sequence analysis refers to the process of processing video data of tool operations to detect the order in which repair tools are picked up and placed. Tool operation sequence information refers to the information obtained after tool operation sequence analysis, used to indicate the order in which repair tools are picked up and placed. For example, in this embodiment of the application, after determining the repair tool information, a virtual tool placement area can be projected onto a real tool table using a target virtual reality device. The repair tools corresponding to the repair tool information are then pre-placed within the virtual tool placement area. Next, the tool placement area status is detected based on tool operation video data to obtain information on changes in the repair tool's presence status. Based on this information, retrieval and placement events are identified to obtain a sequence of retrieval events and a sequence of placement events. The retrieval order and placement order of the repair tools are determined based on these sequences. Finally, the tool operation sequence information is determined based on the retrieval order and placement order. Here, the information on changes in the repair tool's presence status can refer to the change in the repair tool's presence in the tool placement area from existing to non-existent or from non-existent to existing. The retrieval event sequence can refer to a time-series record of repair tool retrieval events. The placement event sequence can refer to a time-series record of repair tools being placed back into the tool placement area. Understandably, if there are multiple repair tools corresponding to the repair tool information, different repair tools can be pre-placed in different tool placement areas. Each tool placement area corresponds to a tool type label. During tool placement area status detection, the status of each tool placement area is detected independently, and the corresponding tool type label is attached to the status change information obtained from the tool placement area status detection. Then, during the identification of retrieval and placement events, retrieval and placement events are identified based on the status change information with tool type labels, resulting in a retrieval event sequence and a placement event sequence for each repair tool with tool type labels. Next, the retrieval event sequences with tool type labels for each repair tool are integrated into a whole repair tool retrieval event sequence, and the placement event sequences with tool type labels for each repair tool are integrated into a whole repair tool placement event sequence. Finally, the repair tool retrieval order and repair tool placement order are determined based on the integrated repair tool retrieval event sequence and repair tool placement event sequence, and the tool operation sequence information is determined based on the repair tool retrieval order and repair tool placement order. Here, the tool type label can refer to identification information used to distinguish different repair tool types. For example, the tool type label could be "adjustable wrench" or "screwdriver," etc., and the specific one can be determined based on the corresponding repair tool information.

[0041] It should be noted that, in this embodiment, a virtual reality device projects a virtual tool placement area onto a real tool table. This can be achieved by acquiring a continuous sequence of images corresponding to the real tool table using the target virtual reality device, then extracting environmental feature points from the image sequence using a simultaneous localization and mapping (SLAM) algorithm. A sparse environment map is then constructed based on these extracted feature points. Next, feature matching is performed between the sparse environment map and a pre-stored environment template containing specific markers (such as a QR code or feature pattern on the real tool table) to calculate the six-degrees-of-freedom pose of the target virtual reality device relative to the real world. Finally, the target virtual reality device overlays and renders the boundaries and icons of the virtual tool placement area onto the real object based on this six-degrees-of-freedom pose, allowing the operator to observe the virtual tool placement area projected onto the real tool table through the target virtual reality device. Furthermore, if the operator's work behavior information related to the repair tools does not meet preset repair compliance conditions, the target virtual reality device can highlight the correct tool placement area, indicate incorrect tool usage, or play tool usage instructions.

[0042] Alternatively, embodiments of this application can also continuously identify and track the repair tools using target detection and tracking algorithms. This involves detecting the repair tools based on tool operation video data to obtain repair tool identification information; tracking the repair tools based on the identification information to obtain their motion trajectory information; identifying retrieval and placement events based on the motion trajectory information to obtain a sequence of retrieval and placement events; and determining the retrieval order and placement order based on these sequences to obtain tool operation sequence information. Here, the repair tool identification information can refer to identity information used to identify and distinguish different repair tools, and the repair tool motion trajectory information can refer to the position sequence information of the repair tools as they change over time during retrieval, operation, and placement. It is understood that the specific implementation of the tool operation sequence analysis in this application embodiment can be adjusted according to actual needs.

[0043] In steps S202 to S203 of some embodiments, tool operation duration analysis can refer to the process of processing tool operation video data and detecting the time interval from tool retrieval to placement. Tool operation duration information can refer to information obtained after tool operation duration analysis, used to indicate the usage duration of the repair tool. For example, in embodiments of this application, after determining the repair tool information, the corresponding repair tool can be pre-placed in a pre-marked tool placement area. The tool placement area status can be detected based on the tool operation video data to obtain information on changes in the existence status of the repair tool. Retrieval events and placement events can be identified based on the information on changes in the existence status of the repair tool to obtain a sequence of repair tool retrieval events and a sequence of repair tool placement events. The time difference can be calculated based on the retrieval timestamp in the repair tool retrieval event sequence and the placement timestamp in the corresponding repair tool placement event sequence to obtain the tool operation duration information. The retrieval event sequence can refer to a sequence of events recorded in chronological order in which the repair tool is retrieved, with each retrieval event corresponding to a retrieval timestamp; the placement event sequence can refer to a sequence of events recorded in chronological order in which the repair tool is placed back into the tool placement area, with each placement event corresponding to a placement timestamp; the retrieval timestamp and the placement timestamp correspond one-to-one in the order of event occurrence. Alternatively, embodiments of this application can also perform hand keypoint detection based on tool operation video data to obtain a hand keypoint sequence; perform repair tool target detection based on tool operation video data to obtain repair tool location information; perform retrieval status recognition based on the hand keypoint sequence and repair tool location information to obtain repair tool retrieval status information; determine the retrieval time point and placement time point based on the repair tool retrieval status information to obtain repair tool retrieval time point information and repair tool placement time point information; calculate the time difference based on the repair tool retrieval time point information and repair tool placement time point information to obtain tool operation duration information. Among them, the hand key point sequence can refer to the position information sequence of hand joints in the image, the repair tool position information can refer to the position information of the repair tool in the image, and the retrieval status information can refer to the status information used to indicate whether the repair tool has been retrieved by the hand. It is understood that the specific implementation method of tool operation time analysis in this application embodiment can be adjusted according to actual needs. Work behavior information can refer to behavioral feature information that integrates tool operation sequence information and tool operation time information.

[0044] It is understood that this application embodiment obtains tool operation sequence information by analyzing the tool operation video data, and obtains tool operation duration information by analyzing the tool operation video data. The operation sequence information and the operation duration information are then used together to determine the work behavior information. In this way, the order and duration of tool retrieval and placement throughout the entire process can be detected from both sequence and duration dimensions. The detection results from these two dimensions are then fused to accurately detect tool retrieval and placement behavior, providing more accurate data support for subsequent maintenance compliance assessments.

[0045] Please see Figure 3 In some embodiments, step S203 may include, but is not limited to, steps S301 to S302: Step S301: Analyze the tool operation actions based on the tool operation video data to obtain tool operation action information; Step S302: Determine the work behavior information corresponding to the repair object information based on the tool operation action information, tool operation sequence information, and tool operation duration information.

[0046] In steps S301 to S302 of some embodiments, tool operation motion analysis can refer to the process of processing tool operation video data to detect the motion characteristics of the maintenance tool during use. Tool operation motion information can refer to information obtained after tool operation motion analysis, used to indicate the operation motion characteristics of the maintenance tool. For example, embodiments of this application can perform target detection of the repair tool based on tool operation video data to obtain the repair tool's position information; perform posture estimation of the repair tool based on the repair tool's position information to obtain the repair tool's posture information; perform key component detection of the target equipment based on the tool operation video data to obtain the position information of the key components of the equipment; perform relative pose analysis based on the repair tool's posture information and the position information of the key components of the equipment to obtain the relative pose information between the tool and the equipment; and perform operation action recognition based on the relative pose information to obtain tool operation action information. The correspondence between the repair tool information and the key components of the equipment is pre-established and used to indicate the component to be operated corresponding to each repair tool. The repair tool's position information can refer to the spatial position information of the repair tool in the image; the repair tool's posture information can refer to the orientation and angle information of the repair tool in space; the key component's position information can refer to the spatial position information of the key component to be operated on the target equipment in the image; the relative pose information can refer to the relative position and relative angle information between the repair tool and the key components of the equipment; and the tool operation action information can refer to the positional offset and angular offset of the repair tool relative to the key components of the equipment. Alternatively, embodiments of this application can perform hand keypoint detection based on tool operation video data to obtain a hand keypoint sequence; perform repair tool target detection based on the tool operation video data to obtain repair tool position information; perform gripping state analysis based on the hand keypoint sequence and repair tool position information to obtain tool gripping state information; after determining that the repair tool is in use based on the tool gripping state information, perform hand motion trajectory tracking based on the tool operation video data to obtain hand motion trajectory information; and perform operation action recognition based on the hand motion trajectory information to obtain tool operation action information. Here, the hand keypoint sequence can refer to the position information sequence of hand joints in the image; the repair tool position information can refer to the spatial position information of the repair tool in the image; the tool gripping state information can refer to information indicating whether the hand is holding the repair tool and the gripping method; the hand motion trajectory information can refer to the movement path information of the hand during operation; and the tool operation action information can refer to the starting point position, ending point position, movement path length, and change in movement direction of the hand motion trajectory. It is understood that the specific implementation method of tool operation action analysis in embodiments of this application can be adjusted according to actual needs. Job behavior information can refer to behavioral characteristic information that integrates tool operation sequence information, tool operation duration information, and tool operation action information.

[0047] It is understood that the embodiments of this application analyze tool operation actions based on tool operation video data to obtain tool operation action information, and jointly determine work behavior information based on tool operation action information, tool operation sequence information, and tool operation duration information. In this way, it is possible to detect the sequence, usage duration, and operation actions of maintenance tools throughout the entire process of picking up, placing, and using them from three dimensions: sequence, duration, and action. The detection results from these three dimensions are then fused to achieve accurate detection of maintenance tool operation behavior, providing more accurate data support for subsequent maintenance compliance judgments.

[0048] Please see Figure 4 In some embodiments, the repair object information includes repair part information, and the repair object operation video data includes the part installation video data corresponding to the repair part information. Step S104 may include, but is not limited to, steps S401 to S403: Step S401: Analyze the component installation sequence based on the component installation video data to obtain component installation sequence information; Step S402: Analyze the installation position of the parts based on the video data of the parts installation to obtain the installation position information of the parts; Step S403: Determine the operation behavior information corresponding to the maintenance object information based on the part installation sequence information and the part installation location information.

[0049] In step S401 of some embodiments, the repair part information may refer to the information of the repair parts required for the current repair node. The part installation video data may refer to video data obtained by continuously capturing images of the entire process of the repair part from picking up, installing, to placing. For example, in the maintenance process of a vacuum pump, if the current repair node information is "installing a metal ring," then the repair part information may include information about parts such as the metal ring, and the part installation video data may be video data obtained by continuously capturing images of the complete process of the metal ring from picking up, installing, to placing; or, if the current repair node information is "inspecting a sealing ring," then the repair part information may include information about parts such as the sealing ring, and the part installation video data may be video data obtained by continuously capturing images of the complete process of the sealing ring from picking up, inspecting, to placing. It should be noted that in the embodiments of this application, the current repair node information only involves the operation of repair parts.

[0050] Parts installation sequence analysis refers to the process of processing video data on parts installation to detect the order in which parts are picked up and placed. Parts installation sequence information refers to the information obtained after parts installation sequence analysis, used to indicate the order in which parts are picked up and placed. It should be noted that the specific implementation principle of parts installation sequence analysis can be the same as that of tool operation sequence analysis; that is, it can also obtain the sequence of parts picking up and placing events by performing state detection on a pre-defined parts placement area; or, it can obtain the sequence of parts picking up and placing events by continuously identifying and tracking parts using object detection and tracking algorithms.

[0051] In steps S402 to S403 of some embodiments, the component installation position analysis can refer to the process of processing component installation video data to detect the spatial position of the component relative to the target device during installation. Component installation position information can refer to information obtained after component installation position analysis, used to indicate the actual installation position of the component. For example, embodiments of this application can perform component target detection based on component installation video data to obtain component position information; perform device installation area detection based on component installation video data to obtain device installation area information; and calculate the position deviation based on the component position information and device installation area information to obtain component installation position information. Wherein, component position information can refer to the spatial position information of the component in the image; device installation area information can refer to the spatial range information of the preset installation area on the target device in the image; and component installation position information can refer to the position deviation value between the actual position of the component and the device installation area. Alternatively, embodiments of this application can also perform component contour detection based on component installation video data to obtain component contour information; perform device installation feature detection based on component installation video data to obtain device installation feature information; and perform feature matching based on the component contour information and device installation feature information to obtain component installation position information. Among them, part contour information can refer to the edge feature information of the part; equipment installation feature information can refer to the geometric feature information of the preset installation position on the target equipment; and part installation position information can refer to the matching degree information between the part contour and the equipment installation features. It is understood that the specific implementation method of part installation position analysis in this application embodiment can be adjusted according to actual needs. Operation behavior information can refer to information determined jointly by part installation sequence information and part installation position information, used to indicate the characteristics of part installation behavior.

[0052] It is understood that the embodiments of this application obtain part installation sequence information by analyzing the part installation sequence based on the part installation video data, and obtain part installation position information by analyzing the part installation position based on the part installation video data. The part installation sequence information and the part installation position information are then used together to determine the work behavior information. In this way, the order and installation position deviation of the repair parts in the entire process of picking, installing, and placing can be detected from both the sequence and position dimensions. The detection results from the two dimensions are then fused to achieve accurate detection of the repair part installation behavior, thereby providing more accurate data support for subsequent repair compliance judgment.

[0053] In some embodiments, the repair object information includes repair tool information and repair part information, and the repair object operation video data includes tool operation video data corresponding to the repair tool information and part installation video data corresponding to the repair part information. Operation behavior analysis is performed based on the repair object operation video data to obtain operation behavior information corresponding to the repair object information, including: The tool operation sequence is analyzed based on the tool operation video data to obtain tool operation sequence information; The tool operation duration is analyzed based on the tool operation video data to obtain tool operation duration information; The component installation sequence information is obtained by analyzing the component installation video data. The installation location of the parts is analyzed based on the video data of the parts installation to obtain the installation location information of the parts; The corresponding work behavior information for the repaired object is determined based on the tool operation sequence information, tool operation duration information, part installation sequence information, and part installation location information.

[0054] In this embodiment, the repair object information can refer to the collection of information on repair tools and repair parts involved in the current repair node. Repair object operation video data can refer to a collection of video data including tool operation video data obtained by continuously capturing images of the entire process of taking, operating, and placing repair tools, and part installation video data obtained by continuously capturing images of the entire process of taking, installing, and placing repair parts. Operation behavior information can refer to information obtained by first determining tool-side behavior characteristics based on tool operation sequence information and tool operation duration information, then determining part-side behavior characteristics based on part installation sequence information and part installation position information, and finally merging the tool-side and part-side behavior characteristics to obtain information indicating the comprehensive behavior characteristics of repair tools and repair parts. For example, in the "installing a sealing ring" repair node, tool-side behavior characteristics can include records of the taking sequence and usage duration of the rubber hammer and special sleeve, while part-side behavior characteristics can include records of the sealing ring installation sequence and installation position. The merged operation behavior information can simultaneously reflect the complete behavior data of both the tool operation process and the part installation process. It is understood that, when the current maintenance node involves both maintenance tools and maintenance parts, the embodiments of this application independently detect the operation sequence and usage time on the tool side and the installation sequence and installation position on the part side to obtain the behavioral characteristics on the tool side and the behavioral characteristics on the part side. The behavioral characteristics of the two dimensions are then merged to obtain comprehensive behavioral data that simultaneously reflects the tool operation process and the part installation process, providing more complete data support for subsequent maintenance compliance judgment.

[0055] It should be noted that, in the embodiments of this application, when the current maintenance node involves both maintenance tools and maintenance parts, the tool operation action analysis can be further performed based on the tool operation video data on the basis of the tool-side behavioral characteristics to obtain tool operation action information. The tool operation action information, together with the tool operation sequence information and the tool operation duration information, are determined as the tool-side behavioral characteristics, and then merged with the part-side behavioral characteristics to obtain more refined and comprehensive integrated behavioral data.

[0056] Please see Figure 5 In some embodiments, step S101 may also include, but is not limited to, steps S501 to S502: Step S501: Use the target virtual reality device to acquire images of the maintenance scene corresponding to the target device; Step S502: Perform maintenance node detection based on the maintenance scene image to obtain the current maintenance node information of the target device.

[0057] In step S501 of some embodiments, the maintenance scene image may refer to image information collected by the target virtual reality device that includes the target device and its surrounding operating environment. For example, if the target device is a vacuum pump that needs to have its seals replaced, the maintenance scene image may be an image including the overall structure of the vacuum pump, the state of its casing, and the location information of key components; or, if the target device is a centrifugal compressor that needs to have its impeller replaced, the maintenance scene image may be an image including the main body of the centrifugal compressor and the location information of the components to be disassembled. It is understood that the specific content of the maintenance scene image may vary depending on the type of target device and the maintenance task.

[0058] In step S502 of some embodiments, maintenance node detection can refer to the process of processing the maintenance scene image and identifying the current maintenance node information. The current maintenance node information can refer to information obtained after maintenance node detection, used to indicate the specific operation stage currently being performed. For example, embodiments of this application can identify the structural state of the target equipment based on the maintenance scene image to obtain equipment structural state information; and determine the current maintenance node information based on the equipment structural state information. The equipment structural state information can refer to the state feature information of the target equipment on its appearance or key components. Alternatively, embodiments of this application can also obtain the execution result information of the previous maintenance node on the target equipment based on the maintenance scene image, and perform node recursion in combination with the current maintenance scene image to obtain the current maintenance node information. For example, if the previous maintenance node has completed "removing the outer casing," then if the current maintenance scene image detects that the outer casing has been removed and the rotor is in place, it can be deduced that the current maintenance node is "removing the claw-type rotor." It is understood that the specific implementation method of maintenance node detection in embodiments of this application can be adjusted according to actual needs.

[0059] Please see Figure 6 In some embodiments, step S502 may include, but is not limited to, steps S601 to S603: Step S601: Detect the main body status of the equipment based on the maintenance scene image to obtain the main body status information of the equipment; Step S602: Perform equipment component status detection based on the maintenance scene image to obtain equipment component status information; Step S603: Determine the current maintenance node information of the target equipment based on the equipment main body status information and equipment component status information.

[0060] In step S601 of some embodiments, the device body state detection can refer to the process of processing the maintenance scene image and detecting the state features of the target device in its overall structure. Device body state information can refer to information obtained after device body state detection, used to indicate the state of the target device's main structure. For example, embodiments of this application can locate the device body region based on the maintenance scene image to obtain device body region information; identify the shell state based on the device body region information to obtain shell state information; and determine the device body state information based on the shell state information. Here, device body region information can refer to the range of the target device's main body position in the image; shell state information can refer to state information such as whether the shell is in place, intact, or loose; and device body state information can refer to the overall structural state information of the device determined based on the shell state. Alternatively, embodiments of this application can also perform device body contour detection based on the maintenance scene image to obtain device body contour information; compare the device body contour information with a preset standard body contour to obtain contour matching degree information; and determine the device body state information based on the contour matching degree information. Among them, the equipment body contour information can refer to the outer edge feature information of the target equipment body; the preset body standard contour can refer to the body contour feature information of the equipment in an intact state; the contour matching degree information can refer to the similarity information between the actual contour and the standard contour; and the equipment body status information can refer to the structural integrity status information of the equipment body determined based on the contour matching degree. It is understood that the specific implementation method of equipment body status detection in the embodiments of this application can be adjusted according to actual needs.

[0061] In step S602 of some embodiments, equipment component status detection can refer to the process of processing a maintenance scene image to detect the status features of key components inside or outside the target equipment. Equipment component status information can refer to information obtained after equipment component status detection, used to indicate the status of key components of the target equipment. For example, embodiments of this application can locate key component areas based on the maintenance scene image to obtain component area information; identify the component's presence status based on the component area information to obtain component presence status information; and determine equipment component status information based on the component presence status information. Here, component area information can refer to the location range information of key components on the target equipment in the image; component presence status information can refer to status information such as whether the component is in place, complete, or disassembled; and equipment component status information can refer to the key component status information determined based on the component's presence status. Alternatively, embodiments of this application can also perform component feature detection based on the maintenance scene image to obtain component feature information; compare the component feature information with preset component standard features to obtain feature matching degree information; and determine equipment component status information based on the feature matching degree information. Among them, component feature information can refer to visual feature information such as the shape, color, and texture of key components; preset component standard features can refer to the standard visual feature information of the component in an intact state; feature matching degree information can refer to the similarity information between actual features and standard features; and device component status information can refer to the component status information determined based on feature matching degree. It is understood that the specific implementation method of device component status detection in the embodiments of this application can be adjusted according to actual needs.

[0062] In step S603 of some embodiments, the current maintenance node information may refer to information determined comprehensively based on the equipment body status information and equipment component status information, used to indicate the specific operation stage currently being performed. For example, if the equipment body status information indicates that the equipment housing has been disassembled, and the equipment component status information indicates that the pump housing is still in place and the rotor has not been disassembled, then the current maintenance node can be determined as "removing the pump housing"; or, if the equipment body status information indicates that the equipment housing has been installed, and the equipment component status information indicates that the sealing ring has been installed in place, then the current maintenance node can be determined as "testing the gap".

[0063] It is understood that the embodiments of this application obtain equipment main body status information by detecting the equipment main body status based on the maintenance scene image, and obtain equipment component status information by detecting the equipment component status based on the maintenance scene image. The current maintenance node information is then determined by combining the equipment main body status information and the equipment component status information. In this way, the current overall status and key component status of the equipment can be detected from two dimensions: the main structure and internal components. The detection results from the two dimensions are then fused to achieve accurate judgment of the current maintenance node, providing a reliable data foundation for subsequent maintenance object matching and operation behavior analysis.

[0064] Please see Figure 7 In some embodiments, after step S106, steps S701 to S703 may also be included, but are not limited to: Step S701: Obtain a repair operation query instruction for the repaired item information; Step S702: Based on the maintenance operation query instruction and maintenance auxiliary guidance information, the query intent is parsed to obtain maintenance operation guidance information; Step S703: Output maintenance operation guidance information using the target virtual reality device.

[0065] In step S701 of some embodiments, the maintenance operation query instruction can refer to an interactive instruction issued by the operator for the maintenance object involved in the current maintenance node, used to inquire about the operation method. For example, the maintenance operation query instruction can be "How to install the sealing ring" entered by the operator via voice, or "Sealing ring installation steps" entered by the operator via text. It is understood that the specific form of the maintenance operation query instruction can be adjusted according to the actual interaction method.

[0066] In steps S702 to S703 of some embodiments, query intent parsing can refer to the process of identifying the operator's intent and the target repair object based on the maintenance operation query command and maintenance auxiliary guidance information, and generating corresponding operation guidance information. Maintenance operation guidance information can refer to the operation method information for the queried repair object obtained after query intent parsing, used to display to the operator. For example, when an operator issues a query command "How to install a sealing ring?", query intent parsing can identify the operator's intent to obtain the sealing ring installation method based on the sealing ring installation steps corresponding to the current maintenance node in the maintenance auxiliary guidance information, and generate a 3D animation demonstration of sealing ring installation, which is then output using the target virtual reality device; or, when an operator issues a query command "How to use an Allen wrench?", query intent parsing can identify the operator's intent to obtain the Allen wrench usage method based on the tool usage instructions corresponding to the current maintenance node in the maintenance auxiliary guidance information, and generate voice prompts for using the Allen wrench, which are then output using the target virtual reality device. It is understood that the specific output format of the maintenance operation guidance information can be adjusted according to actual needs.

[0067] It is understood that this application embodiment obtains a repair operation query instruction for information on the repaired object, and parses the query intent based on the repair operation query instruction and repair assistance guidance information to obtain repair operation guidance information, and then outputs the repair operation guidance information using the target virtual reality device. In this way, it can respond to the operator's proactive query needs, identify the operator's query intent and the target repaired object, and generate targeted operation guidance information, thereby achieving timely response to the operator's proactive interaction and providing users with more flexible and accurate repair assistance.

[0068] It should be noted that, in the implementation of query intent parsing in this embodiment, if the repair operation query instruction is ambiguous, such as the operator only issuing instructions like "this ring" or "this tool" which do not directly identify the specific repair item, the target virtual reality device can output prompt information to guide the operator to clarify the repair item being inquired about. Based on the operator's response text, the device determines the target repair item information and then generates corresponding repair operation guidance information. This effectively addresses situations where operator instructions are ambiguous, improving the accuracy of query intent parsing and the interactive experience.

[0069] In some embodiments, the maintenance process of the target equipment typically consists of multiple maintenance nodes executed sequentially, and different maintenance nodes may involve different types of maintenance objects. For example, "cleaning the cavity" or "testing the gap" only involves the operation of maintenance tools; while "installing the metal ring" or "inspecting the sealing ring" only involves the operation of maintenance parts; and "installing the sealing ring" or "removing the claw-type rotor" involves the operation of both maintenance tools and maintenance parts. For different types of maintenance nodes, the embodiments of this application can employ corresponding detection methods. For maintenance nodes involving only maintenance tools, the aforementioned detection steps for the operation sequence and duration of maintenance tools can be performed, and can be further combined with the detection of maintenance tool operation actions to obtain behavioral characteristics on the tool side; for maintenance nodes involving only maintenance parts, the aforementioned detection steps for the installation sequence and installation position of maintenance parts can be performed to obtain behavioral characteristics on the part side; for maintenance nodes involving both maintenance tools and maintenance parts, the detection steps on both the tool side and the part side can be performed simultaneously, and the detection results from the two dimensions can be merged to obtain comprehensive operational behavior information. Through the above flexible configuration, the embodiments of this application can select the appropriate testing method according to the actual needs of the current maintenance node, thereby meeting the diverse testing requirements of different maintenance nodes for tool operation and parts installation.

[0070] Please see Figure 8 This application also provides an equipment maintenance assistance device, applied to a target virtual reality device, which can implement the above-mentioned equipment maintenance assistance method. The device includes: The maintenance node determination unit 801 is used to determine the current maintenance node information of the target equipment; The maintenance object matching unit 802 is used to match maintenance objects according to the current maintenance node information to obtain the maintenance object information of the target equipment. The maintenance video acquisition unit 803 is used to acquire maintenance operation video data corresponding to the maintenance object information using the target virtual reality device; The maintenance behavior analysis unit 804 is used to perform operation behavior analysis based on the video data of the maintenance object to obtain the operation behavior information corresponding to the maintenance object information. The maintenance compliance judgment unit 805 is used to judge the maintenance compliance based on the work behavior information and obtain the maintenance compliance judgment result; The maintenance assistance unit 806, in response to the fact that the maintenance compliance judgment result does not meet the preset maintenance compliance conditions, uses the target virtual reality device to output maintenance assistance guidance information corresponding to the current maintenance node information.

[0071] The specific implementation method of the equipment maintenance auxiliary device is basically the same as the specific implementation method of the above-mentioned equipment maintenance auxiliary method, and will not be described again here.

[0072] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the aforementioned device maintenance assistance method. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0073] Please see Figure 9 , Figure 9 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes: The processor 901 can be implemented using a general-purpose central processing unit (CPU), microprocessor, application specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application. The memory 902 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 902 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 902 and is called and executed by the processor 901 to execute the device maintenance assistance method of the embodiments of this application. The input / output interface 903 is used to implement information input and output; The communication interface 904 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 905 transmits information between various components of the device (e.g., processor 901, memory 902, input / output interface 903, and communication interface 904); The processor 901, memory 902, input / output interface 903, and communication interface 904 are connected to each other within the device via bus 905.

[0074] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described equipment maintenance assistance method.

[0075] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0076] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0077] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0078] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0079] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0080] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0081] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0082] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0083] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0084] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0085] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0086] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A method for assisting in equipment maintenance, characterized in that, Applied to a target virtual reality device, the method includes: Determine the current maintenance status of the target equipment; Based on the current maintenance node information, the maintenance items are matched to obtain the maintenance item information of the target equipment; The target virtual reality device is used to collect video data of the repair work corresponding to the repair work information; Based on the video data of the repaired object, the operation behavior is analyzed to obtain the operation behavior information corresponding to the repaired object information; Based on the work behavior information, a maintenance compliance judgment is made to obtain the maintenance compliance judgment result; In response to the fact that the maintenance compliance judgment result does not meet the preset maintenance compliance conditions, the target virtual reality device outputs maintenance assistance guidance information corresponding to the current maintenance node information.

2. The method according to claim 1, characterized in that, The repair item information includes repair tool information, and the repair item operation video data includes tool operation video data corresponding to the repair tool information. The step of performing operation behavior analysis based on the repair item operation video data to obtain operation behavior information corresponding to the repair item information includes: The tool operation sequence is analyzed based on the tool operation video data to obtain tool operation sequence information; Based on the tool operation video data, the tool operation duration is analyzed to obtain tool operation duration information; The operation behavior information corresponding to the repaired object information is determined based on the tool operation sequence information and the tool operation duration information.

3. The method according to claim 2, characterized in that, The step of determining the work behavior information corresponding to the repair item information based on the tool operation sequence information and the tool operation duration information includes: Based on the tool operation video data, analyze the tool operation actions to obtain tool operation action information; The work behavior information corresponding to the repair object information is determined based on the tool operation action information, the tool operation sequence information, and the tool operation duration information.

4. The method according to claim 1, characterized in that, The repair object information includes repair part information, and the repair object operation video data includes part installation video data corresponding to the repair part information. The step of performing operation behavior analysis based on the repair object operation video data to obtain operation behavior information corresponding to the repair object information includes: Based on the component installation video data, the component installation sequence is analyzed to obtain component installation sequence information; Based on the video data of the component installation, the component installation position is analyzed to obtain the component installation position information; The operation behavior information corresponding to the maintenance object information is determined based on the component installation sequence information and the component installation location information.

5. The method according to claim 1, characterized in that, The determination of the current maintenance node information of the target equipment includes: The target virtual reality device is used to acquire images of the maintenance scene corresponding to the target device; Based on the maintenance scene image, maintenance node detection is performed to obtain the current maintenance node information of the target device.

6. The method according to claim 5, characterized in that, The step of detecting maintenance nodes based on the maintenance scene image to obtain the current maintenance node information of the target device includes: Based on the maintenance scene image, the main body status of the equipment is detected to obtain the main body status information of the equipment; Based on the maintenance scene image, the equipment component status is detected to obtain the equipment component status information; The current maintenance node information of the target device is determined based on the device body status information and the device component status information.

7. The method according to claim 1, characterized in that, After responding to the fact that the maintenance compliance judgment result does not meet the preset maintenance compliance conditions, and outputting maintenance assistance guidance information corresponding to the current maintenance node information using the target virtual reality device, the method further includes: Obtain a repair operation query instruction for the information of the repaired item; Based on the maintenance operation query command and the maintenance auxiliary guidance information, the query intent is parsed to obtain maintenance operation guidance information; The maintenance operation guidance information is output using the target virtual reality device.

8. An auxiliary device for equipment maintenance, characterized in that, Applied to a target virtual reality device, the device includes: The maintenance node determination unit is used to determine the current maintenance node information of the target equipment; A maintenance part matching unit is used to match maintenance parts according to the current maintenance node information to obtain maintenance part information of the target equipment. A maintenance video acquisition unit is used to acquire maintenance operation video data corresponding to the maintenance object information using the target virtual reality device; The maintenance behavior analysis unit is used to perform operation behavior analysis based on the operation video data of the maintenance object to obtain the operation behavior information corresponding to the maintenance object information. The maintenance compliance judgment unit is used to judge the maintenance compliance based on the work behavior information and obtain the maintenance compliance judgment result; The maintenance assistance unit, in response to the maintenance compliance judgment result not meeting the preset maintenance compliance conditions, uses the target virtual reality device to output maintenance assistance guidance information corresponding to the current maintenance node information.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the equipment maintenance assistance method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the equipment maintenance assistance method according to any one of claims 1 to 7.

Images