An industrial equipment maintenance system based on virtual reality

Abstract

The present invention relates to the field of virtual reality maintenance technology, and in particular to an industrial equipment maintenance system based on virtual reality. The present invention sets a model construction unit, a pre-analysis unit, an association unit, a model comparison unit and a model response unit. The model construction unit constructs a model to be uploaded of a device to be detected. The pre-analysis unit screens dynamic sub-models and determines dynamic presentation categories. The association unit establishes associations between the model to be uploaded and a sample model in a model database, and between each dynamic sub-model and the sample sub-model. The model comparison unit selects a comparison method for each dynamic sub-model in the model to be uploaded. The model response unit determines whether to repair the device to be detected. Thus, the system can save computing power while ensuring that the system can run stably for a long time, and the response efficiency of the industrial equipment maintenance system is improved.

Description

Technical Field

[0001] The present invention relates to the field of virtual reality maintenance technology, and in particular to an industrial equipment maintenance system based on virtual reality. Background Art

[0002] With the continuous improvement of the degree of industrial automation, the complexity and precision of industrial equipment are also increasing. The traditional industrial equipment maintenance method faces many challenges when facing these complex equipment, such as high maintenance difficulty, high training cost, low maintenance efficiency, etc. The industrial equipment maintenance system based on virtual reality provides an innovative solution to these problems. It uses the immersiveness, interactivity and imagination of virtual reality technology to reduce maintenance costs and improve maintenance efficiency. When facing actual industrial equipment, the industrial equipment maintenance system based on virtual reality can be used to detect the health level of equipment and reasonably formulate maintenance plans. Different maintenance plans can be simulated and evaluated in a virtual environment to select the best maintenance plan.

[0003] However, industrial equipment maintenance systems based on virtual reality usually require high-performance hardware to support complex graphics rendering and interactive computing. In the process of rendering complex virtual models of industrial equipment and physical simulation, by optimizing algorithms and reducing unnecessary calculations, it can be ensured that the system can run stably for a long time, reduce interruptions due to system failures during maintenance, and improve the reliability of maintenance work. Therefore, in the process of industrial equipment monitoring and maintenance based on virtual reality, while ensuring the reliability of detection and maintenance, saving computing power and improving the efficiency of monitoring and maintenance are technical issues that need to be solved urgently.

[0004] For example, China Patent Application Publication No.: CN115220567A, the invention discloses a coupler maintenance simulation system based on VR technology, the coupler maintenance simulation system includes a simulation modeling module, a maintenance step creation module, a coupler maintenance simulation module and a step management module; the simulation modeling module is used to perform three-dimensional modeling according to the collected coupler equipment data to establish a coupler simulation model; the maintenance step creation module creates a VR maintenance demonstration plan for the coupler simulation model according to the coupler disassembly process; the VR maintenance demonstration plan includes multiple disassembly and assembly steps of the coupler simulation model; the coupler maintenance simulation module constructs the maintenance process of the coupler simulation model based on VR technology to realize human-computer interaction for coupler maintenance; the step management module is used to manage the various disassembly and assembly steps of the coupler simulation model in the VR maintenance demonstration plan, adjust the data and time of each disassembly and assembly step, and the coupler maintenance simulation module modifies the maintenance process of the coupler simulation model according to the adjusted VR maintenance demonstration plan.

[0005] The prior art still has the following problems:

[0006] The existing technology does not take into account the problems of data update delays and slow operation response caused by excessive system computing power consumption during model rendering and physical simulation of complex industrial equipment. The existing technology cannot save computing power while ensuring that the system can run stably for a long time, and cannot adaptively adjust the system response method according to specific equipment conditions, affecting the response efficiency of the industrial equipment maintenance system. Summary of the invention

[0007] To this end, the present invention provides an industrial equipment maintenance system based on virtual reality to overcome the problems that the prior art cannot save computing power while ensuring that the system can run stably for a long time, and cannot adaptively adjust the system response mode according to specific equipment conditions.

[0008] To achieve the above object, the present invention provides an industrial equipment maintenance system based on virtual reality, comprising:

[0009] A model building unit, which is used to obtain a video stream of the operation process of the device to be detected, and build a model to be uploaded of the device to be detected according to the video stream of the operation process;

[0010] A pre-analysis unit connected to the model building unit, for selecting dynamic sub-models corresponding to a plurality of moving parts on the model to be uploaded, and determining a dynamic presentation category of the dynamic sub-model based on working condition content information of the dynamic sub-model;

[0011] Wherein, the working condition content information includes sound information within a number of time periods;

[0012] An association unit, which is connected to the model building unit and is used to pre-establish an association relationship between the model to be uploaded and the sample model in the model database, and an association relationship between each dynamic sub-model and the sample sub-model;

[0013] A model comparison unit, which is connected to the association unit and the pre-analysis unit respectively, is used to select a comparison method for each dynamic sub-model in the model to be uploaded according to different dynamic presentation categories, including:

[0014] Determine the position of the special frame component in different frames of the running process video stream corresponding to the dynamic sub-model, and compare the position of the special frame component with the reference position of the special frame component corresponding to the sample sub-model;

[0015] Or, the key trajectory nodes of the dynamic sub-model are screened according to the device instruction triggering condition, and the virtual trajectory fitted by the key trajectory node is compared with the running sample trajectory corresponding to the sample sub-model;

[0016] The model response unit is connected to the model comparison unit and is used to determine whether to send a signal for repairing the equipment corresponding to the dynamic sub-model according to the model comparison result.

[0017] Furthermore, the pre-analysis unit is used to determine the dynamic presentation category of the dynamic sub-model, wherein:

[0018] If the working condition content information of the dynamic sub-model meets the dynamic law presentation condition, the pre-analysis unit determines that the dynamic presentation category of the dynamic sub-model is the first dynamic presentation category;

[0019] If the working condition content information of the dynamic sub-model does not meet the dynamic law presentation condition, the pre-analysis unit determines that the dynamic presentation category of the dynamic sub-model is the second dynamic presentation category.

[0020] Furthermore, the dynamic law presentation condition is that the sequence composed of the area values ​​of the sound intensity fluctuation curve within a number of time periods is a periodic sequence;

[0021] The area value of the sound intensity fluctuation curve is the area of ​​a region enclosed by the sound intensity fluctuation curve and the horizontal axis of the coordinate system where the sound intensity fluctuation curve is located.

[0022] Furthermore, the model comparison unit is used to select a comparison method for each dynamic sub-model in the model to be uploaded, wherein:

[0023] If the dynamic presentation category is the first dynamic presentation category, the model comparison unit determines the position of the special frame component in different frames of the running process video stream corresponding to the dynamic sub-model, and compares the position of the special frame component with the reference position of the special frame component corresponding to the sample sub-model;

[0024] If the dynamic presentation category is the second dynamic presentation category, the model comparison unit selects key trajectory nodes of the dynamic sub-model according to the device instruction trigger condition, and compares the virtual trajectory fitted by the key trajectory node with the running sample trajectory corresponding to the sample sub-model.

[0025] Furthermore, the model comparison unit is used to determine the position of the special frame component, wherein:

[0026] The model comparison unit obtains the periodic running trajectory of the moving component in the dynamic sub-model at the start time and the end time in the pictures corresponding to the running process video stream, and determines the position of the moving component in each picture as the special frame component position.

[0027] Furthermore, the model comparison unit is used to compare the special frame component position with the special frame component reference position corresponding to the sample sub-model, wherein:

[0028] If the position of the special frame component is inconsistent with the reference position of the special frame component corresponding to the sample sub-model, the model comparison unit determines that the to-be-detected equipment component corresponding to the dynamic sub-model is abnormal.

[0029] Furthermore, the device instruction triggering condition is the moment when the moving component executes each action execution instruction.

[0030] Furthermore, the model comparison unit is further used to determine the trajectory points corresponding to the device instruction triggering conditions in the running trajectory as key trajectory nodes, fit the key trajectory nodes into virtual trajectories in chronological order, and compare the virtual trajectory with the running sample trajectory corresponding to the sample sub-model, wherein:

[0031] If the virtual trajectory and the running sample trajectory meet the equipment abnormality condition, the model comparison unit determines that the to-be-detected equipment component corresponding to the virtual trajectory is abnormal;

[0032] If the virtual trajectory and the running sample trajectory do not meet the equipment abnormality condition, the model comparison unit determines that the to-be-detected equipment component corresponding to the virtual trajectory is normal;

[0033] The device abnormal condition is that the overlap between the virtual trajectory and the running sample trajectory does not exceed a preset overlap threshold.

[0034] Furthermore, the model response unit is used to determine whether to send a signal to repair the component corresponding to the dynamic sub-model, wherein:

[0035] If the model comparison unit determines that the component of the equipment to be detected is abnormal, the model response unit determines to send a signal to repair the component corresponding to the dynamic sub-model, and simultaneously sends an equipment maintenance data packet of the dynamic sub-model.

[0036] Furthermore, the equipment maintenance data packet includes a component image corresponding to the dynamic sub-model and a wide-angle image with the component image as the image center.

[0037] Compared with the prior art, the beneficial effects of the present invention lie in that a model building unit, a pre-analysis unit, an association unit, a model comparison unit and a model response unit are provided in the present invention; the model to be uploaded of the equipment to be detected is built by the model building unit; a plurality of dynamic sub-models are screened and the dynamic presentation category of the dynamic sub-model is determined by the pre-analysis unit; an association relationship is established between the model to be uploaded and the sample model in the model database and between each dynamic sub-model and the sample sub-model respectively by the association unit; a comparison method for each dynamic sub-model in the model to be uploaded is selected by the model comparison unit; and whether to repair the equipment to be detected is determined by the model response unit. Thus, it is achieved that computing power is saved while ensuring that the system can run stably for a long time, the system response method is adaptively adjusted according to the specific equipment situation, and the response efficiency of the industrial equipment maintenance system is improved.

[0038] In particular, the present invention selects several dynamic sub-models through the pre-analysis unit. It can be understood that during the operation of the equipment, due to the continuous movement of the moving parts, friction, collision and other situations between the parts frequently occur. The moving parts are usually the most prone to failure in industrial equipment. For example, in the mechanical gripper on the automated production line, the motion trajectory of its end effector is accurately set under normal circumstances to complete various grasping, placement and other operations. If the joint of the mechanical gripper fails, such as the motor stalls or the transmission parts are loose, the motion trajectory of its end effector will deviate from the normal path. By selecting the end effector as a dynamic sub-model for analysis, signs of failure can be discovered in time. At the same time, the overall model of the industrial equipment contains a large amount of information, and the moving parts will show specific behavior patterns and physical characteristics. Screening out the dynamic sub-model corresponding to the moving parts helps to extract relevant fault features, reduce a large amount of unnecessary data interference, and make the detection focus clearer. The present invention selects several dynamic sub-models in the model corresponding to the equipment to be detected, thereby saving computing power while ensuring that the system can operate stably for a long time, and improving the response efficiency of the industrial equipment maintenance system.

[0039] In particular, the present invention determines the dynamic presentation category of the dynamic sub-model based on the working condition content information of the dynamic sub-model through the pre-analysis unit. It can be understood that the trajectory shape and sound information can intuitively reflect the moving state of the moving parts. The trajectory and sound of the reciprocating parts are repeatable. The sound generation of the moving parts is closely related to its motion state. For regular moving parts, the periodicity of their movement will lead to the periodicity of sound generation. Due to the uncertainty of movement, the sound generation mechanism of irregular moving parts is more complex and changeable, which will produce irregular impact sounds and friction sounds. The present invention classifies the dynamic presentation category of the dynamic sub-model through the working condition content information of the dynamic sub-model, and then realizes the classification processing of the dynamic sub-model, thereby improving the response efficiency of the industrial equipment maintenance system.

[0040] In particular, under the condition of dynamically presenting the first category, the present invention determines the position of special frame components in different frames of the video stream of the operation process corresponding to the dynamic sub-model, and compares the position of the special frame component with the reference position of the special frame component corresponding to the sample sub-model. It can be understood that under the condition of dynamically presenting the first category, the moving parts in the equipment have certain regularity and repeatability. The key frames such as the starting point and the end point of each cycle of the moving parts can be extracted to characterize its movement process. There is no need to process a large amount of data at each time point, which significantly reduces the burden of data storage and calculation. The present invention greatly reduces the amount of data to be processed by extracting only special frames, thereby achieving the goal of saving computing power while ensuring that the system can run stably for a long time, adaptively adjusting the system response mode according to the specific equipment conditions, and improving the response efficiency of the industrial equipment maintenance system.

[0041] In particular, under the condition of dynamically presenting the second category, the present invention selects the key trajectory nodes of the dynamic sub-model according to the device instruction trigger condition, and compares the virtual trajectory fitted by the key trajectory node with the running sample trajectory corresponding to the sample sub-model. It can be understood that for moving parts without obvious periodic rules, their movement may be affected by the interaction of multiple complex factors, such as irregular external loads, complex and changeable working environments or complex task processes, etc. The device instruction is the key factor in controlling the movement of the moving parts, reflecting the device's expectations and planning for the movement of the parts. Through the instruction triggering moment, the starting point, turning point and important action moment points of the moving parts can be grasped, and these points constitute the basic movement of the parts. The framework can provide key clues for subsequent analysis in the absence of periodic rules. Multiple key trajectory nodes combined can approximately depict the overall motion process of the moving parts. The constructed virtual trajectory of the moving parts can reflect the motion trend and behavioral characteristics of the moving parts to a certain extent. The virtual trajectory is compared with the motion sample trajectory in the sample library to detect whether the equipment is abnormal. The present invention greatly reduces the amount of data to be processed by extracting key trajectory nodes to fit the virtual trajectory, thereby achieving the goal of saving computing power while ensuring the long-term stable operation of the system, adaptively adjusting the system response mode according to the specific equipment situation, and improving the response efficiency of the industrial equipment maintenance system.

[0042] In particular, the model response unit of the present invention sends an equipment maintenance data packet corresponding to the dynamic sub-model, including a component image corresponding to the dynamic sub-model and a wide-angle image with the component image as the image center. It can be understood that when a fault occurs in the equipment and an alarm is triggered, the maintenance personnel receive a wide-angle image including the component image corresponding to the dynamic sub-model and with the component image as the image center, and can quickly find the exact location where the fault occurred based on the information provided therein, without spending a lot of time to check the entire equipment system to find the problem. The wide-angle image can more conveniently locate the specific position of the moving parts that need to be repaired due to its wider viewing angle, especially for those large and complex industrial equipment. Accurate image information can greatly shorten the time for maintenance personnel to arrive at the fault site and quickly start maintenance work. The present invention sends a data packet containing multi-dimensional image information, so that the entire maintenance process from locating the fault to actually carrying out the operation is smoother and more efficient, reducing the useless time consumption in the maintenance process, thereby reducing production losses caused by equipment failure and downtime, and improving the response efficiency of the industrial equipment maintenance system. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 This is a functional block diagram of an industrial equipment maintenance system based on virtual reality according to an embodiment of the present invention;

[0044] Figure 2 A logic flow chart of a pre-analysis unit determining a dynamic presentation category of a dynamic sub-model according to an embodiment of the present invention;

[0045] Figure 3 A logic flow chart of a comparison method for selecting each dynamic sub-model in a model to be uploaded by a model comparison unit in an embodiment of the present invention;

[0046] Figure 4 The present invention is a logic flow chart for the model response unit to determine whether to send a signal to repair the component corresponding to the dynamic sub-model according to an embodiment of the present invention. DETAILED DESCRIPTION

[0047] In order to make the objects and advantages of the present invention more clearly understood, the present invention is further described below in conjunction with embodiments; it should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0048] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the protection scope of the present invention.

[0049] It should be noted that, in the description of the present invention, terms such as "upper", "lower", "inside" and "outside" indicating directions or positional relationships are based on the directions or positional relationships shown in the drawings. This is only for the convenience of description and does not indicate or imply that the device or element must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it cannot be understood as a limitation on the present invention.

[0050] In addition, it should be noted that in the description of the present invention, unless otherwise clearly specified and limited, the terms "installation" and "connection" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be an indirect connection through an intermediate medium, or it can be the internal communication of two components. For those skilled in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0051] See also Figure 1 As shown, it is a functional block diagram of an industrial equipment maintenance system based on virtual reality according to an embodiment of the present invention. An industrial equipment maintenance system based on virtual reality according to the present invention comprises:

[0052] A model building unit, which is used to obtain a video stream of the operation process of the device to be detected, and build a model to be uploaded of the device to be detected according to the video stream of the operation process;

[0053] Specifically, the present invention does not limit the specific structure of the model building unit. Preferably, it can be a camera with built-in video acquisition software that cooperates with a microprocessor to realize the construction of the model to be uploaded of the device to be detected. According to the characteristics of the device to be detected, a suitable model architecture is selected, the data is normalized and standardized, and the model to be uploaded of the device to be detected is constructed. It will not be repeated here.

[0054] A pre-analysis unit connected to the model building unit, for selecting dynamic sub-models corresponding to a plurality of moving parts on the model to be uploaded, and determining a dynamic presentation category of the dynamic sub-model based on working condition content information of the dynamic sub-model;

[0055] Wherein, the working condition content information includes sound information within a number of time periods;

[0056] Specifically, the present invention does not limit the specific structure of the pre-analysis unit. Preferably, it can be constructed using logic components. The logic components can be field programmable logic components, microprocessors, processors used in computers, etc. The models corresponding to the components that undergo displacement changes in the device to be detected are screened as dynamic sub-models, and the dynamic presentation category of the dynamic sub-model is determined. No further details are given here.

[0057] Specifically, the present invention does not specifically limit the method for obtaining sound information. Preferably, it can be obtained through a capacitive microphone, and the sound signal can be accurately captured through the frequency response curve and low noise characteristics to obtain the sound information of the dynamic sub-model, which will not be repeated here.

[0058] Specifically, the present invention selects several dynamic sub-models through a pre-analysis unit. It can be understood that during the operation of the equipment, due to the continuous movement of the moving parts, friction, collision and other situations between the parts frequently occur. The moving parts are usually the most prone to failure in industrial equipment. For example, in the mechanical gripper on the automated production line, the motion trajectory of its end effector is accurately set under normal circumstances to complete various grasping, placement and other operations. If the joint of the mechanical gripper fails, such as the motor stalls or the transmission parts are loose, the motion trajectory of its end effector will deviate from the normal path. By selecting the end effector as a dynamic sub-model for analysis, signs of failure can be discovered in time. At the same time, the overall model of the industrial equipment contains a large amount of information, and the moving parts will show specific behavior patterns and physical characteristics. Screening out the dynamic sub-model corresponding to the moving parts helps to extract relevant fault features, reduce a large amount of unnecessary data interference, and make the detection focus clearer. The present invention selects several dynamic sub-models in the model corresponding to the equipment to be detected, thereby saving computing power while ensuring that the system can operate stably for a long time, and improving the response efficiency of the industrial equipment maintenance system.

[0059] An association unit, which is connected to the model building unit and is used to pre-establish an association relationship between the model to be uploaded and the sample model in the model database, and an association relationship between each dynamic sub-model and the sample sub-model;

[0060] Specifically, the present invention does not limit the specific structure of the association unit. Preferably, it can be constructed using logic components. The logic components can be field programmable logic components, microprocessors, processors used in computers, etc., which will not be repeated here.

[0061] A model comparison unit, which is connected to the association unit and the pre-analysis unit respectively, is used to select a comparison method for each dynamic sub-model in the model to be uploaded according to different dynamic presentation categories, including:

[0062] Determine the position of the special frame component in different frames of the running process video stream corresponding to the dynamic sub-model, and compare the position of the special frame component with the reference position of the special frame component corresponding to the sample sub-model;

[0063] Or, the key trajectory nodes of the dynamic sub-model are screened according to the device instruction triggering condition, and the virtual trajectory fitted by the key trajectory node is compared with the running sample trajectory corresponding to the sample sub-model;

[0064] The model response unit is connected to the model comparison unit and is used to determine whether to send a signal for repairing the equipment corresponding to the dynamic sub-model according to the model comparison result.

[0065] Specifically, the present invention does not limit the specific structure of the model comparison unit and the model response unit. Preferably, they can be constructed using logic components, which can be field programmable logic components, microprocessors, processors used in computers, etc., which will not be repeated here.

[0066] Specifically, the electrical signal can be converted into a sound signal through a buzzer to issue a maintenance signal alarm, and the equipment maintenance data packet can be displayed on a display to help maintenance personnel quickly identify the parts that need maintenance.

[0067] Specifically, see Figure 2 As shown, it is a logic flow chart of the pre-analysis unit of an embodiment of the present invention for determining the dynamic presentation category of the dynamic sub-model, wherein the pre-analysis unit is used to determine the dynamic presentation category of the dynamic sub-model, wherein:

[0068] If the working condition content information of the dynamic sub-model meets the dynamic law presentation condition, the pre-analysis unit determines that the dynamic presentation category of the dynamic sub-model is the first dynamic presentation category;

[0069] If the working condition content information of the dynamic sub-model does not meet the dynamic law presentation condition, the pre-analysis unit determines that the dynamic presentation category of the dynamic sub-model is the second dynamic presentation category.

[0070] Specifically, the dynamic law presentation condition is that the sequence composed of the area values ​​of the sound intensity fluctuation curve within a number of time periods is a periodic sequence;

[0071] The area value of the sound intensity fluctuation curve is the area of ​​a region enclosed by the sound intensity fluctuation curve and the horizontal axis of the coordinate system where the sound intensity fluctuation curve is located.

[0072] Specifically, in an embodiment of the present invention, the area values ​​of the sound intensity fluctuation curve within several time periods of the dynamic sub-model are sorted in time as follows: 60.8, 52, 70.23, 65, 60.8, 52, 70.23, 65, 60.8, 52, 70.23, 65...65. At this time, the sequence composed of the area values ​​of the sound intensity fluctuation curve within several time periods of the dynamic sub-model is a periodic sequence. The area values ​​of the sound intensity fluctuation curve within several time periods of the dynamic sub-model are sorted in time as follows: 60.8, 74.67, 70.23, 63.98, 60.8, 51.26, 70.23, 72, 60.8, 79.4, 70.23, 60.8...60.7. At this time, the sequence composed of the area values ​​of the sound intensity fluctuation curve within several time periods of the dynamic sub-model is not a periodic sequence.

[0073] Specifically, the present invention can obtain the sound intensity fluctuation curve through a data acquisition card in conjunction with a capacitive microphone. The data acquisition card converts the analog sound signal collected by the microphone into a digital signal, and transmits it to a computer for processing and analysis to generate a sound intensity fluctuation curve of a dynamic sub-model, which will not be repeated here.

[0074] Specifically, the present invention can obtain the area value of the sound intensity fluctuation curve within several time periods through a microprocessor or a processor used in a computer, and determine the dynamic presentation category of the dynamic sub-model according to the area value, which will not be repeated here.

[0075] Specifically, the present invention determines the dynamic presentation category of the dynamic sub-model based on the working condition content information of the dynamic sub-model through the pre-analysis unit. It can be understood that the trajectory shape and sound information can intuitively reflect the moving state of the moving parts. The trajectory and sound of the reciprocating parts are repeatable. The sound generation of the moving parts is closely related to its motion state. For regular moving parts, the periodicity of their movement will lead to the periodicity of sound generation. Due to the uncertainty of movement, the sound generation mechanism of irregular moving parts is more complex and changeable, which will produce irregular impact sounds and friction sounds. The present invention classifies the dynamic presentation category of the dynamic sub-model through the working condition content information of the dynamic sub-model, and then realizes the classification processing of the dynamic sub-model, thereby improving the response efficiency of the industrial equipment maintenance system.

[0076] Specifically, see Figure 3 As shown, it is a logic flow chart of the model comparison unit of the embodiment of the present invention selecting the comparison method of each dynamic sub-model in the model to be uploaded, and the model comparison unit is used to select the comparison method of each dynamic sub-model in the model to be uploaded, wherein:

[0077] If the dynamic presentation category is the first dynamic presentation category, the model comparison unit determines the position of the special frame component in different frames of the running process video stream corresponding to the dynamic sub-model, and compares the position of the special frame component with the reference position of the special frame component corresponding to the sample sub-model;

[0078] If the dynamic presentation category is the second dynamic presentation category, the model comparison unit selects key trajectory nodes of the dynamic sub-model according to the device instruction trigger condition, and compares the virtual trajectory fitted by the key trajectory node with the running sample trajectory corresponding to the sample sub-model.

[0079] Specifically, under the condition of dynamically presenting the first category, the present invention determines the position of special frame components in different frames of the video stream of the operation process corresponding to the dynamic sub-model, and compares the position of the special frame component with the reference position of the special frame component corresponding to the sample sub-model. It can be understood that under the condition of dynamically presenting the first category, the moving parts in the equipment have certain regularity and repeatability. The key frames such as the starting point and the end point of each cycle of the moving parts can be extracted to characterize its movement process. There is no need to process a large amount of data at each time point, which significantly reduces the burden of data storage and calculation. The present invention greatly reduces the amount of data to be processed by extracting only special frames, thereby achieving the goal of saving computing power while ensuring that the system can run stably for a long time, adaptively adjusting the system response mode according to the specific equipment conditions, and improving the response efficiency of the industrial equipment maintenance system.

[0080] Specifically, under the condition of dynamically presenting the second category, the present invention selects the key trajectory nodes of the dynamic sub-model according to the device instruction trigger condition, and compares the virtual trajectory fitted by the key trajectory node with the running sample trajectory corresponding to the sample sub-model. It can be understood that for moving parts without obvious periodic rules, their movement may be affected by the interaction of multiple complex factors, such as irregular external loads, complex and changeable working environments or complex task processes, etc. The device instruction is the key factor in controlling the movement of the moving parts, reflecting the device's expectations and planning for the movement of the parts. Through the instruction triggering moment, the starting point, turning point and important action moment points of the moving parts can be grasped, and these points constitute the basis of the movement of the parts. This framework can provide key clues for subsequent analysis in the absence of periodic patterns. Multiple key trajectory nodes combined can approximately depict the overall motion process of the moving parts. The constructed virtual trajectory of the moving parts can reflect the motion trend and behavioral characteristics of the moving parts to a certain extent. Comparing the virtual trajectory with the motion sample trajectory in the sample library can detect whether the equipment is abnormal. The present invention greatly reduces the amount of data to be processed by extracting key trajectory nodes to fit the virtual trajectory, thereby achieving the goal of saving computing power while ensuring the long-term stable operation of the system, adaptively adjusting the system response mode according to the specific equipment situation, and improving the response efficiency of the industrial equipment maintenance system.

[0081] Specifically, the model comparison unit is used to determine the position of the special frame component, wherein:

[0082] The model comparison unit obtains the periodic running trajectory of the moving component in the dynamic sub-model at the start time and the end time in the pictures corresponding to the running process video stream, and determines the position of the moving component in each picture as the special frame component position.

[0083] Here is a specific embodiment. The running trajectory of the moving part performing reciprocating motion is A1-B1-C1-D1-A2-B2-C2-D2...Dn, wherein n is the number of cycles of the moving part performing reciprocating motion, the position points of A1 and A2 correspond to the same, the position points of B1 and B2 correspond to the same, the position points of C1 and C2 correspond to the same, the position points of D1 and D2 correspond to the same, and so on. The special frame component position of the moving part is the position of the component in the running process video stream screen corresponding to time A1 and time D1, the position of the component in the running process video stream screen corresponding to time A2 and time D2...the position of the component in the running process video stream screen corresponding to time An and time Dn.

[0084] Specifically, the model comparison unit is used to compare the special frame component position with the special frame component reference position corresponding to the sample sub-model, wherein:

[0085] If the position of the special frame component is inconsistent with the reference position of the special frame component corresponding to the sample sub-model, the model comparison unit determines that the component of the device to be detected corresponding to the dynamic sub-model is abnormal;

[0086] If the position of the special frame component is consistent with the reference position of the special frame component corresponding to the sample sub-model, the model comparison unit determines that the to-be-detected device component corresponding to the dynamic sub-model is normal.

[0087] Specifically, the device instruction triggering condition is the moment when the moving component executes each action execution instruction.

[0088] Specifically, the model comparison unit is further used to determine the trajectory points corresponding to the device instruction triggering conditions in the running trajectory as key trajectory nodes, fit the key trajectory nodes into virtual trajectories in chronological order, and compare the virtual trajectory with the running sample trajectory corresponding to the sample sub-model, wherein:

[0089] If the virtual trajectory and the running sample trajectory meet the equipment abnormality condition, the model comparison unit determines that the to-be-detected equipment component corresponding to the virtual trajectory is abnormal;

[0090] If the virtual trajectory and the running sample trajectory do not meet the equipment abnormality condition, the model comparison unit determines that the to-be-detected equipment component corresponding to the virtual trajectory is normal;

[0091] The device abnormal condition is that the overlap between the virtual trajectory and the running sample trajectory does not exceed a preset overlap threshold.

[0092] Specifically, the preset overlap threshold can be set by technical personnel in this field according to the operating accuracy requirements of the equipment to be detected. The higher the operating accuracy requirements of the equipment to be detected, the larger the value of the overlap threshold. The value range of the overlap threshold can be [0.88, 0.95]. Preferably, in implementation, the overlap threshold can be 0.9.

[0093] Specifically, see Figure 4 As shown, it is a logic flow chart of the model response unit of an embodiment of the present invention determining whether to send a signal to repair the component corresponding to the dynamic sub-model, wherein the model response unit is used to determine whether to send a signal to repair the component corresponding to the dynamic sub-model, wherein:

[0094] If the model comparison unit determines that the component of the equipment to be detected is abnormal, the model response unit determines to send a signal to repair the component corresponding to the dynamic sub-model, and simultaneously sends a device maintenance data packet of the dynamic sub-model;

[0095] If the model comparison unit determines that the component of the equipment to be detected is normal, the model response unit does not send a signal to repair the component corresponding to the dynamic sub-model.

[0096] Specifically, the equipment maintenance data packet includes a component image corresponding to the dynamic sub-model and a wide-angle image with the component image as the image center.

[0097] Specifically, the component image and the wide-angle image can be obtained through the running process video stream. By locating the position of the component image in the running process video stream, the target detection algorithm is used to obtain a wide-angle image with the component image as the image center. This is a prior art and will not be repeated here.

[0098] Specifically, the model response unit of the present invention sends an equipment maintenance data packet corresponding to the dynamic sub-model, including a component image corresponding to the dynamic sub-model and a wide-angle image with the component image as the image center. It can be understood that when a fault occurs in the equipment and an alarm is triggered, the maintenance personnel receive a wide-angle image including the component image corresponding to the dynamic sub-model and with the component image as the image center, and can quickly find the exact location where the fault occurred based on the information provided therein, without spending a lot of time to check the entire equipment system to find the problem. The wide-angle image can more conveniently locate the specific position of the moving parts that need to be repaired due to its wider viewing angle, especially for those large and complex industrial equipment. Accurate image information can greatly shorten the time for maintenance personnel to arrive at the fault site and quickly start maintenance work. The present invention sends a data packet containing multi-dimensional image information, so that the entire maintenance process from locating the fault to actually carrying out the operation is smoother and more efficient, reducing the useless time consumption in the maintenance process, thereby reducing production losses caused by equipment failure and downtime, and improving the response efficiency of the industrial equipment maintenance system.

[0099] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

[0100] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An industrial equipment maintenance system based on virtual reality, characterized in that: include: A model building unit, which is used to obtain a video stream of the operation process of the device to be detected, and build a model to be uploaded of the device to be detected according to the video stream of the operation process; A pre-analysis unit connected to the model building unit, for selecting dynamic sub-models corresponding to a plurality of moving parts on the model to be uploaded, and determining a dynamic presentation category of the dynamic sub-model based on working condition content information of the dynamic sub-model; Wherein, the working condition content information includes sound information within a number of time periods; If the working condition content information of the dynamic sub-model meets the dynamic law presentation condition, the pre-analysis unit determines that the dynamic presentation category of the dynamic sub-model is the first dynamic presentation category; If the working condition content information of the dynamic sub-model does not meet the dynamic law presentation condition, the pre-analysis unit determines that the dynamic presentation category of the dynamic sub-model is the second dynamic presentation category; The dynamic law presentation condition is that the sequence composed of the area values ​​of the sound intensity fluctuation curve within a certain time period is a periodic sequence; The area value of the sound intensity fluctuation curve is the area of ​​the region enclosed by the sound intensity fluctuation curve and the horizontal axis of the coordinate system where the sound intensity fluctuation curve is located; An association unit, which is connected to the model building unit and is used to pre-establish an association relationship between the model to be uploaded and the sample model in the model database, and an association relationship between each dynamic sub-model and the sample sub-model; A model comparison unit, which is connected to the association unit and the pre-analysis unit respectively, is used to select a comparison method for each dynamic sub-model in the model to be uploaded according to different dynamic presentation categories, including: If the dynamic presentation category is the first dynamic presentation category, determining the position of the special frame component according to the running process video stream corresponding to the dynamic sub-model in different frames, and comparing the position of the special frame component with the reference position of the special frame component corresponding to the sample sub-model; If the dynamic presentation category is the second dynamic presentation category, the key trajectory nodes of the dynamic sub-model are screened according to the device instruction trigger condition, and the virtual trajectory fitted by the key trajectory node is compared with the running sample trajectory corresponding to the sample sub-model; A model response unit, connected to the model comparison unit, for determining whether to send a signal for repairing the equipment corresponding to the dynamic sub-model according to the model comparison result; If the model comparison unit determines that the component of the equipment to be detected is abnormal, the model response unit determines to send a signal to repair the component corresponding to the dynamic sub-model, and simultaneously sends a device maintenance data packet of the dynamic sub-model; The equipment maintenance data packet includes a component image corresponding to the dynamic sub-model and a wide-angle image with the component image as the image center.

2. The virtual reality-based industrial equipment maintenance system according to claim 1, characterized in that: The model comparison unit is used to determine the position of the special frame component, wherein: The model comparison unit obtains the periodic running trajectory of the moving component in the dynamic sub-model at the start time and the end time in the pictures corresponding to the running process video stream, and determines the position of the moving component in each picture as the special frame component position.

3. The virtual reality-based industrial equipment maintenance system according to claim 2, characterized in that: The model comparison unit is used to compare the special frame component position with the special frame component reference position corresponding to the sample sub-model, wherein: If the position of the special frame component is inconsistent with the reference position of the special frame component corresponding to the sample sub-model, the model comparison unit determines that the to-be-detected equipment component corresponding to the dynamic sub-model is abnormal.

4. The virtual reality-based industrial equipment maintenance system according to claim 3 is characterized in that: The device instruction triggering condition is the moment when the moving component executes each action execution instruction.

5. The virtual reality-based industrial equipment maintenance system according to claim 4, characterized in that: The model comparison unit is further used to determine the trajectory points corresponding to the device instruction triggering conditions in the running trajectory as key trajectory nodes, fit the key trajectory nodes into virtual trajectories in chronological order, and compare the virtual trajectory with the running sample trajectory corresponding to the sample sub-model, wherein: If the virtual trajectory and the running sample trajectory meet the equipment abnormality condition, the model comparison unit determines that the to-be-detected equipment component corresponding to the virtual trajectory is abnormal; If the virtual trajectory and the running sample trajectory do not meet the equipment abnormality condition, the model comparison unit determines that the to-be-detected equipment component corresponding to the virtual trajectory is normal; The device abnormal condition is that the overlap between the virtual trajectory and the running sample trajectory does not exceed a preset overlap threshold.

Images