Inspection work assistance guidance system based on wearable pda

Abstract

The application discloses a check operation auxiliary guiding system based on a wearable PDA, belongs to the technical field of check operation auxiliary, and comprises a preparation module and an operation module; the preparation module is used for analyzing a network environment of a work area according to task information of a check operation before the check operation auxiliary guiding is carried out, and obtaining a corresponding operation preparation report; the operation module is used for guiding a user to carry out the check operation, configuring operation content, wearing AR glasses and a PDA device, instructing the user to check according to a set step through a graphical interface, sequentially displaying a next check point by the wearable PDA and the AR glasses, and marking a check result, so that the user checks; an operation condition, an abnormal condition and a check position image in a check process are temporarily stored in the wearable PDA terminal; the preparation module is set, so that the wearable PDA system is prevented from being stopped due to network problems in an actual application process.

Description

Technical Field

[0001] This invention belongs to the field of inspection operation assistance technology, specifically an inspection operation assistance and guidance system based on wearable PDA. Background Technology

[0002] Many industries and professions today require efficient completion of inspection tasks, such as manufacturing, maintenance, and healthcare. Operators are expected to complete tasks accurately within limited timeframes, and some inspection tasks may involve complex procedures and procedures, such as the inspection and maintenance of medical equipment and the routine inspection of industrial equipment. Sometimes operators may forget certain steps or procedures, leading to errors or risks. Furthermore, some tasks may require guidance and support from remote experts, such as maintenance and emergency rescue in remote areas. Traditional communication methods may have limitations in terms of time and space. Therefore, to address these issues, this invention provides an inspection task assistance and guidance system based on a wearable PDA. Summary of the Invention

[0003] To address the problems of the above solutions, this invention provides an inspection operation assistance and guidance system based on a wearable PDA.

[0004] The objective of this invention can be achieved through the following technical solutions:

[0005] An inspection operation assistance and guidance system based on wearable PDA, including a preparation module and an operation module;

[0006] The preparation module is used to analyze the network environment of the work area based on the task information of the inspection operation before conducting the inspection operation, and obtain the corresponding operation preparation report.

[0007] Furthermore, the working method of the preparation module includes:

[0008] Obtain task information, locate the corresponding job position in the preset network graph based on the task information, and obtain the corresponding network score based on the job position;

[0009] Based on the task information, determine the prediction function corresponding to the inspection task, as well as the usage probability and network score range of each prediction function, and integrate the obtained prediction function, usage probability and network score range into task detail data;

[0010] A task preparation report is generated based on task details and online ratings, and the obtained task preparation report is sent to the relevant staff.

[0011] Furthermore, the methods for setting up a network diagram include:

[0012] An initial map is set up based on the available work areas, and the initial map is updated according to changes in the work areas; network environment data is collected and analyzed for each work area in the initial map to obtain the corresponding network score; the obtained network score is marked at the corresponding position in the initial map, and the current initial map is marked as the network map.

[0013] Furthermore, the network score for the work location is adjusted based on the current network influencing factors.

[0014] Furthermore, add feasible values ​​to the job preparation report.

[0015] Furthermore, the methods for calculating feasible values ​​include:

[0016] Based on the task details data and network scores, generate individual evaluation values ​​for each prediction function, label the prediction function as i, where i = 1, 2, ..., n, and n is a positive integer; label the individual evaluation value as DPi, label the probability of use as LVi, and calculate the corresponding feasible value according to the feasibility formula KP = ∑(LVi × DPi).

[0017] The operation module is used to guide users to perform inspection tasks, configure the task content, and guide users to wear AR glasses and PDA devices. Through a graphical interface, the module instructs users to perform inspections according to the set steps. The wearable PDA and AR glasses sequentially display the next inspection point and mark the inspection results, which the user then uses to perform the inspection. The operation status, abnormal situations, and images of the inspected parts during the inspection process are temporarily stored on the wearable PDA terminal.

[0018] Furthermore, during the inspection of the corresponding parts, the AR glasses provide synchronized voice prompts based on safety precautions, technical standards, inspection content, and operating methods.

[0019] Furthermore, while performing the part inspection, the AR glasses capture images of the inspected part and compare them with pre-stored photos of the part to determine whether the user has inspected the correct part. The image recognition uses a grayscale block matching method.

[0020] Furthermore, the AR glasses use WIFI to communicate with the wearable PDA for inspection and operation control. The wearable PDA communicates with the field terminal computer via USB data cable. The field terminal computer and the field base server communicate via Ethernet, and all data is transmitted after encryption.

[0021] Compared with the prior art, the beneficial effects of the present invention are:

[0022] It adopts head-mounted AR glasses and wearable PDAs, making it convenient for staff to carry and use; it guides and assists staff in carrying out inspection work, improving the accuracy and efficiency of inspection work; it provides inspection route guidance function, making it easy for staff to quickly find the inspection location; it records data and images of the inspection area, facilitating the review of the inspection process or assisting in post-event accountability; the data can be stored and transmitted synchronously, providing a raw data source for future big data analysis.

[0023] By setting up a preparation module, it is ensured that the wearable PDA system will not cause the operation to stop due to network problems during actual application. The module analyzes the task situation in advance, helps users understand the usage details of the wearable PDA system in advance, which functions may have usage restrictions, and prompts users to prepare in advance to avoid the inspection work being affected by the restriction of certain functions when they are needed during the task. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a block diagram illustrating the principle of the present invention. Detailed Implementation

[0026] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0027] like Figure 1 As shown, the inspection operation assistance and guidance system based on wearable PDA includes a preparation module and an operation module;

[0028] The preparation module is used to analyze the network environment of the corresponding work area based on the inspection task information before using the wearable PDA system to assist in the inspection task guidance. This analysis determines whether the network environment meets the requirements of the inspection task, as some functions of the wearable PDA system will be limited when a network connection is unavailable. Therefore, it is necessary to analyze the task information in advance to avoid situations where certain functions are required during the task but are restricted, affecting the inspection process and potentially causing interruption. This preparation analysis is crucial and includes the following steps:

[0029] Obtain task information, including task content, job location, and other data. Locate the corresponding job location in the network graph based on the task information, and obtain the corresponding network score based on the job location.

[0030] Based on the task information, identify the functions that may be applied in this inspection task and mark them as predicted functions. Also, determine the usage probability and network score range for each predicted function. The usage probability is the probability of applying the function based on the task information. The network score range refers to the network score required for the function to be used. Because the types of task information are limited, the predicted functions, usage probabilities, and network score ranges for different task scenarios can be preset manually. During subsequent application, matching can be performed based on the task information. The obtained predicted functions, usage probabilities, and network score ranges are then integrated into task detail data.

[0031] A task preparation report is generated based on task details and online ratings, and the obtained task preparation report is sent to the relevant staff.

[0032] The network diagram is set up based on the areas of responsibility of enterprises, etc. An initial diagram is set up according to the existing work areas, and the initial diagram is updated according to changes in the work areas, such as the location of various equipment that need to be inspected. Therefore, the initial diagram may not be a continuous whole. Network environment data of each work area in the initial diagram is collected and analyzed to obtain the corresponding network score. The network environment data of each work area can be obtained by combining various network interference situations, relevant historical data, on-site detection and other methods, and then combined with the corresponding network scoring method for scoring. Existing network scoring methods can be used for evaluation, or a custom scoring method can be defined. The obtained network score is marked at the corresponding position in the initial diagram and continuously updated according to subsequent data, such as the network score of the work position detected on-site during the inspection process. The current initial diagram is marked as the network diagram.

[0033] It is also possible to dynamically correct the network score by combining relevant weather data and other data that affect the network environment. That is, based on the original network score, the current network influencing factors are analyzed to obtain the corresponding correction coefficient, and the network score is corrected based on the correction coefficient; or, an artificial intelligence model is built by combining existing neural networks, and a corresponding training set is built manually for training. The trained artificial intelligence model is then analyzed to realize the correction based on the current network influencing factors; other existing methods can also be used for correction.

[0034] Methods for generating task preparation reports based on task details data and online ratings include:

[0035] Based on the task details data and the relationship between the network rating range and the network score for each prediction function, analyze whether the prediction function can operate normally. If it can operate normally, output a normal operation label. If it cannot operate normally, output the corresponding pre-preparation measures when the prediction function is not used but may be needed. Enterprises and other entities can then pre-set corresponding preparation methods based on the actual situation. Integrate the output results to generate a work preparation report, indicating what needs to be prepared, what does not need to be prepared, and suggesting how to prepare if necessary.

[0036] In one embodiment, because the job preparation report lacks a holistic evaluation, users and managers cannot quickly and intuitively gain an understanding of the task; therefore, a feasible value is added to the job preparation report.

[0037] The methods for calculating feasible values ​​include:

[0038] Based on the task details data and network scores, generate individual evaluation values ​​for each prediction function, label the prediction function as i, where i = 1, 2, ..., n, and n is a positive integer; label the individual evaluation value as DPi, label the probability of use as LVi, and calculate the corresponding feasible value according to the feasibility formula KP = ∑(LVi × DPi).

[0039] Methods for generating individual ratings for each prediction function based on task details data and network scores include:

[0040] The evaluation is based on the relationship between the network score and each network score interval. When the score is within the network score interval, including the boundary, it is the full value of the single item rating. When the score is below the lower boundary of the network score interval, the difference between the lower boundary of the interval and the network score is calculated. The corresponding single item rating is matched according to the calculated difference. The single item threshold matching table corresponding to each prediction function is preset, including the single item rating corresponding to each difference. It can also be combined with the corresponding interpolation method for matching and setting.

[0041] By setting up a preparation module, it is ensured that the wearable PDA system will not cause the operation to stop due to network problems during actual application. The module analyzes the task situation in advance, helps users understand the usage details of the wearable PDA system in advance, which functions may have usage restrictions, and prompts users to prepare in advance to avoid the inspection work being affected by the restriction of certain functions when they are needed during the task.

[0042] The task module is used to guide users in performing inspection tasks;

[0043] Operational instructions are provided through videos, 3D animations, and text; the inspection outline, item order, precautions, technical standards, inspection content, and operation methods are displayed on the wearable PDA; and AR augmented reality technology is used to overlay the virtual and real worlds to achieve visual augmented reality operation instructions.

[0044] Configurable job content: Provides a visual configuration function for job content, enabling configuration of operation sequence, precautions, technical standards, inspection content and methods, text, images, videos, animations and other operation content; staff can transfer job content to wearable PDA via USB wired connection at the training base to solve the problem of job content version updates.

[0045] Inspection sequence instruction: The graphical interface instructs staff to perform inspections according to the set steps. Wearable PDAs and AR glasses sequentially display the next inspection point and mark the inspection results. Staff can then perform the inspections accordingly. Different inspection items can be displayed in different colors according to their importance level.

[0046] Inspection route guidance: Through technologies such as 3D modeling and real-world scene fusion, precise positioning based on the base point location is achieved. AR glasses guide staff to the inspection work location with a graphical interface, avoiding the situation of finding the wrong inspection location.

[0047] Synchronized voice prompts: When inspecting the corresponding parts, the wearable AR glasses provide synchronized voice prompts based on safety precautions, technical standards, inspection content and operating methods, improving work efficiency;

[0048] Data storage and transmission: The system can temporarily store the work status, abnormal situations, and images of the inspected areas of the staff during the inspection process on the wearable PDA terminal. When necessary, the data information can be transmitted to the base terminal or central control room via USB wired connection.

[0049] While inspecting the affected area, the AR glasses device captures images of the area to be inspected and compares them with pre-stored photos of the area to confirm the accuracy of the inspection. The image recognition uses a grayscale block matching method, which can register images not only after horizontal and vertical displacement but also after rotation or scaling. Block matching involves selecting a square area from the image to be matched as a block template for matching another image. This method is a high-precision, computationally efficient matching method. The entire matching process focuses on a single block within the overlapping area of ​​the images, primarily searching for the most similar matching block in the other image. Image information best reflects grayscale information; after block extraction, the image's features are more concentrated on its edge information. Therefore, compared to grayscale information, edge information provides stronger resistance to grayscale and geometric distortions, and edge information also exhibits greater stability in image stitching when constructing templates. Therefore, we use the image's edges as features to find the reference block.

[0050] When necessary, it runs on the field terminal computer and the central control room computer (server) to collect data transmitted from different AR glasses devices, understand the on-site inspection situation in real time, store the data, and establish an intelligent diagnostic platform.

[0051] By accumulating a large amount of fault data, a rich fault model knowledge base has been established. Combined with advanced artificial intelligence and big data processing technologies, an expert system has been built on the basis of existing maintenance manuals, experience cases and data. This system has the advantage of diagnosing complex faults, can improve the accuracy of equipment fault diagnosis, and further improve the efficiency of "condition-based maintenance".

[0052] The equipment consists of AR glasses and a wearable PDA device. Workers perform inspections according to the steps and methods indicated by the wearable PDA software. Simultaneously, the AR glasses provide virtual-real 3D work guidance, video guidance, and route directions, indicating the steps and methods for inspection. Inspection data and anomaly information are recorded and stored in the system. When necessary, this data can be transmitted to a field terminal computer for comparison, analysis, judgment, storage, and remote transmission.

[0053] The AR glasses device is designed as a head-mounted unit, consisting of a lithium battery, a computer, a camera, and a data cable. It features touch, voice, wireless transmission, and data storage capabilities. The computer handles all power supply, operational status control commands, image acquisition, and voice functions. The lithium battery and computer are integrated into a single unit, with a reserved external power supply interface for extended power and working time.

[0054] Power supply, data transmission, and voice information are integrated into the AR glasses device, making it easy to carry.

[0055] The wearable PDA device is an ergonomically designed handheld terminal with a compact, wearable design that helps workers easily free their hands, speed up operation, and improve work efficiency. It uses a color industrial-grade capacitive screen with fine display, high touch sensitivity, and supports multi-touch and glove-based touch.

[0056] The device features a low-power design and a professional magnetic interface for easy charging and data transfer; it also meets IP65 industrial protection standards and 1.5-meter drop resistance, ensuring the device's durability.

[0057] The data processing center includes field terminal computers and field base servers. The field base servers and field terminal computers receive and store data transmitted from different terminals. When necessary, they can store records of field problems and handling methods, accumulate diagnostic experience, realize the sharing of maintenance and inspection technologies, and provide support for data trend analysis and data mining analysis, thereby improving work efficiency.

[0058] Data transmission: The AR glasses device uses Wi-Fi to communicate with the wearable PDA for inspection and operation control. The wearable PDA communicates with the field terminal computer via USB data cable. The field terminal computer and the field base server communicate via Ethernet. For security reasons, all data is encrypted before transmission.

[0059] The above formulas are all numerical calculations after removing dimensions. The formulas are obtained by software simulation based on a large amount of data and are closest to the real situation. The preset parameters and preset thresholds in the formulas are set by those skilled in the art according to the actual situation or obtained by simulation based on a large amount of data.

[0060] The above embodiments are only used to illustrate the technical methods of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical methods of the present invention without departing from the spirit and scope of the technical methods of the present invention.

Claims

1. An inspection operation assistance and guidance system based on a wearable PDA, characterized in that, Includes a preparation module and an assignment module; The preparation module is used to analyze the network environment of the work area based on the task information of the inspection operation before conducting the inspection operation guidance, and obtain the corresponding operation preparation report. The operation module is used to guide users to perform inspection tasks, configure the task content, wear AR glasses and PDA devices, and instruct users to perform inspections according to the set steps through a graphical interface. The wearable PDA and AR glasses sequentially display the next inspection point and mark the inspection results, which the user then uses to perform the inspection. The operation status, abnormal situations, and images of the inspected areas during the inspection process will be temporarily stored on the wearable PDA terminal; The working methods for preparing the module include: Obtain task information, locate the corresponding job position in the preset network graph based on the task information, and obtain the corresponding network score based on the job position; Based on the task information, determine the prediction function corresponding to the inspection task, as well as the usage probability and network score range of each prediction function, and integrate the obtained prediction function, usage probability and network score range into task detail data; A task preparation report is generated based on task details and online ratings, and the obtained task preparation report is sent to the relevant staff.

2. The inspection operation assistance and guidance system based on wearable PDA according to claim 1, characterized in that, Methods for setting up a network diagram include: An initial map is set up based on the available work areas, and the initial map is updated according to changes in the work areas; network environment data is collected and analyzed for each work area in the initial map to obtain the corresponding network score; the obtained network score is marked at the corresponding position in the initial map, and the current initial map is marked as the network map.

3. The inspection operation assistance and guidance system based on wearable PDA according to claim 2, characterized in that, The network score for the work location is adjusted based on the current network influencing factors.

4. The inspection operation assistance and guidance system based on wearable PDA according to claim 2, characterized in that, Add feasible values ​​to the job preparation report.

5. The inspection operation assistance and guidance system based on wearable PDA according to claim 4, characterized in that, The methods for calculating feasible values ​​include: Based on the task details data and network scores, generate individual ratings for each prediction function, label the prediction function as i, where i = 1, 2, ..., n, and n is a positive integer; label the individual rating as DPi, label the probability of use as LVi, and calculate the corresponding feasible value according to the feasibility formula KP = ∑(LVi × DPi).

6. The inspection operation assistance and guidance system based on wearable PDA according to claim 1, characterized in that, When inspecting the corresponding parts, the AR glasses provide synchronized voice prompts based on safety precautions, technical standards, inspection content, and operating methods.

7. The inspection operation assistance and guidance system based on wearable PDA according to claim 1, characterized in that, While performing the area inspection, the AR glasses capture images of the area to be inspected and compare them with pre-stored photos of the area to determine whether the user has inspected the correct area. The image recognition uses a grayscale block matching method.

8. The inspection operation assistance and guidance system based on wearable PDA according to claim 1, characterized in that, The AR glasses use WIFI to communicate with the wearable PDA for inspection and operation control. The wearable PDA communicates with the field terminal computer via USB data cable. The field terminal computer and the field base server communicate via Ethernet, and all data is transmitted after encryption.

Images