Portable knowledge storage method based on AR glasses

Abstract

The invention discloses a carry-on knowledge storage method based on AR glasses, and relates to the technical field of AR glasses, and the AR glasses comprise a head hoop type tightening belt, a turnover shaft body, a host and power supply assembly, an AR display assembly, a tightening adjustment assembly and a camera assembly. According to the method, carry-on knowledge management in a hydropower station scene is realized through seven steps: an AR glasses acquisition function is triggered to obtain field data and external files, a hydropower station exclusive knowledge graph is constructed after processing and is converted into a low-dimensional vector through a TransR model, and the low-dimensional vector is uploaded to a platform through a TCP protocol and is locally cached; association retrieval and intelligent recommendation are realized by combining a knowledge graph embedded vector and a VSM model, data consistency is guaranteed through offline-online collaborative synchronization, and collaborative sharing of multiple devices is supported. The method is adaptive to special environments such as weak light and offline environments of the hydropower station plant, the timeliness and accuracy of knowledge acquisition are improved, the hands of a user are liberated, and the operation and maintenance efficiency and the team cooperation capability are remarkably improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of AR, in particular to a method for carrying knowledge reserve based on AR glasses. BACKGROUND

[0002] Nowadays, knowledge reserve is particularly important for an enterprise, and knowledge base systems have been widely used. The importance of knowledge base systems for enterprises mainly lies in improving work efficiency, improving customer service quality, risk management, training and knowledge inheritance, decision support, collaboration and communication, and innovation and research and development, etc. Therefore, many enterprises have begun to pay attention to the construction and management of knowledge base systems to ensure that they can create greater value for enterprises.

[0003] In the ordinary knowledge reserve process, it is often necessary to create, consult and manage knowledge content through a computer. The operation area of a hydropower station plant is wide, and some areas lack network coverage. Moreover, during the operation process, the hands of the staff need to be focused on the operation of the equipment, and they cannot carry and use a computer, which leads to the inability to obtain knowledge base support in time when encountering technical problems, causing work stagnation and wasting time cost; secondly, poor adaptability to special environments. The light distribution inside the hydropower station plant is uneven, and the light in some areas is dim. Traditional shooting equipment cannot clearly capture the details of the equipment, which leads to inaccurate on-site knowledge collection, and further affects knowledge matching and reuse; thirdly, the knowledge reserve is disconnected from the actual application. The content of the traditional knowledge base is uploaded through the computer terminal, and there is a lack of rapid supplement of first-hand data on site. Moreover, the knowledge retrieval is mostly single keyword matching, which cannot excavate the association relationship between knowledge, and it is difficult to meet the multi-dimensional knowledge demand in the operation process; fourthly, knowledge cannot be used in offline scenarios. There is a weak or interrupted network signal in some remote operation areas of the hydropower station. The traditional knowledge base relies on real-time network connection, and cannot be accessed in offline state, which seriously affects the continuity of operation.

[0004] To solve the above problems, the present application provides a method for carrying knowledge reserve based on AR glasses, which combines the portability of AR glasses, intelligent hardware configuration and knowledge graph technology, builds a portable knowledge management system adapted to the scene of a hydropower station, breaks the use limit of traditional knowledge base, and realizes the collection, efficient storage, accurate retrieval and convenient sharing of knowledge anytime and anywhere. SUMMARY

[0005] The technical problem to be solved by this invention is to provide a portable knowledge storage method based on AR glasses, which overcomes the shortcomings of existing knowledge base systems in terms of ease of use, immediacy, accuracy, and collaboration in special work scenarios. Through the deep integration of AR glasses hardware and software algorithms, it solves problems such as inaccurate knowledge collection in low-light environments of hydropower plant buildings, inability to use knowledge offline, and insufficient knowledge association retrieval. It realizes portable knowledge management throughout the entire process, improves the practicality and efficiency of knowledge application, reduces the time cost of knowledge acquisition, and improves the work efficiency and teamwork ability of enterprise employees.

[0006] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: A portable knowledge storage method based on AR glasses, the AR glasses include a headband-style tightening strap, a flip axis, a main unit and power supply assembly, an AR display assembly, a tightening adjustment assembly and a camera assembly. The headband-style tightening strap can be adjusted for tightness through the tightening adjustment assembly. The flip axis can adjust the usage status of the AR display assembly and the tilt angle of the glasses. The main unit and power supply assembly (3) provides power to each functional component and controls the AR. The portable knowledge storage method includes the following steps: S1: Knowledge collection starts. Users can trigger the collection function of AR glasses through voice commands or gestures. The camera component and sensor work together to take photos or record videos of the equipment scene inside the hydropower station plant. It also supports importing external drawings, manuals, experience summary files, etc. S2: File processing: Users can rename photos or videos taken via voice commands, compress all knowledge files using compression algorithms, and detect and remove duplicate data in the files using big data analysis technology. S3: Construction and embedding of a knowledge graph specific to hydropower stations. A top-down construction process is adopted, which constructs a knowledge graph through ontology design, data acquisition and preprocessing, knowledge extraction, knowledge fusion and knowledge storage. The TransR model is used to transform the knowledge graph into a low-dimensional vector representation. S4: Data transmission and storage. The processed knowledge files and knowledge graphs are embedded in vectors and uploaded to the knowledge reserve platform via the Transmission Control Protocol (TCP). The platform automatically classifies and stores the files according to their types. At the same time, the AR glasses cache commonly used knowledge files and knowledge graph fragments locally. S5: Knowledge graph-based related retrieval and intelligent recommendation. Users input search keywords via voice commands, and the system combines knowledge graph embedded vectors and vector space model (VSM) for retrieval matching and simultaneously pushes related knowledge. S6: Offline-online collaborative synchronization. It supports local knowledge operations in offline mode, and updates data through incremental synchronization technology after the network is restored to ensure knowledge consistency. S7: Collaborative sharing application. In remote collaboration, teaching and training or meeting scenarios, users can access specified knowledge files and related graph information through AR glasses. It supports simultaneous display on multiple devices and interactive annotation by multiple people. In step S3 above, the ontology design steps of the hydropower station-specific knowledge graph include: abstracting six core entities: equipment, fault type, solution, accessories, operation scenario, and operation and maintenance personnel; defining four core relationships between entities: adaptation relationship, correspondence relationship, association relationship, and matching relationship; and forming the schema layer framework of the knowledge graph.

[0007] The knowledge extraction steps in the above knowledge graph construction adopt a three-level extraction mechanism: entity extraction, relation extraction, and attribute extraction. Entity extraction uses the BERT model to extract entity information such as equipment model and fault name. Relationship extraction uses the attention mechanism to mine the association between entities. Attribute extraction obtains attribute data such as equipment parameters and fault characteristics.

[0008] In step S3 above, the process of establishing the TransR model includes: Step S3-1: Map the entities in the knowledge graph to the d-dimensional entity space to obtain entity vectors h and t; Step S3-2: Introduce an independent k-dimensional relation space and projection matrix M for each relation r. r ∈R k ×d; Step S3-3: Project the entity vector onto the relation space to obtain the projected entity vector h. r =h•M r t r =t•M r Step S3-4: Optimize the vector representation of entities and relations using a scoring function, which is: f r (h,t)=-‖h r +rt r ||2²; Where r is the relation vector.

[0009] In step S5 above, the associated retrieval includes the following process: Step S5-1: Convert the user's search keywords into a query vector q using the BERT model; Step S5-2: Calculate the cosine similarity between the query vector and the entity embedding vector in the knowledge graph. The similarity formula is: , where e is the entity embedding vector; Step S5-3: Combining the calculation formula of the Vector Space Model (VSM) To achieve content matching, knowledge files corresponding to multi-hop related entities in the knowledge graph are simultaneously mined.

[0010] In step S5 above, intelligent recommendation employs an intent-aware relationship path aggregation strategy to construct an intent graph of users, intents, and items, using a formula... Modeling intent embedding, where For the intent vector, For relational attention weights, Let r be the trainable weights for relation r and intention q. This is the relation embedding vector.

[0011] In step S6 above, the offline-online collaborative synchronization adopts a contrastive learning framework to optimize the knowledge graph fragment caching strategy, generating two enhanced views by randomly deleting knowledge graph nodes. , Where V is the set of entities, , Let R be a mask vector and R be a set of relations. This calculates the structural consistency of the project across different views. s(•) is the cosine similarity function. , This allows for the embedded representation of a project across different views.

[0012] In step S6 above, incremental synchronization technology is implemented through the following process: Step S6-1: Record knowledge operation logs offline, including new files, annotation modifications, and search records; Step S6-2: After the network is restored, calculate the difference feature values ​​between local data and cloud data; Step S6-3: Upload only the newly added and modified difference data, and use the breakpoint resume protocol to ensure transmission reliability, with a synchronization delay of no more than 500ms.

[0013] In step S1 above, the camera component supports infrared fill light function, which is automatically activated when the light intensity is below 300 lux. The fill light intensity adjustment range is 50-300 lux. Combined with the ResNet50 model to extract device image features, it improves the accuracy of entity recognition in low light environment.

[0014] In step S3 above, the knowledge fusion step uses coreference resolution, entity disambiguation, and attribute alignment techniques. It eliminates ambiguity by calculating the similarity of entity features, and the entity matching accuracy is no less than 99%. The knowledge graph storage uses a graph database, which supports efficient querying and updating of triples (h,r,t).

[0015] In step S5 above, the comprehensive preference score of the intelligent recommendation is calculated using the formula... Calculation, where For multimodal preference scoring, For structural preference scores, , These are the final embedded representations for users and projects, respectively. β is the weighting parameter, 0 < β < 1.

[0016] In step S6 above, the knowledge graph fragments cached locally by the AR glasses adopt a multi-hop neighbor aggregation strategy, through the formula... Aggregate first-order neighbor information, where Let i be the set of neighboring nodes of project i. For relational embedding, Initial embedding of neighboring entities, It is an aggregate function.

[0017] In step S3 above, the knowledge graph supports dynamic updates. When a new device or knowledge file is added, it is updated using a formula. Update the embedding vector, where The set of adjacent entities of the newly added entity. For attention weights, This is the initial embedding vector.

[0018] In step S5 above, the retrieval process supports fuzzy matching correction. By calculating the edit distance between the retrieval keyword and the knowledge graph entity name, the matching result is automatically corrected and returned when the edit distance is ≤2, and the retrieval response time does not exceed 2 seconds.

[0019] In step S1 above, the annotation information from multi-person annotation interaction is stored in association through a knowledge graph, and a mapping relationship is established between the annotation content and the corresponding entity node, using a formula. Update entity embedding, where For labeled vectors, The weighting coefficients are labeled.

[0020] The portable knowledge storage method based on AR glasses mentioned in this invention has the following beneficial effects: 1. Adaptable to special working environments: The infrared supplementary lighting function of the AR glasses camera component, combined with the ResNet50 model, solves the problem of unclear equipment detail acquisition in the low light environment of the hydropower station plant, ensuring the accuracy of on-site knowledge acquisition; the offline-online collaborative synchronization technology adapts to the scenario of network interruption in some areas of the plant, ensuring the normal use of knowledge in offline state.

[0021] 2. Accurate and efficient knowledge retrieval: By constructing a knowledge graph specific to hydropower stations and combining the low-dimensional vector embedding and association retrieval algorithm of the TransR model, multi-dimensional association matching of knowledge is achieved. It can not only quickly locate target knowledge, but also push related knowledge to meet the various knowledge needs in the operation process. The retrieval response time is no more than 2 seconds.

[0022] 3. Convenient Knowledge Management: The portability of AR glasses and the voice and gesture interaction methods enable hands-free operation throughout the entire knowledge management process. Users can complete knowledge collection, retrieval, and sharing without interrupting their work, significantly improving work efficiency. The combination of local caching and cloud storage solves the problem of inconvenient data carrying, enabling knowledge to be accessed anytime.

[0023] 4. Highly efficient collaboration and knowledge transfer: Simultaneous display across multiple devices and interactive annotation features enhance team communication efficiency in scenarios such as remote collaboration, on-site training, and meetings; dynamic updates and associated storage of the knowledge graph facilitate the rapid accumulation and transfer of on-site experience, forming a standardized knowledge base system.

[0024] 5. Reliable data transmission and storage: The TCP protocol is used to optimize data transmission, combined with incremental synchronization and breakpoint resume technology to ensure the reliability and efficiency of data transmission; the knowledge graph is stored in a graph database, supporting efficient querying and updating of triples, ensuring the stable operation of the knowledge base. Attached Figure Description

[0025] The present invention will be further described below with reference to the accompanying drawings and embodiments: Figure 1 This is a schematic diagram of the AR glasses structure of the present invention; Figure 2 This is a schematic diagram illustrating the on-the-go knowledge storage and retrieval process of the AR glasses in this invention; Figure 3 This is a schematic diagram of the vector space algorithm model of the present invention.

[0026] The components include: 1. Headband-type tightening strap; 2. Flip shaft; 3. Main unit and power supply assembly; 4. AR display assembly; 5. Tightening adjustment assembly; and 6. Camera assembly. Detailed Implementation

[0027] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0028] A method for portable knowledge storage based on AR glasses. The AR glasses include a headband-style tightening strap 1, a flip axis 2, a main unit and power supply assembly 3, an AR display assembly 4, a tightening adjustment assembly 5, and a camera assembly 6. The headband-style tightening strap 1 allows for adjustment of the wearing tightness via the tightening adjustment assembly 5. The flip axis 2 can adjust the usage status of the AR display assembly 4 and the tilt angle of the glasses. The main unit and power supply assembly 3 provides power to each functional component and controls the AR. This portable knowledge storage method includes the following steps: S1: Knowledge collection starts. Users can trigger the collection function of AR glasses through voice commands or gestures. Camera component 6 works in conjunction with sensors to take photos or record videos of equipment scenes inside the hydropower station plant. It also supports importing external drawings, manuals, experience summary files, etc. S2: File processing: Users can rename photos or videos taken via voice commands, compress all knowledge files using compression algorithms, and detect and remove duplicate data in the files using big data analysis technology. S3: Construction and embedding of a knowledge graph specific to hydropower stations. A top-down construction process is adopted, which constructs a knowledge graph through ontology design, data acquisition and preprocessing, knowledge extraction, knowledge fusion and knowledge storage. The TransR model is used to transform the knowledge graph into a low-dimensional vector representation. S4: Data transmission and storage. The processed knowledge files and knowledge graphs are embedded in vectors and uploaded to the knowledge reserve platform via the Transmission Control Protocol (TCP). The platform automatically classifies and stores the files according to their types. At the same time, the AR glasses cache commonly used knowledge files and knowledge graph fragments locally. S5: Knowledge graph-based related retrieval and intelligent recommendation. Users input search keywords via voice commands, and the system combines knowledge graph embedded vectors and vector space model (VSM) for retrieval matching and simultaneously pushes related knowledge. S6: Offline-online collaborative synchronization. It supports local knowledge operations in offline mode, and updates data through incremental synchronization technology after the network is restored to ensure knowledge consistency. S7: Collaborative sharing application. In remote collaboration, teaching and training or meeting scenarios, users can access specified knowledge files and related graph information through AR glasses. It supports simultaneous display on multiple devices and interactive annotation by multiple people. Using AR glasses as the hardware platform, this system integrates the TransR model, VSM model, and incremental synchronization technology through seven steps: "collection, processing, knowledge map construction, transmission and storage, retrieval and recommendation, collaboration and synchronization, and sharing." This creates a comprehensive, portable knowledge management system. Breaking away from the traditional reliance on computers and networks, it enables portable knowledge management in hydropower station scenarios, freeing up users' hands and improving overall operational efficiency.

[0029] In step S3 above, the ontology design steps for the hydropower station-specific knowledge graph include: abstracting six core entities: equipment, fault types, solutions, spare parts, operating scenarios, and maintenance personnel; defining four core relationships between entities: adaptation, correspondence, association, and matching; and forming the schema layer framework of the knowledge graph. A top-down ontology design approach is adopted to abstract the core entities and relationships of the hydropower station scenario, building the schema layer framework of the knowledge graph. This clarifies the core components of the knowledge graph, ensuring its adaptation to the specific scenarios of hydropower station equipment operation and maintenance, fault handling, etc., laying the foundation for subsequent association retrieval. The knowledge extraction steps in the aforementioned knowledge graph construction employ a three-level extraction mechanism: entity extraction, relation extraction, and attribute extraction. Entity extraction uses the BERT model to extract entity information such as equipment model and fault name; relation extraction uses an attention mechanism to mine the relationships between entities; and attribute extraction obtains attribute data such as equipment parameters and fault characteristics. This three-level knowledge extraction mechanism, combining entity extraction using the BERT model, relation mining using the attention mechanism, and attribute extraction using features, achieves accurate extraction of multiple types of knowledge. It efficiently extracts key information from structured and unstructured data related to hydropower stations, ensuring the integrity and accuracy of the knowledge graph data.

[0030] In step S3 above, the process of establishing the TransR model includes: Step S3-1: Map the entities in the knowledge graph to the d-dimensional entity space to obtain entity vectors h and t; Step S3-2: Introduce an independent k-dimensional relation space and projection matrix M for each relation r. r ∈R k ×d; Step S3-3: Project the entity vector onto the relation space to obtain the projected entity vector h. r =h•M r t r =t•M r Step S3-4: Optimize the vector representation of entities and relations using a scoring function, which is: f r (h,t)=-‖h r +rt r ||2²; Where r is the relation vector.

[0031] The TransR model separates entity and relation spaces, projecting entity vectors onto the corresponding relation space and optimizing vector representations through a scoring function. This addresses the issue of inaccurate semantic representation of entities in complex relationships, improves the rationality of low-dimensional vector embedding in knowledge graphs, and provides reliable support for association retrieval.

[0032] In step S5 above, the associated retrieval includes the following process: Step S5-1: Convert the user's search keywords into a query vector q using the BERT model; Step S5-2: Calculate the cosine similarity between the query vector and the entity embedding vector in the knowledge graph. The similarity formula is: , where e is the entity embedding vector; Step S5-3: Combining the calculation formula of the Vector Space Model (VSM) To achieve content matching, knowledge files corresponding to multi-hop related entities in the knowledge graph are simultaneously mined.

[0033] Search keywords are vectorized using the BERT model and combined with cosine similarity calculation and the VSM model to achieve dual matching between keywords and knowledge graph entities and content. This improves search accuracy, uncovers multi-hop related knowledge, meets the multi-dimensional knowledge requirements in the assignment process, and avoids knowledge omissions caused by single matching.

[0034] In step S5 above, intelligent recommendation employs an intent-aware relationship path aggregation strategy to construct an intent graph of users, intents, and items, using a formula... Modeling intent embedding, where For the intent vector, For relational attention weights, Let r be the trainable weights for relation r and intention q. This is the relation embedding vector.

[0035] We construct a user-intent-project intention graph, assign relationship weights through an attention mechanism, model intent embedding vectors, and quantify user structural preferences. This accurately captures users' potential needs, enables personalized knowledge recommendations, reduces irrelevant knowledge interference, and improves knowledge acquisition efficiency.

[0036] In step S6 above, the offline-online collaborative synchronization adopts a contrastive learning framework to optimize the knowledge graph fragment caching strategy, generating two enhanced views by randomly deleting knowledge graph nodes. , Where V is the set of entities, , Let R be a mask vector and R be a set of relations. This calculates the structural consistency of the project across different views. s(•) is the cosine similarity function. , This approach embeds the project's representation across different views. A contrastive learning framework is employed to generate knowledge graph-enhanced views, calculate project structural consistency, and optimize local caching strategies. This reduces the impact of knowledge graph noise, ensuring the high value of locally cached knowledge graph fragments and improving retrieval performance in offline scenarios.

[0037] In step S6 above, incremental synchronization technology is implemented through the following process: Step S6-1: Record knowledge operation logs offline, including new files, annotation modifications, and search records; Step S6-2: After the network is restored, calculate the difference feature values ​​between local data and cloud data; Step S6-3: Upload only the newly added and modified difference data, using a breakpoint resume protocol to ensure transmission reliability, with a synchronization latency not exceeding 500ms. Record operation logs in offline mode; after network recovery, calculate data difference characteristic values, and synchronize only the newly added and modified data, in conjunction with the breakpoint resume protocol. This reduces data transmission volume, lowers network dependence, ensures no loss of offline operation data, controls synchronization latency within 500ms, and improves data synchronization efficiency and reliability.

[0038] In step S1 above, the camera component 6 supports infrared supplementary lighting, which automatically activates when the light intensity is below 300 lux. The supplementary lighting intensity is adjustable from 50 to 300 lux. Combined with the ResNet50 model to extract device image features, this improves the accuracy of entity recognition in low-light environments. The camera component integrates an infrared supplementary lighting module, which automatically activates supplementary lighting in low-light environments, working in conjunction with the ResNet50 model to extract image features. This solves the problem of unclear equipment detail acquisition in low-light environments at hydropower plants, improves entity recognition accuracy, and ensures the effectiveness of on-site knowledge acquisition.

[0039] In step S3 above, the knowledge fusion step employs coreference resolution, entity disambiguation, and attribute alignment techniques. It eliminates ambiguity by calculating entity feature similarity, achieving an entity matching accuracy of no less than 99%. The knowledge graph is stored in a graph database, supporting efficient querying and updating of triples (h,r,t). The knowledge fusion stage utilizes coreference resolution, entity disambiguation, and attribute alignment techniques, eliminating ambiguity through similarity calculation, and storing triple data in a graph database. This ensures the consistency and uniqueness of the knowledge graph data, achieving an entity matching accuracy of no less than 99%, and supporting efficient graph querying and updating.

[0040] In step S5 above, the comprehensive preference score of the intelligent recommendation is calculated using the formula... Calculation, where For multimodal preference scoring, For structural preference scores, , These are the final embedded representations for users and projects, respectively. β represents the weighting parameter, where 0 < β < 1. A comprehensive preference evaluation model is constructed by weighted fusion of multimodal preference scores and structural preference scores to balance user preferences across different dimensions. This improves the comprehensiveness and accuracy of intelligent recommendations, taking into account users' preferences for knowledge formats and their structural needs, thus optimizing the recommendation experience.

[0041] In step S6 above, the knowledge graph fragments cached locally by the AR glasses adopt a multi-hop neighbor aggregation strategy, through the formula... Aggregate first-order neighbor information, where Let i be the set of neighboring nodes of project i. For relational embedding, Initial embedding of neighboring entities, This is an aggregation function. A multi-hop neighbor aggregation strategy is employed to aggregate the relationships and entity embeddings of first-order neighbor nodes, generating a compact representation of the knowledge graph fragment. This reduces the amount of data cached locally while retaining key association information, ensuring the availability and retrieval efficiency of the knowledge graph fragment in offline scenarios.

[0042] In step S3 above, the knowledge graph supports dynamic updates. When a new device or knowledge file is added, it is updated using a formula. Update the embedding vector, where The set of adjacent entities of the newly added entity. For attention weights, This serves as the initial embedding vector. When a new entity is added, its vector representation is dynamically updated based on the embeddings of neighboring entities and attention weights, enabling incremental embedding of the knowledge graph. It supports dynamic expansion of the knowledge graph without retraining the entire graph, reducing update costs and ensuring that new knowledge is quickly integrated into the existing system.

[0043] In step S5 above, the retrieval process supports fuzzy matching correction. By calculating the edit distance between the search keyword and the entity name in the knowledge graph, the system automatically corrects and returns the matching result when the edit distance is ≤2, with a retrieval response time of no more than 2 seconds. By calculating the edit distance between the search keyword and the entity name, misspelled or incomplete keywords are corrected, and an edit distance threshold is set to control the correction range. This improves retrieval fault tolerance, avoids retrieval failures caused by keyword input errors, ensures a retrieval response time of no more than 2 seconds, and guarantees smooth retrieval.

[0044] In step S1 above, the annotation information from multi-person annotation interaction is stored in association through a knowledge graph, and a mapping relationship is established between the annotation content and the corresponding entity node, using a formula. Update entity embedding, where For labeled vectors, Weighting coefficients are assigned to the annotations. A mapping relationship is established between the annotation information and entity nodes in the knowledge graph, and the annotation information is integrated into the entity embedding through a vector update formula. This enables the effective accumulation and reuse of annotated knowledge, enriches the semantic representation of entities, improves the accuracy of subsequent retrieval and recommendation, and promotes collaborative optimization of knowledge.

[0045] Example 1: Application of equipment fault diagnosis and repair in hydropower stations During maintenance work on turbine group A at a hydroelectric power station, employees wore the AR glasses of this invention. The work area was dimly lit, with a light intensity of 220 lux. The AR glasses' camera component 6 automatically activated its infrared supplementary lighting function, adjusting the intensity to 150 lux. The employee discovered a vibration fault in the turbine and, via voice command "Take a picture of the fault site," the camera component 6 clearly captured an image of the faulty area. The ResNet50 model extracted the equipment features. Subsequently, via voice command "Name it Turbine Group A Vibration Fault - 20240810," the image was compressed using the LZ77 algorithm and temporarily stored locally.

[0046] Since there was no network in the work area, employees used the voice command "Search for turbine vibration faults." The AR glasses then accessed locally cached knowledge graph fragments and related knowledge files. The system converted the search keywords into query vectors using a BERT model, calculated the cosine similarity between these vectors and the entity embedding vectors in the knowledge graph, and combined this with a VSM model to quickly match the knowledge files "turbine vibration fault solutions," "vibration sensor calibration manual," and the associated "parts replacement process." Employees then used the voice command "Multi-window display," and the AR display component 4 simultaneously displayed the three files. They used "page turning" and "zoom in" commands to view key content and quickly determine the cause of the fault and its solution.

[0047] After the task is completed, the employee moves to a network-covered area. The system automatically triggers incremental synchronization, uploading only newly added fault scene images and operation logs. Reliable data transmission is ensured through TCP protocol and breakpoint resume technology, with a synchronization latency of 320ms. New images and related operations are synchronized to the cloud-based knowledge reserve platform, and the knowledge graph automatically updates its embedded vectors to support subsequent handling of similar faults.

[0048] Example 2: Remote Expert Collaboration Scenario Application When employees at a hydroelectric power station encountered technical difficulties while overhauling electrical control instruments, they initiated a remote expert consultation using the AR glasses of this invention. At the time, the network signal in the work area was weak. The employee used the voice command "Retrieve drawing B for electrical control instrument." The AR glasses then accessed the locally cached drawing file and corresponding knowledge graph fragments, simultaneously synchronizing the file to the expert's computer with a synchronization delay of 450ms.

[0049] Experts reviewed the drawings on a computer and discovered that the instrument malfunction was related to sensor parameters. They then marked key parameter areas with annotations, synchronized the annotation information to the employees' AR display components (4), and used formulas... Update the embedding vectors of the corresponding entities in the knowledge graph. Following expert guidance, employees use the voice command "Search for sensor parameter calibration methods." The system, based on the association search of the knowledge graph, pushes "Electrical Control Instrument Sensor Calibration Manual" and related cases, and prioritizes displaying highly relevant content based on the comprehensive preference score of intelligent recommendation.

[0050] Under the guidance of experts, the employees successfully completed the maintenance work. At the same time, they recorded a high-definition video of the calibration process by using the voice command "record calibration process video". The video was compressed by the LZW algorithm and stored locally. After the network was restored, it was automatically synchronized to the cloud knowledge base, and the knowledge graph was dynamically updated to reflect the maintenance case relationships of the equipment.

[0051] Example 3: Application in on-site training and teaching scenarios A hydropower station conducted equipment operation training for new employees, with both instructors and new employees wearing the AR glasses of this invention. The instructor used voice commands to "demonstrate the turbine operation specifications," and the AR glasses accessed the operation specifications document and corresponding knowledge graph from the knowledge base, projecting the document onto a large screen on-site. New employees could then simultaneously access the document through their respective AR glasses.

[0052] During training, instructors used gestures to mark key operational steps in the document, establishing a mapping between the marked information and the "water turbine operation" entity node in the knowledge graph. Simultaneously, the instructor used voice commands to "shoot an operation demonstration video," and camera component 6 recorded the demonstration process, using infrared illumination to ensure clarity in low-light areas. New employees can also retrieve relevant knowledge points using voice commands during training; the system, based on an intent-aware relationship path aggregation strategy, pushes personalized related knowledge.

[0053] After the training is completed, all training materials are automatically categorized and stored in the knowledge base, and the knowledge graph is updated with the entity relationships related to the training. New employees can retrieve and review the training materials at any time in subsequent tasks, enabling rapid knowledge transfer.

[0054] The present invention provides a portable knowledge storage method based on AR glasses. Through the deep integration of AR glasses hardware with knowledge graphs, comparative learning and other technologies, it realizes portable, efficient and precise knowledge management in the special scenario of hydropower stations. It effectively solves the limitations of traditional knowledge bases, improves work efficiency and collaboration capabilities, and has broad application prospects.

Claims

1. A method for portable knowledge storage based on AR glasses, characterized in that, AR glasses include a headband (1), a flip axis (2), a main unit and power supply assembly (3), an AR display assembly (4), a tightening adjustment assembly (5), and a camera assembly (6). The headband (1) adjusts the tightness of the glasses through the tightening adjustment assembly (5). The flip axis (2) adjusts the usage status of the AR display assembly (4) and the tilt angle of the glasses. The main unit and power supply assembly (3) provides power to each functional component and controls the AR. This portable knowledge storage method includes the following steps: S1: Knowledge collection starts. The user triggers the collection function of the AR glasses through voice commands or gestures. The camera component (6) works in conjunction with the sensor to take photos or record videos of the equipment scene in the hydropower plant. It also supports the import of external drawings, manuals, experience summary files, etc. S2: File processing: Users can rename photos or videos taken via voice commands, compress all knowledge files using compression algorithms, and detect and remove duplicate data in the files using big data analysis technology. S3: Construction and embedding of a knowledge graph specific to hydropower stations. It adopts a top-down construction process, and constructs a knowledge graph through ontology design, data acquisition and preprocessing, knowledge extraction, knowledge fusion and knowledge storage. The TransR model is used to transform the knowledge graph into a low-dimensional vector representation. S4: Data transmission and storage. The processed knowledge files and knowledge graphs are embedded in vectors and uploaded to the knowledge reserve platform via the Transmission Control Protocol (TCP). The platform automatically classifies and stores them according to file type. At the same time, the AR glasses cache commonly used knowledge files and knowledge graph fragments locally. S5: Knowledge graph-based related retrieval and intelligent recommendation. Users input search keywords via voice commands, and the system combines knowledge graph embedding vectors and vector space model (VSM) for retrieval matching and simultaneously pushes related knowledge. S6: Offline-online collaborative synchronization. It supports local knowledge operations in offline mode, and updates data through incremental synchronization technology after the network is restored to ensure knowledge consistency. S7: Collaborative sharing application. In remote collaboration, teaching and training or meeting scenarios, users can access specified knowledge files and related graph information through AR glasses. It supports simultaneous display on multiple devices and interactive annotation by multiple people.

2. The method for storing portable knowledge based on AR glasses according to claim 1, characterized in that, In step S3, the ontology design steps of the hydropower station-specific knowledge graph include: abstracting six core entities: equipment, fault type, solution, accessories, operation scenario, and operation and maintenance personnel; defining four core relationships between entities: adaptation relationship, correspondence relationship, association relationship, and matching relationship; and forming the schema layer framework of the knowledge graph.

3. The method for storing portable knowledge based on AR glasses according to claim 1, characterized in that, The knowledge extraction steps in the knowledge graph construction adopt a three-level extraction mechanism: entity extraction, relation extraction, and attribute extraction. Entity extraction uses the BERT model to extract entity information such as equipment model and fault name. Relationship extraction uses the attention mechanism to mine the association between entities. Attribute extraction obtains attribute data such as equipment parameters and fault characteristics.

4. The method for storing portable knowledge based on AR glasses according to claim 1, characterized in that, In step S3, the process of establishing the TransR model includes: Step S3-1: Map the entities in the knowledge graph to the d-dimensional entity space to obtain entity vectors h and t; Step S3-2: Introduce an independent k-dimensional relation space and projection matrix M for each relation r. r ∈R k ×d; Step S3-3: Project the entity vector onto the relation space to obtain the projected entity vector h. r =h•M r t r =t•M r Step S3-4: Optimize the vector representation of entities and relations using a scoring function, which is: f r (h,t)=-‖h r +r-t r ‖2²; Where r is the relation vector.

5. The method for storing portable knowledge based on AR glasses according to claim 1, characterized in that, In step S5, the associated retrieval includes the following process: Step S5-1: Convert the user's search keywords into a query vector q using the BERT model; Step S5-2: Calculate the cosine similarity between the query vector and the entity embedding vector in the knowledge graph. The similarity formula is: , where e is the entity embedding vector; Step S5-3: Combining the calculation formula of the Vector Space Model (VSM) To achieve content matching, knowledge files corresponding to multi-hop related entities in the knowledge graph are simultaneously mined.

6. The method for storing portable knowledge based on AR glasses according to claim 1, characterized in that, In step S5, the intelligent recommendation employs an intent-aware relationship path aggregation strategy to construct an intent graph of users, intents, and items, using a formula... Modeling intent embedding, where For the intent vector, For relational attention weights, Let r be the trainable weights for relation r and intention q. This is the relation embedding vector.

7. The method for storing portable knowledge based on AR glasses according to claim 1, characterized in that, In step S6, the offline-online collaborative synchronization adopts a contrastive learning framework to optimize the knowledge graph fragment caching strategy, generating two enhanced views by randomly deleting knowledge graph nodes. , Where V is the set of entities, , Let R be a mask vector and R be a set of relations. This calculates the structural consistency of the project across different views. s(•) is the cosine similarity function. , This allows for the embedded representation of a project across different views.

8. The method for storing portable knowledge based on AR glasses according to claim 1, characterized in that, In step S6, the incremental synchronization technique is implemented through the following process: Step S6-1: Record knowledge operation logs offline, including new files, annotation modifications, and search records; Step S6-2: After the network is restored, calculate the difference feature values ​​between local data and cloud data; Step S6-3: Upload only the newly added and modified difference data, and use the breakpoint resume protocol to ensure transmission reliability, with a synchronization delay of no more than 500ms.

9. A method for storing portable knowledge based on AR glasses according to claim 1, characterized in that, In step S1, the camera component (6) supports infrared supplementary lighting function, which is automatically activated when the light intensity is below 300 lux. The supplementary lighting intensity is adjustable from 50 to 300 lux. It works with the ResNet50 model to extract device image features and improve the accuracy of entity recognition in low light environment.

10. A method for storing portable knowledge based on AR glasses according to claim 1, characterized in that, In step S3, the knowledge fusion step employs coreference resolution, entity disambiguation, and attribute alignment techniques. It eliminates ambiguity by calculating entity feature similarity, achieving an entity matching accuracy of no less than 99%. The knowledge graph is stored using a graph database, supporting efficient querying and updating of triples (h,r,t).

11. The method for storing portable knowledge based on AR glasses according to claim 1, characterized in that, In step S5, the comprehensive preference score of the intelligent recommendation is calculated using the formula... Calculation, where For multimodal preference scoring, For structural preference scores, , These are the final embedded representations for users and projects, respectively. β is the weighting parameter, 0 < β < 1.

12. The method for storing portable knowledge based on AR glasses according to claim 1, characterized in that, In step S6, the knowledge graph fragments cached locally by the AR glasses employ a multi-hop neighbor aggregation strategy, using the formula... Aggregate first-order neighbor information, where Let i be the set of neighboring nodes of project i. For relational embedding, Initial embedding of neighboring entities, It is an aggregate function.

13. The method for storing portable knowledge based on AR glasses according to claim 1, characterized in that, In step S3, the knowledge graph supports dynamic updates. When a new device or knowledge file is added, it is updated using a formula. Update the embedding vector, where The set of adjacent entities of the newly added entity. For attention weights, This is the initial embedding vector.

14. The method for storing portable knowledge based on AR glasses according to claim 1, characterized in that, In step S5, the retrieval process supports fuzzy matching correction. By calculating the edit distance between the retrieval keyword and the entity name in the knowledge graph, the matching result is automatically corrected and returned when the edit distance is ≤2, and the retrieval response time does not exceed 2 seconds.

15. A method for storing portable knowledge based on AR glasses according to claim 1, characterized in that, In step S1, the annotation information from multi-user annotation interaction is stored in association through a knowledge graph, and a mapping relationship is established between the annotation content and the corresponding entity node, using a formula. Update entity embedding, where For labeled vectors, The weighting coefficients are labeled.

Images