An ar remote assistance method and system for installation and debugging of a communication device

Abstract

This invention provides an AR remote assistance method and system for the installation and debugging of communication equipment. By constructing a three-in-one collaborative architecture of AR smart terminal, expert collaboration platform, and cloud service brain, it achieves remote immersive assistance throughout the entire process of communication equipment installation and debugging. The method includes: the AR smart terminal collecting first-person perspective data and environmental information from the site, and uploading it to the cloud service brain via ultra-low latency transmission technology; the cloud service brain achieving spatial positioning and virtual-real overlay based on multi-source fusion SLAM technology, and completing equipment identification, instrument reading extraction, and fault diagnosis through AI models; the expert collaboration platform supporting multi-user collaborative annotation and real-time interaction, combined with guidance pushed from a communication industry-specific knowledge base; and achieving three-dimensional spatial consistency of multi-terminal virtual annotations through spatial anchor point sharing technology. This invention solves the problems of poor immersion, inaccurate information transmission, and low utilization of expert resources in traditional remote assistance, improving the efficiency and accuracy of communication equipment installation and debugging.

Description

Technical Field

[0001] This invention relates to the intersection of communication technology, augmented reality (AR) technology and artificial intelligence technology, and specifically to an AR remote assistance method and system for the installation and debugging of communication equipment. Background Technology

[0002] The installation and commissioning of communication equipment (such as 5GAAU, BBU, ODF racks, etc.) is a core part of communication network construction and operation and maintenance. Its operation is highly specialized and complex, requiring extremely high technical skills from on-site engineers. Currently, the following prominent problems exist in the installation and commissioning process of communication equipment:

[0003] Expert resources are scarce and scheduling costs are high: There are many models of communication equipment and complex failure scenarios. Many field engineers lack experience in handling complex problems, while the number of senior experts in the industry is limited, making it difficult to achieve timely coverage of installation and commissioning sites across the country. Experts need to spend a lot of time and travel costs to provide on-site guidance.

[0004] Traditional remote assistance methods are ineffective: Current remote assistance mainly relies on telephone, video calls, and text / image communication, which suffers from inaccurate information delivery and a lack of immersion. On-site engineers cannot visually demonstrate equipment details and the installation environment, and experts cannot accurately mark operation locations and steps, resulting in low communication efficiency and long problem-solving cycles.

[0005] Insufficient accuracy in spatial positioning and virtual-real fusion: When existing AR technology is applied in communication equipment installation and debugging scenarios, it faces complex situations such as textureless environments (equipment panels), low light (computer rooms), and dynamic occlusion (cables), which can easily lead to positioning drift and misalignment between virtual annotations and real equipment, thus failing to effectively assist in precise operation.

[0006] Low level of intelligence and lack of dedicated knowledge base: Existing solutions rely heavily on manual interaction and do not fully integrate AI technology to achieve automatic equipment identification and intelligent fault diagnosis; at the same time, there is a lack of standardized knowledge base for communication equipment, which cannot provide structured and accurate guidance and support for installation and debugging.

[0007] Low efficiency of multi-party collaboration: Complex installation and debugging tasks often require the collaborative guidance of multiple experts from different fields, but existing systems cannot enable collaborative annotation and interaction among multiple experts in the same virtual space, resulting in chaotic collaboration processes and untimely information synchronization.

[0008] Therefore, how to build an AR remote assistance system with high immersion, low latency, high precision, intelligence, and collaboration to solve the above-mentioned pain points in the installation and debugging process of communication equipment has become a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0009] This invention provides an AR remote assistance method and system for the installation and commissioning of communication equipment, aiming to solve the problems of low utilization rate of expert resources, poor immersion in remote assistance, insufficient positioning accuracy, low level of intelligence and low collaborative efficiency in the prior art, and to achieve high efficiency, precision and intelligence in the installation and commissioning of communication equipment.

[0010] To achieve the above objectives, the present invention is implemented through the following technical solution:

[0011] In a first aspect, the present invention provides an AR remote assistance method for the installation and debugging of communication equipment, comprising the following steps:

[0012] Collect first-person audio and video data, environmental perception data, and equipment status data at the installation and commissioning site of communication equipment;

[0013] Spatial computing technology based on multi-source information fusion enables the overlay of virtual and real scenes, constructing an immersive AR interactive environment;

[0014] The device status data is analyzed using AI intelligent recognition technology to obtain key device information and abnormal characteristics;

[0015] Based on a dedicated knowledge base for the communications industry and a fault diagnosis reasoning engine, it generates installation and debugging guidance plans or fault solutions.

[0016] Real-time audio and video communication and collaborative annotation in 3D space between on-site personnel and remote experts are achieved through multi-party collaborative interaction technology.

[0017] The AR smart terminal receives the guidance scheme and 3D spatial collaborative annotation information, and performs a visual display in a virtual-real fusion scene to assist in on-site installation and debugging operations.

[0018] As a further improvement to the technical solution of the present invention, the first-person perspective audio and video data, environmental perception data, and equipment status data collected at the installation and debugging site of the communication equipment specifically include:

[0019] Collected via the high-definition camera of the AR smart terminal First-person view video stream and 48kHz audio stream;

[0020] The AR smart terminal collects environmental pose data, position data, and distance data through its built-in inertial measurement unit (IMU), GPS module, and UWB module.

[0021] The image acquisition module acquires images of the device panel, instrument display, and port connections as device status data.

[0022] By adopting a hybrid transmission mode of P2P and Relay based on WebRTC, combined with efficient video coding algorithms and forward error correction strategies, ultra-low latency transmission of acquired data is achieved.

[0023] As a further improvement to the technical solution of the present invention, the spatial computing technology based on multi-source information fusion to realize the overlay of virtual and real scenes specifically includes:

[0024] A multi-source SLAM algorithm model fusing visual inertial odometry (VIO), GPS, and UWB is constructed, and spatial positioning results are calculated using a weighted fusion formula:

[0025] ;

[0026] in, For VIO location results, This is the GPS positioning result. For UWB positioning results, , , To merge weights and satisfy , Here are the covariance matrices for each positioning method. This is the error correction factor. It is the identity matrix;

[0027] Based on lighting estimation and environmental texture analysis techniques, lighting rendering and pose adjustment are performed on virtual 3D models to achieve a realistic fusion of virtual objects and the real environment.

[0028] As a further improvement to the technical solution of the present invention, the step of parsing the device status data using AI intelligent recognition technology specifically includes:

[0029] A deep learning object detection model is used to perform millisecond-level identification of the communication device body, port type, and indicator light status in device status data, and output the identification confidence score.

[0030] Based on OCR technology, digital images are extracted from instrument display images, and image distortion is eliminated through character correction algorithms to obtain accurate instrument readings.

[0031] The deep learning target detection model is a lightweight model optimized for communication equipment scenarios. Its backbone network adopts a deep separable convolutional structure, and the detection head adopts a multi-scale feature fusion design.

[0032] As a further improvement to the technical solution of the present invention, the generation scheme based on a communication industry-specific knowledge base and a fault diagnosis reasoning engine specifically includes:

[0033] The dedicated knowledge base for the communications industry includes 3D models of mainstream communication equipment (5GAAU, BBU, ODF rack, etc.), standardized installation and debugging process animations, and a knowledge graph of fault phenomena, causes, and solutions.

[0034] The fault diagnosis inference engine calculates the confidence level of the fault solution using the following formula:

[0035] ;

[0036] in, To improve the accuracy of AI recognition, For the knowledge graph node correlation degree, These are the weighting coefficients. For the first The influence coefficient of each failure factor For the first Uncertainty of each failure factor; output sorted by confidence level. Several solutions are available for on-site personnel and experts to choose from.

[0037] As a further improvement to the technical solution of the present invention, the multi-party collaborative interaction technology specifically includes:

[0038] Based on the cloud-based spatial anchor point sharing protocol, a three-dimensional spatial coordinate system mapping relationship is established for multiple terminals (AR smart terminals, expert operating consoles) to ensure the consistency of virtual annotation positions;

[0039] It supports multi-user online collaborative annotation, with annotation types including 3D arrows, virtual 3D models, structured text, and voice annotations. It resolves concurrent annotation conflicts through a state synchronization protocol.

[0040] The expert control panel supports switching between multiple live feeds, interactive whiteboards, and sharing of installation and debugging documents, enabling collaborative guidance from multiple experts.

[0041] As a further improvement to the technical solution of the present invention, the AR smart terminal is AR glasses or... The mobile terminal supports multiple interaction methods, including voice, gesture, and eye tracking. The visualization includes step-by-step animation of 3D models, highlighted fault points, and text guidance for operation steps.

[0042] A second aspect of the present invention provides an AR remote assistance system for the installation and debugging of communication equipment, comprising:

[0043] The data acquisition and transmission module is used to acquire first-person perspective audio and video data, environmental perception data, and equipment status data on site, and to achieve ultra-low latency transmission.

[0044] The spatial computing and virtual-real fusion module is used to achieve spatial positioning based on SLAM technology of multi-source information fusion and to complete the fusion rendering of virtual objects and real environment;

[0045] The AI ​​intelligent recognition module is used to analyze equipment status data and identify key equipment information and abnormal features.

[0046] The knowledge base and diagnostic module includes a telecommunications industry-specific knowledge base and a fault diagnosis reasoning engine, which are used to generate installation and debugging guidance or fault solutions.

[0047] The multi-party collaborative interaction module is used to enable real-time communication and collaborative annotation in three-dimensional space between on-site personnel and remote experts.

[0048] The AR visualization module is used to visualize and display guidance schemes and collaborative annotation information on AR smart terminals.

[0049] A third aspect of the present invention provides a computer device including a memory and a processor, the memory storing a computer program, and the processor being configured to execute the computer program to implement the AR remote assistance method for installation and debugging of communication equipment as described above.

[0050] A fourth aspect of the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the AR remote assistance method for the installation and debugging of communication equipment as described above.

[0051] The technical solution of the present invention has the following advantages over the prior art:

[0052] This invention integrates core elements such as multi-dimensional on-site data acquisition, multi-source fusion spatial computing, AI intelligent analysis, industry-specific knowledge base support, and multi-party collaborative interaction to construct a closed-loop process encompassing data acquisition, virtual-real fusion, intelligent analysis, collaborative guidance, and visualized execution. It effectively addresses industry pain points in traditional communication equipment installation and debugging, including scarce expert resources and high scheduling costs, poor immersion in remote assistance and inaccurate information transmission, spatial positioning drift, insufficient intelligence, and low efficiency in multi-party collaboration. It ensures real-time interaction through ultra-low latency audio and video transmission, achieves precise alignment between virtual annotations and real equipment through high-precision virtual-real fusion, rapidly outputs accurate guidance solutions based on AI intelligent recognition and a dedicated knowledge base, and ensures consistency in multi-expert collaborative annotation through spatial anchor point sharing technology. Finally, through immersive visualization on an AR terminal, it significantly reduces reliance on on-site personnel experience, improves the efficiency and accuracy of installation and debugging, and significantly saves on expert on-site costs. It also possesses excellent compatibility and scalability, adapting to different types of communication equipment and scenarios, providing efficient, intelligent, and precise technical support for communication network construction and maintenance. Attached Figure Description

[0053] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0054] Figure 1 This is a schematic diagram of the framework of an AR remote assistance method for the installation and debugging of communication equipment according to an embodiment of the present invention;

[0055] Figure 2 This is a schematic diagram of the module framework of an AR remote assistance system for the installation and debugging of communication equipment according to an embodiment of the present invention;

[0056] Figure 3 This is a schematic diagram of the composition of a computing device according to an embodiment of the present invention. Detailed Implementation

[0057] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0058] The present invention will be further described in detail below with reference to the accompanying drawings.

[0059] Reference Figure 1 In a first aspect, the present invention provides an AR remote assistance method for the installation and debugging of communication equipment, comprising the following steps:

[0060] Collect first-person audio and video data, environmental perception data, and equipment status data at the installation and commissioning site of communication equipment;

[0061] Spatial computing technology based on multi-source information fusion enables the overlay of virtual and real scenes, constructing an immersive AR interactive environment;

[0062] The device status data is analyzed using AI intelligent recognition technology to obtain key device information and abnormal characteristics;

[0063] Based on a dedicated knowledge base for the communications industry and a fault diagnosis reasoning engine, it generates installation and debugging guidance plans or fault solutions.

[0064] Real-time audio and video communication and collaborative annotation in 3D space between on-site personnel and remote experts are achieved through multi-party collaborative interaction technology.

[0065] The AR smart terminal receives the guidance scheme and 3D spatial collaborative annotation information, and performs a visual display in a virtual-real fusion scene to assist in on-site installation and debugging operations.

[0066] In specific implementation, the method of this invention achieves AR remote assistance for the installation and commissioning of communication equipment through six core steps: First, it collects first-person perspective audio and video data, environmental perception data, and equipment status data to provide basic data support for subsequent processing; then, it uses spatial computing technology with multi-source information fusion to accurately overlay the virtual scene with the real installation and commissioning environment to construct an immersive AR interactive scene; subsequently, it uses AI intelligent recognition technology to deeply analyze the collected equipment status data and extract key information such as equipment model, port type, instrument readings, and abnormal characteristics; based on a dedicated knowledge base for the communication industry and combined with a fault diagnosis reasoning engine, it generates targeted installation and commissioning guidance plans or fault solutions based on the key information obtained from the analysis; it establishes a real-time communication channel between on-site personnel and remote experts through multi-party collaborative interaction technology, supporting multi-user three-dimensional spatial collaborative annotation; finally, the AR smart terminal receives the guidance plan and collaborative annotation information and visualizes it in the virtual-real fusion scene, providing intuitive assistance for on-site operations.

[0067] This invention constructs a closed-loop AR remote assistance system, effectively addressing industry pain points in traditional communication equipment installation and debugging, such as scarce expert resources and high scheduling costs, poor immersion in remote assistance, inaccurate information transmission, spatial positioning drift, insufficient intelligence, and low efficiency of multi-party collaboration. Through the integrated application of multiple technologies, it significantly improves the utilization rate of expert resources and reduces collaboration costs; high-precision virtual-real fusion and AI intelligent recognition ensure the accuracy of operational guidance and reduce human error; the multi-party collaboration mechanism supports efficient collaboration on complex tasks, comprehensively improving installation and debugging efficiency and accuracy, and providing strong technical support for communication network construction and maintenance.

[0068] In some embodiments, the acquisition of first-person audio and video data, environmental perception data, and equipment status data at the installation and commissioning site of the communication equipment specifically includes:

[0069] Collected via the high-definition camera of the AR smart terminal First-person view video stream and 48kHz audio stream;

[0070] The AR smart terminal collects environmental pose data, position data, and distance data through its built-in inertial measurement unit (IMU), GPS module, and UWB module.

[0071] The image acquisition module acquires images of the device panel, instrument display, and port connections as device status data.

[0072] By adopting a hybrid transmission mode of P2P and Relay based on WebRTC, combined with efficient video coding algorithms and forward error correction strategies, ultra-low latency transmission of acquired data is achieved.

[0073] It should be noted that the data acquisition stage utilizes the high-definition camera and omnidirectional microphone configured on the AR smart terminal to capture first-person view audio and video streams conforming to high-definition standards, employing a specific encoding format to ensure data quality. The terminal's built-in inertial measurement unit, GPS module, and UWB module collect environmental pose, position, and distance data at a set sampling frequency, and image preprocessing algorithms optimize the quality of device status images. The transmission stage employs a hybrid P2P and Relay transmission mode of WebRTC, prioritizing direct P2P transmission to reduce latency, and automatically switching to Relay forwarding mode when network conditions are insufficient. Video encoding uses... A high-efficiency encoding algorithm, combined with forward error correction and adaptive bitrate adjustment, dynamically adjusts the transmission bitrate using a specific bitrate calculation formula to ensure stable data transmission in complex network environments. This approach guarantees the integrity, high definition, and timeliness of the collected data, providing high-quality data input for subsequent AI recognition and virtual-real fusion processes. The combination of hybrid transmission mode with high-efficiency encoding, forward error correction, and adaptive bitrate adjustment strategies keeps end-to-end audio and video latency stable within 200ms, effectively addressing complex network fluctuations, ensuring real-time and smooth remote interaction, and preventing data transmission issues from affecting installation and debugging progress.

[0074] In some embodiments, the implementation of virtual-real scene overlay using spatial computing technology based on multi-source information fusion specifically includes:

[0075] A multi-source SLAM algorithm model fusing visual inertial odometry (VIO), GPS, and UWB is constructed, and spatial positioning results are calculated using a weighted fusion formula:

[0076] ;

[0077] in, For VIO location results, This is the GPS positioning result. For UWB positioning results, , , To merge weights and satisfy , Here are the covariance matrices for each positioning method. This is the error correction factor. It is the identity matrix;

[0078] Based on lighting estimation and environmental texture analysis techniques, lighting rendering and pose adjustment are performed on virtual 3D models to achieve a realistic fusion of virtual objects and the real environment.

[0079] It should be noted that the spatial computing stage constructs a multi-source SLAM algorithm model that integrates visual inertial odometry, GPS, and UWB. Each module acquires relative pose changes, absolute position data, and relative distance data of the device. After time synchronization, an adaptive weighted fusion algorithm is used to calculate the final positioning result using a specific fusion formula. The fusion weights of each sensor are dynamically adjusted, and an error correction mechanism is employed to improve positioning accuracy. In the virtual-real fusion rendering stage, image segmentation algorithms are used to extract the device and background regions from the real environment. Ambient lighting parameters are obtained based on a lighting estimation model, and the virtual 3D model is matched and rendered. Plane detection and collision detection technologies are used to ensure the virtual model fits the real device, supporting flexible operation of the virtual model to adapt to different scenario requirements.

[0080] The SLAM technology, which integrates multi-source information, solves the positioning defects of single sensors in complex communication scenarios such as no texture, low light, and dynamic occlusion. The positioning error is controlled within 5cm, ensuring accurate alignment between virtual labels and real devices. The lighting matching and collision detection technologies make the virtual model blend with the real environment realistic, avoid floating or penetration phenomena, enhance immersion, provide intuitive and accurate virtual auxiliary references for on-site personnel, and reduce the difficulty of operation.

[0081] In some embodiments, the step of parsing the device status data using AI intelligent recognition technology specifically includes:

[0082] A deep learning object detection model is used to perform millisecond-level identification of the communication device body, port type, and indicator light status in device status data, and output the identification confidence score.

[0083] Based on OCR technology, digital images are extracted from instrument display images, and image distortion is eliminated through character correction algorithms to obtain accurate instrument readings.

[0084] The deep learning target detection model is a lightweight model optimized for communication equipment scenarios. Its backbone network adopts a deep separable convolutional structure, and the detection head adopts a multi-scale feature fusion design.

[0085] It should be noted that device recognition employs a lightweight deep learning object detection model optimized for communication equipment scenarios. This model reduces computational load through depthwise separable convolutional structures and improves detection accuracy through multi-scale feature fusion design. The model is trained on a large-scale dataset of device image samples covering various scenarios, enabling rapid identification and classification of devices and components. Instrument reading recognition uses an improved text detection algorithm to locate digit regions. After perspective transformation correction to eliminate distortion, a text recognition algorithm extracts accurate readings, and a reasonableness verification mechanism removes abnormal data. Indicator light status recognition quickly determines the on / off state and color of the indicator light through color feature extraction and shape feature matching. The optimized lightweight object detection model achieves a device recognition speed of over 30fps and an accuracy rate of no less than [percentage missing]. The accuracy of instrument reading recognition is no less than The indicator light status recognition delay is controlled within 10ms, enabling millisecond-level, high-precision extraction of key equipment information. This reduces errors from manual observation and recording, lowers reliance on the professional experience of on-site personnel, and provides accurate and efficient data support for subsequent fault diagnosis and guidance solution generation.

[0086] In some embodiments, the scheme generated based on a communication industry-specific knowledge base and a fault diagnosis reasoning engine specifically includes:

[0087] The dedicated knowledge base for the communications industry includes 3D models of mainstream communication equipment (5GAAU, BBU, ODF rack, etc.), standardized installation and debugging process animations, and a knowledge graph of fault phenomena, causes, and solutions.

[0088] The fault diagnosis inference engine calculates the confidence level of the fault solution using the following formula:

[0089] ;

[0090] in, To improve the accuracy of AI recognition, For the knowledge graph node correlation degree, These are the weighting coefficients. For the first The influence coefficient of each failure factor For the first Uncertainty of each failure factor; output sorted by confidence level. Several solutions are available for on-site personnel and experts to choose from.

[0091] It should be noted that the dedicated knowledge base for the communications industry comprises three core components: a high-precision 3D model library covering mainstream communication equipment, supporting component disassembly and assembly animation; a standardized process animation library containing full-process installation and debugging animations, supporting step-by-step playback; and a structured knowledge graph library centered on fault phenomena, causes, and solutions, storing a large amount of information related to common faults. The fault diagnosis reasoning engine first fuses the abnormal features identified by AI with structured text converted from on-site personnel's voice descriptions to generate a fault query vector; then, it filters candidate fault nodes using a graph embedding-based similarity matching algorithm; finally, using a specific confidence calculation formula, combined with AI recognition accuracy, knowledge graph node correlation, and the influence coefficient and uncertainty of fault factors, it calculates the confidence score of each candidate solution and outputs the optimal solution in order of confidence score. The construction of the dedicated knowledge base achieves standardized and structured storage of communication equipment-related data and expert experience, providing a rich and reliable data source for guiding solution generation; the fault diagnosis reasoning engine, through multi-dimensional information fusion and scientific confidence calculation, ensures the accuracy and rational prioritization of the output solutions. The solution has a confidence level of no less than 0.85, which significantly shortens the time for troubleshooting and problem solving, and improves the efficiency and standardization of installation and commissioning.

[0092] In some embodiments, the multi-party collaborative interaction technology specifically includes:

[0093] Based on the cloud-based spatial anchor point sharing protocol, a three-dimensional spatial coordinate system mapping relationship is established for multiple terminals (AR smart terminals, expert operating consoles) to ensure the consistency of virtual annotation positions;

[0094] It supports multi-user online collaborative annotation, with annotation types including 3D arrows, virtual 3D models, structured text, and voice annotations. It resolves concurrent annotation conflicts through a state synchronization protocol.

[0095] The expert control panel supports switching between multiple live feeds, interactive whiteboards, and sharing of installation and debugging documents, enabling collaborative guidance from multiple experts.

[0096] It should be noted that spatial anchor point sharing, through a cloud-based spatial anchor point service, standardizes local spatial anchor points generated by AR smart terminals into global spatial anchor points. After the expert operating platform downloads these global anchor points, it establishes a 3D spatial coordinate system mapping with the AR terminals, ensuring consistent virtual annotation positions across multiple devices. Collaborative annotation supports various types, including 3D arrows, virtual 3D models, structured text, and voice annotations. It employs an optimistic locking mechanism to process concurrent annotations from multiple users in timestamp order, avoiding conflicts and supporting annotation history review and undoing. The expert collaboration platform supports switching between multiple live feeds, real-time interactive whiteboards for multiple users, and file upload and sharing. It also features a three-level access control system, clearly defining the operating permissions for experts, general collaborators, and on-site personnel.

[0097] Spatial anchor point sharing technology enables the consistency error of three-dimensional space in multi-terminal virtual annotation to be controlled within 3cm, achieving the effect of multi-expert "on-site" collaboration; diversified collaborative annotation functions and conflict resolution mechanisms ensure the flexibility and orderliness of collaboration; three-level permission management ensures that the collaboration process is standardized and efficient, supports up to 8 people to collaborate at the same time, greatly improves the multi-party collaboration efficiency of complex installation and debugging tasks, and gives full play to the collaborative advantages of experts in multiple fields.

[0098] In some embodiments, the AR smart terminal is AR glasses or The mobile terminal supports multiple interaction methods, including voice, gesture, and eye tracking. The visualization includes step-by-step animation of 3D models, highlighted fault points, and text guidance for operation steps.

[0099] It should be noted that the AR smart terminal includes mainstream AR glasses and Android / iOS mobile terminals, adapting to different usage scenarios; voice interaction is activated by preset wake words, supports Chinese voice command recognition, and realizes functions such as annotation creation, solution query, and operation confirmation; gesture interaction recognizes preset gestures such as raising hands, clenching fists, and pointing, which are used to control the scaling, rotation, translation, and annotation selection of virtual models; eye tracking captures the focus of gaze through the eye-tracking module built into the AR glasses, and confirms operations in conjunction with blinking actions, adapting to operation scenarios where tools are held in both hands; the visualization display adopts a layered display strategy, distinguishing display colors and methods according to the importance of information, and supports automatic step-by-step display and manual switching of solutions.

[0100] The diverse range of AR smart terminal types expands the applicability of the technology, meeting the equipment needs of different installation and debugging scenarios; multimodal interaction methods adapt to different operating situations, and the voice recognition accuracy is no less than [a certain percentage]. The gesture recognition response time is no more than 200ms, making operation convenient and efficient, and reducing the operating threshold for on-site personnel; the layered visual display highlights key information, improves information acquisition efficiency, ensures that on-site personnel can quickly and accurately receive guidance information, and improves operational standardization and efficiency.

[0101] Reference Figure 2 The second aspect of the present invention provides an AR remote assistance system for the installation and debugging of communication equipment, comprising:

[0102] The data acquisition and transmission module is used to acquire first-person perspective audio and video data, environmental perception data, and equipment status data on site, and to achieve ultra-low latency transmission.

[0103] The spatial computing and virtual-real fusion module is used to achieve spatial positioning based on SLAM technology of multi-source information fusion and to complete the fusion rendering of virtual objects and real environment;

[0104] The AI ​​intelligent recognition module is used to analyze equipment status data and identify key equipment information and abnormal features.

[0105] The knowledge base and diagnostic module includes a telecommunications industry-specific knowledge base and a fault diagnosis reasoning engine, which are used to generate installation and debugging guidance or fault solutions.

[0106] The multi-party collaborative interaction module is used to enable real-time communication and collaborative annotation in three-dimensional space between on-site personnel and remote experts.

[0107] The AR visualization module is used to visualize and display guidance schemes and collaborative annotation information on AR smart terminals.

[0108] It should be noted that the system achieves AR remote assistance through the collaborative work of six functional modules: the data acquisition and transmission module is responsible for collecting multi-dimensional data on-site and transmitting it with ultra-low latency; the spatial computing and virtual-real fusion module achieves high-precision spatial positioning and realistic fusion of virtual models; the AI ​​intelligent recognition module analyzes equipment status data and extracts key information and abnormal features; the knowledge base and diagnostic module stores standardized data and expert experience to generate targeted guidance solutions; the multi-party collaborative interaction module establishes real-time communication and collaborative annotation channels to achieve efficient collaboration among multiple users; and the AR visualization display module receives guidance information and provides immersive guidance to on-site personnel through multimodal interaction. All modules are seamlessly connected through data interfaces to form a complete technical closed loop.

[0109] The system architecture of this invention is scientifically and rationally designed, with clear functional divisions and efficient collaboration among modules, ensuring the stable operation of the overall technical solution. The seamless connection between modules enables smooth data flow and function calls, improving the overall system response speed. The open interface design allows the system to be integrated with existing operation and maintenance management systems and equipment management platforms in the communications industry, adapting to communication equipment from different manufacturers and models, and has broad compatibility and scalability, covering various scenarios for the installation and debugging of communication equipment.

[0110] Reference Figure 3 The third aspect of the present invention provides a computer device including a memory and a processor, the memory storing a computer program, and the processor being configured to execute the computer program to implement the AR remote assistance method for installation and debugging of communication equipment as described above.

[0111] It should be noted that the computer equipment includes a memory and a processor. The memory stores the computer program and related data used to implement the AR remote assistance method for installing and debugging communication equipment, including collected field data, knowledge base data, model parameters, etc. The processor obtains the computer program and data stored in the memory through the communication bus, and executes steps such as data acquisition and parsing, spatial computation and fusion, AI recognition, fault diagnosis, collaborative interaction, and visualization according to the program instructions to complete the entire process of AR remote assistance. The processor's computing power provides hardware support for the efficient operation of each algorithm. Through the collaborative design of hardware and software, the stable and efficient operation of the AR remote assistance method for installing and debugging communication equipment is ensured. The high-performance computing power of the processor meets the needs of computationally intensive tasks such as AI recognition and multi-source data fusion, while the large-capacity storage capacity of the memory ensures the secure storage of knowledge base data, field-collected data, etc., adapting to installation and debugging tasks of different scales and providing reliable hardware support for the implementation of the technical solution.

[0112] In some embodiments, the AR remote assistance method for installing and debugging communication equipment in the above embodiments can be implemented by a computer device, which includes at least one processor, a communication bus, a memory, and at least one communication interface.

[0113] A processor can be a general-purpose central processing unit (CPU) or an application-specific integrated circuit (ASIC).

[0114] A communication bus can be used to transmit information between the aforementioned components.

[0115] The memory can be read-only memory (ROM) or other types of static storage devices capable of storing static information and instructions, random access memory (RAM) or other types of dynamic storage devices capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, universal optical discs, Blu-ray discs, etc.), magnetic disks or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited to these. The memory can exist independently and be connected to the processor via a communication bus. The memory can also be integrated with the processor.

[0116] The memory stores program code for executing the scheme of this application, and its execution is controlled by a processor. The processor executes the program code stored in the memory. The program code may include one or more software modules. The AR remote assistance method for installing and debugging communication equipment in the above embodiments can be implemented by the processor and one or more software modules in the program code in the memory.

[0117] A communication interface is a device that uses any transceiver or similar device to communicate with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area networks (WLAN), etc.

[0118] In a specific implementation, as one example, a computer device may include multiple processors, each of which may be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. Here, a processor may refer to one or more devices, circuits, and / or processing cores used to process data (e.g., computer program instructions).

[0119] The aforementioned computer device can be a general-purpose computer device or a special-purpose computer device. In specific implementations, the computer device can be a desktop computer, a portable computer, a network server, a handheld digital assistant (PDA), a mobile phone, a tablet computer, a wireless terminal device, a communication device, or an embedded device. This application does not limit the type of computer device.

[0120] A fourth aspect of this invention provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the AR remote assistance method for installing and debugging communication equipment as described above. This computer-readable storage medium employs a suitable storage medium type to store the computer program implementing the AR remote assistance method for installing and debugging communication equipment. When the storage medium is connected to a computer device, the processor can read and load the computer program in the storage medium, driving the computer device to execute various functions of AR remote assistance according to the program instructions, including data acquisition, processing, analysis, collaborative interaction, and visualization, thus achieving complete implementation of the method. The computer-readable storage medium facilitates the storage, transmission, and deployment of programs related to the AR remote assistance method, lowering the barriers to program promotion and application, and enabling the rapid implementation of related technical solutions on different computer devices. The stability of the storage medium ensures the secure storage of the program and related data, improving the portability and practicality of the technical solution, and facilitating its widespread application in the communications industry.

[0121] To provide a clearer understanding of the invention, the invention is further described below:

[0122] Reference Figure 1 In a first aspect, this application provides an AR remote assistance method for the installation and debugging of communication equipment, comprising the following steps:

[0123] Collect first-person audio and video data, environmental perception data, and equipment status data at the installation and commissioning site of communication equipment;

[0124] Spatial computing technology based on multi-source information fusion enables the overlay of virtual and real scenes, constructing an immersive AR interactive environment;

[0125] The device status data is analyzed using AI intelligent recognition technology to obtain key device information and abnormal characteristics;

[0126] Based on a dedicated knowledge base for the communications industry and a fault diagnosis reasoning engine, it generates installation and debugging guidance plans or fault solutions.

[0127] Real-time audio and video communication and collaborative annotation in 3D space between on-site personnel and remote experts are achieved through multi-party collaborative interaction technology.

[0128] The AR smart terminal receives the guidance scheme and collaborative annotation information, and performs a visual display in a virtual-real fusion scene to assist in on-site installation and debugging operations.

[0129] In some embodiments, the acquisition of first-person audio and video data, environmental perception data, and equipment status data at the installation and commissioning site of the communication equipment specifically includes:

[0130] AR smart terminals are equipped with high-definition cameras (resolution) Frame rate ), omnidirectional microphone (sampling rate 48kHz, signal-to-noise ratio) Real-time acquisition of first-person perspective video and audio streams, with the video stream using YUV420 encoding format and the audio stream using AAC encoding format;

[0131] AR smart terminals have a built-in inertial measurement unit (IMU, including a three-axis gyroscope, a three-axis accelerometer, and a three-axis magnetometer) and a GPS module (for positioning accuracy). ), UWB module (range measurement accuracy) The system collects pose data (angular velocity, acceleration, attitude angle), position data, and distance data between devices in the field environment at a sampling frequency of 100Hz.

[0132] The image acquisition module periodically captures images of the device panel, instrument display, and port connections. The image resolution is [missing information]. Image quality is optimized through image preprocessing algorithms (Gaussian filtering, histogram equalization);

[0133] The transmission layer adopts a hybrid P2P and Relay transmission mode based on WebRTC, prioritizing direct P2P data transmission and automatically switching to Relay forwarding mode when network conditions are unsatisfactory; video encoding uses... High-efficiency encoding algorithm, encoding code rate range is Combining forward error correction (FEC) technology, RS(255,223) codes are used for data redundancy coding to reduce the impact of packet loss; the transmission bitrate is dynamically adjusted through an adaptive rate adjustment (ADR) algorithm. The target bitrate calculation formula for the ADR algorithm is as follows:

[0134] ;

[0135] in, The base bitrate is 5Mbps (default). For real-time network bandwidth, This is the bandwidth threshold (default 2Mbps). For real-time packet loss rate, Packet loss rate weight (value) ), For real-time end-to-end latency, This is the delay threshold (default 200ms). Delay weights (values) This formula enables dynamic adjustment of the bitrate, ensuring that the end-to-end audio and video latency remains stable within 200ms.

[0136] In some embodiments, the implementation of virtual-real scene overlay using spatial computing technology based on multi-source information fusion specifically includes:

[0137] To construct a multi-source SLAM algorithm model integrating VIO, GPS, and UWB, and to address the positioning limitations of single sensors in complex scenarios:

[0138] The VIO module matches IMU data with image feature points to calculate the relative pose change of the device and outputs the result. ;

[0139] GPS module outputs absolute position data Time synchronization is achieved by combining timestamps with VIO data;

[0140] The UWB module communicates with pre-installed anchor base stations on-site to obtain relative distance data and calculates location information. ;

[0141] The final localization result is calculated using an adaptive weighted fusion algorithm. The fusion formula is as follows:

[0142] ;

[0143] in, , , The fusion weights are dynamically adjusted based on sensor confidence, with initial values ​​of 0.6, 0.2, and 0.2, respectively, and are updated in real time using a Kalman filter algorithm. This is the covariance matrix for each positioning method, reflecting the positioning accuracy; Error correction coefficient (value) ), for The identity matrix; this formula improves positioning accuracy and reduces positioning error in textureless, low-light, and dynamically occluded scenes by complementing multi-source information. ;

[0144] Virtual-Real Fusion Rendering: The device area and background area in the real environment are extracted through image segmentation algorithms. Based on the lighting estimation model, the ambient lighting parameters (brightness, color temperature, direction) are calculated to render the virtual 3D model, so that the lighting effect of the virtual object is consistent with the real environment. Plane detection and collision detection technologies are used to ensure that the virtual model fits the surface of the real device without floating or penetrating. The virtual model can be scaled, rotated and translated to meet the visualization needs of different installation and debugging scenarios.

[0145] In some embodiments, the step of parsing the device status data using AI intelligent recognition technology specifically includes:

[0146] Communication equipment and component identification: A lightweight deep learning target detection model optimized for communication equipment scenarios is adopted. The backbone network of the model adopts a deep separable convolutional structure of MobileNetV3, and the detection head adopts a multi-scale feature fusion design of FPN to reduce the computational load of the model and meet the real-time requirements of AR terminals. The training dataset contains image samples of more than 20 mainstream communication equipment and components such as 5GAAU, BBU, ODF frame, and optical module (total amount). (Tens of thousands of images), covering different angles, lighting, and occlusion conditions; the model input is a device status image, and the output is the category of the identified target, the bounding box coordinates, and the confidence score, with a recognition speed of [missing information]. Recognition accuracy ;

[0147] Instrument reading recognition: Based on the improved EAST text detection algorithm and CRNN text recognition algorithm, automatic extraction of instrument readings is achieved.

[0148] The EAST algorithm detects digital regions in instrument images and outputs the minimum bounding rectangle.

[0149] A perspective transformation algorithm is used to correct the digital region and eliminate image distortion;

[0150] The CRNN algorithm performs feature extraction and sequence recognition on the corrected digital image and outputs accurate readings;

[0151] Introducing a reading rationality verification mechanism, combining instrument range and historical data, eliminates abnormal readings, improving reading accuracy. ;

[0152] Indicator light status recognition: By extracting color features (HSV color space threshold segmentation) and matching shape features, the indicator light's on / off state and color (red, green, yellow) are identified, and an indicator light status vector is output. Recognition delay is also considered. .

[0153] In some embodiments, the scheme generated based on a communication industry-specific knowledge base and a fault diagnosis reasoning engine specifically includes:

[0154] Construction of a dedicated knowledge base for the telecommunications industry:

[0155] 3D Model Library: High-precision 3D digital modeling of mainstream communication equipment (5G AAU, BBU, ODF rack, optical modem, etc.), with high model accuracy. It supports component disassembly and assembly animation display;

[0156] Process Animation Library: Creates standardized installation and commissioning process animations, covering the entire process from equipment unpacking, fixed installation, cable connection, parameter configuration, and functional testing, with high animation frame rates. It supports step-by-step playback and pause;

[0157] Knowledge Graph Base: Centered on "fault phenomenon-cause-solution", a structured knowledge graph is constructed, containing 500+ common fault types. Each fault node includes information such as fault description, possible causes (weighted sorting), troubleshooting steps, solutions, and precautions. The knowledge graph is stored using the Neo4j graph database, supporting efficient querying and updating.

[0158] Fault diagnosis inference engine:

[0159] Input processing: The abnormal features of the equipment identified by AI (such as indicator lights remaining red, instrument readings exceeding limits, and ports not being recognized) are fused with the voice descriptions of on-site personnel (converted into structured text through NLP technology) to generate a fault query vector;

[0160] Knowledge graph matching: A graph embedding-based similarity matching algorithm is used to calculate the similarity between the fault query vector and the fault nodes in the knowledge graph, and to filter out candidate fault nodes.

[0161] Confidence score calculation: The confidence score of each candidate solution is calculated using the following formula:

[0162] ;

[0163] in, AI recognition accuracy (value) ), The correlation between the fault query vector and the knowledge graph node (values) ), This is the weighting coefficient (default 0.7). For the first The influence coefficient of each failure factor (set based on expert experience, with a value of...) ), For the first Uncertainty of individual failure factors (calculated based on data volume, values) );

[0164] Output results: sorted in descending order of confidence level. Each solution includes operation steps, diagrams, and precautions, and supports manual adjustment and supplementation by experts.

[0165] In some embodiments, the multi-party collaborative interaction technology specifically includes:

[0166] Spatial Anchor Point Sharing: Based on cloud-based spatial anchor point services, the AR smart terminal uploads the spatial anchor point data generated by local SLAM to the cloud. The cloud standardizes the anchor point data to generate global spatial anchor points. The expert operation platform downloads the global spatial anchor points and establishes a mapping relationship with the 3D spatial coordinate system of the AR smart terminal, minimizing mapping errors. This ensures that the virtual annotations seen on multiple devices are in completely consistent positions in three-dimensional space.

[0167] Collaborative annotation feature: Supports multi-user online collaborative annotation, with annotation types including:

[0168] 3D arrows: can point to specific locations on a device in virtual space, and support adjustments for color, thickness, and length;

[0169] Virtual 3D Models: Standard component 3D models can be retrieved from the knowledge base and placed next to real devices for comparison or demonstration;

[0170] Structured text: Supports rich text format, allowing you to add operation instructions, parameter values, warning messages, etc.

[0171] Voice annotation: Supports real-time voice recording and playback, and binds voice data to annotation locations;

[0172] An optimistic locking mechanism is used to resolve conflicts in concurrent annotation by multiple users. When multiple users annotate the same location, the annotations are saved in timestamp order, and annotation history can be backtracked and revoked.

[0173] Expert Collaboration Platform: Based on Client-side development, supports:

[0174] Multi-channel live feed switching: It can simultaneously receive video streams from multiple AR smart terminals and supports single-screen and multi-screen display modes;

[0175] Interactive whiteboard: Supports real-time drawing, text input, and file upload (PDF, Word, image) by multiple users; whiteboard content is synchronized with AR terminal in real time.

[0176] Access control: Set up three levels of access: experts, general collaborators, and on-site personnel. Experts have the authority to edit annotations and approve plans, general collaborators have the authority to view and comment on annotations, and on-site personnel have the authority to perform operations and provide feedback on problems.

[0177] In some embodiments, the AR smart terminal is AR glasses (such as HoloLens2, Nreal) or... Mobile terminals support multiple interaction methods:

[0178] Voice interaction: Supports Chinese voice command recognition (wake word "AR assistant"), enabling functions such as annotation creation, solution query, and operation confirmation. Voice recognition accuracy is high. ;

[0179] Gesture interaction: Supports preset gestures such as raising hand, clenching fist, and pointing, used to control the scaling, rotation, translation, and annotation selection of the virtual model. Gesture recognition response time is fast. ;

[0180] Eye tracking: The AR glasses have a built-in eye tracking module that supports focusing on virtual objects with your eyes and confirming actions with blinking. This is suitable for scenarios where you hold tools with both hands.

[0181] Visualization: The AR terminal overlays installation and debugging guidance and collaborative annotation information onto the real environment, adopting a layered display strategy: key operation steps are highlighted in red, secondary information is displayed in blue, and warning information is displayed in yellow flashing; it supports step-by-step display of the solution, automatically displaying the next step after completing the current step, or it can be manually switched.

[0182] Secondly, this application provides an AR remote assistance system for the installation and debugging of communication equipment, comprising:

[0183] Data acquisition and transmission module: including high-definition cameras, microphones, IMU, GPS, UWB and other sensors of AR smart terminals, and a WebRTC-based transmission submodule; used to acquire first-person perspective audio and video data, environmental perception data and equipment status data on site, and achieve ultra-low latency transmission through optimized transmission strategies;

[0184] Spatial Computing and Virtual-Real Fusion Module: Includes a multi-source SLAM submodule and a rendering submodule; the multi-source SLAM submodule is used to fuse VIO, GPS, and UWB data to calculate spatial positioning results; the rendering submodule is used to realize the fusion rendering of virtual 3D models and the real environment;

[0185] AI intelligent recognition module: includes equipment recognition submodule, instrument reading recognition submodule, and indicator light status recognition submodule; used to analyze equipment status data and extract key equipment information and abnormal features;

[0186] Knowledge base and diagnostic module: including a telecommunications industry-specific knowledge base (3D model library, process animation library, knowledge graph library) and a fault diagnosis reasoning engine; used to store standardized data and expert experience, and generate installation and debugging guidance plans or fault solutions;

[0187] Multi-party collaborative interaction module: including spatial anchor point sharing sub-module, collaborative annotation sub-module, and expert operation platform sub-module; used to realize real-time audio and video communication between on-site personnel and remote experts, 3D spatial collaborative annotation, and access control;

[0188] AR visualization module: Deployed on AR smart terminals, it is used to receive guidance plans and collaborative annotation information, and provide immersive visual guidance to on-site personnel through interactive methods such as voice, gestures, and eye tracking.

[0189] To make the technical solution of the present invention clearer and easier to understand, the present invention will be described in detail below with reference to specific embodiments.

[0190] Example 1: Hardware Deployment of an AR Remote Assistance System

[0191] In this embodiment, the AR smart terminal uses HoloLens2AR glasses, configured as follows:

[0192] Display: 4K resolution for both eyes, field of view ;

[0193] Sensors: IMU (sampling rate 100Hz), GPS (positioning accuracy) ), UWB module (range measurement accuracy) 13-megapixel RGB camera (60fps).

[0194] Interaction: Supports gesture recognition, voice recognition, and eye tracking;

[0195] Performance: Snapdragon 850 processor, 4GB RAM, 64GB storage;

[0196] Network: Supports Wi-Fi 6, 5G, and 4G networks.

[0197] The expert collaboration platform is deployed on a PC and configured as follows:

[0198] Processor: Intel Core i7-12700K;

[0199] Memory: 32GB DDR5;

[0200] Graphics card: NVIDIA RTX 3080 Ti (12GB VRAM);

[0201] Monitor: 4K resolution, supports multi-screen display;

[0202] Network: Wired gigabit Ethernet + 5G module.

[0203] The cloud service is deployed on an Alibaba Cloud server cluster, configured as follows:

[0204] Compute nodes: 8 ECS instances (8 cores, 16GB memory);

[0205] Storage nodes: Cloud database RDS (MySQL 8.0) + object storage OSS (stores 3D models, images, and video data);

[0206] Network: SLB (Solution Load Balancer) + Elastic Public IP Address;

[0207] Services: Real-time communication server (based on WebRTC), spatial anchor service, AI inference service, knowledge base service.

[0208] Example 2: AR Remote Assistance Process for Installation and Debugging of Communication Equipment

[0209] Taking the installation and debugging of a 5GBBU device as an example, the implementation process of the method in this application is explained in detail:

[0210] On-site preparation: On-site engineers wear HoloLens2AR glasses, turn on the device and connect to the cloud service; experts log in to the expert collaboration platform via PC, enter the collaboration code to join the on-site assistance session;

[0211] Data Acquisition and Transmission: AR glasses acquire first-person view video streams from the BBU device ( The audio stream is collected using IMU, GPS, and UWB to acquire environmental pose and location data; the video stream is encoded using H.265 encoding and transmitted to the cloud via WebRTCP2P mode, and then forwarded to the expert console, with an end-to-end latency of 150ms.

[0212] Spatial positioning and virtual-real fusion: The AR glasses' multi-source SLAM module integrates VIO, GPS, and UWB data to calculate the current position and attitude, with a positioning error of 3cm; the cloud sends the 3D model of the BBU device to the AR glasses, and the AR glasses render the 3D model based on the illumination estimation results and overlay it next to the real BBU device to achieve virtual-real fusion display;

[0213] AI Intelligent Recognition: The AI ​​intelligent recognition module analyzes the collected BBU device images and identifies the device model as Huawei BBU5900, the port types as CPRI ports (8) and GE ports (4), the indicator light status as a solid green power light and a flashing red running light (abnormal feature), and the instrument reading (power supply voltage) as -48V (normal range).

[0214] Fault Diagnosis: The fault diagnosis inference engine matches the "running light flashing red" anomaly feature with the knowledge graph to calculate the Top-3 solutions.

[0215] Solution 1 (confidence level 0.92): Check the CPRI cable connection between the BBU and AAU, which may be loose or plugged into the wrong port;

[0216] Solution 2 (Confidence level 0.85): Check if the baseband board of the BBU device is properly inserted;

[0217] Solution 3 (Confidence level 0.78): Check if the software version of the BBU device is compatible;

[0218] Multi-party collaborative annotation: Experts view the on-site video and AI recognition results through the control panel, select Solution 1, add a red 3D arrow annotation to the CPRI port of the BBU, enter the text "Check the cable connection of this port", and explain with voice annotation "Please make sure the cable is fully inserted until you hear a click"; the annotation information is synchronized to the AR glasses in real time;

[0219] Visual guidance and operation: On-site engineers can see arrow markings, text and voice prompts superimposed on the real BBU through AR glasses. Following the guidance, they check the CPRI cables and find that one of the cables is loose. After reconnecting the cable, the AR glasses capture images of the equipment in real time, and the AI ​​recognizes that the operation light turns green (normal state). The system automatically prompts "The fault has been resolved. Please continue with the subsequent installation steps."

[0220] Subsequent operations: Experts push out an animation of the cable connection process for the BBU device through the control panel, and AR glasses show the cable connection sequence and method for each port step by step. The on-site engineer completes all cable connections according to the animation guidance, and the installation and debugging task is completed.

[0221] Example 3: System Performance Testing

[0222] Performance tests were conducted on the AR remote assistance system of this application. The test scenario was the installation and debugging of a 5GAAU and BBU in a communication equipment room. The test results are as follows:

[0223]

[0224] Test results show that all performance indicators of the system in this application meet the design goals and can meet the actual needs of communication equipment installation and debugging scenarios.

[0225] The AR remote assistance method and system for installing and debugging communication equipment provided in this application has the following beneficial effects:

[0226] Improve the utilization rate of expert resources and reduce collaboration costs: AR remote assistance enables senior experts to provide precise guidance without physical presence, solving the problems of scarce and unevenly distributed expert resources, significantly reducing travel and time costs. Testing has shown improved efficiency in solving complex problems. above.

[0227] Enhance the immersiveness and accuracy of remote assistance: Based on multi-source fusion SLAM technology, high-precision spatial positioning and virtual-real fusion are achieved. The alignment error between virtual annotations and real devices is ≤5cm. Combined with first-person view video transmission, experts can feel as if they are "on-site" and accurately locate the problem location and operation node, solving the problem of inaccurate information transmission in traditional remote assistance.

[0228] Enhance the intelligence level of installation and commissioning: Integrate AI intelligent recognition technology and a dedicated knowledge base to achieve automatic equipment identification, instrument reading extraction, and intelligent fault diagnosis, reducing reliance on on-site personnel experience, lowering the operational error rate, and improving the accuracy of installation and commissioning. above.

[0229] Supports efficient multi-party collaboration: Through spatial anchor point sharing and collaborative annotation technology, it enables collaborative guidance from multiple experts in the same virtual space, solving the problem of multi-domain expert collaboration for complex tasks and improving collaboration efficiency. above.

[0230] It has good scalability and compatibility: the system supports mainstream AR glasses and mobile terminals, provides open API interfaces, can be integrated with existing operation and maintenance management systems and equipment management platforms in the communications industry, and is compatible with communication equipment from different manufacturers and models, with a wide range of application scenarios.

[0231] The technical solutions provided by the embodiments of the present invention have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of the embodiments of the present invention. The descriptions of the embodiments above are only for helping to understand the principles of the embodiments of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the embodiments of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. An AR remote assistance method for the installation and debugging of communication equipment, characterized in that, Includes the following steps: Collect first-person audio and video data, environmental perception data, and equipment status data at the installation and commissioning site of communication equipment; Spatial computing technology based on multi-source information fusion enables the overlay of virtual and real scenes, constructing an immersive AR interactive environment; The device status data is analyzed using AI intelligent recognition technology to obtain key device information and abnormal characteristics; Based on a dedicated knowledge base for the communications industry and a fault diagnosis reasoning engine, it generates installation and debugging guidance plans or fault solutions. Real-time audio and video communication and collaborative annotation in 3D space between on-site personnel and remote experts are achieved through multi-party collaborative interaction technology. The AR smart terminal receives the guidance scheme and 3D spatial collaborative annotation information, and performs a visual display in a virtual-real fusion scene to assist in on-site installation and debugging operations.

2. The AR remote assistance method for installation and debugging of communication equipment according to claim 1, characterized in that: The first-person audio and video data, environmental perception data, and equipment status data collected at the installation and commissioning site of the communication equipment specifically include: Collected via the high-definition camera of the AR smart terminal First-person view video stream and 48kHz audio stream; The AR smart terminal collects environmental pose data, location data, and distance data through its built-in inertial measurement unit, GPS module, and UWB module. The image acquisition module acquires images of the device panel, instrument display, and port connections as device status data. By adopting a hybrid transmission mode of P2P and Relay based on WebRTC, combined with efficient video coding algorithms and forward error correction strategies, ultra-low latency transmission of acquired data is achieved.

3. The AR remote assistance method for installation and debugging of communication equipment according to claim 1, characterized in that: The spatial computing technology based on multi-source information fusion for overlaying virtual and real scenes specifically includes: A multi-source SLAM algorithm model integrating visual inertial odometry, GPS, and UWB is constructed, and the spatial positioning result is calculated using a weighted fusion formula: ； in, For VIO location results, This is the GPS positioning result. For UWB positioning results, , , To merge weights and satisfy , Here are the covariance matrices for each positioning method. This is the error correction factor. It is the identity matrix; Based on lighting estimation and environmental texture analysis techniques, lighting rendering and pose adjustment are performed on virtual 3D models to achieve a realistic fusion of virtual objects and the real environment.

4. The AR remote assistance method for installation and debugging of communication equipment according to claim 1, characterized in that: The step of parsing the device status data using AI intelligent recognition technology specifically includes: A deep learning object detection model is used to perform millisecond-level identification of the communication device body, port type, and indicator light status in device status data, and output the identification confidence score. Based on OCR technology, digital images are extracted from instrument display images, and image distortion is eliminated through character correction algorithms to obtain accurate instrument readings. The deep learning target detection model is a lightweight model optimized for communication equipment scenarios. Its backbone network adopts a deep separable convolutional structure, and the detection head adopts a multi-scale feature fusion design.

5. The AR remote assistance method for installation and debugging of communication equipment according to claim 1, characterized in that: The solution generated based on a communication industry-specific knowledge base and a fault diagnosis reasoning engine specifically includes: The dedicated knowledge base for the communications industry includes 3D models of mainstream communication equipment, standardized installation and debugging process animations, and a knowledge graph of fault phenomena, causes, and solutions. The fault diagnosis inference engine calculates the confidence level of the fault solution using the following formula: ； in, To improve the accuracy of AI recognition, For the knowledge graph node correlation degree, These are the weighting coefficients. For the first The influence coefficient of each failure factor For the first Uncertainty of each failure factor; output sorted by confidence level. Several solutions are available for on-site personnel and experts to choose from.

6. The AR remote assistance method for installation and debugging of communication equipment according to claim 1, characterized in that: The multi-party collaborative interaction technology specifically includes: Based on the cloud-based spatial anchor point sharing protocol, a multi-terminal three-dimensional spatial coordinate system mapping relationship is established to ensure the consistency of virtual annotation positions; It supports multi-user online collaborative annotation, with annotation types including 3D arrows, virtual 3D models, structured text, and voice annotations. It resolves concurrent annotation conflicts through a state synchronization protocol. The expert control panel supports switching between multiple live feeds, interactive whiteboards, and sharing of installation and debugging documents, enabling collaborative guidance from multiple experts.

7. The AR remote assistance method for installation and debugging of communication equipment according to claim 1, characterized in that: The AR smart terminal is AR glasses or... The mobile terminal supports multiple interaction methods, including voice, gesture, and eye tracking. The visualization includes step-by-step animation of 3D models, highlighted fault points, and text guidance for operation steps.

8. An AR remote assistance system for the installation and debugging of communication equipment, used to implement the AR remote assistance method for the installation and debugging of communication equipment as described in any one of claims 1-7, characterized in that, include: The data acquisition and transmission module is used to acquire first-person perspective audio and video data, environmental perception data, and equipment status data on site, and to achieve ultra-low latency transmission. The spatial computing and virtual-real fusion module is used to achieve spatial positioning based on SLAM technology of multi-source information fusion and to complete the fusion rendering of virtual objects and real environment; The AI ​​intelligent recognition module is used to analyze equipment status data and identify key equipment information and abnormal features. The knowledge base and diagnostic module includes a telecommunications industry-specific knowledge base and a fault diagnosis reasoning engine, which are used to generate installation and debugging guidance or fault solutions. The multi-party collaborative interaction module is used to enable real-time communication and collaborative annotation in three-dimensional space between on-site personnel and remote experts. The AR visualization module is used to visualize and display guidance schemes and collaborative annotation information on AR smart terminals.

9. A computer device, characterized in that, It includes a memory and a processor, the memory storing a computer program, and the processor being configured to execute the computer program to implement the AR remote assistance method for installation and commissioning of a communication device as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the AR remote assistance method for installing and debugging communication equipment as described in any one of claims 1 to 7.

Images