Visual management method based on AR live-action monitoring

Abstract

The invention relates to a visual management method based on AR live-action monitoring, and aims to improve management efficiency and decision quality. According to the method, through comprehensive application of data acquisition, processing, display and feedback mechanisms, a set of complete solution is provided. The core module is an independently researched and developed intelligent data fusion and coordinate conversion algorithm (IDFCTA), and the functions of the core module comprise data preprocessing, feature extraction, intelligent data fusion and coordinate conversion. The data acquisition module is provided with various sensors and high-definition cameras, so that comprehensive monitoring is ensured; the data processing module uses an advanced algorithm to eliminate noise, identify abnormity and integrate information; the AR display module superposes the processed data and an actual scene through equipment, and real-time visualization is achieved; the user interaction module provides a visual interface and supports gesture and voice operation; and the feedback and optimization module records operation data and continuously improves system functions. Compared with the prior art, by means of efficient data processing and intelligent user interaction, the AR application immersion experience and management decision accuracy are greatly improved, and the method has wide market application prospects.

Description

Technical Field

[0001] The present invention relates to the field of augmented reality (AR) technology, and in particular to a visual management method based on AR real-scene monitoring. Background Art

[0002] In the traditional management process, operators often rely on static data and manual records, which leads to a lag in information transmission and affects the timeliness of decision-making. In complex environments such as industry and construction, the status of management objects changes frequently, and it is difficult for operators to obtain real-time information, which restricts the management effect. This situation has caused the phenomenon of information islands, and various types of data are scattered in different systems, making it difficult to share and integrate. In addition, relying on manual operations and records is also prone to errors and omissions, and the lack of effective real-time monitoring means makes it impossible for managers to respond in time at critical moments.

[0003] To solve these problems, traditional management methods have taken some measures, such as establishing a centralized monitoring system to integrate various types of information. However, this still relies on manual operations and information updates are not timely enough. Although regular inspections can obtain certain data, they cannot achieve real-time monitoring. At the same time, although information management software has been introduced to record and calculate statistics, it is often not combined with actual scenarios, resulting in a disconnect between data and reality.

[0004] In recent years, with the rapid development of information technology and sensor technology, especially the rise of the Internet of Things (IoT) and augmented reality (AR) technology, management methods have gradually changed. More and more companies have begun to deploy IoT devices to achieve interconnection between devices, obtain data in real time, and centrally manage them through cloud platforms. In addition, the application of AR technology in visual management has gradually gained attention. By combining real-time data with actual scenes, the intuitiveness of information display has been improved, helping operators to make decisions quickly. At the same time, intelligent decision support systems based on big data analysis and machine learning technology have also become popular, providing managers with data-driven decision-making basis.

[0005] The advancement of these technologies provides new solutions for modern management, makes real-time monitoring and intelligent management possible, and lays a solid foundation for the implementation of the present invention. Summary of the invention

[0006] The present invention aims to provide a visual management method based on AR real-scene monitoring. Through innovative algorithms and technical means, the real-time collected data can be efficiently processed and visualized to help managers make efficient and accurate management decisions in a complex and changing environment. Specific goals include: enhancing the accuracy and real-time performance of data processing, simplifying the user interface, improving the interactive experience, and optimizing the data analysis and feedback mechanism, thereby improving the operating efficiency and decision-making quality of the entire system.

[0007] The technical implementation scheme of the present invention is a real-time monitoring management method based on augmented reality (AR). By configuring a variety of sensors and cameras, the status information and environmental parameters of the management object are collected in real time; the collected real-time data is analyzed and processed by data processing algorithms, and converted into a format suitable for AR display; the processed data is superimposed and displayed with the actual scene through AR equipment to achieve real-time visualization of information; a user operation interface is provided to support operators to interact with the AR system through gestures or voice commands to quickly obtain the required information and implement management; the data during the operation is recorded, and the effects of management decisions are analyzed to continuously optimize the management strategy.

[0008] A visual management method based on AR real-scene monitoring, comprising:

[0009] Step 1: Obtain the basic information of the managed object, configure sensors and high-definition cameras, collect the status information and environmental parameters of the managed object in real time, and transmit the data to the data processing center through wireless communication protocols.

[0010] Step 2: Preprocess the collected data using an improved Kalman filter and self-developed outlier detection algorithm (AVDA) to eliminate noise and abnormal data and ensure the accuracy and completeness of the data.

[0011] Step 3: Perform statistical analysis and frequency domain feature extraction on the data through the feature extraction module, and use the self-developed feature extraction function (FETF) to extract feature information that is helpful for data fusion.

[0012] Step 4: Use the intelligent data fusion algorithm (IDF) to perform weighted fusion on multi-source data, and convert the data to the coordinate system of the AR system through the coordinate conversion module to prepare for subsequent scene overlay.

[0013] Step 5: The processed data is superimposed on the actual scene on the AR device and an interactive interface is provided. Users can obtain real-time information and operation suggestions through gestures or voice. At the same time, the interaction data is recorded for system optimization and analysis.

[0014] In step 1, the modules include:

[0015] Data acquisition module: Various sensors (such as temperature and humidity sensors, smoke sensors, vibration sensors, etc.) and high-definition cameras are deployed in the monitoring area to achieve comprehensive environment and equipment monitoring. Data is transmitted to the data processing center in real time through wireless communication protocols (such as Wi-Fi or Bluetooth) to ensure the timeliness and accuracy of data transmission.

[0016] In steps 2, 3, and 4, the modules include:

[0017] Data processing module: This module is the core of the entire system, relying on the self-developed "intelligent data fusion and coordinate conversion algorithm (IDFCTA)" for data processing. The specific process includes: first, eliminating noise through improved Kalman filtering, and combining the self-developed outlier detection algorithm (AVDA) to process abnormal data to ensure data quality; then, using the self-developed feature extraction function (FETF) to analyze and extract the key features of the data, such as rate of change, acceleration and frequency components; then, integrating data sources from different sensors through the weighted data fusion algorithm (WDFA) to improve data integrity and reliability; finally, mapping the processed data to the AR coordinate system through the coordinate conversion function (CTF) to ensure accurate matching of data in the virtual and real worlds.

[0018] In step 5, the modules include:

[0019] AR display module: The processed data is superimposed on the actual scene through AR devices (such as smart glasses or mobile devices) to achieve real-time visual display. For example, the temperature value of the device is superimposed on its physical location in digital form, and different colors are used to indicate the status of the device. Users can interact through the AR interface and click or point to different devices to obtain detailed operating data and suggestions.

[0020] User interaction module: A simple and intuitive user interface is designed to support gesture control (such as sliding, clicking) and voice commands (such as "show current temperature") to reduce the difficulty of user operation. In addition, the system will automatically push task reminders to users based on real-time data analysis, such as providing maintenance suggestions when equipment abnormalities are detected.

[0021] Feedback and Optimization Module: This module is responsible for recording detailed data of each operation, including operation time, results, and environmental changes, providing a basis for subsequent data analysis and optimization. Through machine learning algorithms, the system will regularly analyze operation data, identify patterns and trends in management, and continuously optimize system functions and user experience, thereby improving overall management efficiency.

[0022] Compared with the prior art, the present invention has the following significant advantages:

[0023] 1. The independently developed IDFCTA algorithm ensures the efficiency and accuracy of data processing, especially in multi-sensor fusion and coordinate transformation, which greatly improves data quality and real-time performance.

[0024] 2. Efficient anomaly detection and data preprocessing methods (such as AVDA and Kalman filtering) ensure the quality of sensor data and accurately capture effective information even in dynamic environments.

[0025] 3. AR visualization and interaction functions display real-time data through an intuitive AR interface, simplifying the operation process and improving the interaction efficiency between users and the system.

[0026] 4. The feedback optimization module based on machine learning continuously improves the intelligence level of the system and provides more accurate decision support by analyzing and learning the operation data.

[0027] 5. The modular design enables the system to have good scalability in different application scenarios. In the future, it can be widely used in technical fields such as AR, VR, and MR, further expanding its market application.

[0028] Through the above-mentioned innovative technical solutions and module designs, the present invention significantly improves the functionality, accuracy and user experience of the AR real-scene monitoring system, and has broad market prospects and application potential. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the modules of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0030] Figure 1 This is a flow chart of a visual management method for real-scene monitoring based on AR technology.

[0031] Figure 2 Figure 1 is a flow chart of the modules used.

[0032] Figure 3 This is the flow chart of the data acquisition module.

[0033] Figure 4 Flowchart of the data processing module.

[0034] Figure 5 This is a flowchart of the AR display module.

[0035] Figure 6 Flowchart of the user interaction module.

[0036] Figure 7 Flowchart of the feedback and optimization module. DETAILED DESCRIPTION

[0037] The modules used in the present invention are described clearly and completely below in conjunction with the accompanying drawings. The description is only used to illustrate and explain the present invention and is not used to limit the present invention.

[0038] This paper provides a visual management method based on AR real-scene monitoring, such as Figure 1 As shown, including:

[0039] Step 1: Obtain the basic information of the managed object, configure sensors and high-definition cameras, collect the status information and environmental parameters of the managed object in real time, and transmit the data to the data processing center through wireless communication protocols.

[0040] Step 2: Preprocess the collected data using an improved Kalman filter and self-developed outlier detection algorithm (AVDA) to eliminate noise and abnormal data and ensure the accuracy and completeness of the data.

[0041] Step 3: Perform statistical analysis and frequency domain feature extraction on the data through the feature extraction module, and use the self-developed feature extraction function (FETF) to extract feature information that is helpful for data fusion.

[0042] Step 4: Use the intelligent data fusion algorithm (IDF) to perform weighted fusion on multi-source data, and convert the data to the coordinate system of the AR system through the coordinate conversion module to prepare for subsequent scene overlay.

[0043] Step 5: The processed data is superimposed on the actual scene on the AR device and an interactive interface is provided. Users can obtain real-time information and operation suggestions through gestures or voice. At the same time, the interaction data is recorded for system optimization and analysis.

[0044] Among them, the specific implementation modules in the steps are as follows:

[0045] Data acquisition module:

[0046] This data acquisition module achieves efficient and accurate real-time data acquisition through sensor layout, data transmission and signal monitoring. Sensors cover key areas, data is transmitted in real time through wireless networks, and a signal monitoring and feedback mechanism is set up to ensure data integrity and provide support for subsequent processing and decision-making.

[0047] 1. Sensor layout:

[0048] (1) Determine the monitoring area: Evaluate the management area and identify key monitoring locations (such as equipment side, entrance, vent, etc.).

[0049] (2) Select sensor type:

[0050] Temperature and humidity sensor: Choose a sensor that is suitable for the environmental conditions (such as a high-precision digital temperature and humidity sensor) to ensure that the temperature and humidity can be accurately measured.

[0051] Smoke Sensor: Choose a smoke sensor with high sensitivity to respond quickly to fire hazards.

[0052] Vibration sensors: Install vibration sensors on machinery and critical infrastructure to monitor abnormal vibrations.

[0053] (3) Install high-definition cameras: Install high-definition cameras at key locations (such as entrances and exits, equipment control panels) to achieve comprehensive monitoring and configure appropriate viewing angles and focal lengths.

[0054] 2. Data transmission:

[0055] (1) Select a wireless communication protocol: Select a suitable wireless communication protocol (such as Wi-Fi, Bluetooth, LoRa, etc.) based on actual needs and environmental conditions.

[0056] (2) Configure the sensor network:

[0057] Connection configuration: Configure a unique identification code for each sensor to ensure data uniqueness and security.

[0058] Network topology design: Design the network topology of sensors to ensure that data can be effectively transmitted to the data processing center.

[0059] (3) Real-time data collection and transmission:

[0060] Data collection frequency setting: Set the data collection frequency of the sensor according to management requirements (such as every second, every minute).

[0061] Data packaging and transmission: The collected data is packaged and transmitted to the data processing center in real time using the selected communication protocol to ensure timeliness and accuracy.

[0062] 3.Signal monitoring and feedback:

[0063] Monitor the quality of data transmission signals in real time to ensure the integrity of data during transmission, and set up a feedback mechanism to deal with possible signal interruptions.

[0064] Through the above specific steps, the data acquisition module can achieve efficient real-time monitoring and data transmission, providing a reliable basis for subsequent data processing and management decisions. The module flow chart is as follows: Figure 3 shown.

[0065] Data processing module:

[0066] The core of this module is a set of self-developed algorithms, called "Intelligent Data Fusion and Coordinate Transformation Algorithm" (IDFCTA), which aims to efficiently process real-time data and convert it into the format required by the AR system. The algorithm integrates four key technical links: data preprocessing, feature extraction, intelligent data fusion and coordinate transformation, ensuring the accuracy and real-time performance of data processing, and providing a solid foundation for AR applications.

[0067] 1. Data preprocessing (DP):

[0068] In the data preprocessing stage, we adopted a method that combines an improved Kalman filter and a self-developed outlier detection algorithm (AVDA) to eliminate random noise and outliers in the data and ensure data quality.

[0069] (1) Improved Kalman filter: By optimizing the parameters of the Kalman filter, we can more accurately predict data trends and correct observations to reduce errors in dynamic environments. The improved Kalman filter consists of two main stages: prediction and update. By introducing a dynamic parameter adjustment mechanism and adaptive noise estimation, this algorithm can effectively reduce the impact of system noise on data and ensure the accuracy of data processing.

[0070] Prediction stage:

[0071] In the prediction phase, the algorithm uses historical state data to estimate the current state. This step can get the estimated value in advance, thereby reducing the delay in the data collection process. Let xk be the current state and Pk be the covariance, then the prediction of the Kalman filter can be expressed as:

[0072]

[0073] Among them, Fk is the state transfer matrix, Bk is the control matrix, uk is the control vector, and Qk is the process noise covariance.

[0074] By adaptively adjusting the value of Qk, the sensitivity to burst noise can be reduced and the robustness of the system can be improved.

[0075] Update phase:

[0076] After acquiring new observation data, the update phase begins, using the observation results to correct the predicted state and covariance to improve the accuracy of the estimate. The update process includes the following formula:

[0077] K k =P k∣k-1 H k T (H k P k∣k-1 H k T +R k ) -1

[0078]

[0079] P k =(IK k H k ) k∣k-1

[0080] Where: Kk is the Kalman gain, which determines the weight of the influence of the observation data on the final estimation result; Hk is the observation matrix, which maps the prediction in the state space to the observation space; Rk is the observation noise covariance matrix, which can be dynamically adjusted in the improved Kalman filter to adapt to different sensor errors.

[0081] Dynamic noise adjustment and parameter optimization

[0082] The improved Kalman filter also includes the function of dynamic noise adjustment, that is, adaptively adjusting the values ​​of Qk and Rk according to the changes in the real-time environment to adapt to different noise levels. The core of this improvement is to correct the covariance matrix in real time according to the fluctuation trend in historical data, thereby improving the performance of the filter under different environmental conditions.

[0083] Adaptive adjustment of process noise: Automatically adjust Qk according to the amplitude of data fluctuations, enhance the adaptability to sudden fluctuations, and reduce errors in high-noise scenarios.

[0084] Adaptive adjustment of observation noise: Use sensor characteristics to automatically set Rk so that it changes with the real-time accuracy of the sensor. For example, when sensor noise increases, increase Rk to reduce dependence on current observations and keep the system stable.

[0085] The improved Kalman filter shows remarkable stability and accuracy when processing sensor data in dynamic environments, providing high-quality basic data for feature extraction and data fusion. This algorithm optimizes real-time performance and is suitable for real-time monitoring scenarios of AR applications, which helps to improve the reliability and accuracy of the overall data processing of the system.

[0086] (2) Self-developed outlier detection algorithm (AVDA):

[0087] The patented self-developed outlier detection algorithm (AVDA) is an efficient algorithm for data processing modules, which is designed to identify and process abnormal data points in data streams in real time. Through the flexible adjustment of dynamic thresholds, AVDA can ensure high-accuracy anomaly detection and data quality improvement under different environments and data fluctuation conditions. The algorithm includes three key steps: data statistical analysis, dynamic threshold calculation, and outlier point determination and processing.

[0088] Data Statistical Analysis:

[0089] In the initial step of the AVDA algorithm, the key features of the data, such as mean and standard deviation, are obtained by statistically analyzing the real-time data stream. This process provides basic information about data distribution for subsequent anomaly detection.

[0090] According to the following formula, the central value μ of the data is obtained.

[0091]

[0092] The following formula is used to calculate the degree of dispersion σ of the data.

[0093]

[0094] Among them, N represents the total number of data points, xi is the i-th data point, μ represents the center position of the data, and σ represents the discrete characteristics of the data.

[0095] Dynamic threshold calculation:

[0096] The core of the AVDA algorithm is to calculate the dynamic threshold in real time according to the changes in data distribution to adapt to the volatility of different environments. A flexible threshold factor α is used to achieve dynamic adjustment of the threshold range, thereby ensuring the high adaptability and accuracy of the algorithm. The dynamic threshold calculation formula is as follows:

[0097] Threshold=μ±ασ

[0098] The threshold factor α is adjusted to achieve an adaptive range, so that the algorithm can accurately identify anomalies for different types of data. This setting ensures that the algorithm maintains stable performance under diverse data fluctuation conditions.

[0099] Abnormal point determination and processing:

[0100] In the AVDA algorithm, if a data point exceeds the dynamic threshold range, it is identified as an outlier and immediately enters the processing flow to avoid interference with downstream data processing modules. The specific operations are as follows:

[0101] According to the following formula, determine whether the data point xi is an outlier.

[0102] |x i -μ>α·σ

[0103] There are three strategies for dealing with outliers: Replacement strategy: replace outliers with the mean or median of the previous time point to eliminate mutations. Smoothing strategy: weighted average adjacent data points through a smoothing algorithm to improve the smoothness of the data curve. Deletion strategy: when the abnormal data deviates seriously, delete it directly to ensure the continuity and overall trend of the data. And it also has three technical advantages: High adaptability: compatibility with multiple data environments is achieved through dynamic threshold adjustment. Strong real-time performance: algorithm design ensures low-latency data processing, which is suitable for the application requirements of real-time monitoring systems. High-precision detection: while filtering data efficiently, ensure the quality of output data and provide reliable data support for subsequent algorithms.

[0104] The self-developed outlier detection algorithm (AVDA) makes data preprocessing more efficient and stable through dynamic adaptation and real-time processing, which is especially suitable for AR real-time monitoring systems that have strict requirements for high-quality data. This algorithm effectively reduces the negative impact of abnormal data on system functions and decision-making, and provides an accurate data foundation for real-time display in AR applications.

[0105] 2. Feature Extraction (FE):

[0106] Feature extraction is an important step in the data processing module, which aims to extract representative features from the raw data to support subsequent data fusion and analysis. This patent uses a self-developed feature extraction function (FETF), which can extract key features from complex data sets and help the system understand the inherent laws of the data.

[0107] (1) Self-developed feature extraction function (FETF): By analyzing the rate of change, acceleration, and frequency components of data, FETF can reveal the dynamic characteristics and periodicity of data, providing key data support for AR applications. The specific formula is as follows:

[0108]

[0109] Among them, Δxk and Δtk are the change amount and time interval of the data respectively, and FFT is the fast Fourier transform.

[0110] FETF is one of the core innovations of this patent and has the following features:

[0111] Change rate analysis: FETF calculates the change rate of data through a self-developed algorithm to reveal the dynamic characteristics of data in real time. The algorithm can quickly identify the trend of data changes and provide important dynamic information.

[0112] Acceleration calculation: FETF further uses a self-developed algorithm to calculate the difference of the rate of change to obtain the acceleration of the data, thereby revealing the speed and direction of the change. Through this acceleration analysis, FETF can capture the instantaneous characteristics of data changes and provide a basis for in-depth understanding of the data.

[0113] Frequency component extraction: To analyze the periodicity of data, FETF uses a self-developed fast Fourier transform algorithm to convert time series data into frequency domain signals. This self-developed algorithm can effectively identify the main frequency and periodic characteristics in the data, providing important support for subsequent analysis.

[0114] The feature vectors finally output by FETF are highly representative. These vectors can not only effectively describe the dynamic characteristics and periodicity of the original data, but also provide an important information basis for subsequent data fusion. These feature vectors play a key role in augmented reality (AR) applications, supporting the system to make intelligent decisions in multiple fields such as real-time monitoring, user interaction and equipment status analysis.

[0115] FETF is designed to perform well in a variety of application scenarios, including:

[0116] Environmental monitoring: Use FETF to analyze the dynamic changes of sensor data and realize real-time environmental status monitoring. User behavior analysis: Extract the characteristics of user interaction data through FETF to optimize user experience. Equipment status monitoring: Real-time analysis of equipment operation data and use FETF to predict potential failures.

[0117] In summary, the self-developed feature extraction function (FETF) has greatly improved the efficiency and accuracy of data processing through unique algorithm design. The innovation of FETF makes it outstanding in complex data analysis and provides accurate decision support for AR applications. Its self-developed part not only demonstrates the uniqueness of the technology, but also brings new solutions to the industry.

[0118] 3. Intelligent Data Fusion (IDF):

[0119] Intelligent data fusion is the core of this algorithm, which integrates data from different sensors and sources through the self-developed weighted data fusion algorithm (WDFA) to improve data integrity and reliability.

[0120] (1) Self-developed weighted data fusion algorithm (WDFA): WDFA dynamically assigns weights to each data source based on its quality and importance, and synthesizes a more accurate data representation by weighted averaging to provide a basis for decision-making in the AR environment. Let w1, w2, ..., wn be the weights of each data source, and x1, x2, ..., xn be the values ​​of each data source. The fused data X is:

[0121]

[0122] WDFA is one of the key innovations of this patent and has the following features:

[0123] Dynamic weight assignment: WDFA dynamically assigns weights based on the quality, reliability, and importance of each data source. Unlike traditional static weighting methods, WDFA evaluates the status of each data source in real time to ensure that weights are adjusted in a timely manner when data quality changes. This mechanism significantly improves the adaptability of data fusion, enabling the system to maintain efficient decision-making capabilities in different scenarios.

[0124] Weighted average mechanism: WDFA uses weighted average to synthesize the information of each data source. Specifically, the algorithm calculates the product of each data source value and its corresponding weight, sums these products, and finally divides them by the sum of the weights to obtain a more accurate representation of the fused data. This method makes full use of the advantages of different data sources and reduces the noise impact that may be brought by individual data sources.

[0125] Real-time performance optimization: WDFA is optimized for real-time application scenarios and can quickly process large amounts of data with low latency. This performance improvement enables the system to respond instantly to user needs in an augmented reality (AR) environment, ensuring the real-time and accuracy of data.

[0126] Adaptive features: WDFA has adaptive capabilities and can automatically adjust the weight distribution strategy based on the results of real-time data analysis. For example, when the environment changes or the data quality fluctuates, the algorithm can promptly identify and adjust the weight of each data source to ensure that the fused data is always in a highly reliable state.

[0127] Multi-source data integration: WDFA supports obtaining information from multiple heterogeneous data sources, including sensors, network APIs, and databases. By efficiently integrating data from different sources, WDFA provides a comprehensive and accurate data view, laying a solid foundation for subsequent decision-making processes.

[0128] The fused data processed by WDFA not only accurately reflects the real-time status of each data source, but also provides a reliable basis for decision-making in the AR environment. These fused data play a vital role in various application scenarios such as intelligent monitoring, real-time analysis and user interaction, ensuring the system's decision-making ability in the face of complex environments.

[0129] WDFA is designed to excel in a variety of application scenarios, including: Smart cities: real-time integration of traffic, environmental and public safety data to improve city management efficiency. Industrial monitoring: real-time monitoring of equipment operating status to quickly identify potential failures. Health monitoring: data collected from a variety of physiological monitoring devices to provide comprehensive health assessments.

[0130] In summary, the self-developed weighted data fusion algorithm (WDFA) significantly improves the accuracy and reliability of data fusion through dynamic weight allocation and weighted average mechanism. The innovative design of WDFA provides strong support for decision-making in AR environment, ensuring the efficient response and accurate judgment of the system in complex scenarios.

[0131] 4. Coordinate transformation (CT):

[0132] Coordinate transformation is a necessary step to map the processed data into the AR coordinate system. Our self-developed coordinate transformation function (CTF) ensures the correct position and orientation of the data in the virtual space.

[0133] (1) Self-developed coordinate transformation function (CTF): CTF realizes the precise transformation of data from the physical coordinate system to the AR coordinate system by considering the mapping relationship between the real world and the AR world, including translation, rotation, and scaling.

[0134] x AR =(x real -x offset )·cos(θ)-(y real - offset )·sin(θ)

[0135] y AR =(x real -x offset )·sin(θ)+(y real - offset )·cos(θ)

[0136] The advantage of the IDFCTA algorithm is its high adaptability and accuracy, which can cope with complex and changing data environments and provide stable and reliable data streams for AR applications. The algorithm has broad application prospects and is not limited to current AR technology. In the future, it can also be extended to virtual reality (VR), mixed reality (MR) and other fields that require high-precision data processing.

[0137] Through the implementation of this module, we can efficiently convert real-time data into AR format, greatly improving the user's immersive experience and laying a solid technical foundation for the development of more advanced AR applications. The module flow chart is as follows Figure 4 shown.

[0138] AR display module:

[0139] This module improves the monitoring and management efficiency of operators through real-time visualization and interactive display. AR devices receive and overlay processed data to display device status and environmental parameters in real time. Operators can select devices through gestures or clicks to view detailed information and obtain operation suggestions, while voice feedback is also supported. These features simplify the operation process and improve work efficiency.

[0140] 1. Real-time visualization:

[0141] (1) Data reception:

[0142] AR devices (such as smart glasses, tablet computers, etc.) receive processed data from the data processing module in real time, including device status, environmental parameters, etc.

[0143] (2) Scene overlay:

[0144] The received data is superimposed with the actual scene using AR technology. The real scene is captured by the camera, and the processed data is associated with the corresponding object or area in the scene based on the positioning information. For example, the temperature sensor data of a device will be matched with the location of the device and displayed above the device.

[0145] (3) Information display:

[0146] On the AR interface, operators can see real-time data such as temperature, humidity, equipment status, etc. These data will be displayed in different colors or graphics according to the set rules:

[0147] Normal status: Displayed as a green number or icon, indicating that the device is operating normally.

[0148] Alarm status: Displayed as a red number or icon with flashing prompts to alert operators to potential problems.

[0149] (4) Status update:

[0150] The data will be automatically refreshed at a set time interval (such as once a second) to ensure that operators always see the latest real-time status information.

[0151] 2. Interactive display:

[0152] (1) Target selection:

[0153] In the AR interface, operators can select specific equipment or management objects by gestures, clicks, or line of sight. When an operator selects an object, the system will enlarge and highlight the object, and a detailed information window will pop up.

[0154] (2) Information hierarchy display:

[0155] After clicking on a device, you can see multiple levels of detailed information. The first level is usually the key status information of the device, such as temperature, running time, etc. The second level may show more detailed operation logs, fault records or recommended maintenance measures.

[0156] (3) Operational suggestions:

[0157] When the device is in an abnormal state, the system will automatically generate recommended operation steps according to the preset rules and pop up prompts in the AR interface. For example, if the device temperature is too high, the interface will display "Temperature abnormal, please check the cooling system."

[0158] (4) Feedback and adjustment:

[0159] Operators can also provide feedback on the AR interface through voice or gesture control. For example, they can use the voice command "see next device" to switch the displayed device information, or use gestures to slide the screen to view different device states.

[0160] Through these specific steps, operators can easily and quickly conduct real-time monitoring and management in the AR interface, effectively improving work efficiency and reducing operational difficulty. The flow chart is as follows Figure 5 shown.

[0161] User interaction module:

[0162] This module improves the user experience through a simple and intuitive interface design, highlighting the core functions and device status. Supporting gesture and voice command operations, users can easily switch devices and view information, and the system will provide immediate feedback on operations. The task reminder function monitors the device status in real time, pushes problem descriptions and recommended operations when abnormalities occur, and classifies reminder content by priority. Users can confirm tasks through gestures or voice to ensure effective tracking and management, thereby significantly improving management efficiency.

[0163] 1. Operation interface design:

[0164] (1) Interface layout design:

[0165] The user interface (UI) uses a simple and intuitive layout, placing core functions and information in a prominent position. Key status information of the device (such as temperature, humidity, and operating time) will be displayed in an easy-to-read format, and colors and icons help users quickly identify the status of the device.

[0166] (2) Gesture control:

[0167] The system supports operator interaction through simple gestures, such as:

[0168] Slide gesture: Users can slide their fingers on the AR device screen or AR glasses to switch between different devices or manage objects. Tap gesture: Users can view detailed information by tapping a target area, similar to the action of clicking the screen. The AR device will recognize the user's gesture and enlarge the status information of the device.

[0169] Double-tap gesture: Double-tap a specific area to open additional features, such as viewing historical data for a device or generating a maintenance report.

[0170] (3) Voice command support:

[0171] The system integrates voice recognition technology, allowing users to control commands through natural language. Common voice commands include:

[0172] "Show current temperature": The system will immediately display the temperature information of the current device and provide color indication according to the temperature conditions (such as green for normal and red for warning).

[0173] "Switch to next device": When the user uses voice command to switch devices, the AR interface will automatically jump to the status display interface of the next device.

[0174] “Show maintenance log”: Users can directly view the maintenance records of the device using voice commands.

[0175] (4) User feedback:

[0176] When the user performs an operation through gestures or voice, the system will provide immediate feedback, such as animation or sound confirmation that the operation has been performed. For example, when the user successfully switches devices through voice commands, the system may emit a prompt sound and perform the corresponding switching operation on the interface.

[0177] 2. Task reminder:

[0178] (1) Real-time data monitoring:

[0179] The system background will continuously analyze sensor data and monitor the operating status and environmental parameters of the equipment. If it detects that the key indicators of the equipment (such as temperature, humidity, vibration, etc.) exceed the set safety threshold, the system will trigger a task reminder.

[0180] (2) Task push rules:

[0181] According to the monitoring requirements of different devices, the system sets different trigger rules for each state. For example:

[0182] Abnormal temperature reminder: When the temperature of a device exceeds the safe range, the system will immediately push a reminder to the operator and give corresponding maintenance suggestions.

[0183] Periodic maintenance reminder: The system automatically reminds operators to perform regular inspections and maintenance based on the equipment's operating hours, and pops up task reminders in the AR interface.

[0184] (3)Task reminder content:

[0185] Task reminders include an overview of the device's status, a description of the problem, and recommended actions. For example, when the temperature is abnormal, the reminder will display "Temperature abnormal (current temperature: 90°C)" and prompt "Please check the cooling system."

[0186] (4) Priority classification:

[0187] The system will classify tasks according to their urgency, presenting high-priority tasks (such as safety hazards or equipment failures) in the form of pop-up windows, warning sounds, etc., while low-priority tasks (such as recommended maintenance operations) will be pushed in the form of notification bars.

[0188] (5) Operation confirmation and task tracking:

[0189] After receiving a task reminder, the user can confirm the task by voice or gesture. The system will record the confirmation time and operation steps, and update the status after the task is completed to ensure effective tracking and closed-loop management of the task.

[0190] Through these specific steps, users can intuitively and conveniently operate the system through gestures and voice, and receive real-time task reminders for system analysis, thereby improving management efficiency. The specific flow chart is as follows: Figure 6 shown.

[0191] Feedback and optimization module:

[0192] This module improves system performance and user experience through comprehensive data recording and analysis. The system automatically records operation time, results, and environmental changes, and generates detailed operation logs for review. Through data cleaning and machine learning algorithms, the system identifies operation patterns and trends, predicts failures, and provides optimization suggestions. At the same time, the system continues to learn to improve functions, evaluate user experience, and adjust interface design. These steps significantly improve management efficiency.

[0193] 1. Data Recording:

[0194] (1) Operational data collection:

[0195] Whenever a user performs an operation through gestures or voice, the system automatically records detailed data of the operation, including the following: Operation time: Record the specific timestamp of the operation, so as to analyze the correlation between the operation time and the system response time. Operation result: The system records the execution result of the operation, such as whether the task was successfully completed, whether the alarm was triggered, etc. If the operation fails, the reason for the failure (such as network interruption, equipment failure) will also be recorded.

[0196] Environmental changes: Real-time monitoring of environmental parameter changes during operation (such as temperature, humidity, vibration data, etc.) in order to analyze the correlation between operation and environment. For example, when the temperature is too high, the operation may be more frequent or the response time may be slower.

[0197] (2) Storage and management:

[0198] The system will store detailed records of each operation in the background database and archive them in chronological order. In order to facilitate subsequent analysis, the system will classify these data (such as by equipment, by operator, by operation type, etc.).

[0199] (3) Operation log generation:

[0200] The system will automatically generate an operation log, including the start and end time of the operation, the operation object, the result feedback and other detailed information, for management personnel to review at any time. The operation log can be summarized by day, week or month, and a report can be generated for review and record.

[0201] 2. Analysis and improvement:

[0202] (1) Data preprocessing:

[0203] The collected operational and environmental data will first be cleaned and preprocessed to eliminate outliers and noise. This step ensures that the input analysis data is accurate and can truly reflect the operation of the management system.

[0204] (2) Data pattern recognition:

[0205] By introducing machine learning algorithms (such as clustering algorithms, decision tree algorithms, etc.), the system automatically analyzes large amounts of operational data and identifies operational patterns and trends in management. For example:

[0206] Operation frequency analysis: The system will analyze which devices have a higher operation frequency within a specific time period and infer the reasons behind it (such as device aging, environmental changes, etc.).

[0207] Efficiency Assessment: By analyzing the response time and success rate of operators, the system will assess the efficiency of each operator and identify areas that may need improvement.

[0208] (3) Trend forecast:

[0209] Based on historical data, the system uses time series analysis or prediction algorithms to predict possible future events. For example, failure prediction: If the system finds that the operating frequency or environmental parameters (such as vibration, temperature) of certain equipment gradually deviates from normal values, the system will issue an alarm in advance to remind the operator that the equipment may be about to fail.

[0210] Resource optimization: Based on the operation frequency and task distribution, the system will make resource optimization suggestions, such as adjusting the maintenance cycle of equipment, optimizing equipment configuration or improving operation processes.

[0211] (4) Feedback and adjustment suggestions:

[0212] Based on the analysis, the system will provide suggestions for improvement of operation and management. For example:

[0213] Improve operating procedures: If a certain type of operation has a low success rate or takes a long time, the system will make suggestions for process improvements, such as simplifying the operating steps or introducing faster operating methods (such as more voice control).

[0214] Device optimization suggestions: The system will provide specific maintenance or upgrade suggestions based on the device's status and historical operation records. For example, if a device frequently experiences abnormal temperatures, the system may recommend upgrading its cooling system or adjusting its operating environment.

[0215] (5) Automatic optimization system function:

[0216] As data accumulates, the system can continuously learn and optimize its own functions. For example, the system can automatically adjust the frequency and threshold of task reminders, or improve the interface layout to make the operation more concise and intuitive.

[0217] (6) User experience evaluation:

[0218] The system can also evaluate the user experience by analyzing the operator's feedback (such as operation time, task completion efficiency, etc.), and adjust the system interface design or optimize the interaction method based on the evaluation results. For common problems or operational difficulties, the system can also provide real-time help or training modules.

[0219] Through these specific steps, the feedback and optimization module not only realizes the comprehensive recording and analysis of the operation process, but also optimizes system performance and user experience based on data, thereby continuously improving management efficiency. The specific flow chart is as follows: Figure 7 shown.

[0220] The present invention provides a visual management method based on AR real-scene monitoring, which realizes real-time monitoring and efficient management of management objects through the collaborative work of various modules, thereby improving management efficiency and decision-making accuracy. The method has broad prospects and potential in practical applications.

Claims

1. A visual management method based on AR real-scene monitoring, characterized in that: The following steps are involved: Step 1: Obtain the basic information of the managed object, configure sensors and high-definition cameras, collect the status information and environmental parameters of the managed object in real time, and transmit the data to the data processing center through wireless communication protocols. Step 2: Preprocess the collected data using an improved Kalman filter and self-developed outlier detection algorithm (AVDA) to eliminate noise and abnormal data and ensure the accuracy and completeness of the data. Step 3: Perform statistical analysis and frequency domain feature extraction on the data through the feature extraction module, and use the self-developed feature extraction function (FETF) to extract feature information that is helpful for data fusion. Step 4: Use the intelligent data fusion algorithm (IDF) to perform weighted fusion on multi-source data, and convert the data to the coordinate system of the AR system through the coordinate conversion module to prepare for subsequent scene overlay. Step 5: The processed data is superimposed on the actual scene on the AR device and an interactive interface is provided. Users can obtain real-time information and operation suggestions through gestures or voice. At the same time, the interaction data is recorded for system optimization and analysis.

2. The method according to claim 1, characterized in that: The data processing module adopts the independently developed intelligent data fusion and coordinate transformation algorithm (IDFCTA), which includes four steps: data preprocessing, feature extraction, intelligent data fusion and coordinate transformation to ensure the efficiency and accuracy of data processing.

3. The method according to claim 2, characterized in that: Data preprocessing uses improved Kalman filtering and self-developed outlier detection algorithm (AVDA) to identify and process abnormal data by setting dynamic thresholds to ensure data quality.

4. The method according to claim 2, characterized in that: The feature extraction module uses a self-developed feature extraction function (FETF) to analyze the statistical characteristics and frequency domain characteristics of the time series and provide support for data fusion.

5. The method according to claim 2, characterized in that: The intelligent data fusion module uses the self-developed weighted data fusion algorithm (WDFA) to perform weighted fusion on data from different sensors by dynamically adjusting weights to improve data integrity and accuracy.

6. The method according to claim 2, characterized in that: The coordinate conversion module uses the self-developed coordinate conversion function (CTF) to map data from the physical coordinate system to the AR coordinate system, so that the processed data can be accurately presented in the virtual space.

7. The method according to claim 1, characterized in that: The AR display module superimposes the processed data with the actual scene through the AR device to form a real-time visual interface, and users can interact through gestures or voice commands to obtain detailed information and operation suggestions.

8. The method according to claim 1, characterized in that: The feedback and optimization module automatically records the data of each operation, analyzes the data through machine learning algorithms, identifies management patterns and trends, and continuously optimizes system functions and user experience.

9. The method according to claim 1, characterized in that: The system supports personalized settings, allowing users to adjust the refresh frequency and display content of sensor data, providing more flexible monitoring options.

10. The method according to claim 1, characterized in that: Each module is designed to be independently upgraded and expanded, so that the system can be applied to different management scenarios and achieve good adaptability and compatibility.

Images