Integrated AR device performance optimization method and device based on multi-core processor

Abstract

The present invention relates to the field of AR performance optimization, and in particular to an integrated AR device performance optimization method and device based on a multi-core processor. The method includes the following steps: obtaining real-time AR display scene images and real-time user interaction data; performing scene object visual recognition on the real-time AR display scene images, and encapsulating image rendering data to generate multiple AR scene rendering tasks; performing visual rendering computing power calculations on multiple AR scene rendering tasks one by one, and identifying rendering requirements of different scenes to generate rendering requirement data for each task; identifying multi-core processor parameters in the system, performing image rendering computing power perception to generate image rendering computing power for each processor; performing computing power resource pre-allocation matching on the image rendering computing power of each processor according to the rendering requirement data of each task, and constructing a dynamic computing power resource allocation strategy. The present invention realizes dynamic AR device computing power resource allocation and improves the operating performance of AR devices.

Description

Technical Field

[0001] The present invention relates to the field of AR performance optimization, and in particular to a method and device for optimizing the performance of an integrated AR device based on a multi-core processor. Background Art

[0002] With the rapid development of augmented reality (AR) technology, AR devices have been widely used in many fields such as industrial manufacturing, medical health, education and training. AR technology provides users with an immersive interactive experience by combining virtual information with the real environment. However, AR devices face tremendous computing pressure in real-time rendering, image processing, user interaction response, etc., especially when running in complex environments. The bottleneck problem of processing performance is becoming increasingly prominent, affecting the user experience and device stability.

[0003] AR devices usually need to complete a large number of data processing tasks in a short period of time, including sensor data fusion, image rendering, object recognition, and real-time interactive response. These tasks have extremely high requirements for computing power and system resources. With the continuous expansion of AR application scenarios, the performance requirements of devices are also increasing. Traditional single-core processors are prone to bottlenecks when processing multiple tasks and large amounts of data, resulting in slow system response, freezes, and even crashes. Therefore, the key to improving the performance of AR devices lies in how to efficiently utilize hardware resources, especially the parallel computing capabilities of multi-core processors.

[0004] In recent years, with the advancement of multi-core processor technology, integrated AR devices have gradually adopted multi-core processors to cope with complex computing tasks. These multi-core processors can assign different tasks to different cores for parallel processing, thereby improving overall computing efficiency. However, how to reasonably allocate tasks, dynamically adjust computing resources, avoid processor overload, and improve computing and rendering efficiency are still the core challenges in the current AR device performance optimization. Traditional performance optimization methods usually rely on fixed task scheduling strategies and lack real-time perception and dynamic scheduling capabilities of device loads, resulting in insufficient utilization of system resources and inability to effectively respond to changing load requirements. Summary of the invention

[0005] In order to solve the above technical problems, the present invention proposes a method and device for optimizing the performance of an integrated AR device based on a multi-core processor to solve at least one of the above technical problems.

[0006] To achieve the above object, the present invention provides an integrated AR device performance optimization method based on a multi-core processor, comprising the following steps: Step S1: obtaining a real-time AR display scene image and real-time user interaction data; performing scene object visual recognition on the real-time AR display scene image, and performing image rendering data encapsulation to generate multiple AR scene rendering tasks; Step S2: Calculate the visual rendering computing power of each of the multiple AR scene rendering tasks, and identify the rendering requirements of different scenes to generate rendering requirement data for each task; Step S3: Identify the multi-core processor parameters in the system, and then perform image rendering computing power perception to generate the image rendering computing power of each processor; Step S4: pre-allocate and match the image rendering computing power of each processor according to the rendering demand data of each task, make dynamic resource allocation decisions, and build a dynamic computing power resource allocation strategy; Step S5: Perform dynamic interaction behavior analysis on the user's real-time interaction data and the real-time scene interaction response changes to generate a rendered image after the interaction response; Step S6: According to the rendered image after the interactive response, the dynamic computing resource allocation strategy is optimized for real-time interactive response and the operating parameters are fine-tuned to build an intelligent AR performance optimization model.

[0007] By acquiring real-time AR scene images and user interaction data, the system can accurately perceive the current environment and user behavior, and provide data support for subsequent task division. This real-time performance ensures that the AR device can interact highly with the user and the environment, and enhances the immersiveness of the user experience. By visually identifying the scene objects, the system identifies each object and its spatial relationship, and encapsulates the image rendering task. This not only reduces the complexity of the task, but also makes each rendering task more refined in subsequent processing, improving rendering accuracy and efficiency. Calculating the visual rendering computing power of multiple AR scene rendering tasks one by one helps to clarify the computing requirements of each task and identify the rendering priority and complexity of the task. This provides a basis for subsequent resource allocation, allowing the system to allocate resources more accurately and ensure real-time rendering of high-priority tasks. According to the computing requirements of each task, the rendering requirement data of the task is generated to ensure that each rendering task can be processed on the appropriate computing resources to avoid delays or freezes caused by uneven resource allocation. Through parameter identification of multi-core processors, the system can dynamically perceive the computing power, memory bandwidth and processing power of each core. This provides important information for the optimal scheduling of tasks, making resource allocation more refined and personalized, and avoiding resource waste or overload. Through the perception of image rendering computing power, the system can accurately match the rendering capabilities of each processor core with the task requirements, ensuring that each task can run on the best core, thereby improving the image rendering efficiency and stability of the system. According to the rendering demand data of the task and the computing power resources of the processor, the system dynamically allocates resources. This allocation strategy can ensure that computing resources are efficiently allocated among tasks, avoiding the delay or failure of other tasks due to excessive resources occupied by a task. By pre-allocating computing power resources, the system can make reasonable resource predictions and allocations before the task starts, avoiding frequent adjustments during operation, reducing scheduling overhead, and improving the response speed and operation efficiency of AR devices. By analyzing the real-time interaction data of users, the system can monitor and predict user behavior in real time and respond to user needs quickly. By analyzing data such as user line of sight tracking and gesture recognition, the system quickly adjusts the elements and perspectives in the scene to improve the smoothness and naturalness of the interaction between the user and the AR environment. Based on the analysis of interactive behavior, the system can generate rendered images after interactive response in real time. Such a response not only ensures real-time performance, but also optimizes scene rendering content based on user interaction feedback and improves the dynamic expressiveness of the scene. Through the rendered images after real-time interactive response, the system can adjust the allocation strategy of computing resources in real time according to changes in tasks and feedback from user behavior. Such a dynamic optimization mechanism ensures the adaptability and flexibility of the system, and can automatically optimize according to real-time load to avoid performance bottlenecks. Based on the real-time adjustment of interactive optimization strategies and resource allocation, the system can gradually build an intelligent AR performance optimization model, accumulate task and resource management data over a long period of time, and provide a basis for future optimization.This model can not only improve the overall performance of system operation, but also intelligently adjust system resources according to different usage scenarios and load conditions, improve energy efficiency and extend the use time of equipment.

[0008] In this specification, an integrated AR device performance optimization device based on a multi-core processor is provided, which is used to execute the integrated AR device performance optimization method based on a multi-core processor as described above, including: The rendering task encapsulation module is used to obtain real-time AR display scene images and real-time user interaction data; perform scene object visual recognition on the real-time AR display scene images, and encapsulate image rendering data to generate multiple AR scene rendering tasks; The rendering requirement module is used to calculate the visual rendering computing power of multiple AR scene rendering tasks one by one, and identify the rendering requirements of different scenes to generate rendering requirement data for each task; The computing power perception module is used to identify the parameters of the multi-core processors in the system, and then perform image rendering computing power perception to generate the image rendering computing power of each processor; The dynamic resource allocation module is used to pre-allocate and match the image rendering computing power of each processor according to the rendering demand data of each task, make dynamic resource allocation decisions, and build a dynamic computing power resource allocation strategy; The user interaction response analysis module is used to analyze the dynamic interaction behavior of the user's real-time interaction data and the real-time scene interaction response changes to generate a rendered image after the interaction response; The performance optimization module is used to optimize the dynamic computing resource allocation strategy in real time based on the rendered image after the interactive response and fine-tune the operating parameters to build an intelligent AR performance optimization model.

[0009] The present invention collects and analyzes user interactions and environmental changes in real time, so that the system can accurately understand the current AR scene, improve the processing accuracy of scene information, and after generating multiple AR tasks, it can ensure that tasks in different scenes are processed in parallel, thereby providing task units and data basis for subsequent optimization, avoiding waste of global computing resources, ensuring that each rendering task is accurately evaluated for computing power, avoiding rendering delays or system overload caused by unreasonable resource allocation, providing data support for resource pre-allocation and scheduling in subsequent steps, ensuring that high-load tasks can obtain sufficient computing resources, improving the overall task execution efficiency, accurately perceiving the computing power of different cores, helping to more accurately match task requirements when making subsequent resource allocation decisions, avoiding overload operation of certain tasks, and avoiding resource bottlenecks by fully understanding the capabilities of the processor, ensuring that tasks can be efficiently executed on appropriate cores, and improving the stability and performance of the system. The dynamic resource allocation strategy can flexibly adjust resources according to the complexity and priority of the task to avoid overloading of a single core. Through reasonable pre-allocation of computing resources, each rendering task can get corresponding resource support, reduce the delay in task processing, and ensure the smoothness of the rendering process. Through dynamic analysis of interactive data, the system can respond quickly to user operations, enhance the immersive user experience, respond to user interactions in real time, and ensure that AR devices maintain efficient rendering updates and low latency when users interact with the virtual environment, avoiding poor experience caused by slow response. Real-time interactive response optimization and operating state parameter fine-tuning can enable AR devices to dynamically adjust according to the actual user operation, ensuring that the effect of each interaction can be responded to and optimized in a timely manner. Through intelligent performance optimization models, AR devices can automatically adjust and achieve continuous and efficient operation, improving the long-term stability of the system and user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 A schematic diagram of the steps of a multi-core processor-based integrated AR device performance optimization method of the present invention; Figure 2 Detailed implementation flow chart of step S1; Figure 3 Detailed implementation flow chart of step S2; Figure 4 Detailed implementation flow chart of step S3. DETAILED DESCRIPTION

[0011] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0012] The present application example provides an integrated AR device performance optimization method and device based on a multi-core processor. The execution subject of the integrated AR device performance optimization method and device based on a multi-core processor includes but is not limited to the following: mechanical equipment, data processing platform, cloud server node, network upload device, etc. equipped with the system can be regarded as the general computing node of the present application, and the data processing platform includes but is not limited to: at least one of an audio image management system, an information management system, and a cloud data management system.

[0013] See also Figures 1 to 4 The present invention provides an integrated AR device performance optimization method based on a multi-core processor, and the integrated AR device performance optimization method based on a multi-core processor includes the following steps: Step S1: obtaining a real-time AR display scene image and real-time user interaction data; performing scene object visual recognition on the real-time AR display scene image, and performing image rendering data encapsulation to generate multiple AR scene rendering tasks; Step S2: Calculate the visual rendering computing power of each of the multiple AR scene rendering tasks, and identify the rendering requirements of different scenes to generate rendering requirement data for each task; Step S3: Identify the multi-core processor parameters in the system, and then perform image rendering computing power perception to generate the image rendering computing power of each processor; Step S4: pre-allocate and match the image rendering computing power of each processor according to the rendering demand data of each task, make dynamic resource allocation decisions, and build a dynamic computing power resource allocation strategy; Step S5: Perform dynamic interaction behavior analysis on the user's real-time interaction data and the real-time scene interaction response changes to generate a rendered image after the interaction response; Step S6: According to the rendered image after the interactive response, the dynamic computing resource allocation strategy is optimized for real-time interactive response and the operating parameters are fine-tuned to build an intelligent AR performance optimization model.

[0014] By acquiring real-time AR scene images and user interaction data, the system can accurately perceive the current environment and user behavior, and provide data support for subsequent task division. This real-time performance ensures that the AR device can interact highly with the user and the environment, and enhances the immersiveness of the user experience. By visually identifying the scene objects, the system identifies each object and its spatial relationship, and encapsulates the image rendering task. This not only reduces the complexity of the task, but also makes each rendering task more refined in subsequent processing, improving rendering accuracy and efficiency. Calculating the visual rendering computing power of multiple AR scene rendering tasks one by one helps to clarify the computing requirements of each task and identify the rendering priority and complexity of the task. This provides a basis for subsequent resource allocation, allowing the system to allocate resources more accurately and ensure real-time rendering of high-priority tasks. According to the computing requirements of each task, the rendering requirement data of the task is generated to ensure that each rendering task can be processed on the appropriate computing resources to avoid delays or freezes caused by uneven resource allocation. Through parameter identification of multi-core processors, the system can dynamically perceive the computing power, memory bandwidth and processing power of each core. This provides important information for the optimal scheduling of tasks, making resource allocation more refined and personalized, and avoiding resource waste or overload. Through the perception of image rendering computing power, the system can accurately match the rendering capabilities of each processor core with the task requirements, ensuring that each task can run on the best core, thereby improving the image rendering efficiency and stability of the system. According to the rendering demand data of the task and the computing power resources of the processor, the system dynamically allocates resources. This allocation strategy can ensure that computing resources are efficiently allocated among tasks, avoiding the delay or failure of other tasks due to excessive resources occupied by a task. By pre-allocating computing power resources, the system can make reasonable resource predictions and allocations before the task starts, avoiding frequent adjustments during operation, reducing scheduling overhead, and improving the response speed and operation efficiency of AR devices. By analyzing the real-time interaction data of users, the system can monitor and predict user behavior in real time and respond to user needs quickly. By analyzing data such as user line of sight tracking and gesture recognition, the system quickly adjusts the elements and perspectives in the scene to improve the smoothness and naturalness of the interaction between the user and the AR environment. Based on the analysis of interactive behavior, the system can generate rendered images after interactive response in real time. Such a response not only ensures real-time performance, but also optimizes scene rendering content based on user interaction feedback and improves the dynamic expressiveness of the scene. Through the rendered images after real-time interactive response, the system can adjust the allocation strategy of computing resources in real time according to changes in tasks and feedback from user behavior. Such a dynamic optimization mechanism ensures the adaptability and flexibility of the system, and can automatically optimize according to real-time load to avoid performance bottlenecks. Based on the real-time adjustment of interactive optimization strategies and resource allocation, the system can gradually build an intelligent AR performance optimization model, accumulate task and resource management data over a long period of time, and provide a basis for future optimization.This model can not only improve the overall performance of system operation, but also intelligently adjust system resources according to different usage scenarios and load conditions, improve energy efficiency and extend the use time of equipment.

[0015] In the embodiment of the present invention, refer to Figure 1 , is a schematic flow chart of the steps of a method for optimizing the performance of an integrated AR device based on a multi-core processor of the present invention. In this example, the steps of the method for optimizing the performance of an integrated AR device based on a multi-core processor include: Step S1: obtaining a real-time AR display scene image and real-time user interaction data; performing scene object visual recognition on the real-time AR display scene image, and performing image rendering data encapsulation to generate multiple AR scene rendering tasks; In this embodiment, computer vision technology is applied to the acquired scene image for visual recognition of scene objects. A deep learning model (such as YOLO, Faster R-CNN or OpenPose) is used to identify objects in the scene. When training the model, a labeled data set containing various objects (such as COCO, ImageNet) is used to ensure that the model can accurately identify objects of different categories. The recognition accuracy of the model is improved by transfer learning, and a threshold (such as 0.5) is set to filter unnecessary detection results. The identified scene objects and their attributes (such as position, size, category) are encapsulated as image rendering data. Object information, depth information and image metadata are integrated using a data structure (such as JSON or XML). For each scene object, its coordinate position in three-dimensional space, relative viewing angle, and user interaction are recorded. Ensure that the encapsulated data can support subsequent rendering task scheduling. Based on the encapsulated rendering data, multiple AR scene rendering tasks are generated. These tasks are classified and prioritized according to the complexity of the object, rendering requirements and the real-time nature of user interaction data. Use a task scheduling algorithm (such as a priority queue or a load balancing algorithm) to allocate rendering tasks to ensure that high-priority tasks (such as objects currently being focused on by the user) can be processed in a timely manner. Record the parameters of each task, including object ID, rendering time requirements, resource requirements, etc.

[0016] Step S2: Calculate the visual rendering computing power of each of the multiple AR scene rendering tasks, and identify the rendering requirements of different scenes to generate rendering requirement data for each task; In this embodiment, in the generated AR scene rendering task list, each task is first identified and classified. The classification of tasks is based on factors such as scene complexity, number of objects, lighting effects, and user interaction types. A preset classification standard is used, such as dividing tasks into high, medium, and low complexity. High-complexity tasks include a large number of dynamic objects and complex lighting effects, while low-complexity tasks only contain static objects. In this way, preparations are made for subsequent rendering demand calculations. A visual rendering computing power calculation model is constructed to determine the computing power resources required for rendering tasks. The model should consider multiple factors, such as scene resolution, rendering algorithm complexity (such as rasterization, ray tracing), number of polygons per object, and material complexity. Computing power requirement = (resolution × number of polygons × complexity coefficient) / frame rate, and the complexity coefficient is defined based on the type of rendering algorithm to ensure that the formula can reflect the actual resource requirements. For each AR scene rendering task, the computing power requirement is calculated using the above model. First, the parameters of each task, such as resolution, lighting settings, number of objects, etc., are extracted, and the computing power requirement is calculated. The collected parameters are obtained through the rendering data encapsulated by the task. For a task, the parameters include: Resolution: 1920x1080 Number of subjects: 15 Average number of polygons per object: 2000 Rendering algorithm type: Rasterization (complexity factor = 1.5) Substitute these parameters into the computing power requirement calculation formula to obtain the required computing power resources for each task. After calculating the computing power requirements of each task, identify the rendering requirements. Based on the resource requirements of the task, determine whether additional resource allocation or optimization is required. For each task, generate rendering requirement data and record the following information: computing power requirements (for example: 250GFLOPS), expected frame rate (for example: 60 fps), resource priority (based on task complexity and user interaction importance) to verify the accuracy of computing power calculation and rendering requirement identification. Through actual rendering tests, check the performance of tasks under different conditions to ensure that the calculation results are consistent with actual requirements. Make necessary optimizations based on the test results, such as adjusting the complexity coefficient and updating the computing power requirement calculation model. Ensure the accuracy and efficiency of computing power calculation through continuous iteration.

[0017] Step S3: Identify the multi-core processor parameters in the system, and then perform image rendering computing power perception to generate the image rendering computing power of each processor; In this embodiment, a system information acquisition tool (such as CPU-Z, lscpu or Windows TaskManager) is used to collect the basic parameters of the multi-core processor. These parameters include the number of cores, the number of threads, the main frequency, the cache size and the architecture type. The information is obtained by calling a system command (such as lscpu) and the following parameters are recorded: Number of cores (e.g. 8 cores) Number of threads (e.g. 16 threads) Main frequency (e.g. 3.5 GHz) Cache (e.g. L3 cache size is 16 MB) Build an image rendering computing power calculation model to determine the computing power of the processor in image rendering tasks. The model should consider multiple factors, such as the number of cores, number of threads, main frequency, instruction set architecture (such as AVX2, AVX512), etc.

[0018] The calculation formula for computing power is set as follows: Rendering computing power = number of cores × number of threads × main frequency × instruction set efficiency. Instruction set efficiency is an empirical value, usually between 1 and 2, and is selected according to the architecture and optimization level of the processor. For each core, use performance monitoring tools (such as Intel VTune, AMD uProf, or NVIDIA Nsight) to perform real-time computing power perception. The monitoring tool can provide real-time load, power consumption, temperature and other information for each core. When perceiving, run a standard benchmark test (such as SPEC CPU or Blender Benchmark) to simulate image rendering scenarios and obtain the performance of the processor in actual rendering tasks. During the test, record the utilization and rendering performance of each core. Calculate the actual computing power of each core by analyzing the benchmark data. Record the number and time of tasks completed by each core under a specific rendering task, and make corrections based on the previous computing power model.

[0019] Combine the identified multi-core processor parameters with the performance data obtained from the benchmark test to generate the image rendering computing power data for each processor. Assuming that a processor has 8 cores, a main frequency of 3.5 GHz, and an instruction set efficiency of 1.5, the rendering computing power can be calculated as: Rendering computing power = 8 × 3.5 × 1.5 ≈ 42 GFLOPS. Record the rendering computing power of all processors, including the independent computing power of each core and the total computing power, for subsequent resource allocation and scheduling.

[0020] Step S4: pre-allocate and match the image rendering computing power of each processor according to the rendering demand data of each task, make dynamic resource allocation decisions, and build a dynamic computing power resource allocation strategy; In this embodiment, for each task, its rendering requirement data is extracted and compared with the identified processor computing power data. A loop structure is used to traverse all tasks and processors to find the best match. For task A, its computing power requirement is 250 GFLOPS and the expected frame rate is 60 fps. Matching is performed in the following manner: traverse all processors and check the computing power of each processor (such as processor 1 is 42 GFLOPS, processor 2 is 80 GFLOPS, etc.). Processor 2 is selected for resource pre-allocation because its computing power meets the requirements of task A. After the task matching is completed, dynamic resource allocation decisions are made. This process requires real-time monitoring of resource usage in the system and evaluation of the current processor load and performance. Use monitoring tools (such as Prometheus and Grafana) to obtain the load of each processor in real time and record parameters such as CPU utilization, memory usage, and temperature. If the load of a processor is too high, the system should consider assigning new tasks to processors with lower loads to optimize resource usage. If the load of processor 2 reaches 85%, transfer subsequent tasks to processor 1. Build a dynamic computing resource allocation strategy based on the task priority and processor load. The strategy should include the following elements: Task priority adjustment: High-priority tasks are preferentially allocated to the processor with the highest available computing power.

[0021] Load balancing: timely adjust task allocation to avoid overloading a processor. Resource recycling: after the task is completed, release the processor resources in time for subsequent tasks to use. Task rendering requirements example: Task A: computing power requirement 250 GFLOPS, high priority Task B: computing power requirement 144 GFLOPS, medium priority Processor power example: Processor 1: 42 GFLOPS, 70% load Processor 2: 80 GFLOPS, 85% load Processor 3: 100 GFLOPS, 50% load Dynamic resource allocation decision example: For Task A, it is matched to Processor 3 for pre-allocation (100 GFLOPS>250 GFLOPS), and its priority is higher than other tasks.

[0022] For task B, it is matched to processor 2, which still meets the computing power requirements despite the high load. Step S5: Perform dynamic interaction behavior analysis on the user's real-time interaction data and the real-time scene interaction response changes to generate a rendered image after the interaction response; In this embodiment, real-time interaction data of users are collected, including touch, gesture, voice command, etc. These interaction data are captured by sensors (such as touch screen, camera, microphone, etc.).

[0023] During data collection, set the data sampling frequency (e.g., 30 times per second) to ensure that the details of user interactions are captured. Use data preprocessing techniques (e.g., denoising, smoothing) to improve data quality for subsequent analysis. Structure the processed interaction data and store it in a database or memory for efficient dynamic analysis. Use JSON or CSV format to store data for easy parsing and use. Build a dynamic interaction behavior analysis model and use machine learning techniques (e.g., decision trees, support vector machines, or deep learning models) to identify user interaction patterns. When training the model, use historical interaction datasets and annotate user behaviors (e.g., selection, movement, zooming, etc.) so that the model can learn and identify different types of interactions. Introduce a real-time feedback mechanism in the model to dynamically adjust the analysis results based on the user's latest interaction status. If a user is detected to be sliding their fingers quickly, the model should determine that the user wants to quickly browse the objects in the scene. According to the results of the dynamic interaction behavior analysis, adjust the elements in the AR scene in real time. If the user looks at an object for more than a certain time (e.g., 2 seconds), the system should highlight or enlarge the object. Use an event-driven programming model to ensure that the system can respond to user interactions instantly. By setting up event listeners, monitor changes in the user's interaction status and trigger corresponding scene updates. Determine the thresholds and rules for interactive responses. After the user clicks on an object, the system needs to update the scene within 200 milliseconds to reflect the interaction. After completing the scene interaction response changes, call the rendering engine to generate a rendered image after the interactive response. Use a graphics library such as OpenGL, DirectX, or Vulkan for image rendering. Extract the updated scene state, including object position, status, lighting, and other information, and pass it to the rendering engine for processing. During the rendering process, ensure that the system can maintain a high frame rate (for example, 60 fps) to provide a smooth user experience. Use a frame rate control algorithm to dynamically adjust the rendering quality to adapt to the current computing resources.

[0024] Step S6: According to the rendered image after the interactive response, the dynamic computing resource allocation strategy is optimized for real-time interactive response and the operating parameters are fine-tuned to build an intelligent AR performance optimization model.

[0025] In this embodiment, the rendered image after the interactive response is analyzed to evaluate its quality and user experience. Use image quality evaluation indicators (such as PSNR, SSIM) to quantitatively evaluate the rendered image to ensure the clarity and details of the image, record any problems that occur during the image rendering process, such as delays, freezes, or image quality degradation, record the processing time and system performance parameters (such as CPU and GPU utilization) of each rendered frame through monitoring tools, store the analysis results in a database for subsequent use and optimization, use SQL databases or NoSQL databases (such as MongoDB) to manage and query data, optimize the dynamic computing power resource allocation strategy based on the analysis results of the rendered image, use computer vision technology (such as object detection and tracking) to monitor the user's interactive behavior in real time, and determine the object or scene area that the user is paying attention to. If it is found that the user frequently interacts with an object (such as clicking, zooming in), more computing power resources can be allocated to the rendering task of the object first. By implementing a priority queue, the priority of the task is dynamically adjusted to achieve flexible resource allocation, and adaptive algorithms (such as Q-learning) are introduced. Or genetic algorithm) to optimize resource allocation strategy so that it can be automatically adjusted according to the user's real-time interactive behavior. By continuously learning the user's behavior pattern, adjust the resource allocation strategy to maximize the user experience. In the real-time optimization process, fine-tune the system's operating parameters to ensure stability and performance under high load conditions. Dynamically adjust the resolution and quality settings of image rendering to adapt to the current computing resources. Set thresholds. When the CPU or GPU utilization exceeds 80%, automatically reduce the rendering quality (such as from 1080p to 720p) to ensure that the system can maintain a smooth user experience under different loads. Record the effect of each adjustment, including rendering frame rate, response time, and user satisfaction feedback, for subsequent model optimization. Build an intelligent AR based on the collected historical data and real-time feedback. Performance optimization model: The model should include multiple sub-modules, such as user behavior prediction module, resource allocation module and performance monitoring module. Machine learning algorithms (such as random forest, support vector machine or deep learning model) should be used to train the performance optimization model so as to dynamically adjust the resource allocation strategy according to the user's interactive behavior and system status. A real-time feedback mechanism should be introduced into the model to optimize the allocation strategy by continuously learning user behavior and system performance, ensuring that the model can adapt to different scenarios and user needs and realize personalized optimization solutions.

[0026] In this embodiment, refer to Figure 2 , is a flowchart of detailed implementation steps of step S1. In this embodiment, the detailed implementation steps of step S1 include: Step S11: Acquire real-time AR display scene images and real-time user interaction data based on the AR device; Step S12: performing visual recognition of scene objects on the real-time AR display scene image, and marking each scene object; Step S13: annotating the interactive elements of the real-time AR display scene image to obtain all the interactive elements in the scene; Step S14: Calculating the scene object density based on each scene object and all interactive elements in the scene, thereby generating a scene density value; Step S15: dividing the real-time AR display scene image into multiple position areas according to the scene density value, thereby obtaining multiple AR scene areas; Step S16: encapsulate image rendering data for multiple AR scene areas to generate multiple AR scene rendering tasks.

[0027] In this embodiment, the real-time AR display scene image in the AR system is obtained based on the AR device. During the capture process, the device's processor is used to perform preliminary image processing, filter out background noise, and enhance image clarity to improve the efficiency of subsequent processing. Design a user interaction interface to allow users to interact with the AR scene through gestures, voice or touch. Record the user's interaction data in real time, such as click location, gesture trajectory, and voice instructions. Synchronize the user interaction data with the captured image to ensure that the user's behavior and intention can be accurately reflected in the subsequent analysis. Preprocess the captured real-time scene image, including denoising, contrast enhancement, and edge detection, to improve the accuracy of subsequent visual recognition. Use image processing libraries such as OpenCV for processing. Ensure that the preprocessed image can clearly display the object features in the scene, especially in complex environments, to improve the recognizability of the target object. Apply a deep learning model (such as YOLO or Faster R-CNN) for visual recognition of scene objects. First, load the trained model and perform inference on the real-time image. Identify each object in the scene and generate a bounding box and label for each object. Record the location information and category of each object for subsequent processing. Overlay markers on the scene image and present the identified objects in the form of borders and labels to form a visual recognition result. Ensure that the markers are clearly visible and do not interfere with the user's vision. Combine the marker information with the image data to form structured data containing all the identified object information for subsequent analysis and processing. Analyze the user's real-time interaction data to determine which objects in the scene the user interacted with. Identify interactive objects through gesture recognition or click events. Combine the marker information of the scene objects to identify all elements that the user interacts with and record the type and location of these elements. Annotate the identified interactive elements and display them with different colors or styles of borders to distinguish between ordinary objects and interactive elements. Ensure that the annotation information is clear and easy to understand. Add interaction prompts to the image, such as text instructions or icons, to indicate the user's actions. This will enhance the user experience and make it easier to understand how to interact with the scene. Organize all the annotation information into a structured data format, such as JSON or XML, for subsequent processing and analysis. Ensure that the data contains detailed properties of each interactive element, such as location, type, and available operations. Design a density calculation model that takes into account the number and type of scene objects and their relative positions. Use density functions or statistical methods to evaluate the density of the scene. Determine the calculation formula, such as density value = number of objects / scene range, to ensure that the calculation result can reflect the actual density of the scene. Extract the number and location data of scene objects and interactive elements from the previous annotation information. Input this data into the density calculation model for calculation. Record the density value of each area for subsequent area division and rendering task allocation.The calculated density values ​​are presented in a visual way, such as a heat map or chart, to intuitively show the density of objects in different areas. This will help users understand the complexity of the scene. Through the visualization results, the areas with high density are identified, which facilitates the subsequent area division and rendering task allocation. Design a region division algorithm to divide the scene according to the density value. Use the K-means clustering algorithm or the threshold-based region growing algorithm to determine the region boundaries. Determine the division criteria, such as the density threshold, to divide the scene into high-density and low-density areas. Apply the calculated density value to the region division algorithm to perform the division operation. Ensure that the division of each area can accurately reflect the distribution of objects. Output the location information and characteristics of each divided area, including the number of objects in the area, interactive elements, etc. Verify the division results to check whether each area is reasonable and whether there are any inaccurate divisions. Make manual adjustments as needed to ensure that the division effect of each area meets expectations. Record the key parameters and adjustment results in the division process to provide a basis for subsequent rendering tasks. Design image rendering tasks for each area based on the multiple divided AR scene areas. Determine the rendering parameters for each task, such as resolution, lighting, shadows, and materials. Ensure that each rendering task can be executed independently to improve the parallelism and efficiency of rendering. Encapsulate the rendering data of each area to form a unified data structure for easy management and calling. Use JSON format to include area information and rendering parameters. Ensure that the data can be quickly transmitted and processed after encapsulation to avoid data redundancy and improve system performance. Schedule multiple AR scene rendering tasks according to the system's resource management strategy to ensure that the tasks can be executed efficiently. Use multi-threading or asynchronous processing technology to improve rendering efficiency. During the rendering process, monitor the task execution status in real time to ensure that each task can be completed on time, and record the rendering results for subsequent analysis.

[0028] In this embodiment, refer to Figure 3 , is a flowchart of detailed implementation steps of step S2. In this embodiment, the detailed implementation steps of step S2 include: Step S21: Calculate the visual rendering computing power of each of the multiple AR scene rendering tasks to generate the scene image rendering computing power of each task; Step S22: Calculate the number of GPU resources for multiple AR scene rendering tasks to generate the number of GPU resources for each task; Step S23: performing processor memory capacity analysis on multiple AR scene rendering tasks to obtain memory capacity data for each task; Step S24: quantifying bandwidth resources for multiple AR scene rendering tasks to obtain a bandwidth resource value for each task; Step S25: Identify different scene rendering requirements for the scene image rendering computing power of each task, the number of GPU resources of each task, the memory capacity data of each task, and the bandwidth resource value of each task to generate rendering requirement data for each task.

[0029] In this embodiment, a rendering computing power calculation model is designed to consider the main factors affecting scene image rendering, such as scene complexity, lighting effects, material characteristics, and the number of dynamic elements. Rendering computing power is usually based on the number of frames per second (FPS) and the computing resources required for each frame. Determine the computing power calculation formula, such as computing power = (number of polygons × material computing complexity × lighting computing complexity) / rendering time, to ensure that the formula effectively reflects the actual rendering requirements. Perform a detailed analysis of each AR scene rendering task to obtain relevant parameters, such as the number of polygons, material types, number of light sources, and dynamic elements in the scene. These data are obtained through the scene description file. Input the analysis results into the rendering computing power model, calculate the computing power value required for each rendering task, and record it. Verify the rendering computing power of each task calculated to ensure its rationality. Compare it with the frame rate in the actual rendering process to verify the accuracy of the computing power calculation. Record the rendering computing power results of each task in the database to form structured data for subsequent use. Determine the computing requirements of GPU resources, including the number of GPU cores, video memory size, and computing power. Refer to the GPU specifications and market standards to understand the performance of different GPU models. Design a calculation formula, such as GPU resource number = (scene complexity × rendering power) / GPU performance, to determine the GPU resources required for each rendering task. Apply the GPU resource calculation model to each AR scene rendering task, input the previously calculated rendering power value and scene complexity data, and obtain the required GPU resource number. Record the GPU resource requirements for each task and ensure the accuracy of the data. Verify the calculation results to ensure that the number of GPU resources for each task is consistent with the actual requirements. Compare the measured data with the calculation results to identify potential errors. Make necessary adjustments and optimizations based on the verification results to ensure that the number of GPU resources for each task can meet the rendering requirements. Design a memory capacity analysis model to consider the memory requirements of the rendering task, including scene data, texture data, and dynamic object data. Use a memory usage calculation formula, such as memory capacity = (scene data + texture data + dynamic object data) × safety factor, to ensure the safety of memory allocation. Determine the safety factor, usually 1.2 to 1.5, to avoid memory overflow. Perform a detailed analysis of each AR scene rendering task to obtain the specific size of the scene data, texture data, and dynamic object data. This data is obtained through the scene editor or the resource management tool of the rendering engine. Input this data into the memory requirement model, calculate the memory capacity required for each task, and record the results. Verify the calculated memory capacity to ensure its rationality. Compare the memory usage during the actual rendering process by running a memory monitoring tool (such as Windows Performance Monitor).Make necessary adjustments based on the verification results to ensure that the memory capacity of each task can meet the rendering requirements. Design a bandwidth resource quantification model to consider factors that affect bandwidth requirements, such as scene data transfer rate, texture loading rate, and user interaction data. Use the bandwidth requirement formula, bandwidth = (scene data size + texture data size) / rendering time, to calculate the bandwidth requirement of each task. Determine a reasonable value for rendering time, which is usually set to the time per frame, such as 16.67 ms for 60 FPS. Apply the bandwidth requirement model to each AR scene rendering task, input the scene data size, texture data size, and rendering time, and derive the required bandwidth resource value. Record the bandwidth requirement of each task and ensure the accuracy of the data. Verify the calculation results to ensure that the bandwidth resource value is consistent with the actual network performance. Verify with tools that monitor network traffic (such as Wireshark). Make necessary adjustments based on the verification results to ensure that the bandwidth resource value of each task can meet the rendering requirements. Design a comprehensive demand identification model to integrate the scene image rendering computing power, GPU resource number, memory capacity, and bandwidth resource value to form the rendering demand data for each task. Use a weighted algorithm to assign different weights to each parameter to reflect its importance in the rendering process. Determine the weighting coefficients, computing power accounts for 40%, GPU resources account for 30%, memory accounts for 20%, and bandwidth accounts for 10%. Apply the comprehensive demand identification model to each AR scene rendering task, input various resource data, and calculate the rendering demand data for each task. Use a simple linear combination method to calculate the total demand value. Record the rendering demand data of each task and ensure that the data is complete. Verify the calculated rendering demand data to ensure its rationality. Compare the demand with the actual usage through performance monitoring during the actual rendering process to identify potential differences. Perform necessary optimization based on the verification results to ensure that the rendering demand data of each task can accurately reflect its resource requirements.

[0030] In this embodiment, the specific steps of step S21 are: Perform scene object calculations on multiple AR scene rendering tasks to obtain object data of the AR scene; Perform spatial distribution analysis on the object data of the AR scene to generate spatial distribution data of the scene objects; Perform scene rendering texture mining on multiple AR scene rendering tasks to extract the rendering texture features of each scene; Identify lighting effects for multiple AR scene rendering tasks; Define the target frame rate for rendering the scene; Based on the target frame rate of the rendered scene, the spatial distribution data of the scene objects, the rendering texture characteristics of each scene and the lighting effect are estimated for the number of floating-point operations per frame, thereby generating the number of floating-point operations per frame; The task visual rendering computing power is calculated based on the number of floating-point operations in each frame to generate the scene image rendering computing power of each task.

[0031] In this embodiment, relevant data of all objects in each AR scene are collected, including the type, size, position, shape and material of the objects. These data are usually extracted through scene description files or 3D models. The object data is structured to generate a database containing object attributes for subsequent analysis and calculation. Use JSON or XML format to store these data for easy parsing. Feature extraction is performed on each object to calculate the geometric features (such as volume, surface area) and visual features (such as color, texture) of each object. Computer vision techniques (such as edge detection and shape analysis) are used to enhance the extraction of object features to ensure that objects in complex scenes can be identified. The extracted object feature information is sorted to generate an object data set for each scene, including information such as the position, type and feature value of each object. The generated object data is verified to ensure the integrity and accuracy of the data. The object distribution is displayed through visualization tools (such as Matplotlib or Unity) to check whether the data meets expectations. Design a spatial distribution analysis model, considering the distribution characteristics of objects in the scene, including density, aggregation and relative position relationship. Use statistical methods (such as uniformity test) to analyze the spatial distribution of objects. Determine key indicators for analysis, such as the number of objects, distribution range, and spatial density. Calculate the spatial distribution of object data for each AR scene, and use the K-means clustering algorithm or heat map analysis method to identify the clustering areas of objects in space. Generate spatial distribution data, record the number of objects in each area and their position coordinates, and form a database containing spatial distribution information. Visualize the results of spatial distribution analysis, such as generating heat maps or three-dimensional scatter plots to intuitively display the distribution of objects. Use visualization tools such as Tableau or D3.js to display the results. Verify the analysis results to ensure the accuracy and rationality of the spatial distribution data, and make adjustments if necessary. Extract texture data from the rendering task of each AR scene, including texture images, resolution, texture type, and material properties applied to objects. Use image processing tools such as OpenCV to preprocess texture images to improve the accuracy of subsequent feature extraction. Use texture analysis techniques such as Gabor filters, LBP features, or texture energy to extract key features of each texture. These features should effectively describe the details and changes of the texture. Generate a texture feature dataset for each scene, and record the statistical features of each texture (such as mean, variance, and spectral characteristics). Build a lighting model, taking into account the type of light source (such as point light source, parallel light source, and spotlight) and lighting settings (such as intensity, color, and position). The model uses a physically based lighting model (such as Phong or Blinn-Phong model). Determine the key parameters of lighting calculation, including the number of light sources, lighting intensity, and shadow effects.Perform lighting effect analysis for each AR scene rendering task and calculate the rendering effect of each object under different lighting conditions. This is achieved through lighting simulation tools such as Unity or Unreal Engine. Record the lighting configuration of each scene and its impact on the rendering effect to form a lighting effect dataset. Verify the calculation results of the lighting effect to ensure its rationality and accuracy, especially in complex scenes where the lighting effect is affected by multiple factors. Make adjustments based on the verification results to ensure that the lighting effect of each task meets expectations. Define the target frame rate for the rendering scene based on application requirements and user experience. Generally speaking, 60 FPS is the standard frame rate for games and highly interactive applications, while 30 FPS is suitable for applications with lower requirements. When setting the target frame rate, consider the performance limitations of the device and the network bandwidth to ensure that the target frame rate is within the technically achievable range. Evaluate the processing power of the device to ensure that it can support the set target frame rate. Evaluate the performance of the CPU and GPU by running benchmarks. Record the relevant parameters for setting the target frame rate to provide a basis for subsequent calculations. Adjust the target frame rate based on the feedback from actual test results. If the device cannot run smoothly at the set frame rate, you need to lower the target frame rate or optimize the scene content. Ensure that the target frame rate setting can meet the user experience requirements and match the system performance. Design a floating-point operation estimation model to consider the impact of the spatial distribution of scene objects, texture characteristics, and lighting effects on the floating-point operations of each frame. Use linear or nonlinear regression analysis methods to build the model. Determine the calculation formula, such as floating-point operations = (number of objects × complexity coefficient) + (texture feature complexity × texture weight) + (lighting impact × lighting weight). Based on the set target frame rate and various parameters, estimate the number of floating-point operations for each frame frame by frame. Input the parameters of the number of objects, texture characteristics, and lighting effects into the model for calculation. Record the number of floating-point operations for each frame for subsequent use and analysis. Verify whether the estimated number of floating-point operations is consistent with the number of operations during actual runtime, and verify it through performance monitoring tools (such as NVIDIA Nsight or IntelVTune). Make necessary adjustments to the model based on the verification results to ensure that the number of floating-point operations estimated for each frame is accurate. Design a scene image rendering computing power calculation model based on the number of floating-point operations per frame. Consider the processing power and rendering efficiency of the device and determine the computing power calculation formula, such as rendering computing power = number of floating-point operations per frame / rendering time, where the rendering time is the time corresponding to the target frame rate. Ensure that the formula can accurately reflect the computing power requirements of each task in the actual rendering process. Apply the computing power calculation model to each AR scene rendering task, input the number of floating-point operations per frame and the set rendering time, and obtain the scene image rendering computing power of each task. Record the rendering computing power of each task and ensure the accuracy of the data.

[0032] In this embodiment, refer to Figure 4 , is a flowchart of detailed implementation steps of step S3. In this embodiment, the detailed implementation steps of step S3 include: Step S31: Identify multi-core processor parameters in the system, wherein the multi-core processor parameters include the number of CPU / GPU cores, the computing power of each core, the clock frequency of the processor, the cache capacity, the cache capacity and bandwidth, the bus bandwidth and the data transmission capacity; Step S32: quantify the total computing power of the processor according to the number of CPU / GPU cores and the computing power of each core to obtain the computing power value of each processor; Step S33: Calculate the maximum memory throughput based on the cache capacity and bandwidth to generate the maximum memory throughput of each processor; Step S34: performing real-time memory access delay analysis based on the processor's clock frequency and cache capacity, and extracting real-time memory access delay data; Step S35: Perform maximum data transmission analysis according to bus bandwidth and data transmission capacity, and generate a maximum data transmission value for each processor; Step S36: image rendering computing power perception is performed on the computing power value of each processor, the maximum memory throughput of each processor, the real-time memory access delay data and the maximum data transmission value of each processor to generate the image rendering computing power of each processor.

[0033] In this embodiment, the detailed parameters of the multi-core processor are obtained by using system information tools (such as CPU-Z, GPU-Z or the command line tools provided by the system). These tools can provide information such as the number of cores of the CPU and GPU, the computing power of each core, the clock frequency, and the cache capacity. For the CPU, use the lscpu command (in the Linux system) or access the performance options of the task manager in Windows to obtain information such as the number of cores and frequency. The obtained parameters are classified and recorded, including: Number of CPU / GPU cores: record the number of cores of the CPU and GPU. The computing power of each core: usually in GFLOPS (billion floating-point operations per second), according to the specification documents provided by the manufacturer. Clock frequency: in GHz, indicating the operating frequency of each core. Cache capacity: including the size of L1, L2 and L3 caches, in MB. Bandwidth: record the memory bandwidth, usually in GB / s. Bus bandwidth and data transmission capacity: record the bandwidth information of the system bus. Design a computing power quantification model, the formula is the total computing power of the processor = the number of CPU / GPU cores × the computing power of each core. This formula can quickly calculate the total computing power of the processor, reflecting its processing performance. Determine the unit of computing power, such as GFLOPS or TFLOPS, and adjust it according to the characteristics of the processor. For each identified CPU and GPU, apply the above computing power model, input the number of cores and the computing power of each core, and calculate its total computing power value. Record the total computing power of each processor. For multi-core processors, record the computing power of the CPU and GPU separately for subsequent comparison and analysis. Verify the calculated total computing power of the processor to ensure that it is consistent with the actual performance of the processor. Verify the calculation results through performance benchmarks such as Geekbench or Cinebench. Make necessary adjustments based on the verification results to ensure the accuracy of the total computing power value of the processor. Establish a memory throughput calculation model with the formula of maximum memory throughput = cache bandwidth × (cache capacity / response time). This formula can reflect the maximum memory throughput that the processor can achieve under high load conditions. Determine the unit of bandwidth as GB / s and the unit of cache capacity as MB. Collect the cache capacity and bandwidth data of each processor, and apply the memory throughput model to calculate the maximum memory throughput. Record the maximum memory throughput value of each processor to ensure data integrity. Verify that the calculated maximum memory throughput is consistent with the actual performance by using a memory performance testing tool (such as Stream Benchmark) for comparison. Make necessary adjustments based on the verification results to ensure that the maximum memory throughput is calculated accurately.Design a real-time memory access delay analysis model, with the formula of real-time memory access delay = (cache access time + main memory access time) / (clock frequency × cache capacity). This model can reflect the processor's delay when accessing memory. Determine the unit of access time as ns (nanoseconds) to ensure that the calculation results can accurately reflect the delay. Collect the processor's clock frequency and cache capacity data, and apply the delay analysis model to calculate the real-time memory access delay. Record the real-time memory access delay data of each processor for subsequent analysis. Verify the rationality of the delay analysis results by using delay testing tools (such as Cachegrind) for actual measurement verification. Make necessary adjustments based on the verification results to ensure the accuracy of the real-time memory access delay data. Establish a maximum data transfer analysis model, with the formula of maximum data transfer = bus bandwidth × data transfer time. This model can reflect the data transfer capability of the processor under high load. Determine the unit of bandwidth and transfer time, with GB / s as the standard, to ensure the readability of the calculation results. Collect the bus bandwidth and data transfer capability data of each processor, and apply the data transfer model to calculate the maximum data transfer value. Record the maximum data transfer value of each processor for subsequent analysis. Verify whether the calculated maximum data transfer value is consistent with the actual performance. Verify through data transfer test tools (such as iPerf). Make necessary adjustments based on the verification results to ensure the accuracy of the maximum data transfer value. Design an image rendering computing power perception model to combine the processor's computing power, memory throughput, real-time memory access latency, and maximum data transfer to form a comprehensive rendering computing power indicator. Use a weighted average model to set the weight of each indicator. Determine the weight distribution, such as 40% for computing power, 30% for memory throughput, 20% for access latency, and 10% for data transfer. For each processor, apply the image rendering computing power model, input various data, and calculate the image rendering computing power value of each processor. Record the rendering computing power of each processor to ensure the integrity of the data. Verify the rationality of the calculation results and ensure that the computing power calculation results can reflect the actual situation by comparing with the actual rendering performance. Make necessary optimizations based on the verification results to ensure that the image rendering computing power of each processor is accurate and has reference value.

[0034] In this embodiment, step S4 includes the following steps: Step S41: Analyze the complexity of each of the multiple AR scene rendering tasks to obtain the rendering complexity; Step S42: Perform dynamic task priority analysis according to the rendering complexity, thereby generating resource priority for each regional scene; Step S43: pre-allocating and matching the image rendering computing power of each processor according to the rendering demand data of each task and the resource priority of each regional scene, so as to generate pre-allocation data of computing power resources for each task; Step S44: Make dynamic resource allocation decisions based on the computing power resource pre-allocation data of each task and build a dynamic computing power resource allocation strategy.

[0035] In this embodiment, the main factors affecting the rendering complexity are considered, including the number of objects in the scene, the number of polygons of each object, the lighting effect and the complexity of the texture. The complexity is expressed by a comprehensive index, and the formula is rendering complexity = (number of objects × average number of polygons) + (number of lighting × lighting complexity) + (number of textures × texture complexity), where each parameter should be clearly defined. For each AR scene rendering task, collect relevant parameters such as the number of objects in the scene, the number of polygons of each object, the type and number of light sources used, the type and size of textures, etc. This information is obtained through the scene description file or the API provided by the rendering engine. Input this data into the complexity model, calculate the rendering complexity of each task, and record the results in the database. Generate the priority of each task based on the rendering complexity. Using a linear weighted method, set a complexity threshold, and tasks with a complexity higher than a certain value have a higher priority than tasks below this value. Define a priority calculation formula, such as resource priority = rendering complexity / maximum complexity, to ensure that the priority is between 0 and 1. For each AR scene rendering task, priority calculation is performed based on the calculated rendering complexity, and classification is performed based on the preset threshold to generate the resource priority of each task. The resource priority of each task is recorded for subsequent resource allocation. The rationality of the priority calculation result is verified to ensure that the priority can effectively reflect the importance of the task. Comparative analysis is performed through actual rendering performance data (such as response time). Necessary adjustments are made based on the verification results to ensure the accuracy and effectiveness of dynamic task priority analysis. The rendering demand data of each task is combined with the resource priority to determine the computing power resources allocated to each processor. The weighted allocation method can be used to set the weights of rendering demand and priority. Define the pre-allocation formula, such as pre-allocated computing power = (rendering demand × resource priority) / (total number of tasks). According to the rendering demand data of each task and the corresponding resource priority, the image rendering computing power of the processor is matched one by one to calculate the computing power resource pre-allocation data of each task. The computing power resource pre-allocation of each task is recorded and the integrity of the data is ensured. The rationality of the pre-allocation result is verified to ensure that the allocated computing power resources can meet the rendering demand of each task. Comparative analysis is performed based on the actual rendering effect. Make necessary adjustments based on the verification results to ensure the accuracy and effectiveness of the computing resource pre-allocation data. Make decisions based on the computing resource pre-allocation data of each task and the actual rendering situation (such as real-time load, frame rate). Use feedback control theory to optimize resource allocation. Define allocation strategies, such as dynamically adjusting the computing resource allocation when the actual load of a task is higher than expected. Monitor the real-time performance data of each task (such as CPU / GPU usage, frame rate) and compare it with the preset threshold. Update the allocation strategy in real time to ensure the optimal resource utilization of the task.Based on the monitored data, dynamically adjust the computing resource allocation of each task to ensure that high-priority tasks obtain sufficient resources.

[0036] In this embodiment, step S5 includes the following steps: Step S51: performing dynamic interaction behavior analysis on the user real-time interaction data to generate user dynamic interaction behavior data; Step S52: performing real-time sight tracking on the user's real-time interaction data, and extracting the user's sight tracking trajectory; Step S53: locating the user's gaze area based on the user's sight tracking trajectory to obtain a user's gaze area scene; Step S54: performing scene interaction response analysis on the user gaze area scene according to the user dynamic interaction behavior data to generate scene interaction response features; Step S55: performing real-time scene interaction response changes according to the scene interaction response characteristics to generate a rendered image after the interaction response.

[0037] In this embodiment, the sensors of the AR device (such as cameras, gesture recognition devices, and touch screens) are used to collect real-time interaction data of users. These data include information such as user gestures, clicks, drags, and line of sight movements. Select appropriate sensors to ensure the accuracy and real-time nature of the data. Store the collected interaction data in an efficient data structure, such as a time series database, for subsequent analysis. Build a dynamic interaction behavior analysis model and use machine learning methods (such as support vector machines or decision trees) to classify and identify user interaction behaviors. The model should be able to recognize common interaction patterns, such as clicks, slides, zooms, etc. Define behavioral features, such as interaction frequency, duration, changes in interaction patterns, etc., as input to the model. Analyze user interaction data frame by frame, identify user interaction behaviors in real time, and record the recognition results. Use sliding window technology to process real-time data to ensure the fluency of analysis. Generate user dynamic interaction behavior data to form a structured data set containing the timestamp, type, and feature value of each interaction behavior. Select appropriate eye tracking technology, such as using an infrared camera or a depth camera, or using eye tracker technology to achieve high-precision eye tracking. Ensure that the device can provide sufficient resolution and update frequency. Determine the parameter settings for gaze tracking, such as sampling rate (e.g., 60Hz or higher) and tracking accuracy, to ensure data reliability. Capture the user's gaze data in real time as the user interacts with the system. Use computer vision techniques (e.g., facial feature detection and eye location) to identify the user's gaze direction. Extract the gaze tracking trajectory, convert the gaze data into a format that can be used for subsequent analysis, and record the gaze coordinates at each time point. Store the extracted gaze tracking trajectory in a data structure to ensure the time series and integrity of the data. Use arrays or lists to store the gaze coordinates for subsequent analysis. Record the starting point, end point, and path of each gaze trajectory to analyze the user's gaze behavior and attention distribution. Verify the accuracy of gaze tracking by comparing it with the user's actual gaze behavior. Conduct experiments to require users to look at specific targets and check the tracking results. Adjust the gaze tracking algorithm and parameters based on the verification results to optimize the gaze tracking accuracy. Based on the gaze tracking trajectory, set the definition criteria for the gaze area, such as a gaze time exceeding a certain threshold (e.g., 200 milliseconds) is a valid gaze area. Such a definition can effectively identify the content that the user is paying attention to. Set the size and shape parameters of the area to adapt to different scenarios, such as rectangular areas or circular areas. By analyzing the gaze tracking trajectory, extract the areas where the user has been looking for a long time. Determine the key areas where the user is looking by calculating the looking time of each coordinate point. Generate user looking area data, and record the coordinates, size, and looking time of each looking area. Store the extracted user looking areas in structured data to ensure fast access and analysis of the data in each area. Use a database management system (such as SQLite) or JSON format to store data.Record the relationship with objects in the scene, such as which objects are in the user's gaze area, for subsequent interactive response analysis. Verify the accuracy of gaze area extraction, observe the user's gaze behavior through user interviews or experiments, and ensure that the extracted area is consistent with the user's real focus. Adjust the gaze area definition and extraction algorithm based on the verification results to optimize the accuracy of gaze area positioning. Construct a scene interactive response analysis model to combine user dynamic interactive behavior data with the gaze area scene. Use statistical analysis or machine learning methods to identify the relationship between user behavior and interactive response. Determine response characteristics, such as response time, response type (such as visual feedback, sound effects, etc.), user emotional state, etc. For each user's gaze area, analyze its relationship with user interaction behavior. When the user looks at an object, will it generate clicks, drags, or other interactive behaviors? Record the interactive response characteristics of each gaze area, including the number of user interactions in the area, the average response time, etc. Store the interactive response feature data in a structured data set to ensure the accessibility and analyzability of the data. Use a database management system or CSV file for storage. Record the interactive response characteristics of each gaze area so that a rendered image with response changes can be generated later. Verify the validity of the interactive response features, and confirm whether the features reflect the real user behavior through comparative analysis of user feedback and actual data. Adjust the interactive response model based on the verification results, and optimize the feature extraction and analysis methods. Dynamically adjust the performance of the rendered image based on the characteristics of the user's interactive behavior and gaze area. When the user gazes at an object, enhance the feedback through highlighting, color change, or animation effects. Determine the parameters of the response change, such as the type of change (visual, sound, etc.), change intensity, and duration. When the user interacts, detect the user's gaze area and interactive behavior in real time, and generate a new rendered image based on the preset response change rules. Use a rendering engine (such as Unity or Unreal Engine) to achieve dynamic rendering. Ensure that the rendered image can be updated quickly when the user interacts to provide a smooth feedback experience. Store the generated interactive response rendered image data in the database, and record the rendered image corresponding to each interactive behavior for subsequent analysis and optimization. Record the parameters and effects of the rendered image for subsequent performance evaluation and adjustment.

[0038] In this embodiment, step S6 includes the following steps: Step S61: re-evaluating the rendering resource requirement of the rendered image after the interactive response to generate a real-time interactive response rendering resource requirement; Step S62: optimizing the dynamic computing power resource allocation strategy in real-time interactive response according to the real-time interactive response rendering resource demand, and generating an interactive optimization resource allocation strategy; Step S63: Calculate the global execution efficiency of the AR system based on the interactive optimization resource allocation strategy to generate a global execution efficiency value of the AR system; Step S64: fine-tune the operating parameters of the multi-core processor system according to the global execution efficiency value of the AR system, and build an intelligent AR performance optimization model.

[0039] In this embodiment, specific indicators of resource requirements are determined, such as GPU memory requirements, computing power requirements (expressed in GFLOPS), bandwidth requirements, etc. After user interaction, relevant parameters of the rendered image are collected, and real-time rendering performance data is obtained by recording the rendering time, memory usage, and processor load of each frame. Performance analysis tools (such as NVIDIA Nsight or AMD Radeon GPU Profiler) are used to monitor the usage of real-time rendering resources to ensure the accuracy of the data. The collected rendering parameters are input into the rendering resource requirement model to calculate the real-time interactive response rendering resource requirements. The formula is resource requirement = (resolution × scene complexity × lighting effect) / (frame rate × rendering time). Real-time interactive response rendering resource requirement data is generated, and the resource requirements of each scene are recorded for subsequent optimization. Optimization parameters, including processor allocation priority, memory usage, GPU load, etc., are determined to formulate a reasonable resource allocation strategy, monitor the computing power resource status of the current system, and obtain the processor and GPU in real time. and memory usage, use system monitoring tools (such as Grafana and Prometheus) to obtain real-time data, dynamically adjust resource allocation strategies based on real-time interactive response rendering resource requirements, use adaptive algorithms (such as genetic algorithms or particle swarm optimization) to optimize resource allocation, ensure that high-priority tasks obtain sufficient resources, generate interactive optimization resource allocation strategies, record the resource allocation of each task, ensure that the strategy can respond to user interaction in real time, implement optimization strategies in the system, dynamically update resource allocation to meet real-time rendering requirements, determine the calculation formula for global efficiency, such as global execution efficiency = (rendering frame rate × processor utilization) / (average response time + resource requirements), collect system performance data after the implementation of real-time interactive optimization resource allocation strategies, including rendering frame rate, processor and GPU load, use monitoring tools to collect detailed data, record the response time and resource usage of each scene, so as to evaluate the global efficiency, input the collected data into the global efficiency calculation model, and generate AR The global execution efficiency value of the system ensures that the data in the calculation process is accurate and complete, records the trend of changes in the global execution efficiency for subsequent analysis, and determines the optimization strategy, including dynamically adjusting the core frequency, optimizing task scheduling, balancing the load, etc., to achieve performance improvement. Monitor the operating status of the multi-core processor system, obtain the core load and resource usage in real time, use performance analysis tools (such as Intel VTune, AMD uProf) for monitoring, and fine-tune the operating parameters of the multi-core processor according to the global execution efficiency value, dynamically adjust the clock frequency of the high-load core, or reallocate tasks to the core with lower load. Apply the intelligent AR performance optimization model to adjust the operating parameters of the multi-core processor in real time to improve system performance.Monitor the performance changes of the optimized system, record the frame rate increase, response time reduction and other indicators to evaluate the optimization effect, verify the effectiveness of the intelligent AR performance optimization model through continuous monitoring and user feedback, ensure that fine-tuning parameters can effectively improve system performance, perform necessary iterative optimization based on the verification results, and continuously improve the performance optimization model to ensure the best performance of the system in different usage scenarios.

[0040] In this embodiment, an integrated AR device performance optimization device based on a multi-core processor is provided, which is used to execute the integrated AR device performance optimization method based on a multi-core processor as described above, including: The rendering task encapsulation module is used to obtain real-time AR display scene images and real-time user interaction data; perform scene object visual recognition on the real-time AR display scene images, and encapsulate image rendering data to generate multiple AR scene rendering tasks; The rendering requirement module is used to calculate the visual rendering computing power of multiple AR scene rendering tasks one by one, and identify the rendering requirements of different scenes to generate rendering requirement data for each task; The computing power perception module is used to identify the parameters of the multi-core processors in the system, and then perform image rendering computing power perception to generate the image rendering computing power of each processor; The dynamic resource allocation module is used to pre-allocate and match the image rendering computing power of each processor according to the rendering demand data of each task, make dynamic resource allocation decisions, and build a dynamic computing power resource allocation strategy; The user interaction response analysis module is used to analyze the dynamic interaction behavior of the user's real-time interaction data and the real-time scene interaction response changes to generate a rendered image after the interaction response; The performance optimization module is used to optimize the dynamic computing resource allocation strategy in real time based on the rendered image after the interactive response and fine-tune the operating parameters to build an intelligent AR performance optimization model.

[0041] The present invention collects and analyzes user interactions and environmental changes in real time, so that the system can accurately understand the current AR scene, improve the processing accuracy of scene information, and after generating multiple AR tasks, it can ensure that tasks in different scenes are processed in parallel, thereby providing task units and data basis for subsequent optimization, avoiding waste of global computing resources, ensuring that each rendering task is accurately evaluated for computing power, avoiding rendering delays or system overload caused by unreasonable resource allocation, providing data support for resource pre-allocation and scheduling in subsequent steps, ensuring that high-load tasks can obtain sufficient computing resources, improving the overall task execution efficiency, accurately perceiving the computing power of different cores, helping to more accurately match task requirements when making subsequent resource allocation decisions, avoiding overload operation of certain tasks, and avoiding resource bottlenecks by fully understanding the capabilities of the processor, ensuring that tasks can be efficiently executed on appropriate cores, and improving the stability and performance of the system. The dynamic resource allocation strategy can flexibly adjust resources according to the complexity and priority of the task to avoid overloading of a single core. Through reasonable pre-allocation of computing resources, each rendering task can get corresponding resource support, reduce the delay in task processing, and ensure the smoothness of the rendering process. Through dynamic analysis of interactive data, the system can respond quickly to user operations, enhance the immersive user experience, respond to user interactions in real time, and ensure that AR devices maintain efficient rendering updates and low latency when users interact with the virtual environment, avoiding poor experience caused by slow response. Real-time interactive response optimization and operating state parameter fine-tuning can enable AR devices to dynamically adjust according to the actual user operation, ensuring that the effect of each interaction can be responded to and optimized in a timely manner. Through intelligent performance optimization models, AR devices can automatically adjust and achieve continuous and efficient operation, improving the long-term stability of the system and user experience.

[0042] Therefore, the embodiments should be regarded as illustrative and non-restrictive from all points, and the scope of the present invention is limited by the appended claims rather than the above description, and it is therefore intended that all changes falling within the meaning and range of equivalent elements of the application documents are included in the present invention.

[0043] The above is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but should conform to the widest scope consistent with the principles and novel features invented herein.

Claims

1. A method for optimizing the performance of an integrated AR device based on a multi-core processor, characterized in that: The following steps are involved: Step S1: obtaining a real-time AR display scene image and real-time user interaction data; performing scene object visual recognition on the real-time AR display scene image, and performing image rendering data encapsulation to generate multiple AR scene rendering tasks; Step S2: Calculate the visual rendering computing power of each of the multiple AR scene rendering tasks, and identify the rendering requirements of different scenes to generate rendering requirement data for each task; Step S3: Identify the multi-core processor parameters in the system, and then perform image rendering computing power perception to generate the image rendering computing power of each processor; Step S4: pre-allocate and match the image rendering computing power of each processor according to the rendering demand data of each task, make dynamic resource allocation decisions, and build a dynamic computing power resource allocation strategy; Step S5: Perform dynamic interaction behavior analysis on the user's real-time interaction data and the real-time scene interaction response changes to generate a rendered image after the interaction response; Step S6: According to the rendered image after the interactive response, the dynamic computing resource allocation strategy is optimized for real-time interactive response and the operating parameters are fine-tuned to build an intelligent AR performance optimization model.

2. The integrated AR device performance optimization method based on a multi-core processor according to claim 1, characterized in that: The specific steps of step S1 are: Step S11: Acquire real-time AR display scene images and real-time user interaction data based on the AR device; Step S12: performing visual recognition of scene objects on the real-time AR display scene image, and marking each scene object; Step S13: annotating the interactive elements of the real-time AR display scene image to obtain all the interactive elements in the scene; Step S14: Calculating the scene object density based on each scene object and all interactive elements in the scene, thereby generating a scene density value; Step S15: dividing the real-time AR display scene image into multiple position areas according to the scene density value, thereby obtaining multiple AR scene areas; Step S16: encapsulate image rendering data for multiple AR scene areas to generate multiple AR scene rendering tasks.

3. The integrated AR device performance optimization method based on a multi-core processor according to claim 1, characterized in that: The specific steps of step S2 are: Step S21: Calculate the visual rendering computing power of each of the multiple AR scene rendering tasks to generate the scene image rendering computing power of each task; Step S22: Calculate the number of GPU resources for multiple AR scene rendering tasks to generate the number of GPU resources for each task; Step S23: performing processor memory capacity analysis on multiple AR scene rendering tasks to obtain memory capacity data for each task; Step S24: quantifying bandwidth resources for multiple AR scene rendering tasks to obtain a bandwidth resource value for each task; Step S25: Identify different scene rendering requirements for the scene image rendering computing power of each task, the number of GPU resources of each task, the memory capacity data of each task, and the bandwidth resource value of each task to generate rendering requirement data for each task.

4. The integrated AR device performance optimization method based on a multi-core processor according to claim 3 is characterized in that: The specific steps of step S21 are: Perform scene object calculations on multiple AR scene rendering tasks to obtain object data of the AR scene; Perform spatial distribution analysis on the object data of the AR scene to generate spatial distribution data of the scene objects; Perform scene rendering texture mining on multiple AR scene rendering tasks to extract the rendering texture features of each scene; Identify lighting effects for multiple AR scene rendering tasks; Define the target frame rate for rendering the scene; Based on the target frame rate of the rendered scene, the spatial distribution data of the scene objects, the rendering texture characteristics of each scene and the lighting effect are estimated for the number of floating-point operations per frame, thereby generating the number of floating-point operations per frame; The task visual rendering computing power is calculated based on the number of floating-point operations in each frame to generate the scene image rendering computing power of each task.

5. The integrated AR device performance optimization method based on a multi-core processor according to claim 1, characterized in that: The specific steps of step S3 are: Step S31: Identify multi-core processor parameters in the system, wherein the multi-core processor parameters include the number of CPU / GPU cores, the computing power of each core, the clock frequency of the processor, the cache capacity, the cache capacity and bandwidth, the bus bandwidth and the data transmission capacity; Step S32: quantify the total computing power of the processor according to the number of CPU / GPU cores and the computing power of each core to obtain the computing power value of each processor; Step S33: Calculate the maximum memory throughput based on the cache capacity and bandwidth to generate the maximum memory throughput of each processor; Step S34: performing real-time memory access delay analysis based on the processor's clock frequency and cache capacity, and extracting real-time memory access delay data; Step S35: Perform maximum data transmission analysis according to bus bandwidth and data transmission capacity, and generate a maximum data transmission value for each processor; Step S36: image rendering computing power perception is performed on the computing power value of each processor, the maximum memory throughput of each processor, the real-time memory access delay data and the maximum data transmission value of each processor to generate the image rendering computing power of each processor.

6. The integrated AR device performance optimization method based on a multi-core processor according to claim 1, characterized in that: The specific steps of step S4 are: Step S41: Analyze the complexity of each of the multiple AR scene rendering tasks to obtain the rendering complexity; Step S42: Perform dynamic task priority analysis according to the rendering complexity, thereby generating resource priority for each regional scene; Step S43: pre-allocating and matching the image rendering computing power of each processor according to the rendering demand data of each task and the resource priority of each regional scene, so as to generate pre-allocation data of computing power resources for each task; Step S44: Make dynamic resource allocation decisions based on the computing power resource pre-allocation data of each task and build a dynamic computing power resource allocation strategy.

7. The integrated AR device performance optimization method based on a multi-core processor according to claim 1, characterized in that: The specific steps of step S5 are: Step S51: performing dynamic interaction behavior analysis on the user real-time interaction data to generate user dynamic interaction behavior data; Step S52: performing real-time sight tracking on the user's real-time interaction data, and extracting the user's sight tracking trajectory; Step S53: locating the user's gaze area based on the user's sight tracking trajectory to obtain a user's gaze area scene; Step S54: performing scene interaction response analysis on the user gaze area scene according to the user dynamic interaction behavior data to generate scene interaction response features; Step S55: performing real-time scene interaction response changes according to the scene interaction response characteristics to generate a rendered image after the interaction response.

8. The integrated AR device performance optimization method based on a multi-core processor according to claim 1, characterized in that: The specific steps of step S6 are: Step S61: re-evaluating the rendering resource requirement of the rendered image after the interactive response to generate a real-time interactive response rendering resource requirement; Step S62: optimizing the dynamic computing power resource allocation strategy in real-time interactive response according to the real-time interactive response rendering resource demand, and generating an interactive optimization resource allocation strategy; Step S63: Calculate the global execution efficiency of the AR system based on the interactive optimization resource allocation strategy to generate a global execution efficiency value of the AR system; Step S64: fine-tune the operating parameters of the multi-core processor system according to the global execution efficiency value of the AR system, and build an intelligent AR performance optimization model.

9. An integrated AR device performance optimization device based on a multi-core processor, characterized in that: The method for optimizing the performance of an integrated AR device based on a multi-core processor according to claim 1 comprises: The rendering task encapsulation module is used to obtain real-time AR display scene images and real-time user interaction data; perform scene object visual recognition on the real-time AR display scene images, and encapsulate image rendering data to generate multiple AR scene rendering tasks; The rendering requirement module is used to calculate the visual rendering computing power of multiple AR scene rendering tasks one by one, and identify the rendering requirements of different scenes to generate rendering requirement data for each task; The computing power perception module is used to identify the parameters of the multi-core processors in the system, and then perform image rendering computing power perception to generate the image rendering computing power of each processor; The dynamic resource allocation module is used to pre-allocate and match the image rendering computing power of each processor according to the rendering demand data of each task, make dynamic resource allocation decisions, and build a dynamic computing power resource allocation strategy; The user interaction response analysis module is used to analyze the dynamic interaction behavior of the user's real-time interaction data and the real-time scene interaction response changes to generate a rendered image after the interaction response; The performance optimization module is used to optimize the dynamic computing resource allocation strategy in real time based on the rendered image after the interactive response and fine-tune the operating parameters to build an intelligent AR performance optimization model.

Images