A MR virtual image visualization method and system based on AI intelligence

Abstract

The present invention discloses an AI-based intelligent MR virtual screen visualization method and system, which relates to the field of data visualization technology. First, a collection time point is preset, and the MR device is used to collect scene environment data and update the user position to render virtual objects to the user screen. At the same time, the user view data is extracted to adjust the object rendering effect in the user view, and the performance of the MR device is monitored to evaluate the energy consumption and manage it. Finally, the virtual screen is adjusted according to the device performance and the rendering compliance index. By optimizing the rendering effect of the MR virtual screen in real time, it is ensured that the user obtains a high-quality visual experience. At the same time, by intelligently monitoring and managing the energy consumption of the MR device, the energy efficiency ratio of the device is effectively improved, the use time is extended, and a smoother and clearer picture is provided for the user.

Description

Technical Field

[0001] The present invention relates to the technical field of data visualization, and in particular to an AI-based MR virtual image visualization method and system. Background Art

[0002] Virtual reality visualization combines computer vision, 3D modeling and artificial intelligence technologies to create an immersive interactive experience. Through image recognition, depth perception and real-time rendering, the MR system can understand and map the real environment, allowing virtual objects to seamlessly blend with the real world.

[0003] For example, the invention patent with announcement number CN110362209B announces an MR mixed reality intelligent perception interaction system. The system includes perception layer components, central control management layer components and application layer components. The perception layer components include physical perception layer components and virtual perception layer components. The three-dimensional spatial relationship is constructed by inputting the feature data of the perception interaction entity collected in real time and the constructed virtual scene into the 3D game engine, and combining it with the real physical environment to generate a mixed reality three-dimensional interaction relationship and realize the visual display of real-time interaction. The five elements of people, position, object, time and dimension are freely arranged and combined according to any preset digital environment and real-world environment to create a new virtual-reality combined interactive space environment for immersive real-time interaction and precise fault-tolerant error correction mechanism.

[0004] For example, the invention patent with announcement number CN107391793B announces a method for demolishing a building structure based on 3D scanning technology and MR mixed reality technology, the steps of which are as follows: set up scanning points according to the characteristics of the existing building structure, use a 3D laser scanner to scan the building, splice the data information of different scanning points, print the building structure model, and reversely establish the BIM model of the component; convert the model format and import it into the 3D3S force analysis software for force analysis to form the internal force envelope diagram and combined displacement diagram corresponding to the building structure; combine the internal force envelope diagram and combined displacement diagram to form a building structure demolition plan in the BIM model; use mixed reality technology to superimpose the building structure demolition plan formed in the BIM model on the building entity for simulation.

[0005] Based on the above findings, the existing technical solutions have the problem of high virtual image update delay, which may cause unrealistic visual effects and lack of timely feedback in the interaction between users and virtual objects, thereby reducing the sense of immersion and the smoothness of interaction, resulting in a decline in user experience. Summary of the invention

[0006] In view of the deficiencies in the prior art, the present invention provides an AI-based intelligent MR virtual image visualization method and system, which solves the problems designed in the above-mentioned background technology.

[0007] To achieve the above objectives, the present invention is implemented through the following technical solutions: an AI-based MR virtual image visualization method, including presetting each acquisition time point, collecting scene environment data through an MR device at each acquisition time point, obtaining the scene environment data at each acquisition time point, updating the user position, and rendering the virtual object into the user view.

[0008] The scene environment data at each acquisition time point in the user view is extracted to obtain the scene dynamic rendering compliance index of the user view at each acquisition time point, and the object rendering effect in the user view is adjusted according to the scene dynamic rendering compliance index of the user view at each acquisition time point.

[0009] The performance data of each hardware of the MR device is monitored and collected at each collection time point, and the hardware performance evaluation value of the MR device at each collection time point is analyzed to evaluate the energy consumption status of the MR device and manage the energy consumption of the MR device.

[0010] According to the hardware performance evaluation value of the MR device, the scene dynamic rendering compliance index of the user view, and combined with the user interface feature data of the MR device at each acquisition time point, the picture quality evaluation value of the user interface at each acquisition time point is obtained, and the user view is adjusted according to the picture quality evaluation value of the user interface at each acquisition time point.

[0011] Furthermore, the scene environment data at each acquisition time point includes: light intensity, main light source position, light color temperature, camera position, and camera focal length at each acquisition time point.

[0012] Furthermore, the camera position and the main light source position at each acquisition time point are extracted, and the distance between the main light source and the camera is obtained by processing. The scene dynamic rendering compliance index of the user view at each acquisition time point is obtained by combining the light intensity, light color temperature, and camera focal length. The scene dynamic rendering compliance index of the user view at each acquisition time point is used to evaluate the rendering change requirements of the user view.

[0013] Furthermore, adjusting the rendering effect of the object in the user view is specifically analyzed as follows: the adjusting the rendering effect of the object in the user view includes improving the resolution of the user view, the shadow and reflection of the object in the user view, and maintaining the rendering effect of the object in the user view.

[0014] The scene dynamic rendering compliance index of the user view at each acquisition time point is extracted and compared with the scene dynamic rendering compliance threshold of the user view stored in the database. If the scene dynamic rendering compliance index of the user view at a certain acquisition time point is lower than the scene dynamic rendering compliance threshold of the user view stored in the database, the resolution of the user view at the acquisition time point, the shadow clarity and reflection of the objects in the user view are improved.

[0015] If the scene dynamic rendering compliance index of the user view at each collection time point is higher than or equal to the scene dynamic rendering compliance threshold of the user view stored in the database, the rendering effect of the object in the user view is maintained, and the collection scene environment data is continuously monitored.

[0016] Furthermore, the hardware performance evaluation value of the MR device at each acquisition time point is obtained. The specific analysis is as follows: the performance data of each hardware of the MR device at each acquisition time point is extracted, including the remaining battery power, the temperature of each hardware, and the cumulative CPU power consumption and GPU power consumption of the MR device at each adjacent acquisition time point are monitored and counted, and the hardware performance evaluation value of the MR device at each acquisition time point is obtained by processing. The hardware performance evaluation value of the MR device at each acquisition time point is used to evaluate the energy consumption status of the MR device.

[0017] Furthermore, the energy consumption of the MR device is managed. Specifically, the hardware performance evaluation value of the MR device at each acquisition time point is compared with the hardware performance evaluation threshold of the MR device stored in the database. When the hardware performance evaluation value of the MR device at a certain acquisition time point is higher than or equal to the hardware performance evaluation threshold of the MR device stored in the database, the data transmission volume of the MR device at that acquisition time point is maintained.

[0018] When the hardware performance evaluation value of the MR device at a certain acquisition time point is lower than the hardware performance evaluation threshold of the MR device stored in the database, the data transmission volume of the MR device at the acquisition time point is reduced, the processor frequency is reduced, and the hardware in the standby state is turned off.

[0019] Furthermore, the picture quality evaluation value of the user interface at each acquisition time point is obtained, and the specific analysis is as follows: the hardware performance evaluation value of the MR device at each acquisition time point and the scene dynamic rendering compliance index of the user view are extracted, and the user interface feature data of the MR device at each acquisition time point, including the resolution and pixel density of the user view, are extracted, and the picture quality evaluation value of the user interface at each acquisition time point is obtained by processing. The picture quality evaluation value of the user interface at each acquisition time point is used to comprehensively evaluate the quality of the MR device and the user view.

[0020] Furthermore, the picture quality evaluation value of the user interface at each acquisition time point is specifically analyzed as follows:

[0021] ;

[0022] In the formula, represents the picture quality evaluation value of the user interface at the i-th acquisition time point, represents the hardware performance evaluation value of the MR device at the i-th acquisition time point, Indicates the scene dynamic rendering compliance index of the user view at the i-th acquisition time point, represents the resolution of the user view at the i-th acquisition time point, represents the pixel density of the user view at the i-th acquisition time point, Indicates the reference value of the resolution of the set user view. Indicates the reference value of the pixel density of the set user view. represents the weight factor of the hardware performance evaluation value of the set MR device, Indicates the weight factor of the scene dynamic rendering index of the set user view, Indicates the weight factor of the resolution of the set user view. Represents the weight factor of the pixel density of the set user view, , i represents the number of the acquisition time point, and m represents the total number of acquisition time points.

[0023] Furthermore, the user view is adjusted according to the picture quality evaluation value of the user interface at each acquisition time point. The specific analysis is: the picture quality evaluation value of the user interface at each acquisition time point is extracted and matched with the picture quality level corresponding to each interval of the picture quality evaluation value of the user interface in the database to obtain the picture quality level at each acquisition time point. The picture quality level includes full HD, HD, and standard definition. The picture is adjusted according to the picture quality level at each acquisition time point.

[0024] When the picture quality level is full HD and HD, the frame rate, refresh rate and blurriness of the picture are kept unchanged. When the picture quality level is standard definition, the frame rate and refresh rate of the virtual picture are increased and the blurriness is reduced.

[0025] Furthermore, an AI-based MR virtual image visualization system is characterized by comprising:

[0026] The data acquisition module is used to preset each acquisition time point, collect scene environment data through the MR device at each acquisition time point, obtain the scene environment data at each acquisition time point, update the user position, and render the virtual object into the user view.

[0027] The rendering adjustment module is used to extract the scene environment data at each acquisition time point in the user view, obtain the scene dynamic rendering compliance index of the user view at each acquisition time point, and adjust the object rendering effect in the user view according to the scene dynamic rendering compliance index of the user view at each acquisition time point.

[0028] The energy consumption management module is used to monitor and collect the performance data of each hardware of the MR device at each collection time point, analyze and obtain the hardware performance evaluation value of the MR device at each collection time point, evaluate the energy consumption status of the MR device, and manage the energy consumption of the MR device.

[0029] The comprehensive analysis module is used to obtain the picture quality evaluation value of the user interface at each acquisition time point based on the hardware performance evaluation value of the MR device, the scene dynamic rendering compliance index of the user view, and the user interface feature data of the MR device at each acquisition time point, and adjust the user view according to the picture quality evaluation value of the user interface at each acquisition time point.

[0030] The present invention has the following beneficial effects:

[0031] (1) The present invention provides a preset collection time point, uses the MR device to collect scene environment data and update the user position to render virtual objects to the user screen, and extracts user view data to adjust the rendering effect of objects in the user view. The performance of the MR device is monitored to evaluate and manage energy consumption. Finally, the virtual screen is adjusted according to the device performance and the rendering compliance index. By optimizing the rendering effect of the MR virtual screen in real time, it ensures that the user obtains a high-quality visual experience. At the same time, by intelligently monitoring and managing the energy consumption of the MR device, the energy efficiency ratio of the device is effectively improved, the usage time is extended, and a smoother and clearer picture is provided to the user.

[0032] (2) The present invention obtains the scene dynamic rendering compliance index of the user view by collecting scene environment data and processing it, and evaluates and adjusts the visual effect according to the scene dynamic rendering compliance index of the user view, which helps the user to obtain a clear view effect under different lighting conditions and improves the realism and immersion of the picture;

[0033] (3) The present invention obtains a hardware performance evaluation value of the MR device by real-time monitoring of the hardware performance data of the MR device, and compares the hardware performance evaluation value of the MR device with the hardware performance evaluation threshold of the MR device stored in the database, and adjusts the data transmission volume, processor frequency and hardware status according to the comparison result, which helps to optimize energy consumption management and improve the stability of the device;

[0034] (4) The present invention combines the hardware performance evaluation value and the scene dynamic rendering compliance index to comprehensively evaluate the picture quality, and adjusts the picture according to the evaluation result to ensure the optimal performance of the virtual picture. By comprehensively considering the hardware performance evaluation value of the MR device and the scene dynamic rendering compliance index of the user view, it is helpful for the MR device to accurately allocate resources and maintain the stability and consistency of the picture effect;

[0035] Of course, any product implementing the present invention does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 is a flow chart of the method of the present invention;

[0037] Figure 2The hardware performance evaluation value of the MR device of the present invention is a curve of the image quality evaluation value of the user interface;

[0038] Figure 3 It is a flow chart of the system of the present invention. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0040] In the description of the present invention, it should be understood that the terms "opening", "upper", "lower", "thickness", "top", "middle", "length", "inside", "all around" and the like indicating orientation or positional relationship are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the components or elements referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the present invention.

[0041] See also Figure 1 The embodiment of the present invention provides a technical solution: an AI-based intelligent MR virtual screen visualization method, including presetting each acquisition time point, acquiring scene environment data through an MR device at each acquisition time point, obtaining the scene environment data at each acquisition time point, updating the user position, and rendering the virtual object into the user view.

[0042] The scene environment data at each acquisition time point in the user view is extracted to obtain the scene dynamic rendering compliance index of the user view at each acquisition time point, and the object rendering effect in the user view is adjusted according to the scene dynamic rendering compliance index of the user view at each acquisition time point.

[0043] The performance data of each hardware of the MR device is monitored and collected at each collection time point, and the hardware performance evaluation value of the MR device at each collection time point is analyzed to evaluate the energy consumption status of the MR device and manage the energy consumption of the MR device.

[0044] According to the hardware performance evaluation value of the MR device, the scene dynamic rendering compliance index of the user view, and combined with the user interface feature data of the MR device at each acquisition time point, the picture quality evaluation value of the user interface at each acquisition time point is obtained, and the user view is adjusted according to the picture quality evaluation value of the user interface at each acquisition time point.

[0045] It should be noted that the camera of the MR device is used to capture images of the environment, AI is used to identify objects in the scene, and intelligently identify scene requirements to generate corresponding virtual objects. The lighting, shadow, reflection and other effects of the objects are simulated through graphics rendering technology, and the virtual objects are rendered into the user's view.

[0046] It should be noted that the collection time point is preset, and the MR device is used to collect scene environment data and update the user position to render virtual objects to the user screen. At the same time, the user view data is extracted to adjust the rendering effect of objects in the user view, and the performance of the MR device is monitored to evaluate and manage energy consumption. Finally, the virtual screen is adjusted according to the device performance and the rendering compliance index. By optimizing the rendering effect of the MR virtual screen in real time, it is ensured that the user has a high-quality visual experience. At the same time, through intelligent monitoring and management of the energy consumption of MR devices, the energy efficiency ratio of the equipment is effectively improved, the usage time is extended, and a smoother and clearer picture is provided to the user.

[0047] Specifically, the scene environment data at each acquisition time point includes: light intensity, main light source position, light color temperature, camera position, and camera focal length at each acquisition time point.

[0048] Specifically, the camera position and the main light source position at each acquisition time point are extracted, and the distance between the main light source and the camera is obtained by processing. The scene dynamic rendering compliance index of the user view at each acquisition time point is obtained by combining the light intensity, light color temperature, and camera focal length. The scene dynamic rendering compliance index of the user view at each acquisition time point is used to evaluate the rendering change requirements of the user view.

[0049] It should be noted that the light intensity of the environment can be measured by using a photosensitive sensor; the environment image can be captured by a camera, the image is gray-processed to obtain a gray-scale image, the area with the highest brightness in the image is extracted, the main light source and the position of the main light source of the scene are determined, and expressed in three-dimensional coordinates; a color thermometer is used to measure the color temperature of the light; the camera position of the MR device is obtained through the positioning system equipped with the MR system, and expressed in three-dimensional coordinates; the focal length of the camera mounted on the MR device is extracted.

[0050] It should be noted that the three-dimensional coordinates of the main light source position and the camera position are extracted and processed to obtain the distance between the main light source and the camera.

[0051] It should be noted that the specific analysis process of the scene dynamic rendering compliance index of the user view at each collection time point is as follows:

[0052] ;

[0053] In the formula, Indicates the scene dynamic rendering compliance index of the user view at the i-th acquisition time point, Indicates the distance between the main light source and the camera at the i-th acquisition time point, represents the light intensity at the i-th acquisition time point, represents the light color temperature at the i-th acquisition time point, represents the focal length of the camera at the i-th acquisition time point, Indicates the distance between the main light source and the camera at the i-1th acquisition time point, represents the light intensity at the i-1th acquisition time point, represents the light color temperature at the i-1th acquisition time point, Indicates the allowed value of the distance change between the main light source and the camera. Indicates the allowed value of the set light intensity change. Indicates the allowed value of the set light color temperature change. Indicates the reference value of the camera focal length. Indicates the correction factor corresponding to the distance between the set main light source and the camera. Indicates the corresponding correction factor of the set light intensity, Indicates the corresponding correction factor of the set light color temperature, Indicates the corresponding correction factor of the set camera focal length, , i represents the number of each collection time point, and m represents the total number of collection time points.

[0054] It should be noted that there is a correlation between the distance between the main light source and the camera, the light intensity, the light color temperature, and the camera focal length. The greater the distance between the main light source and the camera, the weaker the light intensity; the longer the focal length, the greater the light intensity captured by the camera; the light color temperature affects the spectral distribution of the light source, thereby affecting the lighting, shadows, and reflections of the rendered model.

[0055] It is the correction factor of the distance between the main light source and the camera set in the database, which indicates the degree of correction of the dynamic rendering compliance index of the scene in the user view by the distance between the main light source and the camera. When used, the correction factor of the distance between the main light source and the camera can be directly obtained from the database. The corresponding relationship can be a pre-set mapping relationship. For example, the distance between the main light source and the camera and the correction factor of the distance between the main light source and the camera set in the database form a mapping set, and the real-time distance between the main light source and the camera is input into the mapping set to obtain the correction factor of the distance between the main light source and the camera. The mapping relationship can be a one-to-one correspondence or a many-to-one relationship. In this example, its value range is [0,1].

[0056] It is the correction factor of light intensity set in the database, which indicates the correction degree of light intensity to the dynamic rendering index of the scene in the user view. When used, the correction factor of light intensity can be directly obtained from the database. Its corresponding relationship can be a pre-set mapping relationship. For example, the light intensity and the correction factor of light intensity set in the database form a mapping set, and the real-time light intensity is input into the mapping set to obtain the correction factor of light intensity. The mapping relationship can be a one-to-one correspondence or a many-to-one relationship. In this example, its value range is [0,1].

[0057] It is the correction factor of the light color temperature set in the database, which indicates the correction degree of the light color temperature to the dynamic rendering index of the scene in the user view. When used, the correction factor of the light color temperature can be directly obtained from the database, and its corresponding relationship can be a pre-set mapping relationship. For example, the light color temperature and the correction factor of the light color temperature set in the database form a mapping set, and the real-time light color temperature is input into the mapping set to obtain the correction factor of the light color temperature. The mapping relationship can be a one-to-one correspondence or a many-to-one relationship. In this example, its value range is [0,1].

[0058] It is the correction factor of the camera focal length set in the database, which indicates the correction degree of the dynamic rendering index of the scene in the user view by the camera focal length. When used, the correction factor of the camera focal length can be directly obtained from the database. The corresponding relationship can be a pre-set mapping relationship. For example, the camera focal length and the correction factor of the camera focal length set in the database form a mapping set, and the real-time camera focal length is input into the mapping set to obtain the correction factor of the camera focal length. The mapping relationship can be a one-to-one correspondence or a many-to-one relationship. In this example, its value range is [0,1].

[0059] It should be noted that the scene dynamic rendering compliance index of the user view at each acquisition time point is determined by the distance between the main light source and the camera, the light intensity, the light color temperature and the focal length of the camera.

[0060] Specifically, adjusting the rendering effect of the object in the user view includes improving the resolution of the user view, the shadow and reflection of the object in the user view, and maintaining the rendering effect of the object in the user view.

[0061] Specifically, the scene dynamic rendering compliance index of the user view at each acquisition time point is extracted and compared with the scene dynamic rendering compliance threshold of the user view stored in the database. If the scene dynamic rendering compliance index of the user view at a certain acquisition time point is lower than the scene dynamic rendering compliance threshold of the user view stored in the database, the resolution of the user view at the acquisition time point, and the shadow clarity and reflection of objects in the user view are improved.

[0062] Specifically, if the scene dynamic rendering compliance index of the user view at each acquisition time point is higher than or equal to the scene dynamic rendering compliance threshold of the user view stored in the database, the rendering effect of the object in the user view is maintained, and the scene environment data is continuously monitored and acquired.

[0063] It should be noted that the resolution of the user view at the acquisition time point, the shadow clarity and reflection of the object in the user view are improved. The specific process includes extracting the current resolution of the user view, the shadow clarity and reflection of the object in the user view, and doubling the shadow clarity and reflection of the object in the user view; comparing the resolution of the current user view with the target user view resolution stored in the database, if the resolution of the user view is lower than the target user view resolution, then improving the resolution of the user view to the target user view resolution, if the resolution of the user view is higher than or equal to the target user view resolution, there is no need to adjust the resolution of the user view.

[0064] It should be noted that if the scene dynamic rendering compliance index of the user view at each acquisition time point is higher than or equal to the scene dynamic rendering compliance threshold of the user view stored in the database, then maintaining the rendering effect of the object in the user view helps to ensure the quality of the user view and avoid the degradation of the rendering effect of the object in the user view due to environmental changes.

[0065] Specifically, the performance data of each hardware of the MR device at each acquisition time point is extracted, including the remaining battery power, the temperature of each hardware, and the cumulative CPU power consumption and GPU power consumption of the MR device at each adjacent acquisition time point are monitored and counted, and the hardware performance evaluation value of the MR device at each acquisition time point is obtained by processing. The hardware performance evaluation value of the MR device at each acquisition time point is used to evaluate the energy consumption status of the MR device.

[0066] It should be noted that power consumption viewing software (such as CPU-Z and GPU-Z) is used to obtain CPU power consumption and GPU power consumption; a temperature sensor is used to collect the temperature of each hardware of the MR device at each collection time point; and a battery management system is used to read the remaining battery power at each collection time point.

[0067] It should be noted that the interval between adjacent collection time points is recorded as a monitoring time period, for example: the interval between the first collection time point and the second collection time point is the first monitoring time period, and the interval between the second collection time point and the third collection time point is the second monitoring time period.

[0068] It should be noted that the specific analysis process of the hardware performance evaluation value of the MR device at each acquisition time point is as follows:

[0069] ;

[0070] In the formula, represents the hardware performance evaluation value of the MR device at the i-th acquisition time point, Indicates the cumulative CPU power consumption of the MR device in the qth monitoring time period, represents the cumulative GPU power consumption of the MR device in q monitoring time periods, represents the temperature of the hardware of the jth MR device at the i-th acquisition time point, represents the remaining battery power of the MR device at the i-th acquisition time point, Indicates the reference value of the MR device's CPU cumulative power consumption. Indicates the reference value of the GPU cumulative power consumption of the MR device. Indicates the temperature reference value of the set hardware. Indicates setting the battery power reference value. Indicates the correction factor corresponding to the set CPU power consumption. Indicates the correction factor corresponding to the set GPU power consumption, Indicates the correction factor corresponding to the set hardware temperature. Indicates the correction factor corresponding to the set remaining battery power, e is a natural constant, , i represents the number of each collection time point, m represents the total number of collection time points, , j represents the number of each hardware, n represents the total number of hardware, , q represents the number of each monitoring time period, and r represents the total number of monitoring time periods.

[0071] It should be noted that there is a correlation between the temperature of each hardware of the MR device, the remaining battery power, the CPU power consumption, and the GPU power consumption. When the CPU power consumption and the GPU power consumption increase, the heat generated by each hardware will also increase accordingly; the reduction of battery power will reduce the CPU and GPU power consumption to extend the usage time.

[0072] The correction factor corresponding to the CPU power consumption set in the database represents the degree of correction of the hardware performance evaluation value of the MR device by the CPU power consumption. When used, the correction factor corresponding to the CPU power consumption can be directly obtained from the database. The corresponding relationship can be a pre-set mapping relationship. For example, the CPU power consumption and the correction factor corresponding to the CPU power consumption set in the database form a mapping set. The real-time CPU power consumption is input into the mapping set to obtain the correction factor corresponding to the CPU power consumption. The mapping relationship can be a one-to-one correspondence or a many-to-one relationship. In this example, its value range is [0,1].

[0073] The correction factor corresponding to the GPU power consumption set in the database represents the degree of correction of the hardware performance evaluation value of the MR device by the GPU power consumption. When used, the correction factor corresponding to the GPU power consumption can be directly obtained from the database. The corresponding relationship can be a pre-set mapping relationship. For example, the GPU power consumption and the correction factor corresponding to the GPU power consumption set in the database form a mapping set. The real-time GPU power consumption is input into the mapping set to obtain the correction factor corresponding to the GPU power consumption. The mapping relationship can be a one-to-one correspondence or a many-to-one relationship. In this example, its value range is [0,1].

[0074] The correction factor corresponding to the temperature of the hardware set in the database represents the degree of correction of the hardware temperature to the hardware performance evaluation value of the MR device. When used, the correction factor corresponding to the temperature of the hardware can be directly obtained from the database. The corresponding relationship can be a pre-set mapping relationship. For example, the temperature of the hardware and the correction factor corresponding to the temperature of the hardware set in the database form a mapping set. The real-time hardware temperature is input into the mapping set to obtain the correction factor corresponding to the temperature of the hardware. The mapping relationship can be a one-to-one correspondence or a many-to-one relationship. In this example, its value range is [0,1].

[0075] The correction factor corresponding to the remaining battery power set in the database represents the degree of correction of the hardware performance evaluation value of the MR device by the remaining battery power. The correction factor corresponding to the remaining battery power can be directly obtained from the database when used. The corresponding relationship can be a pre-set mapping relationship. For example, the remaining battery power and the correction factor corresponding to the remaining battery power set in the database form a mapping set. The real-time remaining battery power is input into the mapping set to obtain the correction factor corresponding to the remaining battery power. The mapping relationship can be a one-to-one correspondence or a many-to-one relationship. In this example, its value range is [0,1].

[0076] It should be noted that the hardware performance evaluation value of the MR device at each acquisition time point is obtained by processing the temperature of each hardware of the MR device, the remaining battery power, the CPU power consumption, and the GPU power consumption.

[0077] Specifically, the hardware performance evaluation value of the MR device at each acquisition time point is compared with the hardware performance evaluation threshold of the MR device stored in the database. When the hardware performance evaluation value of the MR device at a certain acquisition time point is higher than or equal to the hardware performance evaluation threshold of the MR device stored in the database, the data transmission volume of the MR device at the acquisition time point is maintained.

[0078] Specifically, when the hardware performance evaluation value of the MR device at a certain acquisition time point is lower than the hardware performance evaluation threshold of the MR device stored in the database, the data transmission amount of the MR device at the acquisition time point is reduced, the processor frequency is reduced, and the hardware in the standby state is turned off.

[0079] It should be noted that maintaining the data transmission volume and monitoring the hardware data of the MR device can timely monitor the hardware performance of the MR device and ensure the operating status of the device.

[0080] It should be noted that the data transmission volume of the MR device at the acquisition time point is reduced, the processor frequency is reduced and the hardware in the standby state is turned off. The specific process includes: when it is detected that the hardware performance evaluation value of the MR device at a certain acquisition time point is lower than the hardware performance evaluation threshold of the MR device stored in the database, the preset acquisition time point is reduced by half, so as to reduce the frequency of data update, thereby reducing the data transmission volume; by gradually reducing the resolution of the user view to reduce the computing demand of the MR device and thus reduce the processor frequency, for example: when it is detected that the hardware performance evaluation value of the MR device at a certain acquisition time point is lower than the hardware performance evaluation threshold of the MR device stored in the database, the resolution of the user view is gradually reduced by 10%, 20%, and 30%, and each time it is reduced, the hardware performance evaluation value of the MR device is reprocessed to obtain the hardware performance evaluation value of the MR device until the hardware performance evaluation value of the MR device is higher than or equal to the hardware performance evaluation threshold of the MR device stored in the database; through the operation instruction, it is checked whether the standby state is enabled for each hardware of the MR device, and the hardware in the standby state is turned off.

[0081] Specifically, the hardware performance evaluation value of the MR device at each acquisition time point and the scene dynamic rendering compliance index of the user view are extracted, and the user interface feature data of the MR device at each acquisition time point is extracted, including the resolution and pixel density of the user view, and the picture quality evaluation value of the user interface at each acquisition time point is obtained by processing. The picture quality evaluation value of the user interface at each acquisition time point is used to comprehensively evaluate the quality of the MR device and the user view.

[0082] Specifically, the picture quality evaluation value of the user interface at each acquisition time point, the specific analysis process is as follows:

[0083] ;

[0084] In the formula, represents the picture quality evaluation value of the user interface at the i-th acquisition time point, represents the hardware performance evaluation value of the MR device at the i-th acquisition time point, Indicates the scene dynamic rendering compliance index of the user view at the i-th acquisition time point, represents the resolution of the user view at the i-th acquisition time point, represents the pixel density of the user view at the i-th acquisition time point, Indicates the reference value of the resolution of the set user view. Indicates the reference value of the pixel density of the set user view. represents the weight factor of the hardware performance evaluation value of the set MR device, Indicates the weight factor of the scene dynamic rendering index of the set user view, Indicates the weight factor of the resolution of the set user view. Represents the weight factor of the pixel density of the set user view, , i represents the number of each collection time point, and m represents the total number of collection time points.

[0085] The weight factor of the hardware performance evaluation value of the MR device set in the database represents the value of the hardware performance evaluation value of the MR device to the picture quality evaluation value of the user interface. When used, the weight factor of the hardware performance evaluation value of the MR device can be directly obtained from the database. The corresponding relationship can be a pre-set mapping relationship. For example, the hardware performance evaluation value of the MR device and the weight factor of the hardware performance evaluation value of the MR device set in the database form a mapping set. The real-time hardware performance evaluation value of the MR device is input into the mapping set to obtain the weight factor of the hardware performance evaluation value of the MR device. The mapping relationship can be a one-to-one correspondence or a many-to-one relationship. In this example, its value range is [0,1].

[0086] The weight factor of the scene dynamic rendering compliance index of the user view set in the database represents the weight of the scene dynamic rendering compliance index of the user view to the picture quality evaluation value of the user interface. When used, the weight factor of the scene dynamic rendering compliance index of the user view can be directly obtained from the database. The corresponding relationship can be a pre-set mapping relationship. For example, the scene dynamic rendering compliance index of the user view and the weight factor of the scene dynamic rendering compliance index of the user view set in the database form a mapping set, and the real-time scene dynamic rendering compliance index of the user view is input into the mapping set to obtain the weight factor of the scene dynamic rendering compliance index of the user view. The mapping relationship can be a one-to-one correspondence or a many-to-one relationship. In this example, its value range is [0,1].

[0087] The weight factor of the user view resolution set in the database represents the numerical value of the weight of the user view resolution to the user view resolution. When used, the weight factor of the user view resolution can be directly obtained from the database. The corresponding relationship can be a pre-set mapping relationship. For example, the user view resolution and the weight factor of the user view resolution set in the database form a mapping set. The real-time user view resolution is input into the mapping set to obtain the weight factor of the user view resolution. The mapping relationship can be a one-to-one correspondence or a many-to-one relationship. In this example, its value range is [0,1].

[0088] The weight factor of the pixel density of the user view set in the database represents the numerical value of the weight degree of the pixel density of the user view to the pixel density of the user view. When used, the weight factor of the pixel density of the user view can be directly obtained from the database. The corresponding relationship can be a pre-set mapping relationship. For example, the pixel density of the user view and the weight factor of the pixel density of the user view set in the database form a mapping set, and the real-time pixel density of the user view is input into the mapping set to obtain the weight factor of the pixel density of the user view. The mapping relationship can be a one-to-one correspondence or a many-to-one relationship. In this example, its value range is [0,1].

[0089] It should be noted that the hardware performance evaluation value of the MR device at each acquisition time point is consistent with the scene dynamic rendering index of the user view, and the picture quality evaluation value of the user interface at each acquisition time point is obtained by comprehensive analysis. The hardware performance of the MR device and the scene dynamic rendering effect of the user view can be comprehensively considered, so as to more comprehensively and accurately evaluate the picture quality and adjust the picture.

[0090] like Figure 2 As shown, Figure 2 represents the picture quality evaluation value of the user interface, where the horizontal axis represents the scene dynamic rendering compliance index of the user view at the i-th acquisition time point, and the vertical axis represents the picture quality evaluation value of the user interface at the i-th acquisition time point. Three different sets of example parameters are defined in the figure, corresponding to different situations of the three curves, represented by solid lines, dashed lines and dotted lines, respectively, and the corresponding curve labels are a, b, and c, respectively. It is assumed that the weight factor corresponding to the hardware performance evaluation value of the MR device =0.3, the weight factor corresponding to the dynamic rendering of the scene in the user view corresponds to the exponent =0.8, weight factor of the user view resolution =0.55, weight factor for the pixel density of the user's view =0.76, the resolution of the user view =720, the pixel density of the user's view =450, set the reference value of the user view resolution =1080, the reference value of the pixel density of the user view =600, when the scene dynamic rendering compliance index of the user view at the i-th acquisition time point is 1.2, the schematic diagram of the picture quality evaluation value of the user interface at the i-th acquisition time point is shown as curve a, when the scene dynamic rendering compliance index of the user view at the i-th acquisition time point is 2.6, the schematic diagram of the picture quality evaluation value of the user interface at the i-th acquisition time point is shown as curve b, and when the scene dynamic rendering compliance index of the user view at the i-th acquisition time point is 3.4, the schematic diagram of the picture quality evaluation value of the user interface at the i-th acquisition time point is shown as curve c.

[0091] As shown in Table 1, Table 1 is example data of the picture quality evaluation value of the user interface, which lists the picture quality evaluation value of the user interface and the scene dynamic rendering compliance index of the user view.

[0092] Table 1 Example data of the picture quality evaluation value of the user interface:

[0093] ;

[0094] As shown in Table 1, in a specific embodiment, it is assumed that the weight factor corresponding to the hardware performance evaluation value of the MR device is =0.3, the weight factor corresponding to the dynamic rendering of the scene in the user view corresponds to the exponent =0.8, weight factor of the user view resolution =0.55, weight factor for the pixel density of the user's view =0.76, the resolution of the user view =720, the pixel density of the user's view =450, set the reference value of the user view resolution =1080, the reference value of the pixel density of the user view =600.

[0095] Specifically, the picture quality assessment value of the user interface of each acquisition time point is extracted and matched with the picture quality level corresponding to each interval of the picture quality assessment value of the user interface in the database to obtain the picture quality level of each acquisition time point, and the picture quality level includes full HD, HD, and standard definition, and the picture is adjusted according to the picture quality level of each acquisition time point.

[0096] Specifically, when the picture quality level is full HD and HD, the frame rate, refresh rate and blurriness of the picture are kept unchanged; when the picture quality level is standard definition, the frame rate and refresh rate of the virtual picture are increased and the blurriness is reduced.

[0097] It should be noted that when the picture quality level is full HD and HD, keeping the frame rate, refresh rate, dynamic range and blurriness of the picture unchanged will help maintain visual continuity and avoid picture blur; when the picture quality level is standard definition, the rendering calculation of objects at the edge of the virtual picture is reduced, thereby increasing the frame rate and refresh rate, sharpening the image and reducing blurriness.

[0098] Specifically, in this embodiment, the present invention provides an AI-based intelligent MR virtual screen visualization system, including: a data acquisition module, used to preset each acquisition time point, collect scene environment data through an MR device at each acquisition time point, obtain the scene environment data at each acquisition time point, and update the user position, and render the virtual object into the user view.

[0099] The rendering adjustment module is used to extract the scene environment data at each acquisition time point in the user view, obtain the scene dynamic rendering compliance index of the user view at each acquisition time point, and adjust the object rendering effect in the user view according to the scene dynamic rendering compliance index of the user view at each acquisition time point.

[0100] The energy consumption management module is used to monitor and collect the performance data of each hardware of the MR device at each collection time point, analyze and obtain the hardware performance evaluation value of the MR device at each collection time point, evaluate the energy consumption status of the MR device, and manage the energy consumption of the MR device.

[0101] The comprehensive analysis module is used to obtain the picture quality evaluation value of the user interface at each acquisition time point based on the hardware performance evaluation value of the MR device, the scene dynamic rendering compliance index of the user view, and the user interface feature data of the MR device at each acquisition time point, and adjust the user view according to the picture quality evaluation value of the user interface at each acquisition time point.

[0102] It should be noted that by pre-setting the collection time point and using the MR device to accurately capture the scene environment data, the user position is updated in real time and the rendering effect of the virtual object is optimized. The object rendering effect is flexibly adjusted according to the dynamic rendering compliance index of the user's view to ensure that the picture is smooth and realistic. By comprehensively monitoring and managing the energy consumption of MR devices, it helps to improve energy efficiency and extend the use time of the equipment, thereby providing users with a higher quality, smoother and clearer MR virtual picture experience.

[0103] It should be noted that an AI-based MR virtual image visualization method also includes a database, which is an important tool for storing and managing scene environment data and hardware performance data. In this embodiment, the database is used to store the set allowable value of light intensity change, the set allowable value of light color temperature change, the set reference value of the camera focal length, the set correction factor corresponding to the distance between the main light source and the camera, the set corresponding correction factor of the light intensity, the set corresponding correction factor of the light color temperature, the set corresponding correction factor of the camera focal length, the scene dynamic rendering compliance threshold of the user view, the set hardware temperature reference value, the set battery power reference value, the set correction factor corresponding to the CPU power consumption, and the set GPU Correction factor corresponding to power consumption, correction factor corresponding to the set hardware temperature, correction factor corresponding to the set remaining battery power, hardware performance evaluation threshold of the MR device, reference value of the resolution of the set user view, reference value of the pixel density of the set user view, weight factor of the hardware performance evaluation value of the set MR device, weight factor of the scene dynamic rendering compliance index of the set user view, weight factor of the resolution of the set user view, weight factor of the pixel density of the set user view, picture quality level, set allowable value of change in the distance between the main light source and the camera, target user view resolution, reference value of the cumulative power consumption of the CPU of the MR device, reference value of the cumulative power consumption of the GPU of the MR device.

[0104] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0105] The preferred embodiments of the present invention disclosed above are only used to help illustrate the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can understand and use the present invention well. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. A MR virtual image visualization method based on AI intelligence, characterized in that: include: Preset each collection time point, collect scene environment data through the MR device at each collection time point, obtain the scene environment data at each collection time point, update the user position, and render the virtual object into the user view; Extracting scene environment data at each acquisition time point in the user view, obtaining a scene dynamic rendering compliance index of the user view at each acquisition time point, and adjusting the object rendering effect in the user view according to the scene dynamic rendering compliance index of the user view at each acquisition time point; Monitor and collect the performance data of each hardware of the MR device at each collection time point, analyze and obtain the hardware performance evaluation value of the MR device at each collection time point, evaluate the energy consumption status of the MR device, and manage the energy consumption of the MR device; According to the hardware performance evaluation value of the MR device, the scene dynamic rendering compliance index of the user view, and the user interface feature data of the MR device at each acquisition time point, the picture quality evaluation value of the user interface at each acquisition time point is obtained, and the user view is adjusted according to the picture quality evaluation value of the user interface at each acquisition time point; The scene dynamic rendering compliance index of the user view at each acquisition time point is obtained, and the specific analysis is as follows: The camera position and the main light source position at each acquisition time point are extracted, and the distance between the main light source and the camera is obtained by processing. The scene dynamic rendering compliance index of the user view at each acquisition time point is obtained by combining the light intensity, light color temperature, and camera focal length. The scene dynamic rendering compliance index of the user view at each acquisition time point is used to evaluate the rendering change requirements of the user view; The user view is adjusted according to the picture quality evaluation value of the user interface at each acquisition time point, and the specific analysis is as follows: Extracting the picture quality evaluation value of the user interface of each acquisition time point and matching it with the picture quality level corresponding to each interval of the picture quality evaluation value of the user interface in the database, to obtain the picture quality level at each acquisition time point, wherein the picture quality level includes full high definition, high definition, and standard definition, and adjusting the picture according to the picture quality level at each acquisition time point; When the picture quality level is full HD and HD, the frame rate, refresh rate and blurriness of the picture are kept unchanged. When the picture quality level is standard definition, the frame rate and refresh rate of the virtual picture are increased and the blurriness is reduced.

2. The AI-based MR virtual image visualization method according to claim 1, characterized in that: The scene environment data at each acquisition time point includes: light intensity, main light source position, light color temperature, camera position, and camera focal length at each acquisition time point.

3. The AI-based MR virtual image visualization method according to claim 1, characterized in that: The adjustment of the rendering effect of objects in the user view is specifically analyzed as follows: The adjusting the rendering effect of the object in the user view includes improving the resolution of the user view, the shadow and reflection of the object in the user view, and maintaining the rendering effect of the object in the user view; Extracting the scene dynamic rendering compliance index of the user view at each acquisition time point, and comparing it with the scene dynamic rendering compliance threshold of the user view stored in the database; if the scene dynamic rendering compliance index of the user view at a certain acquisition time point is lower than the scene dynamic rendering compliance threshold of the user view stored in the database, improving the resolution of the user view at the acquisition time point, and the shadow clarity and reflection of the object in the user view; If the scene dynamic rendering compliance index of the user view at each collection time point is higher than or equal to the scene dynamic rendering compliance threshold of the user view stored in the database, the rendering effect of the object in the user view is maintained, and the collection scene environment data is continuously monitored.

4. The AI-based MR virtual image visualization method according to claim 1, characterized in that: The hardware performance evaluation value of the MR device at each acquisition time point is obtained, and the specific analysis is as follows: The performance data of each hardware of the MR device at each acquisition time point is extracted, including the remaining battery power, the temperature of each hardware, and the cumulative power consumption of the CPU and the cumulative power consumption of the GPU of the MR device at each adjacent acquisition time point is monitored and counted, and the hardware performance evaluation value of the MR device at each acquisition time point is obtained by processing. The hardware performance evaluation value of the MR device at each acquisition time point is used to evaluate the energy consumption status of the MR device.

5. The AI-based MR virtual image visualization method according to claim 4, characterized in that: The energy consumption management of the MR device is specifically analyzed as follows: According to the hardware performance evaluation value of the MR device at each acquisition time point, the hardware performance evaluation threshold of the MR device stored in the database is compared. When the hardware performance evaluation value of the MR device at a certain acquisition time point is higher than or equal to the hardware performance evaluation threshold of the MR device stored in the database, the data transmission volume of the MR device at the acquisition time point is maintained; When the hardware performance evaluation value of the MR device at a certain acquisition time point is lower than the hardware performance evaluation threshold of the MR device stored in the database, the data transmission volume of the MR device at the acquisition time point is reduced, the processor frequency is reduced, and the hardware in the standby state is turned off.

6. The AI-based MR virtual image visualization method according to claim 1, characterized in that: The picture quality evaluation value of the user interface at each acquisition time point is obtained, and the specific analysis is as follows: The hardware performance evaluation value of the MR device at each acquisition time point and the scene dynamic rendering compliance index of the user view are extracted, and the user interface feature data of the MR device at each acquisition time point are extracted, including the resolution and pixel density of the user view, and the picture quality evaluation value of the user interface at each acquisition time point is obtained by processing. The picture quality evaluation value of the user interface at each acquisition time point is used to comprehensively evaluate the quality of the MR device and the user view.

7. The AI-based MR virtual image visualization method according to claim 6, characterized in that: The picture quality evaluation value of the user interface at each acquisition time point, the specific analysis process is as follows: ； In the formula, represents the picture quality evaluation value of the user interface at the i-th acquisition time point, represents the hardware performance evaluation value of the MR device at the i-th acquisition time point, Indicates the scene dynamic rendering compliance index of the user view at the i-th acquisition time point, represents the resolution of the user view at the i-th acquisition time point, represents the pixel density of the user view at the i-th acquisition time point, Indicates the reference value of the resolution of the set user view. Indicates the reference value of the pixel density of the set user view. represents the weight factor of the hardware performance evaluation value of the set MR device, Indicates the weight factor of the scene dynamic rendering index of the set user view, Indicates the weight factor of the resolution of the set user view. Indicates the weight factor of the pixel density of the set user view , i represents the number of the acquisition time point, and m represents the total number of acquisition time points.

8. A system using the MR virtual image visualization method based on AI intelligence as described in any one of claims 1 to 7, characterized in that: include: The data collection module is used to preset each collection time point, collect scene environment data through the MR device at each collection time point, obtain the scene environment data at each collection time point, update the user position, and render the virtual object into the user view; A rendering adjustment module is used to extract scene environment data at each acquisition time point in the user view, obtain the scene dynamic rendering compliance index of the user view at each acquisition time point, and adjust the rendering effect of the object in the user view according to the scene dynamic rendering compliance index of the user view at each acquisition time point; The energy consumption management module is used to monitor and collect the performance data of each hardware of the MR device at each collection time point, analyze and obtain the hardware performance evaluation value of the MR device at each collection time point, evaluate the energy consumption status of the MR device, and manage the energy consumption of the MR device; The comprehensive analysis module is used to obtain the picture quality evaluation value of the user interface at each acquisition time point based on the hardware performance evaluation value of the MR device, the scene dynamic rendering compliance index of the user view, and the user interface feature data of the MR device at each acquisition time point, and adjust the user view according to the picture quality evaluation value of the user interface at each acquisition time point.

Images