Panoramic field rendering method, device and equipment of intelligent glasses and storage medium

Abstract

The invention provides a panorama rendering method and device for intelligent glasses, computer equipment and a readable storage medium, and the method comprises the steps: determining a rendering mode according to the current parameters of a gyroscope and scene information; if the rendering mode is static rendering, the target panorama and the target scene are rendered according to view cone matching parameters between the intelligent glasses and the target scene, and the rendered target panorama and the rendered target scene are projected to a view field of the intelligent glasses for display according to a first projection conversion function; if the rendering mode is glasses change rendering, determining a new view cone matching parameter between the intelligent glasses and the target scene according to the current parameter of the gyroscope, and executing the step of static rendering according to the parameter and a second projection conversion function; and if the rendering mode is scene change rendering, determining a new view cone matching parameter according to the transformed view cone model, and executing the steps during static rendering according to the parameter. The production efficiency and immersion of the panorama can be improved.

Description

Technical Field

[0001] This application relates to the field of smart glasses, specifically to a panoramic image rendering method, apparatus, device, and storage medium for smart glasses. Background Technology

[0002] Smart glasses overlay virtual images onto real-world scenes, allowing users to experience a blend of virtual and real worlds. Smart glasses typically include displays, computer vision technology, sensors, and wireless connectivity. They can be used in various applications such as gaming, education, healthcare, and manufacturing. Their main advantages include a more immersive user experience and increased efficiency, while also helping users better understand and master complex information and skills.

[0003] Panoramic images, through wide-angle techniques and formats such as painting, photography, video, and 3D models, depict as much of the surrounding environment as possible. 360-degree panoramas, on the other hand, are created by capturing images of an entire scene with a professional camera or rendering images using modeling software. These images are then stitched together and played back using a dedicated player. Essentially, a flat photograph or computer-modeled image is transformed into a 360-degree panoramic view for virtual reality browsing, simulating a realistic three-dimensional space for the viewer.

[0004] Currently, smart glasses primarily support large-screen video projection and display of fixed scenes and content. However, they cannot currently support the rendering and display of panoramic images. Therefore, how to achieve panoramic image rendering and display on smart glasses is a problem that urgently needs to be solved. Summary of the Invention

[0005] The purpose of this application is to provide a panoramic image rendering method, apparatus, device, and storage medium for smart glasses, addressing the technical problems in the prior art mentioned above, so as to solve the problem that smart glasses in the prior art cannot support the actual need for panoramic image rendering and display.

[0006] To achieve the above objectives, the technical solutions adopted in the embodiments of this application are as follows:

[0007] In a first aspect, one embodiment of this application provides a panoramic scene rendering method for smart glasses, the method comprising:

[0008] Based on the current parameters of the gyroscope in the smart glasses and the scene information to be displayed by the smart glasses, the current rendering method is determined. The rendering method includes: static rendering, glasses-changing rendering, or scene-changing rendering.

[0009] If the rendering method is static rendering, the target panoramic image and the target scene are rendered according to the frustum matching parameters between the smart glasses and the target scene of the target panoramic image to be rendered, and the rendered target panoramic image and the rendered target scene are projected onto the field of view of the smart glasses for display according to the first projection transformation function from the target scene to the smart glasses;

[0010] If the rendering method is glasses-changing rendering, then a new frustum matching parameter between the smart glasses and the target scene is determined according to the current parameters of the gyroscope of the smart glasses, and the steps of the static rendering are executed according to the new frustum matching parameter and the second projection transformation function from the smart glasses to the target scene;

[0011] If the rendering method is scene-changing rendering, then the new frustum matching parameters between the smart glasses and the target scene are determined according to the frustum model of the transformed target scene, and the steps of the static rendering are performed according to the new frustum matching parameters.

[0012] As an optional implementation, before rendering the target panoramic image and the target scene based on the frustum matching parameters between the smart glasses and the target scene of the target panoramic image to be rendered, the method further includes:

[0013] Obtain the view frustum model of the smart glasses and the view frustum model of the target scene;

[0014] The view frustum model of the smart glasses and the view frustum model of the target scene are matched to obtain the view frustum matching parameters between the smart glasses and the target scene.

[0015] As an optional implementation, before projecting the rendered target panoramic image and the rendered target scene onto the field of view of the smart glasses for display according to the first projection transformation function from the target scene to the smart glasses, the method further includes:

[0016] Obtain the projection parameters of the target scene and the projection parameters of the smart glasses;

[0017] Based on the projection parameters of the target scene and the projection parameters of the smart glasses, a first projection transformation function from the target scene to the smart glasses is constructed, and a second projection transformation function from the smart glasses to the target scene is constructed.

[0018] As an optional implementation, determining new frustum matching parameters between the smart glasses and the target scene based on the current parameters of the smart glasses' gyroscope includes:

[0019] Based on the current parameters of the gyroscope of the smart glasses, determine the variable cone momentum of the smart glasses and the variable cone momentum of the target scene;

[0020] Based on the changing cone momentum of the smart glasses and the changing cone momentum of the target scene, the cone model of the smart glasses and the cone model of the target scene are re-acquired.

[0021] The view frustum model of the smart glasses and the view frustum model of the target scene are re-matched to obtain new view frustum matching parameters between the smart glasses and the target scene.

[0022] As an optional implementation, the step of performing the static rendering based on the new view frustum matching parameters and the second projection transformation function from the smart glasses to the target scene includes:

[0023] Based on the second projection transformation function, obtain the first projection transformation function corresponding to the second projection transformation function;

[0024] Render the target panorama and the target scene based on the new frustum matching parameters;

[0025] According to the first projection conversion function, the rendered target panoramic image and the rendered target scene are projected onto the field of view of the smart glasses for display.

[0026] As an optional implementation, determining the new frustum matching parameters between the smart glasses and the target scene based on the transformed frustum model of the target scene includes:

[0027] Obtain the transformed view frustum model of the target scene;

[0028] The view frustum model of the smart glasses and the transformed view frustum model of the target scene are matched to obtain new view frustum matching parameters between the smart glasses and the target scene.

[0029] As an optional implementation, the step of performing the static rendering based on the new view frustum matching parameters includes:

[0030] Render the target panorama and the target scene based on the new frustum matching parameters;

[0031] According to the first projection conversion function, the rendered target panoramic image and the rendered target scene are projected onto the field of view of the smart glasses for display.

[0032] Secondly, embodiments of this application provide a panoramic image rendering device for smart glasses, applied to smart glasses, the device comprising:

[0033] The determination module is used to determine the current rendering method based on the current parameters of the gyroscope in the smart glasses and the scene information to be displayed by the smart glasses. The rendering method includes: static rendering, glasses-movement rendering, or scene-movement rendering.

[0034] The first rendering module is used to render the target panoramic image and the target scene according to the frustum matching parameters between the smart glasses and the target scene of the target panoramic image to be rendered when the rendering mode is static rendering, and to project the rendered target panoramic image and the rendered target scene onto the field of view of the smart glasses for display according to the first projection transformation function from the target scene to the smart glasses.

[0035] The second rendering module is used to determine new frustum matching parameters between the smart glasses and the target scene based on the current parameters of the gyroscope of the smart glasses when the rendering method is glasses-changing rendering, and to perform the steps of the static rendering based on the new frustum matching parameters and the second projection transformation function from the smart glasses to the target scene.

[0036] The third rendering module is used to determine new frustum matching parameters between the smart glasses and the target scene based on the frustum model of the transformed target scene when the rendering mode is scene change rendering, and to perform the steps of the static rendering based on the new frustum matching parameters.

[0037] As one possible implementation, the determining module is further configured to:

[0038] Obtain the view frustum model of the smart glasses and the view frustum model of the target scene;

[0039] The view frustum model of the smart glasses and the view frustum model of the target scene are matched to obtain the view frustum matching parameters between the smart glasses and the target scene.

[0040] As an optional implementation, the determining module is further configured to:

[0041] Obtain the projection parameters of the target scene and the projection parameters of the smart glasses;

[0042] Based on the projection parameters of the target scene and the projection parameters of the smart glasses, a first projection transformation function from the target scene to the smart glasses is constructed, and a second projection transformation function from the smart glasses to the target scene is constructed.

[0043] As an optional implementation, the second rendering module is specifically used for:

[0044] Based on the current parameters of the gyroscope of the smart glasses, determine the variable cone momentum of the smart glasses and the variable cone momentum of the target scene;

[0045] Based on the changing cone momentum of the smart glasses and the changing cone momentum of the target scene, the cone model of the smart glasses and the cone model of the target scene are re-acquired.

[0046] The view frustum model of the smart glasses and the view frustum model of the target scene are re-matched to obtain new view frustum matching parameters between the smart glasses and the target scene.

[0047] As an optional implementation, the second rendering module is specifically used for:

[0048] Based on the second projection transformation function, obtain the first projection transformation function corresponding to the second projection transformation function;

[0049] Render the target panorama and the target scene based on the new frustum matching parameters;

[0050] According to the first projection conversion function, the rendered target panoramic image and the rendered target scene are projected onto the field of view of the smart glasses for display.

[0051] As an optional implementation, the third rendering module is specifically used for:

[0052] Obtain the transformed view frustum model of the target scene;

[0053] The view frustum model of the smart glasses and the transformed view frustum model of the target scene are matched to obtain new view frustum matching parameters between the smart glasses and the target scene.

[0054] As an optional implementation, the third rendering module is specifically used for:

[0055] Render the target panorama and the target scene based on the new frustum matching parameters;

[0056] According to the first projection conversion function, the rendered target panoramic image and the rendered target scene are projected onto the field of view of the smart glasses for display.

[0057] Thirdly, embodiments of this application provide a computer device, including: a processor, a storage medium, and a bus. The storage medium stores machine-readable instructions executable by the processor. When the computer device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the steps of the panoramic rendering method for smart glasses as described in the first aspect above.

[0058] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the panoramic rendering method for smart glasses as described in the first aspect above.

[0059] The beneficial effects of this application are:

[0060] This application provides a panoramic rendering method, apparatus, device, and storage medium for smart glasses. The method includes: determining the current rendering mode based on the current parameters of the gyroscope in the smart glasses and the scene information to be displayed by the smart glasses; the rendering mode includes: static rendering, glasses-modified rendering, or scene-modified rendering; if the rendering mode is static rendering, then rendering the target panoramic image and the target scene based on the frustum matching parameters between the smart glasses and the target scene of the target panoramic image to be rendered, and projecting the rendered target panoramic image and the rendered target scene onto the field of view of the smart glasses for display according to a first projection transformation function from the target scene to the smart glasses; if the rendering mode is glasses-modified rendering, then determining a new frustum matching parameter between the smart glasses and the target scene based on the current parameters of the gyroscope in the smart glasses, and performing the steps of static rendering based on the new frustum matching parameter and a second projection transformation function from the smart glasses to the target scene; if the rendering mode is scene-modified rendering, then determining a new frustum matching parameter between the smart glasses and the target scene based on the frustum model of the transformed target scene, and performing the steps of static rendering based on the new frustum matching parameter. This method improves the efficiency of panoramic scene production and the immersive experience of panoramic scenes in smart glasses. Attached Figure Description

[0061] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0062] Figure 1 A flowchart illustrating the panoramic rendering method for smart glasses provided in this application embodiment;

[0063] Figure 2 A schematic diagram illustrating the process of determining the frustum matching parameters in the panoramic rendering method for smart glasses provided in this application embodiment;

[0064] Figure 3 A schematic diagram illustrating the construction process of the first projection transformation function for the panoramic rendering method of smart glasses provided in this application embodiment;

[0065] Figure 4 A schematic diagram illustrating the process of determining new cone matching parameters based on the current parameters of the gyroscope of the smart glasses in the panoramic rendering method provided in this application embodiment;

[0066] Figure 5 A schematic diagram illustrating the process of static rendering based on new view frustum matching parameters and a second projection transformation function in the panoramic rendering method for smart glasses provided in this application embodiment;

[0067] Figure 6 A schematic diagram illustrating the process of determining new frustum matching parameters based on the frustum model of the transformed target scene in the panoramic rendering method for smart glasses provided in this application embodiment;

[0068] Figure 7 A schematic diagram illustrating the process of performing static rendering based on new view frustum matching parameters for the panoramic rendering method of smart glasses provided in this application embodiment;

[0069] Figure 8 A module structure diagram of the panoramic rendering device for smart glasses provided in this application embodiment;

[0070] Figure 9 This is a schematic diagram of the computer device structure provided in an embodiment of this application. Detailed Implementation

[0071] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the accompanying drawings in this application are for illustrative and descriptive purposes only and are not intended to limit the scope of protection of this application. Furthermore, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of this application. It should be understood that the operations in the flowcharts may not be implemented in sequence, and steps without logical contextual relationships may be reversed or implemented simultaneously. In addition, those skilled in the art, guided by the content of this application, may add one or more other operations to the flowcharts, or remove one or more operations from the flowcharts.

[0072] Furthermore, the described embodiments are merely some, not all, of the embodiments of this application. The components of the embodiments of this application described and illustrated herein can typically be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0073] It should be noted that the term "comprising" will be used in the embodiments of this application to indicate the presence of the features declared thereafter, but does not exclude the addition of other features.

[0074] In existing technologies, smart glasses mainly support large-screen video projection and display of fixed scenes and content, but cannot support the rendering and display of panoramic images.

[0075] Based on the above-mentioned problems, this application proposes a panoramic image rendering method for smart glasses. By determining the rendering mode of the panoramic image in the smart glasses, and using the corresponding steps for different rendering modes, the panoramic image can be rendered and displayed in the smart glasses.

[0076] Figure 1 This is a flowchart illustrating a panoramic image rendering method for smart glasses provided in an embodiment of this application. The method is applied to smart glasses, which may include a display screen, a computer vision center, sensors, and a wireless connection module, etc. Figure 1 As shown, the method includes:

[0077] S101. Based on the current parameters of the gyroscope in the smart glasses and the scene information to be displayed by the smart glasses, determine the current rendering method, which includes: static rendering, glasses-movement rendering, or scene-movement rendering.

[0078] Optionally, the current parameters of the gyroscope in the aforementioned smart glasses may include, for example, the gyroscope's momentum. The scene information to be displayed by the smart glasses is used to indicate changes in the scene, and this scene information can be determined based on the user's operation of the smart glasses or the image information currently being played by the smart glasses.

[0079] The change in momentum of the gyroscope can determine whether the gyroscope has changed. If the gyroscope has not changed and the scene of the smart glasses has not changed, then the current rendering method can be determined to be static rendering. If the gyroscope has changed and the scene of the smart glasses has not changed, then the current rendering method can be determined to be dynamic rendering. If the gyroscope has not changed and the scene of the smart glasses has changed, then the current rendering method can be determined to be scene rendering.

[0080] S102. If the above rendering method is static rendering, then according to the frustum matching parameters between the smart glasses and the target scene of the target panoramic image to be rendered, the target panoramic image and the target scene are rendered, and according to the first projection conversion function from the target scene to the smart glasses, the rendered target panoramic image and the rendered target scene are projected onto the field of view of the smart glasses for display.

[0081] Optionally, when the rendering mode is static rendering, the target panoramic image and target scene are first rendered based on the aforementioned view frustum matching parameters. After the target panoramic image and target scene are rendered, the target panoramic image and target scene are then projected onto the field of view of the smart glasses using the aforementioned first projection transformation function, so that the user can see the target panoramic image and target scene on the screen of the smart glasses.

[0082] Optionally, the aforementioned target panoramic image may be a panoramic image captured by another imaging device and sent to the smart glasses by that device. The target scene of the target panoramic image refers to the scene in which the target panoramic image was captured. The smart glasses and the target scene have corresponding frustum matching parameters, and the method for determining these frustum matching parameters will be explained in detail in the following embodiments.

[0083] In addition, the target scene and the smart glasses have corresponding projection transformation functions. In this step, the projection transformation function that projects from the target scene to the smart glasses is the first projection transformation function mentioned above. Using this projection transformation function, the rendered target panoramic image and the target scene can be projected onto the field of view of the smart glasses.

[0084] S103. If the above rendering method is glasses-changing rendering, then determine the new frustum matching parameters between the smart glasses and the target scene based on the current parameters of the smart glasses' gyroscope, and execute the steps of static rendering based on the new frustum matching parameters and the second projection transformation function from the smart glasses to the target scene.

[0085] If the rendering method is glasses-motion rendering, it indicates that the position of the smart glasses has changed, and therefore, the view frustum matching parameters between the smart glasses and the target scene will also change accordingly. Therefore, when the rendering method is glasses-motion rendering, the new view frustum parameters are first determined based on the current parameters of the gyroscope. Based on this, the steps for static rendering are performed using the new view frustum matching parameters and the second projection transformation function. The steps for static rendering are the same as the execution process in step S102 above.

[0086] S104. If the above rendering method is scene change rendering, then determine the new view frustum matching parameters between the smart glasses and the target scene based on the view frustum model of the transformed target scene, and perform the steps of static rendering based on the new view frustum matching parameters.

[0087] If the rendering method is scene-shifting rendering, it indicates that the target scene of the target panoramic image has changed. Therefore, the view frustum matching parameters between the smart glasses and the target scene will also change accordingly. Thus, when the rendering method is scene-shifting rendering, new view frustum matching parameters are first determined based on the view frustum model of the transformed target scene. Based on these new view frustum matching parameters, the steps for static rendering are performed. The steps for static rendering are the execution process described in step S102 above.

[0088] The view frustum model of the target scene can be a fisheye camera model, an orthographic camera model, a rectangular camera model, etc.

[0089] In this embodiment, the current rendering mode is first determined based on the gyroscope's current parameters and scene information. When the rendering mode is static rendering, the target panorama and target scene are rendered using the frustum matching parameters between the smart glasses and the target scene, and projected using a first projection transformation function. When the rendering mode is glasses-moving rendering, new frustum matching parameters are determined, and static rendering is performed based on the new frustum matching parameters and a second projection transformation function. When the rendering mode is scene-moving rendering, new frustum matching parameters are determined, and static rendering is performed based on the new frustum matching parameters. This achieves accurate rendering and display of the target panorama in the smart glasses under various rendering modes.

[0090] The following explains the process of determining the cone-matching parameters between the smart glasses and the target scene.

[0091] Figure 2 This is a schematic diagram illustrating the process of determining the view frustum matching parameters in the panoramic rendering method for smart glasses provided in this application embodiment. Figure 2 As shown, before step S102 above, the procedure further includes:

[0092] S201. Obtain the view frustum model of the smart glasses and the view frustum model of the target scene.

[0093] The view frustum model of smart glasses can be, for example, a fisheye camera model, an orthographic camera model, a rectangular camera model, etc.

[0094] Optionally, the smart glasses and the target scene can be modeled in advance to obtain the view frustum models of the smart glasses and the target scene, and these models can be saved. Then, in this step, the view frustum models of the smart glasses and the target scene can be retrieved.

[0095] S202. Perform matching processing on the view frustum model of the smart glasses and the view frustum model of the target scene to obtain the view frustum matching parameters between the smart glasses and the target scene.

[0096] Optionally, the difference between the view frustum model of the target scene and the view frustum model of the smart glasses can be determined based on the view frustum model of the smart glasses, and the view frustum matching parameters between the smart glasses and the target scene can be calculated based on the difference.

[0097] In this embodiment, by matching the view frustum model of the smart glasses with the view frustum model of the target scene, accurate view frustum matching parameters can be obtained, thereby ensuring the rendering effect when using the view frustum matching parameters for subsequent rendering.

[0098] The process of determining the first projection transformation function and the second projection transformation function of the target scene to the smart glasses is described below.

[0099] Figure 3 A schematic diagram illustrating the construction process of the first projection transformation function in the panoramic rendering method for smart glasses provided in this application embodiment is shown below. Figure 3 As shown, before step S102 above, the procedure further includes:

[0100] S301. Obtain the projection parameters of the target scene and the projection parameters of the smart glasses.

[0101] The projection parameters of the target scene can be obtained from the original viewport of the scene, and the projection parameters of the smart glasses can be obtained from the original viewport of the smart glasses.

[0102] Optionally, the original viewports of the smart glasses and the target scene can be tracked and captured in advance to obtain the projection parameters of the smart glasses and the target scene, and these parameters can be saved. Then, the projection parameters of the smart glasses and the target scene can be read in this step.

[0103] S302. Based on the projection parameters of the target scene and the projection parameters of the smart glasses, construct a first projection transformation function from the target scene to the smart glasses, and construct a second projection transformation function from the smart glasses to the target scene.

[0104] Optionally, based on the projection parameters of the smart glasses and the projection parameters of the target scene obtained in step S301 above, a projection transformation matrix is ​​obtained, and then a first projection transformation function from the target scene to the smart glasses and a second projection transformation function from the smart glasses to the target scene are constructed.

[0105] In this embodiment, by obtaining the projection parameters of the target scene and the projection parameters of the smart glasses, a first projection transformation function from the target scene to the smart glasses and a second projection transformation function from the smart glasses to the target scene can be constructed, thereby ensuring the rendering effect when the first projection transformation function and the second projection transformation function are used for rendering.

[0106] The following describes the process of determining new cone-matching parameters between the smart glasses and the target scene based on the current parameters of the smart glasses' gyroscope.

[0107] Figure 4 The flowchart illustrating the process of determining new cone-matching parameters based on the current parameters of the gyroscope of the smart glasses in the panoramic rendering method provided in this application embodiment is as follows: Figure 4 As shown, the process of determining the new view cone matching parameters in step S103 above may include:

[0108] S401. Based on the current parameters of the gyroscope of the smart glasses, determine the changing cone momentum of the smart glasses and the changing cone momentum of the target scene.

[0109] Optionally, the variable cone momentum of the smart glasses can be obtained based on the variable momentum of the gyroscope of the smart glasses. In addition, the scene cone momentum can be obtained based on the user's operation of the smart glasses or the image information currently being played by the smart glasses.

[0110] S402. Based on the changing cone momentum of the smart glasses and the changing cone momentum of the target scene, re-acquire the cone model of the smart glasses and the cone model of the target scene.

[0111] Optionally, based on the changing cone momentum of the smart glasses and the changing cone momentum of the target scene, the smart glasses and the target scene are remodeled to obtain new cone models for the smart glasses and the target scene, which are then saved. This allows for the reading of the new cone models in this step.

[0112] S403. Re-match the view frustum model of the smart glasses and the view frustum model of the target scene to obtain new view frustum matching parameters between the smart glasses and the target scene.

[0113] Optionally, the difference between the new target scene's view frustum model and the new smart glasses' view frustum model can be determined based on the new smart glasses' view frustum model, and the view frustum matching parameters between the smart glasses and the target scene can be recalculated based on this difference to obtain new view frustum matching parameters between the smart glasses and the target scene.

[0114] In this embodiment, by re-acquiring the view frustum models of the smart glasses and the target scene based on the changing view frustum of the smart glasses and the changing view frustum of the target scene, and by re-matching the new view frustum models of the smart glasses and the new view frustum models of the target scene, new view frustum matching parameters can be obtained, thereby ensuring the rendering effect when using the new view frustum matching parameters for subsequent rendering.

[0115] The following describes the process of performing static rendering based on the new view frustum matching parameters and the second projection transformation function.

[0116] Figure 5 The flowchart illustrates the process of static rendering based on new view frustum matching parameters and a second projection transformation function in the panoramic rendering method for smart glasses provided in this application embodiment. Figure 5 As shown, the step S103 above, which involves performing static rendering based on the new view frustum matching parameters and the second projection transformation function, includes:

[0117] S501. Obtain the first projection transformation function corresponding to the second projection transformation function according to the second projection transformation function.

[0118] The first projection transformation function is the projection transformation function from the target scene to the smart glasses, and the second projection transformation function is the projection transformation function from the smart glasses to the target scene.

[0119] Optionally, the second projection transformation function from smart glasses to the target scene can be used as a reference. By establishing the analytical relationship of the first projection transformation function corresponding to the second projection transformation function, the first projection transformation function from the target scene to the smart glasses corresponding to the second projection transformation function from smart glasses to the target scene can be obtained.

[0120] S502. Render the target panorama and target scene based on the new view frustum matching parameters.

[0121] Optionally, based on the new cone-matching parameters between the smart glasses and the target scene obtained in step S403 above, the target panoramic image and the target scene are rendered.

[0122] S503. According to the first projection conversion function, the rendered target panoramic image and the rendered target scene are projected onto the field of view of the smart glasses for display.

[0123] The first projection transformation function is the projection transformation function from the target scene to the smart glasses.

[0124] Optionally, according to the first projection transformation function from the target scene to the smart glasses, the rendered target panoramic image and the rendered target scene are projected and displayed in the field of view of the smart glasses.

[0125] In this embodiment, the target panoramic image and the target scene are rendered based on the new frustum matching parameters between the smart glasses and the target scene. Then, according to the first projection transformation function from the target scene to the smart glasses, the rendered target panoramic image and the rendered target scene are projected onto the field of view of the smart glasses for display. This achieves accurate rendering and display of the target panoramic image in the smart glasses under the glasses change rendering.

[0126] The following describes the process of determining new view frustum matching parameters based on the view frustum model of the transformed target scene.

[0127] Figure 6 The flowchart illustrating the process of determining new frustum matching parameters based on the transformed frustum model of the target scene in the panoramic rendering method for smart glasses provided in this application embodiment is as follows: Figure 6 As shown, the process of determining the new view cone matching parameters in step S104 above may include:

[0128] S601. Obtain the view frustum model of the transformed target scene.

[0129] Optionally, based on the changing view frustum of the target scene, the target scene is remodeled to obtain a new view frustum model, which is then saved. This new view frustum model is then read in this step.

[0130] S602. Perform matching processing on the view frustum model of the smart glasses and the transformed view frustum model of the target scene to obtain new view frustum matching parameters between the smart glasses and the target scene.

[0131] Optionally, the difference between the view frustum model of the new target scene and the view frustum model of the smart glasses can be determined based on the view frustum model of the smart glasses, and the view frustum matching parameters between the smart glasses and the target scene can be recalculated based on the difference to obtain new view frustum matching parameters between the smart glasses and the target scene.

[0132] In this embodiment, by re-acquiring the view frustum model of the target scene based on the changing view frustum momentum of the target scene, and re-matching the view frustum model of the smart glasses with the new view frustum model of the target scene, new view frustum matching parameters can be obtained, thereby ensuring the rendering effect when using the new view frustum matching parameters for subsequent rendering.

[0133] The following describes the process of performing static rendering based on the new view frustum matching parameters in step S104.

[0134] Figure 7 A schematic diagram illustrating the process of performing static rendering based on new view frustum matching parameters for the panoramic rendering method of smart glasses provided in this application embodiment is shown below. Figure 7 As shown, the process of performing static rendering based on the new view frustum matching parameters in step S104 above may include:

[0135] S701. Render the target panorama and target scene based on the new view frustum matching parameters.

[0136] Optionally, based on the new frustum matching parameters between the smart glasses and the target scene obtained in step S602 above, the target panoramic image and the target scene are rendered.

[0137] S702. According to the first projection conversion function, the rendered target panoramic image and the rendered target scene are projected onto the field of view of the smart glasses for display.

[0138] The first projection transformation function is the projection transformation function from the target scene to the smart glasses.

[0139] Optionally, according to the first projection transformation function from the target scene to the smart glasses, the rendered target panoramic image and the rendered target scene are projected and displayed in the field of view of the smart glasses.

[0140] In this embodiment, the target panoramic image and the target scene are rendered based on the new frustum matching parameters between the smart glasses and the target scene. Then, according to the first projection transformation function from the target scene to the smart glasses, the rendered target panoramic image and the rendered target scene are projected onto the field of view of the smart glasses for display. This achieves accurate rendering and display of the target panoramic image in the smart glasses under scene change rendering.

[0141] Based on the same inventive concept, this application also provides a panoramic rendering device for smart glasses corresponding to the panoramic rendering method for smart glasses. Since the principle of the device in this application is similar to the panoramic rendering method for smart glasses described above, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.

[0142] Figure 8 This is a module structure diagram of a panoramic rendering device for smart glasses provided in an embodiment of this application, applied to smart glasses, such as... Figure 8 As shown, the device includes:

[0143] The determining module 801 is used to determine the current rendering mode based on the current parameters of the gyroscope in the smart glasses and the scene information to be displayed by the smart glasses. The rendering mode includes: static rendering, glasses-movement rendering, or scene-movement rendering.

[0144] The first rendering module 802 is used to render the target panoramic image and the target scene according to the frustum matching parameters between the smart glasses and the target scene of the target panoramic image to be rendered when the rendering mode is static rendering, and to project the rendered target panoramic image and the rendered target scene onto the field of view of the smart glasses for display according to the first projection conversion function from the target scene to the smart glasses.

[0145] The second rendering module 803 is used to determine new frustum matching parameters between the smart glasses and the target scene based on the current parameters of the gyroscope of the smart glasses when the rendering mode is glasses-changing rendering, and to perform the steps of the static rendering based on the new frustum matching parameters and the second projection transformation function from the smart glasses to the target scene.

[0146] The third rendering module 804 is used to determine new frustum matching parameters between the smart glasses and the target scene based on the frustum model of the transformed target scene when the rendering mode is scene change rendering, and to perform the steps of the static rendering based on the new frustum matching parameters.

[0147] As an optional implementation, the determining module 801 is further configured to:

[0148] Obtain the view frustum model of the smart glasses and the view frustum model of the target scene.

[0149] The view frustum model of the smart glasses and the view frustum model of the target scene are matched to obtain the view frustum matching parameters between the smart glasses and the target scene.

[0150] As an optional implementation, the determining module 801 is further configured to:

[0151] Obtain the projection parameters of the target scene and the projection parameters of the smart glasses.

[0152] Based on the projection parameters of the target scene and the projection parameters of the smart glasses, a first projection transformation function from the target scene to the smart glasses is constructed, and a second projection transformation function from the smart glasses to the target scene is constructed.

[0153] As an optional implementation, the second rendering module 803 is specifically used for:

[0154] Based on the current parameters of the gyroscope of the smart glasses, determine the variable cone momentum of the smart glasses and the variable cone momentum of the target scene.

[0155] As an optional implementation, the third rendering module 804 is specifically used for:

[0156] Obtain the transformed view frustum model of the target scene.

[0157] The view frustum model of the smart glasses and the transformed view frustum model of the target scene are matched to obtain new view frustum matching parameters between the smart glasses and the target scene.

[0158] As an optional implementation, the third rendering module 804 is specifically used for:

[0159] Based on the new view frustum matching parameters, render the target panorama and the target scene.

[0160] According to the first projection conversion function, the rendered target panoramic image and the rendered target scene are projected onto the field of view of the smart glasses for display.

[0161] This application embodiment also provides a computer device 90, such as... Figure 9 The diagram shown is a schematic representation of the structure of a computer device 90 provided in an embodiment of this application, including: a processor 91, a memory 92, and a bus 93. The memory 92 stores machine-readable instructions executable by the processor 91 (e.g., ...). Figure 8 The device determines the execution instructions corresponding to module 801, first rendering module 802, second rendering module 803 and third rendering module 804, etc. When the computer device 90 is running, the processor 91 communicates with the memory 92 through bus 93. When the machine-readable instructions are executed by the processor 91, the method steps in the above method embodiment are executed.

[0162] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the panoramic image rendering method for smart glasses described above.

[0163] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and devices described above can be referred to the corresponding processes in the method embodiments, and will not be repeated here. In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection can be through some communication interfaces; the indirect coupling or communication connection of devices or modules can be electrical, mechanical, or other forms.

[0164] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. If the functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0165] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A panoramic image rendering method for smart glasses, characterized in that, Applications in smart glasses include: Based on the current parameters of the gyroscope in the smart glasses and the scene information to be displayed by the smart glasses, the current rendering method is determined. The rendering method includes: static rendering, glasses-changing rendering, or scene-changing rendering. If the rendering method is static rendering, the target panoramic image and the target scene are rendered according to the frustum matching parameters between the smart glasses and the target scene of the target panoramic image to be rendered, and the rendered target panoramic image and the rendered target scene are projected onto the field of view of the smart glasses for display according to the first projection transformation function from the target scene to the smart glasses; If the rendering method is glasses-changing rendering, then a new frustum matching parameter between the smart glasses and the target scene is determined according to the current parameters of the gyroscope of the smart glasses, and the steps of the static rendering are executed according to the new frustum matching parameter and the second projection transformation function from the smart glasses to the target scene; If the rendering method is scene-changing rendering, then the new frustum matching parameters between the smart glasses and the target scene are determined according to the frustum model of the transformed target scene, and the steps of the static rendering are performed according to the new frustum matching parameters.

2. The method according to claim 1, characterized in that, Before rendering the target panoramic image and the target scene based on the frustum matching parameters between the smart glasses and the target scene in the target panoramic image to be rendered, the process further includes: Obtain the view frustum model of the smart glasses and the view frustum model of the target scene; The view frustum model of the smart glasses and the view frustum model of the target scene are matched to obtain the view frustum matching parameters between the smart glasses and the target scene.

3. The method according to claim 1, characterized in that, Before projecting the rendered target panoramic image and the rendered target scene onto the field of view of the smart glasses for display according to the first projection conversion function from the target scene to the smart glasses, the method further includes: Obtain the projection parameters of the target scene and the projection parameters of the smart glasses; Based on the projection parameters of the target scene and the projection parameters of the smart glasses, a first projection transformation function from the target scene to the smart glasses is constructed, and a second projection transformation function from the smart glasses to the target scene is constructed.

4. The method according to claim 1, characterized in that, The step of determining new cone-matching parameters between the smart glasses and the target scene based on the current parameters of the smart glasses' gyroscope includes: Based on the current parameters of the gyroscope of the smart glasses, determine the variable cone momentum of the smart glasses and the variable cone momentum of the target scene; Based on the changing cone momentum of the smart glasses and the changing cone momentum of the target scene, the cone model of the smart glasses and the cone model of the target scene are re-acquired. The view frustum model of the smart glasses and the view frustum model of the target scene are re-matched to obtain new view frustum matching parameters between the smart glasses and the target scene.

5. The method according to claim 4, characterized in that, The steps of performing the static rendering based on the new view frustum matching parameters and the second projection transformation function from the smart glasses to the target scene include: Based on the second projection transformation function, obtain the first projection transformation function corresponding to the second projection transformation function; Render the target panorama and the target scene based on the new frustum matching parameters; According to the first projection conversion function, the rendered target panoramic image and the rendered target scene are projected onto the field of view of the smart glasses for display.

6. The method according to claim 1, characterized in that, The step of determining new frustum matching parameters between the smart glasses and the target scene based on the transformed frustum model of the target scene includes: Obtain the transformed view frustum model of the target scene; The view frustum model of the smart glasses and the transformed view frustum model of the target scene are matched to obtain new view frustum matching parameters between the smart glasses and the target scene.

7. The method according to claim 6, characterized in that, The steps for performing the static rendering based on the new view frustum matching parameters include: Render the target panorama and the target scene based on the new frustum matching parameters; According to the first projection conversion function, the rendered target panoramic image and the rendered target scene are projected onto the field of view of the smart glasses for display.

8. A panoramic rendering device for smart glasses, characterized in that, The device, used in smart glasses, includes: The determination module is used to determine the current rendering method based on the current parameters of the gyroscope in the smart glasses and the scene information to be displayed by the smart glasses. The rendering method includes: static rendering, glasses-movement rendering, or scene-movement rendering. The first rendering module is used to render the target panoramic image and the target scene according to the frustum matching parameters between the smart glasses and the target scene of the target panoramic image to be rendered when the rendering mode is static rendering, and to project the rendered target panoramic image and the rendered target scene onto the field of view of the smart glasses for display according to the first projection transformation function from the target scene to the smart glasses. The second rendering module is used to determine new frustum matching parameters between the smart glasses and the target scene based on the current parameters of the gyroscope of the smart glasses when the rendering method is glasses-changing rendering, and to perform the steps of the static rendering based on the new frustum matching parameters and the second projection transformation function from the smart glasses to the target scene. The third rendering module is used to determine new frustum matching parameters between the smart glasses and the target scene based on the frustum model of the transformed target scene when the rendering mode is scene change rendering, and to perform the steps of the static rendering based on the new frustum matching parameters.

9. A computer device, characterized in that, include: The computer device includes a processor, a memory, and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processor executes the machine-readable instructions to perform the steps of the panoramic rendering method for smart glasses as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the panoramic rendering method for smart glasses as described in any one of claims 1 to 7.

Images