A multimedia teaching interactive method

Abstract

The present invention discloses a multimedia teaching interaction method, belonging to the field of scene interaction technology, comprising the following steps: S1, generating a three-dimensional interactive scene and a passive plane image for an initial plane image of a courseware to be interactive; S2, determining the passive RGB values ​​using the original RGB values ​​of pixels in the passive plane image, and determining the characteristic parameters of the initial plane image based on the passive RGB values ​​of all pixels in the passive plane image; S3, determining the roughness of each grid in the three-dimensional interactive scene based on the characteristic parameters of the initial plane image, and generating a final interactive scene for the courseware to be interactive. The present invention uses a shader to render the three-dimensional interactive scene, dynamically adjusting the level of detail of the three-dimensional scene to ensure the readability of key information in the courseware.

Description

Technical Field

[0001] The present invention belongs to the technical field of scene interaction, and in particular relates to a multimedia teaching interaction method. Background Art

[0002] With the convergence of educational informatization and virtual reality technology, multimedia teaching interactions are gradually evolving from two-dimensional surfaces to three-dimensional immersive scenarios. Traditional interactive courseware, limited by static presentation and a single operational dimension, struggles to meet the demands of three-dimensional presentation of complex knowledge points. To overcome this bottleneck, technical approaches based on three-dimensional scene reconstruction and physically realistic rendering have become a hot topic of research. The core of this research is how to leverage image features to drive dynamic optimization of three-dimensional scenes, thereby creating a teaching environment that is both interactive and cognitively immersive.

[0003] In the field of 3D scene construction, existing technologies typically use multi-view stereo (MVS) or deep learning-based monocular reconstruction methods to generate 3D models. However, the rendering effects of 3D interactive scenes in existing courseware are poor and lack deep interaction based on visual features. Summary of the Invention

[0004] In order to solve the above problems, the present invention proposes a multimedia teaching interactive method.

[0005] The technical solution of the present invention is: a multimedia teaching interactive method comprises the following steps:

[0006] S1, generating a three-dimensional interactive scene and a passive plane image for the initial plane image of the interactive courseware;

[0007] S2, determining the passive RGB value using the original RGB value of the pixel in the passive plane image, and determining the characteristic parameters of the initial plane image according to the passive RGB values ​​of all the pixels in the passive plane image;

[0008] S3. Determine the roughness of each grid in the three-dimensional interactive scene based on the characteristic parameters of the initial plane image, and generate a final interactive scene for the interactive courseware.

[0009] Furthermore, S1 includes the following sub-steps:

[0010] S11, obtaining an initial planar image of the interactive courseware, performing three-dimensional imaging on the initial planar image, and generating a three-dimensional interactive scene;

[0011] S12. Obtain passive lighting of the interactive courseware, and use the passive lighting to generate a passive planar image for the initial planar image.

[0012] Furthermore, in S12, the passive plane image A * The expression is: ; Where A represents the initial plane image, T represents the transmittance map, and P represents passive lighting.

[0013] The beneficial effect of this further solution is that, in this invention, the transmittance map acts as a soft mask to achieve a gradual fusion of the original image and passive lighting, avoiding artifacts caused by simple superposition. Pixel-level weight distribution ensures a natural lighting transition that conforms to physical rendering principles. Passive lighting, as an independent parameter, can simulate different light sources (such as natural light, laboratory lighting) or ambient lighting conditions, providing a diverse visual representation of the same 3D scene.

[0014] Furthermore, S2 includes the following sub-steps:

[0015] S21, extracting the variance of the original RGB values ​​of all pixels in each row of the passive plane image as the transmission intensity parameter of each row;

[0016] S22, calculating the passive RGB value of each pixel in the passive plane image according to the transmission intensity parameter of each row and the original RGB value of the pixel;

[0017] S23. Taking the ratio between the standard deviation of the passive RGB values ​​of all pixels in the passive plane image and the standard deviation of the original RGB values ​​of all pixels in the initial plane image as a feature parameter.

[0018] The beneficial effect of the above further solution is that, in the present invention, by calculating the variance of the RGB values ​​of each row of pixels, the color distribution dispersion of that row is quantified. This variance is used as a transmission intensity parameter to provide adaptive weighting based on image content for subsequent processing. Calculating variance on a row-by-row basis can reduce the impact of global noise (such as isolated noise points) while preserving local structural information. The standard deviation ratio reflects the change in color distribution in the passive planar image relative to the initial image. This ratio can be directly used to adjust the roughness of the mesh in the three-dimensional scene.

[0019] Furthermore, in S22, the passive RGB value Y of the pixel in the passive plane image p The calculation formula is: ; Where Y represents the original RGB value of the pixel in the passive plane image, Y max represents the maximum original RGB value of the row where the pixel point is located in the passive plane image, V represents the transmission intensity parameter of the row where the pixel point is located in the passive plane image, and V max Represents the maximum transmission intensity parameter of the passive plane image, V min Represents the minimum transmission intensity parameter of the passive planar image.

[0020] The transmission intensity parameter of each row is normalized to [0, 1] to enable intensity comparison across rows.

[0021] Furthermore, S3 includes the following sub-steps:

[0022] S31, meshing the three-dimensional interactive scene and obtaining the diffuse reflection color of each mesh;

[0023] S32, determining the roughness of each mesh in the PBR model according to the diffuse reflection color of each mesh and the characteristic parameters of the initial plane image;

[0024] S33. Based on the PBR model, the shader is used to render the three-dimensional interactive scene to generate the final interactive scene of the interactive courseware.

[0025] The beneficial effect of this further solution is that, in this invention, diffuse color is a core parameter of PBR materials, directly determining the fundamental hue of an object's surface. By extracting the diffuse color of a mesh, a visual basis is provided for subsequent roughness calculations, ensuring that material properties are consistent with the original courseware content. Through color space conversion (RGB to saturation) and feature parameter modulation, a mapping of color properties to physical material parameters (roughness) is established.

[0026] Furthermore, S32 includes the following sub-steps:

[0027] S321, multiplying the characteristic parameters of the initial plane image by the RGB value corresponding to the diffuse reflection color of the grid;

[0028] S322, calculating the saturation of each grid according to the adjusted RGB value of each grid;

[0029] S323. Determine the roughness of each grid according to the saturation of each grid.

[0030] The beneficial effect of the above further solution is that in the present invention, in S322, the RGB values ​​are converted to HSV / HSL space to extract saturation. Saturation directly reflects the purity of the color and is implicitly associated with the roughness of the material. Highly saturated areas (such as metal parts in the courseware) are automatically assigned low roughness, resulting in high light reflection; low saturated areas (such as wooden structures) are assigned high roughness, resulting in diffuse reflection.

[0031] The beneficial effects of the present invention are:

[0032] (1) The present invention generates a passive plane image through passive illumination, simulating different light sources (such as natural light and laboratory light) or ambient lighting conditions to provide a diverse visual presentation of the same three-dimensional scene. Feature parameters related to roughness are then established by extracting characteristic parameters such as saturation and standard deviation from the passive plane image.

[0033] (2) The present invention maps the characteristic parameters of the initial plane image to the three-dimensional grid roughness, so that high-roughness surfaces (such as rocks) will produce obvious shadows under strong directional light, while low-roughness surfaces (such as metals) will show high light reflection, thereby enhancing material recognition;

[0034] (3) The present invention uses a shader to render the three-dimensional interactive scene and dynamically adjusts the level of detail of the three-dimensional scene to ensure the readability of key information in the courseware. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 This is a flowchart of the multimedia teaching interactive method. DETAILED DESCRIPTION

[0036] The embodiments of the present invention will be further described below with reference to the accompanying drawings.

[0037] like Figure 1 As shown, the present invention provides a multimedia teaching interactive method, comprising the following steps:

[0038] S1, generating a three-dimensional interactive scene and a passive plane image for the initial plane image of the interactive courseware;

[0039] S2, determining the passive RGB value using the original RGB value of the pixel in the passive plane image, and determining the characteristic parameters of the initial plane image according to the passive RGB values ​​of all the pixels in the passive plane image;

[0040] S3. Determine the roughness of each grid in the three-dimensional interactive scene based on the characteristic parameters of the initial plane image, and generate a final interactive scene for the interactive courseware.

[0041] In this embodiment of the present invention, S1 includes the following sub-steps:

[0042] S11, obtaining an initial planar image of the interactive courseware, performing three-dimensional imaging on the initial planar image, and generating a three-dimensional interactive scene;

[0043] S12. Obtain passive lighting of the interactive courseware, and use the passive lighting to generate a passive planar image for the initial planar image.

[0044] In the embodiment of the present invention, in S12, the passive plane image A * The expression is: ; Where A represents the initial plane image, T represents the transmittance map, and P represents passive lighting.

[0045] In this paper, the transmittance map acts as a soft mask to achieve a gradual blending of the original image and passive lighting, avoiding artifacts caused by simple overlay. Pixel-level weighting ensures a natural lighting transition, consistent with physical rendering principles. Passive lighting, as an independent parameter, can simulate different light sources (such as natural light, laboratory lighting) or ambient lighting conditions, providing diverse visual representations of the same 3D scene.

[0046] In this embodiment of the present invention, S2 includes the following sub-steps:

[0047] S21, extracting the variance of the original RGB values ​​of all pixels in each row of the passive plane image as the transmission intensity parameter of each row;

[0048] S22, calculating the passive RGB value of each pixel in the passive plane image according to the transmission intensity parameter of each row and the original RGB value of the pixel;

[0049] S23. Taking the ratio between the standard deviation of the passive RGB values ​​of all pixels in the passive plane image and the standard deviation of the original RGB values ​​of all pixels in the initial plane image as a feature parameter.

[0050] In this paper, the variance of the RGB values ​​of each row of pixels is calculated to quantify the degree of color dispersion within that row. This variance is used as a transmission intensity parameter to provide adaptive weighting based on image content for subsequent processing. Computing variance on a row-by-row basis reduces the impact of global noise (such as isolated noise points) while preserving local structural information. The standard deviation ratio reflects the change in color distribution in the passive planar image relative to the initial image. This ratio can be directly used to adjust the roughness of meshes in 3D scenes.

[0051] In the embodiment of the present invention, in S22, the passive RGB value Y of the pixel point in the passive plane image is p The calculation formula is: ; Where Y represents the original RGB value of the pixel in the passive plane image, Y max represents the maximum original RGB value of the row where the pixel point is located in the passive plane image, V represents the transmission intensity parameter of the row where the pixel point is located in the passive plane image, and V max Represents the maximum transmission intensity parameter of the passive plane image, V min Represents the minimum transmission intensity parameter of the passive planar image.

[0052] The transmission intensity parameter of each row is normalized to [0, 1] to enable intensity comparison across rows.

[0053] In this embodiment of the present invention, S3 includes the following sub-steps:

[0054] S31, meshing the three-dimensional interactive scene and obtaining the diffuse reflection color of each mesh;

[0055] S32, determining the roughness of each mesh in the PBR model according to the diffuse reflection color of each mesh and the characteristic parameters of the initial plane image;

[0056] S33. Based on the PBR model, the shader is used to render the three-dimensional interactive scene to generate the final interactive scene of the interactive courseware.

[0057] In this paper, diffuse color is a core parameter of PBR materials, directly determining the underlying hue of an object's surface. Extracting the diffuse color of a mesh provides a visual basis for subsequent roughness calculations, ensuring that material properties are consistent with the original courseware. A mapping from color space conversion (RGB to saturation) and feature parameter modulation is established to map color properties to physical material parameters (roughness).

[0058] In this embodiment of the present invention, S32 includes the following sub-steps:

[0059] S321, multiplying the characteristic parameters of the initial plane image by the RGB value corresponding to the diffuse reflection color of the grid;

[0060] S322, calculating the saturation of each grid according to the adjusted RGB value of each grid;

[0061] S323. Determine the roughness of each grid according to the saturation of each grid.

[0062] In this invention, in S322, the RGB values ​​are converted to HSV / HSL space to extract saturation. Saturation directly reflects the purity of the color and is implicitly associated with the roughness of the material. Highly saturated areas (such as metal parts in the courseware) are automatically assigned low roughness, resulting in high light reflections; low saturated areas (such as wooden structures) are assigned high roughness, resulting in diffuse reflections.

[0063] Those skilled in the art will appreciate that the embodiments described herein are intended to help readers understand the principles of the present invention, and it should be understood that the scope of protection of the present invention is not limited to such specific descriptions and embodiments. Those skilled in the art can make various other specific variations and combinations based on the technical teachings disclosed in the present invention without departing from the essence of the present invention, and such variations and combinations are still within the scope of protection of the present invention.

Claims

1. A multimedia teaching interactive method, characterized in that: The following steps are involved: S1, generating a three-dimensional interactive scene and a passive plane image for the initial plane image of the interactive courseware; S2, determining the passive RGB value using the original RGB value of the pixel in the passive plane image, and determining the characteristic parameters of the initial plane image according to the passive RGB values ​​of all the pixels in the passive plane image; S3. Determine the roughness of each grid in the three-dimensional interactive scene based on the characteristic parameters of the initial plane image, and generate a final interactive scene for the interactive courseware; The S1 includes the following sub-steps: S11, obtaining an initial planar image of the interactive courseware, performing three-dimensional imaging on the initial planar image, and generating a three-dimensional interactive scene; S12, obtaining passive illumination of the interactive courseware, and generating a passive planar image for the initial planar image using the passive illumination; The S2 includes the following sub-steps: S21, extracting the variance of the original RGB values ​​of all pixels in each row of the passive plane image as the transmission intensity parameter of each row; S22, calculating the passive RGB value of each pixel in the passive plane image according to the transmission intensity parameter of each row and the original RGB value of the pixel; S23, taking the ratio between the standard deviation of the passive RGB values ​​of all pixels in the passive plane image and the standard deviation of the original RGB values ​​of all pixels in the initial plane image as a feature parameter; The S3 includes the following sub-steps: S31, meshing the three-dimensional interactive scene and obtaining the diffuse reflection color of each mesh; S32, determining the roughness of each mesh in the PBR model according to the diffuse reflection color of each mesh and the characteristic parameters of the initial plane image; S33. Based on the PBR model, the shader is used to render the three-dimensional interactive scene to generate the final interactive scene of the interactive courseware.

2. The multimedia teaching interactive method according to claim 1, characterized in that: In said S12, the passive plane image A * The expression is: ; Where A represents the initial plane image, T represents the transmittance map, and P represents passive lighting.

3. The multimedia teaching interactive method according to claim 1, characterized in that: In said S22, the passive RGB value Y of the pixel in the passive plane image p The calculation formula is: ; Where Y represents the original RGB value of the pixel in the passive plane image, Y max represents the maximum original RGB value of the row where the pixel point is located in the passive plane image, V represents the transmission intensity parameter of the row where the pixel point is located in the passive plane image, and V max Represents the maximum transmission intensity parameter of the passive plane image, V min Represents the minimum transmission intensity parameter of the passive planar image.

4. The multimedia teaching interactive method according to claim 1, characterized in that: The S32 includes the following sub-steps: S321, multiplying the characteristic parameters of the initial plane image by the RGB value corresponding to the diffuse reflection color of the grid; S322, calculating the saturation of each grid according to the adjusted RGB value of each grid; S323. Determine the roughness of each grid according to the saturation of each grid.

Images