Multimedia teaching interaction method

Abstract

The invention discloses a multimedia teaching interaction method, which belongs to the technical field of scene interaction, and comprises the following steps: S1, generating a three-dimensional interaction scene and a passive plane image for an initial plane image of courseware to be interacted; determining characteristic parameters of the initial plane image according to the passive RGB values of all pixel points in the passive plane image; and S3, determining the roughness of each grid in the three-dimensional interaction scene according to the characteristic parameters of the initial plane image, and generating a final interaction scene for the courseware to be interacted. According to the method, the shader is used for rendering the three-dimensional interaction scene, the detail level of the three-dimensional scene is dynamically adjusted, and the readability of key information of courseware is ensured.

Description

Technical Field

[0001] The present invention belongs to the technical field of scene interaction, and particularly relates to a multimedia teaching interaction method. Background Art

[0002] With the integrated development of educational informatization and virtual reality technology, multimedia teaching interaction has gradually evolved from a two-dimensional plane to a three-dimensional immersive scene. The traditional interactive form of courseware is limited by static display and a single operation dimension, and it is difficult to meet the three-dimensional expression requirements of complex knowledge points. To break through this bottleneck, the technical path based on three-dimensional scene reconstruction and physical realism rendering has become a research hotspot, and its core lies in how to dynamically optimize the three-dimensional scene driven by image features, so as to construct a teaching environment with both interactivity and cognitive immersion.

[0003] In the field of three-dimensional scene construction, existing technologies usually use multi-view stereo vision (MVS) or monocular reconstruction methods based on deep learning to generate three-dimensional models. However, the rendering effect of the three-dimensional interactive scene of existing courseware is poor, and there is a lack of in-depth interaction based on visual features. Summary of the Invention

[0004] The present invention proposes a multimedia teaching interaction method to solve the above problems.

[0005] The technical solution of the present invention is: A multimedia teaching interaction method includes the following steps:

[0006] S1. Generate a three-dimensional interactive scene and a passive plane image for the initial plane image of the courseware to be interacted;

[0007] S2. Determine the passive RGB value using the original RGB value of the pixel points in the passive plane image, and determine the characteristic parameters of the initial plane image according to the passive RGB values of all the pixel points in the passive plane image;

[0008] S3. Determine the roughness of each grid in the three-dimensional interactive scene according to the characteristic parameters of the initial plane image, and generate the final interactive scene for the courseware to be interacted.

[0009] Further, S1 includes the following sub-steps:

[0010] S11. Obtain the initial plane image of the courseware to be interacted, perform three-dimensional imaging on the initial plane image, and generate a three-dimensional interactive scene;

[0011] S12. Obtain the passive illumination of the courseware to be interacted, and use the passive illumination to generate a passive plane image for the initial plane image.

[0012] Further, in S12, the passive plane image A * has the following expression: ; where A represents the initial planar image, T represents the transmittance map, and P represents the passive illumination.

[0013] The beneficial effects of the above further solution are as follows: In the present invention, the transmittance map serves as a soft mask to achieve the gradual fusion of the original image and the passive illumination, avoiding artifacts caused by simple superposition, ensuring natural light transition through pixel-level weight assignment, and conforming to the physical rendering law. The passive illumination, as an independent parameter, can simulate different light sources (such as natural light, laboratory light) or ambient light conditions, providing diverse visual presentations for the same three-dimensional scene.

[0014] Further, S2 includes the following sub-steps:

[0015] S21. Extract the variance of the original RGB values of all pixel points in each row of the passive planar image as the transmittance intensity parameter for each row.

[0016] S22. Calculate the passive RGB values of each pixel point in the passive planar image according to the transmittance intensity parameter of each row and the original RGB values of the pixel points.

[0017] S23. Take the ratio between the standard deviation of the passive RGB values of all pixel points in the passive planar image and the standard deviation of the original RGB values of all pixel points in the initial planar image as the characteristic parameter.

[0018] The beneficial effects of the above further solution are as follows: In the present invention, by calculating the variance of the RGB values of each row of pixels, the distribution dispersion degree of the color of that row is quantified, and the variance is used as the transmittance intensity parameter to provide an adaptive weight based on the image content for subsequent processing. Calculating the variance in units of rows can reduce the influence of global noise (such as isolated noise points) while retaining local structural information. The standard deviation ratio reflects the change in color distribution of the passive planar image relative to the initial image. This ratio can be directly used to adjust the roughness of the mesh in the three-dimensional scene.

[0019] Further, in S22, the passive RGB value Y p of the pixel point in the passive planar image has the following calculation formula: ; where Y represents the original RGB value of the pixel point in the passive planar image, Y max represents the maximum original RGB value of the row where the pixel point in the passive planar image is located, V represents the transmittance intensity parameter of the row where the pixel point in the passive planar image is located, V max represents the maximum transmittance intensity parameter of the passive planar image, V min represents the minimum transmittance intensity parameter of the passive planar image.

[0020] Normalize the transmittance intensity parameter of each row to [0, 1] to achieve cross-row intensity comparison.

[0021] Further, S3 includes the following sub-steps:

[0022] S31. Mesh the three-dimensional interactive scene and obtain the diffuse color of each grid;

[0023] S32. Determine the roughness of each grid in the PBR model according to the diffuse color of each grid and the characteristic parameters of the initial planar image;

[0024] S33. Based on the PBR model, use a shader to render the three-dimensional interactive scene to generate the final interactive scene of the interactive courseware to be interacted.

[0025] The beneficial effect of the above further solution is: In the present invention, the diffuse color is the core parameter of the PBR material, which directly determines the basic tone of the object surface. By extracting the diffuse color of the grid, a visual basis is provided for subsequent roughness calculation to ensure that the material properties are consistent with the original courseware content. Through color space conversion (RGB → saturation) and characteristic parameter modulation, a mapping from color attributes to physical material parameters (roughness) is established.

[0026] Further, S32 includes the following sub-steps:

[0027] S321. Multiply the characteristic parameters of the initial planar image by the RGB values corresponding to the diffuse color of the grid;

[0028] S322. Calculate the saturation of each grid according to the adjusted RGB values of each grid;

[0029] S323. Determine the roughness of each grid according to the saturation of each grid.

[0030] The beneficial effect of the above further solution is: In the present invention, in S322, the RGB values are converted to the HSV / HSL space to extract the saturation. Saturation directly reflects the purity of the color and has an implicit association with the material roughness. High-saturation areas (such as metal parts in the courseware) are automatically assigned low roughness and exhibit specular reflection; low-saturation areas (such as wooden structures) are assigned high roughness and exhibit diffuse reflection.

[0031] The beneficial effect of the present invention is:

[0032] (1) The present invention generates a passive planar image through passive lighting, simulates different light sources (such as natural light and laboratory lighting) or ambient lighting conditions, and provides a diverse visual presentation for the same three-dimensional scene; then, by extracting characteristic parameters such as the saturation and standard deviation of the passive planar image, characteristic parameters related to roughness are established;

[0033] (2) In the present invention, the characteristic parameters of the initial planar image are mapped to the three-dimensional grid roughness, so that obvious shadows will be generated on the high-roughness surface (such as rock) under strong directional light, while the low-roughness surface (such as metal) presents high-gloss reflection, enhancing the material recognition.

[0034] (3) The present invention uses a shader to render the three-dimensional interactive scene, dynamically adjusting the level of detail of the three-dimensional scene to ensure the readability of the key information in the courseware. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a flowchart of the multimedia teaching interaction method. DETAILED DESCRIPTION OF THE INVENTION

[0036] The embodiments of the present invention will be further described below with reference to the drawings.

[0037] As Figure 1 shown, the present invention provides a multimedia teaching interaction method, including the following steps:

[0038] S1. Generate a three-dimensional interactive scene and a passive planar image for the initial planar image of the courseware to be interacted.

[0039] S2. Determine the passive RGB value using the original RGB value of the pixel points in the passive planar image, and determine the characteristic parameters of the initial planar image according to the passive RGB values of all pixel points in the passive planar image.

[0040] S3. Determine the roughness of each grid in the three-dimensional interactive scene according to the characteristic parameters of the initial planar image, and generate the final interactive scene for the courseware to be interacted.

[0041] In the embodiment of the present invention, S1 includes the following sub-steps:

[0042] S11. Obtain the initial planar image of the courseware to be interacted, perform three-dimensional imaging on the initial planar image, and generate a three-dimensional interactive scene.

[0043] S12. Obtain the passive illumination of the courseware to be interacted, and use the passive illumination to generate a passive planar image for the initial planar image.

[0044] In the embodiment of the present invention, in S12, the passive planar image A * has the following expression: ; where A represents the initial planar image, T represents the transmittance map, and P represents the passive illumination.

[0045] In the present invention, the transmittance map serves as a soft mask to achieve a gradual fusion of the original image and passive illumination, avoiding artifacts caused by simple superposition, ensuring a natural transition of illumination through pixel-level weight assignment, and conforming to the laws of physical rendering. Passive illumination, as an independent parameter, can simulate different light sources (such as natural light, laboratory lighting) or ambient lighting conditions, providing diverse visual presentations for the same three-dimensional scene.

[0046] In an embodiment of the present invention, S2 includes the following sub-steps:

[0047] S21. Extract the variance of the original RGB values of all pixel points in each row of the passive plane image as the transmittance intensity parameter for each row;

[0048] S22. Calculate the passive RGB values of each pixel point in the passive plane image according to the transmittance intensity parameter of each row and the original RGB values of the pixel points;

[0049] S23. Take the ratio between the standard deviation of the passive RGB values of all pixel points in the passive plane image and the standard deviation of the original RGB values of all pixel points in the initial plane image as the characteristic parameter.

[0050] In the present invention, by calculating the variance of the RGB values of each row of pixels, the distribution dispersion degree of the colors in that row is quantified, and the variance is used as the transmittance intensity parameter to provide an adaptive weight based on the image content for subsequent processing. Calculating the variance row by row can reduce the influence of global noise (such as isolated noise points) while retaining local structural information. The standard deviation ratio reflects the change in color distribution of the passive plane image relative to the initial image. This ratio can be directly used to adjust the roughness of the meshes in the three-dimensional scene.

[0051] In an embodiment of the present invention, in S22, the passive RGB value Y p of the pixel point in the passive plane image has the following calculation formula: ; where Y represents the original RGB value of the pixel point in the passive plane image, Y max represents the maximum original RGB value of the row where the pixel point in the passive plane image is located, V represents the transmittance intensity parameter of the row where the pixel point in the passive plane image is located, V max represents the maximum transmittance intensity parameter of the passive plane image, and V min represents the minimum transmittance intensity parameter of the passive plane image.

[0052] Normalize the transmittance intensity parameter of each row to [0, 1] to achieve intensity comparison across rows.

[0053] In an embodiment of the present invention, S3 includes the following sub-steps:

[0054] S31. Mesh the three-dimensional interactive scene and obtain the diffuse color of each mesh;

[0055] S32. Determine the roughness of each grid in the PBR model according to the diffuse color of each grid and the characteristic parameters of the initial planar image.

[0056] S33. Based on the PBR model, use a shader to render the 3D interactive scene to generate the final interactive scene of the interactive courseware to be interacted with.

[0057] In the present invention, the diffuse color is the core parameter of the PBR material, which directly determines the basic tone of the object surface. By extracting the diffuse color of the grid, a visual basis is provided for subsequent roughness calculation to ensure that the material properties are consistent with the original courseware content. Through color space conversion (RGB → saturation) and characteristic parameter modulation, a mapping from color attributes to physical material parameters (roughness) is established.

[0058] In the embodiment of the present invention, S32 includes the following sub-steps:

[0059] S321. Multiply the characteristic parameters of the initial planar image by the RGB values corresponding to the diffuse color of the grid.

[0060] S322. Calculate the saturation of each grid according to the adjusted RGB values of each grid.

[0061] S323. Determine the roughness of each grid according to the saturation of each grid.

[0062] In the present invention, in S322, the RGB values are converted to the HSV / HSL space to extract the saturation. The saturation directly reflects the purity of the color and has an implicit relationship with the material roughness. High-saturation regions (such as metal components in the courseware) are automatically assigned low roughness and exhibit specular reflection; low-saturation regions (such as wooden structures) are assigned high roughness and exhibit diffuse reflection.

[0063] Those of ordinary skill in the art will realize that the embodiments described herein are for helping the reader understand the principles of the present invention, and it should be understood that the protection scope of the present invention is not limited to such specific statements and embodiments. Those of ordinary skill in the art can make various other specific deformations and combinations without departing from the essence of the present invention based on the technical revelations disclosed in the present invention, and these deformations and combinations are still within the protection scope of the present invention.

Claims

1. A multimedia teaching interaction method, characterized in that, It includes the following steps: S1. Generate a three-dimensional interactive scene and a passive planar image for the initial planar image of the courseware to be interacted with; S2. Determine the passive RGB values using the original RGB values of the pixel points in the passive planar image, and determine the characteristic parameters of the initial planar image based on the passive RGB values of all the pixel points in the passive planar image; S3. Determine the roughness of each grid in the three-dimensional interactive scene according to the characteristic parameters of the initial planar image, and generate the final interactive scene for the courseware to be interacted with.

2. The multimedia teaching interaction method according to claim 1, wherein The S1 includes the following sub-steps: S11. Obtain the initial planar image of the courseware to be interacted with, perform three-dimensional imaging on the initial planar image, and generate a three-dimensional interactive scene; S12. Obtain the passive light of the courseware to be interacted with, and use the passive light to generate a passive planar image for the initial planar image.

3. The multimedia teaching interaction method according to claim 2, characterized in that, In the above S12, the passive planar image A * is expressed as: ; where A represents the initial planar image, T represents the transmittance map, and P represents the passive illumination.

4. The multimedia teaching interaction method according to claim 1, characterized in that, The S2 includes the following sub-steps: S21. Extract the variance of the original RGB values of all the pixel points in each row of the passive planar image as the transmission intensity parameter of each row; S22. Calculate the passive RGB values of each pixel point in the passive planar image according to the transmission intensity parameter of each row and the original RGB values of the pixel points; S23. Take the ratio between the standard deviation of the passive RGB values of all the pixel points in the passive planar image and the standard deviation of the original RGB values of all the pixel points in the initial planar image as the characteristic parameter.

5. The multimedia teaching interaction method according to claim 4, characterized in that, In S22, the passive RGB value Y of the pixel in the passive plane image p has the following calculation formula: ; where Y represents the original RGB value of the pixel in the passive plane image, Y max represents the maximum original RGB value of the row where the pixel in the passive plane image is located, V represents the transmission intensity parameter of the row where the pixel in the passive plane image is located, and V max represents the maximum transmission intensity parameter of the passive plane image, and V min represents the minimum transmission intensity parameter of the passive plane image.

6. The multimedia teaching interaction method according to claim 1, characterized in that, The S3 includes the following sub-steps: S31. Mesh the three-dimensional interactive scene and obtain the diffuse color of each grid; S32. Determine the roughness of each grid in the PBR model according to the diffuse color of each grid and the characteristic parameters of the initial planar image; S33. Based on the PBR model, use a shader to render the three-dimensional interactive scene and generate the final interactive scene of the courseware to be interacted with.

7. The multimedia teaching interaction method according to claim 6, wherein The S32 includes the following sub-steps: S321. Multiply the characteristic parameters of the initial planar image by the RGB values corresponding to the diffuse color of the grid; S322. Calculate the saturation of each grid according to the adjusted RGB values of each grid; S323. Determine the roughness of each grid according to the saturation of each grid.

Images