Intelligent blackboard image anti-glare rendering method and system based on three-dimensional light field modeling

Abstract

The application discloses a smart blackboard image anti-glare rendering method and system based on three-dimensional light field modeling, and relates to the field of computer graphics processing. The method comprises the following steps: acquiring multi-view blackboard images, environmental illumination data and camera calibration parameters of a target smart blackboard, and determining image sampling points of a main view image; performing three-dimensional reconstruction on a board surface based on the multi-view images and the calibration parameters, so as to obtain a three-dimensional geometric structure of the board surface and spatial positions of the sampling points; remapping multi-view pixel data to the spatial positions of the board surface and light directions, and constructing a four-dimensional light field model; combining environmental illumination, a micro-face element reflection model and prior estimation of chalk reflectivity to estimate bidirectional reflectance distribution function parameters, and separating diffuse reflection components and specular reflection components through inverse rendering optimization; and adaptively attenuating the specular reflection components based on the specular reflection weight, the illumination and the observation direction, and fusing and rendering. Thus, the board surface glare can be inhibited and the board content can be preserved under complex illumination, and the image readability is improved.

Description

Technical Field

[0001] This application relates to the field of computer graphics processing technology, and in particular to a smart blackboard image anti-glare rendering method and system based on three-dimensional light field modeling. Background Technology

[0002] In modern education and meeting settings, smart blackboards and interactive whiteboards have gradually become important carriers for information display, note-taking, and remote sharing. With the increasing demand for applications such as classroom recording, online meetings, cloud archiving, and intelligent recognition, the content on the blackboard is no longer just for on-site viewing; it also needs to be captured in real-time by cameras, digitally processed, and subsequently analyzed. Therefore, the clarity, stability, and readability of the captured images directly affect the presentation of digital content and the effectiveness of subsequent intelligent processing.

[0003] However, in real-world environments, indoor lighting conditions are often complex, potentially involving multiple light sources such as natural light, artificial lighting, projected light, and reflected light from display devices. When these lights illuminate a blackboard or whiteboard surface with reflective properties, they can easily create localized bright areas, glare, or specular reflections at specific viewing angles. This type of optical interference can cause partial obstruction of handwriting, graphic lines, or projected content, resulting in phenomena such as brightness saturation, weakened texture, and missing content in the image. Furthermore, it can negatively impact image segmentation, feature extraction, character recognition, and remote display performance.

[0004] To address the issue of glare on whiteboards, current solutions generally include hardware-level lighting or whiteboard surface modifications, and software-level image post-processing. Hardware methods primarily reduce the impact of reflections by adjusting the lighting layout, reducing the whiteboard's reflectivity, or adding auxiliary optical devices. However, their effectiveness is easily limited by the spatial structure of the classroom or meeting room, the position of the light source, installation costs, and changes in the usage environment. Software methods mostly involve detecting, suppressing, or repairing bright areas after image acquisition. However, when the whiteboard content has complex textures, significant lighting variations, or when glare covers key text, issues such as insufficient processing of bright areas, weakening of realistic handwriting, or unnatural repair results can easily occur. Therefore, the quality of whiteboard images in practical applications is affected by a combination of factors, including the lighting environment, surface reflectivity, acquisition angle, and the form of the whiteboard content. Summary of the Invention

[0005] This application provides a smart blackboard image anti-glare rendering method, system, storage medium, computer program product, and electronic device based on three-dimensional light field modeling, which at least solves the problem in the prior art that the blackboard image is easily affected by glare interference under complex lighting conditions, resulting in a decrease in content acquisition quality and readability.

[0006] In a first aspect, embodiments of this application provide a method for anti-glare rendering of smart blackboard images based on three-dimensional light field modeling. The method includes: acquiring multi-view blackboard images, ambient illuminance data, and camera calibration parameters corresponding to each acquisition viewpoint of a target smart blackboard simultaneously; determining a main viewpoint image to be rendered from the multi-view blackboard images; and using pixels in the main viewpoint image as image sampling points; performing three-dimensional reconstruction of the target smart blackboard surface based on the multi-view blackboard images and the camera calibration parameters to obtain the three-dimensional geometric structure of the target smart blackboard and the spatial position of the board surface corresponding to each image sampling point; remapping the pixel data in the multi-view blackboard images to the corresponding board surface spatial position and light direction based on the three-dimensional geometric structure and the camera calibration parameters to construct a four-dimensional light field model corresponding to the target smart blackboard; the four-dimensional light field model is used to characterize the radiation intensity information of each board surface spatial position under different light directions; and based on the radiation intensity information, the board surface spatial position corresponding to each image sampling point, and the image sampling point, the method further includes: reconstructing the target smart blackboard surface three-dimensionally based on the radiation intensity information, the image sampling point, and the camera calibration parameters; and reconstructing the target smart blackboard surface three-dimensionally based on the radiation intensity information, the image sampling point, and the camera calibration parameters. The ambient illuminance data and the preset micro-facet reflection model, combined with the chalk reflection a priori condition used to characterize the reflectivity difference between the chalk area and the blackboard background area, are used to estimate the bidirectional reflectance distribution function parameters corresponding to each of the image sampling points. The bidirectional reflectance distribution function parameters include diffuse reflection weights and specular reflection weights. Based on the main view image, the bidirectional reflectance distribution function parameters, and the four-dimensional light field model, an inverse rendering optimization objective function is constructed. By solving the inverse rendering optimization objective function, the diffuse reflection component and specular reflection component corresponding to each of the image sampling points are determined. According to the specular reflection weight, specular reflection component, ambient illuminance data, and the observation direction corresponding to the main view image determined based on the camera calibration parameters, an adaptive suppression factor is determined for each of the image sampling points. The adaptive suppression factor is used to attenuate the specular reflection component, and the diffuse reflection component and the attenuated specular reflection component are fused and rendered to obtain the anti-glare rendering image of the target smart blackboard.

[0007] Secondly, embodiments of this application provide a smart blackboard image anti-glare rendering system based on three-dimensional light field modeling. The system includes: a multi-source sampling unit, used to acquire multi-view blackboard images, ambient illuminance data, and camera calibration parameters corresponding to each acquisition viewpoint of a target smart blackboard simultaneously, and to determine the main viewpoint image to be rendered from the multi-view blackboard images, using the pixels in the main viewpoint image as image sampling points; a board surface reconstruction unit, used to perform three-dimensional reconstruction of the target smart blackboard based on the multi-view blackboard images and the camera calibration parameters, to obtain the three-dimensional geometric structure of the target smart blackboard and the board surface spatial position corresponding to each of the image sampling points; a light field modeling unit, used to remap the pixel data in the multi-view blackboard images to the corresponding board surface spatial position and light direction based on the three-dimensional geometric structure and the camera calibration parameters, to construct a four-dimensional light field model corresponding to the target smart blackboard; the four-dimensional light field model is used to characterize the radiation intensity information of each of the board surface spatial positions under different light directions; and a reflection parameter estimation unit, used to estimate the reflection parameters based on the radiation intensity information and the corresponding image sampling points. The system uses the spatial location of the board surface, the ambient illuminance data, and a preset micro-facet reflection model, combined with a priori conditions for chalk reflection to characterize the reflectivity difference between the chalk area and the blackboard background area, to estimate the bidirectional reflectance distribution function parameters corresponding to each of the image sampling points. The bidirectional reflectance distribution function parameters include diffuse reflection weights and specular reflection weights. A reflection component decomposition unit is used to construct an inverse rendering optimization objective function based on the main view image, the bidirectional reflectance distribution function parameters, and the four-dimensional light field model, and to determine the diffuse reflection component and specular reflection component corresponding to each of the image sampling points by solving the inverse rendering optimization objective function. An anti-glare fusion rendering unit is used to determine the adaptive suppression factor of each of the image sampling points based on the specular reflection weight, specular reflection component, ambient illuminance data, and the observation direction corresponding to the main view image determined based on the camera calibration parameters. The adaptive suppression factor is used to attenuate the specular reflection component, and the diffuse reflection component and the attenuated specular reflection component are fused and rendered to obtain the anti-glare rendering image of the target smart blackboard.

[0008] Thirdly, an electronic device is provided, comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the steps of the smart blackboard image anti-glare rendering method based on three-dimensional light field modeling according to any embodiment of the present application.

[0009] Fourthly, embodiments of this application provide a storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the smart blackboard image anti-glare rendering method based on three-dimensional light field modeling according to any embodiment of this application.

[0010] Fifthly, embodiments of this application provide a computer program product, including a computer program / instructions, which, when executed by a processor, implement the steps of the smart blackboard image anti-glare rendering method based on three-dimensional light field modeling of any embodiment of this application.

[0011] The smart blackboard image anti-glare rendering method and system based on three-dimensional light field modeling provided in this application can produce at least the following technical effects: By simultaneously acquiring multi-view blackboard images, ambient illumination data, and camera calibration parameters, and combining these with the 3D reconstruction results of the blackboard surface, pixels in the main view image are mapped to their actual spatial locations on the blackboard surface. This unifies image information from different viewpoints into the dimensions of spatial location and light direction. The resulting four-dimensional light field model can characterize the variation of radiant light intensity at the same location on the blackboard surface under different viewing directions, providing spatial geometric constraints and directional lighting constraints for image processing. Based on this model, the stable diffuse reflection information of the blackboard surface content and the high-brightness specular reflection information that varies with the viewing angle can be distinguished within a unified physical expression framework. This provides a reliable multi-view data foundation for subsequent anti-glare rendering, thereby improving the accuracy and stability of blackboard image processing under complex lighting conditions.

[0012] By combining ambient illumination data, a micro-area reflection model, and a priori conditions for chalk reflection, the bidirectional reflectance distribution function parameters of each image sampling point are estimated. Based on this, the diffuse and specular reflection components are separated through inverse rendering optimization. Since chalk writing and the board background have inherent differences in physical materials, introducing the chalk reflection priori conditions makes the estimated reflection parameters more consistent with the actual physical reflection laws of the smart blackboard. At the same time, the inverse rendering mechanism decouples the diffuse reflection component carrying the blackboard information from the specular reflection component corresponding to glare interference, avoiding a single linear processing of the overall image brightness, making glare suppression more spatially targeted, and helping to maintain the continuity of the writing edges and the content structure of the board.

[0013] This technical solution determines an adaptive suppression factor based on specular reflection weight, specular reflection component, ambient illuminance data, and the main viewing angle. This factor is then used to differentially attenuate the specular reflection component before fusing it with the diffuse reflection component. This allows for dynamic adjustment of the suppression level based on the actual specular intensity and observation geometry at different board locations. Consequently, a complete processing chain is constructed, from parametric modeling of the light field, physical reflection estimation, inverse rendering decoupling to adaptive specular compensation. This ensures the anti-glare process is based on rigorous geometric optics and material reflection property constraints, effectively preserving realistic blackboard stroke details while reducing glare interference, thus improving the clarity and readability of the digital image of the smart blackboard. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 A flowchart is shown as an example of an anti-glare rendering method for smart blackboard images based on three-dimensional light field modeling according to an embodiment of this application; Figure 2 A flowchart illustrating an example of estimating the bidirectional reflectance distribution function parameters corresponding to each image sampling point in a method according to an embodiment of this application is shown. Figure 3 A flowchart illustrating an example of determining the diffuse reflection component and specular reflection component corresponding to each image sampling point in a method according to an embodiment of this application is shown. Figure 4 A flowchart illustrating an example of obtaining an anti-glare rendered image of a target smart blackboard through fusion rendering in a method according to an embodiment of this application is shown. Figure 5 A schematic diagram illustrating the system operation mechanism of an example of a smart blackboard image anti-glare rendering method based on three-dimensional light field modeling according to an embodiment of this application is shown. Figure 6 A schematic diagram of the comparative experimental simulation results of different methods in anti-glare processing of smart blackboards is shown, illustrating the multidimensional performance comparison and Pareto front distribution of an example. Figure 7 This illustration shows a robustness test simulation of an example of the method provided in this application under different physical boundary conditions; Figure 8A structural block diagram of an example of a smart blackboard image anti-glare rendering system based on three-dimensional light field modeling according to an embodiment of this application is shown. Detailed Implementation

[0016] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0017] It should be noted that, at the current software processing level, smart blackboards or classroom blackboard digitization systems typically employ OpenCV-based image preprocessing workflows, such as grayscale conversion, filtering and noise reduction, thresholding, edge detection, perspective correction, and OCR (Optical Character Recognition) preprocessing. Similar blackboard acquisition and processing workflows can improve text contrast and image regularity under normal lighting conditions, but their processing objects are mainly two-dimensional pixel grayscale, edges, and local brightness distribution. When strong glare or brightness saturation occurs in certain areas of the board, reflected light spots, chalk marks, and light-colored backgrounds are easily compressed into the bright areas, making it difficult to recover the obscured stroke texture through subsequent thresholding or enhancement processing.

[0018] For highlight or glare areas in a single image, current technologies still rely on reflection separation as a core processing approach. Examples include single-image reflection decoupling methods based on color analysis, dark channel features, gradient sparsity, or local contrast, as well as SVG-like methods (Single-image Veiling Glare Removal). These methods typically attempt to decompose the input image into scene layers, diffuse layers, specular layers, or glare layers, then suppress reflective components or enhance underlying content. Because they do not require additional acquisition equipment, deployment costs are relatively low, but the algorithm's effectiveness is highly dependent on the prior conditions of the image. For smart blackboards, chalk writing, erasure marks, formula lines, and projected textures often simultaneously possess characteristics such as brightness, thin lines, strong gradients, and discontinuous structures, easily leading to confusion with specular reflection areas. This results in insufficient glare removal or incorrect weakening of the true edges of the writing.

[0019] In terms of hardware-assisted suppression, some medical imaging and industrial inspection studies employ LPF (Linear Polarized Filter) or multi-polarization state acquisition methods to control specular reflection. For example, LPF angle adjustment schemes switch between different polarization states by rotating the linear polarized filter, thereby reducing some strong reflections during the acquisition phase. This type of method is applicable to endoscopy, microscopy, or controlled optical inspection scenarios, but it usually requires additional optical components, mechanical rotation mechanisms, or time-division acquisition processes. For fixed-installation camera equipment in smart classrooms or conference rooms with a large coverage area and requiring continuous transmission, this type of solution increases the complexity of installation and debugging and may affect the continuity of acquisition and the stability of real-time interaction.

[0020] In terms of local restoration, some studies have proposed BC-SRDR (BrightnessClassification-based Specular Reflection Detection and Removal) methods. These methods divide the image into highlight areas, transition areas, and background areas based on pixel brightness or local contrast, and then use neighboring pixels, edge information, or texture synthesis to restore the highlight areas. While these methods can perform reflection detection and restoration at different brightness levels in endoscopic or natural images, chalk dust, discontinuous strokes, and erasure marks in smart blackboard images inherently possess fragmented, bright, and irregular texture characteristics. Pixel-level brightness classification can easily misjudge real writing structures as reflective noise. Furthermore, texture synthesis or image patch stitching processes may introduce mis-padding, breaks, or artifacts in character, formula, and line areas.

[0021] With the development of multi-view acquisition and light field imaging technologies, there are currently solutions that utilize light field cameras, microlens arrays, or multi-view acquisition devices to obtain information about the direction of light rays. These solutions typically use changes in brightness or color from different viewpoints to help identify specular reflection areas, providing richer observational information compared to single-image processing. However, in practical applications, these solutions often rely on dedicated imaging equipment or complex calibration conditions, and the data acquisition, reconstruction, and post-processing processes place high demands on computational resources, making them suitable primarily for controlled experimental environments or relatively static imaging objects. For smart blackboard scenarios with large surface areas, intricate text textures, and the requirement for continuous acquisition, limitations remain in terms of equipment cost, deployment complexity, and real-time processing capabilities.

[0022] In summary, current technologies have developed into various approaches, including OpenCV preprocessing workflows, SVG single-image reflection separation methods, LPF polarization suppression schemes, BC-SRDR brightness gradation restoration methods, and light field imaging multi-view analysis schemes. However, most of these methods rely on specific image priors, specific hardware conditions, or specific acquisition environments. For smart blackboard scenarios that simultaneously require multi-source illumination, board surface reflection, fine-grained chalk texture, large-format acquisition, and real-time transmission, the board image is still easily affected by the directionality of specular reflection, local brightness saturation, the complexity of text structure, and changes in the acquisition viewing angle, thus limiting the stable acquisition and high-quality presentation of digital blackboard content.

[0023] It should be understood that the above description of the relevant technologies is intended only to help the public better understand the inventive spirit and motivation of this application, and is not intended to limit this application. Furthermore, the technical solutions described in the above-mentioned relevant technologies are not prior art, and may also be undisclosed technical solutions, such as those under research or in the laboratory stage.

[0024] The technical solutions in this application, including the collection, storage, use, processing, transmission, provision, and disclosure of users' personal information, comply with relevant laws and regulations and do not violate public order and good morals.

[0025] Figure 1 A flowchart illustrating an example of an anti-glare rendering method for smart blackboard images based on three-dimensional light field modeling according to an embodiment of this application is shown.

[0026] Regarding the execution subject of the method in the embodiments of this application, it can be any controller or processor with computing or processing capabilities, such as the controller of a smart blackboard image acquisition and rendering processing platform. It implements the method of the embodiments of this application by running programs or instructions stored in a storage medium. In some examples, it can be integrated and configured in an electronic device or terminal through software, hardware, or a combination of both, and the type of terminal or electronic device can be diverse.

[0027] like Figure 1 As shown, in step S110, multi-view blackboard images, ambient illumination data and camera calibration parameters corresponding to each acquisition view are acquired synchronously for the target smart blackboard, and the main view image to be rendered is determined from the multi-view blackboard images, and the pixels in the main view image are used as image sampling points.

[0028] In practical implementation, the target smart blackboard can be a blackboard, whiteboard, or integrated display writing board used for writing, projection, or interactive display in smart classrooms, recording studios, remote interactive classrooms, or large conference rooms. Because such scenarios may simultaneously have natural light from windows, overhead lighting, side lighting, projection light, and reflected light from the display screen, glare patches can easily form on the blackboard surface in localized areas, causing chalk marks, blackboard lines, or displayed content to appear washed out, obscured, or reduced in readability. Therefore, multi-view image acquisition devices can be deployed in front of, above, to the side front of, or on the recording equipment bracket of the target smart blackboard to acquire images of the same target smart blackboard area from different viewing directions. It should be understood that the multi-view image acquisition device can include multiple camera units or an image acquisition structure capable of generating equivalent multi-view acquisition results.

[0029] To ensure that multi-view blackboard images correspond to the same writing state, synchronous exposure, unified triggering, or time synchronization can be used to control the synchronous acquisition of images from different viewpoints. For example, synchronous acquisition can reduce content discrepancies caused by time misalignment between images from different viewpoints when the teacher is writing, erasing, briefly obscuring the blackboard, or when the projected image changes. Ambient illumination data can be obtained by illumination acquisition units installed at the edge of the target smart blackboard, on the top of the classroom, on the side walls, or in the lighting system, and is used to characterize the light intensity or illumination changes in the environment in which the target smart blackboard is located. Camera calibration parameters corresponding to each acquisition viewpoint can be obtained through pre-calibration, and these parameters can include camera intrinsic parameters, distortion parameters, camera extrinsic parameters, relative pose between cameras, and acquisition viewpoint direction.

[0030] After acquiring multi-view blackboard images, the primary view image to be rendered can be determined from these images based on the needs of classroom display, recording output, remote streaming, or image recognition processing. For example, an image that faces the center of the target smart blackboard, has complete board coverage, minimal occlusion, and sufficient clarity for output can be selected as the primary view image. Using the pixels in the primary view image as image sampling points allows the processing results to directly correspond to the coordinates of the final output image, ensuring that the anti-glare rendering result is consistent with the actual display or transmission perspective. Therefore, by simultaneously acquiring multi-view images, ambient illumination data, and camera calibration parameters, the system can simultaneously grasp the multi-directional visual performance of the target smart blackboard, the ambient lighting conditions, and the geometric relationships of the acquisition, thereby improving the problem of insufficient information in glare judgment from single-view images.

[0031] In step S120, based on the multi-view blackboard images and camera calibration parameters, the target smart blackboard is reconstructed in three dimensions to obtain the three-dimensional geometric structure of the target smart blackboard and the spatial position of each image sampling point on the board surface.

[0032] In some implementations, the spatial pose relationship between different acquisition angles can be determined based on camera calibration parameters, and combined with the correspondence between multi-view blackboard images, the target smart blackboard can be reconstructed in three dimensions. While a smart blackboard is generally close to a planar structure, in a real installation environment, it may be affected by installation errors, frame structure, slight panel warping, support stress, lens distortion, or skewed shooting angles, causing projection differences for the same writing position in images from different angles. For example, when a camera located on the side wall or at the back of the classroom acquires the edge area of ​​the blackboard, perspective compression and local deformation are likely to occur. If pixels from different angles are directly compared in a two-dimensional image, the same chalk stroke or glare patch may not correspond accurately.

[0033] Based on this, the spatial constraints between multi-view images can be used to perform 3D modeling of the target smart blackboard area, obtaining a 3D geometric structure that can characterize the board's position, orientation, boundary range, and local geometric relationships. For each image sampling point in the main view image, the camera calibration parameters and the 3D geometric structure can be combined to map it from the main view image coordinates to the board space of the target smart blackboard, determining the corresponding board space position of that image sampling point. Here, the board space position can be used to represent the corresponding physical position of the pixel on the real board surface or the reconstructed board surface. Thus, through 3D reconstruction of the board surface, the sampling points in the 2D images are unified to the board space coordinates of the target smart blackboard, enabling the image content from different acquisition perspectives to correspond based on the physical position of the board surface, reducing matching errors caused by perspective distortion, shooting angle differences, and local geometric deviations.

[0034] In step S130, based on the three-dimensional geometric structure and camera calibration parameters, the pixel data in the multi-view blackboard image is remapped to the corresponding board surface spatial position and light direction to construct a four-dimensional light field model corresponding to the target smart blackboard; the four-dimensional light field model is used to characterize the radiation intensity information of each board surface spatial position under different light directions.

[0035] In some implementations, the three-dimensional geometry of the target smart blackboard can be used as a spatial reference to uniformly remap image pixels from different acquisition perspectives to their corresponding board surface spatial positions. For any image sampling point in the main view image, after determining its corresponding board surface spatial position, the light direction of that board surface spatial position relative to different acquisition perspectives can be determined based on the camera calibration parameters of each acquisition perspective. The corresponding pixel light intensity can then be extracted or obtained from the blackboard image of the corresponding acquisition perspective, thereby establishing the correspondence between the board surface spatial position, light direction, and radiant light intensity.

[0036] Through the above processing, a four-dimensional light field model corresponding to the target smart blackboard can be constructed. This four-dimensional light field model does not only record the image brightness from a single viewpoint, but also records the changes in radiated light intensity at the same spatial location on the blackboard under different light directions. For example, when the overhead lights in the classroom create local reflections in the upper part of the blackboard, this location may appear significantly brighter from a direct camera viewpoint, while its brightness may decrease from a side camera viewpoint; meanwhile, chalk strokes at the same location usually maintain a relatively stable structural appearance. The four-dimensional light field model can organize these light intensity characteristics that change with the viewing direction into a unified data structure.

[0037] Therefore, by constructing a four-dimensional light field model, the system can obtain the distribution of radiation intensity at the same spatial position on the board under different viewing directions, which enhances the ability to express directional reflection phenomena. It can reflect the difference in directional changes between specular glare and diffuse reflection of the blackboard content, thereby improving the completeness of the expression of image reflection characteristics.

[0038] In step S140, based on the radiation intensity information, the spatial position of the board surface corresponding to each image sampling point, the ambient illuminance data, and the preset micro-element reflection model, and combined with the chalk reflection a priori condition used to characterize the difference in reflectivity between the chalk area and the blackboard background area, the bidirectional reflectance distribution function parameters corresponding to each image sampling point are estimated; the bidirectional reflectance distribution function parameters include diffuse reflection weights and specular reflection weights.

[0039] In some implementations, the multi-directional radiant intensity information provided by the four-dimensional light field model, the spatial location of the board surface corresponding to the image sampling point, and the ambient illuminance data can be used together to estimate the surface reflection characteristics of the target smart blackboard. A pre-defined micro-element reflection model is used to describe the reflection response of the target smart blackboard surface under different incident illuminations and different viewing directions. It can characterize the basic scattering response of the board surface and the directional specular response affected by the surface microstructure. Based on the radiant intensity variation at the same spatial location of the board surface under different light directions, the diffuse reflection weight and specular reflection weight of the corresponding image sampling point can be estimated.

[0040] In a smart blackboard scenario, chalk strokes, the blackboard background, and the panel material typically exhibit different reflectivity. For example, chalk strokes are usually formed by powder particles adhering to the board surface, resulting in strong scattering; while glass, resin, or coated board materials are more likely to produce directional highlights under specific lighting and viewing directions. Since both white chalk strokes and specular glare can appear as bright areas in a single image, it is difficult to reliably distinguish them based solely on brightness. Therefore, by incorporating a priori conditions for chalk reflection, the parameter estimation process can reference the reflectivity difference between the chalk area and the blackboard background area.

[0041] Through the above processing, image sampling points that better match the reflective characteristics of chalk strokes can have their reflection parameters more inclined to reflect diffuse reflection contributions; for areas where light intensity varies significantly under different viewing directions and is strongly correlated with ambient lighting conditions, their reflection parameters can be more inclined to reflect specular reflection contributions. Thus, the problem of distinguishing bright areas in smart blackboard images is transformed into a reflection parameter estimation problem, enabling the system to identify the reflection attributes of image sampling points by combining material differences, lighting conditions, and multi-view light intensity variations, reducing the probability of misclassification between bright chalk strokes and specular glare.

[0042] In step S150, an inverse rendering optimization objective function is constructed based on the main view image, bidirectional reflection distribution function parameters, and four-dimensional light field model. By solving the inverse rendering optimization objective function, the diffuse reflection component and specular reflection component corresponding to each image sampling point are determined.

[0043] In some implementations, the imaging results of each image sampling point in the main view image can be considered as being formed by both diffuse reflection and specular reflection components. The diffuse reflection component mainly corresponds to the blackboard background, chalk lines, and other relatively stable board surface content; the specular reflection component mainly corresponds to the high-brightness reflection interference that is greatly affected by ambient light and viewing direction. Based on the aforementioned bidirectional reflection distribution function parameters and four-dimensional light field model, an inverse rendering optimization objective function can be constructed to perform inverse decomposition of the imaging response in the main view image.

[0044] For example, in a main-view image, top lighting or side window lighting may create a bright patch in a certain area of ​​the blackboard, obscuring localized writing. A four-dimensional light field model can reflect the light intensity changes in this area under different viewing directions, while the bidirectional reflectance distribution function parameters can characterize the reflectance properties of this area. The inverse rendering optimization objective function can integrate this information to separate stable blackboard content and directional highlight interference mixed in the same pixel representation, obtaining the diffuse reflection component and specular reflection component corresponding to each image sampling point. Thus, inverse rendering optimization achieves the separation of diffuse and specular reflection components in the main-view image, allowing the real blackboard content and glare interference to be expressed in different component forms. Compared to uniformly suppressing the brightness of the entire image, this embodiment can reduce the simultaneous weakening of blackboard details and improve the targeted nature of image component separation.

[0045] In step S160, based on the specular reflection weight, specular reflection component, ambient illumination data, and the observation direction corresponding to the main view image determined based on the camera calibration parameters, an adaptive suppression factor is determined for each image sampling point. The specular reflection component is attenuated using the adaptive suppression factor, and the diffuse reflection component is fused with the attenuated specular reflection component for rendering to obtain the anti-glare rendering image of the target smart blackboard.

[0046] In some implementations, the degree of specular reflection influence can be comprehensively evaluated for each image sampling point, and an adaptive suppression factor can be determined accordingly. This degree of influence can be related to the specular reflection weight, the intensity of the specular reflection component, ambient illumination data, and the viewing direction corresponding to the main viewpoint image. For example, when an image sampling point has a large specular reflection weight, a strong specular reflection component, high ambient illumination, and the viewing direction corresponding to the main viewpoint image is more likely to receive reflected light from that location, a stronger suppression effect can be configured for that image sampling point; for weakly reflective areas, areas with natural brightness variations, or normal blackboard writing areas, a weaker suppression effect can be configured.

[0047] After obtaining the adaptive suppression factor for each image sampling point, this factor can be used to attenuate the corresponding specular reflection component. The attenuated specular reflection component is then fused with the diffuse reflection component for rendering. This process doesn't simply remove the specular reflection component completely; instead, it adaptively weakens it based on the degree of reflection influence at each image sampling point. For example, when strong light from a side window or a projector causes strong local reflections on the blackboard, the specular reflection component in areas of strong glare can be more significantly suppressed; in areas with natural gloss or weak reflection, a more natural brightness transition can be preserved. Therefore, by performing pixel-level adaptive attenuation of the specular reflection component and fusing it with the diffuse reflection component, local glare can be reduced while maintaining the overall brightness continuity and visual naturalness of the blackboard. The resulting anti-glare rendered image improves the readability of the blackboard content and reduces local distortion, color abruptness, or texture breaks caused by simple specular highlight removal.

[0048] Figure 2 A flowchart illustrating an example of estimating the bidirectional reflectance distribution function parameters corresponding to each image sampling point in a method according to an embodiment of this application is shown.

[0049] like Figure 2 As shown, in step S210, the preset micro-surface element reflection model is configured as a reflection estimation model consisting of a diffuse reflection term and a specular reflection term. The diffuse reflection term is controlled by the diffuse reflection weight to be solved, while the specular reflection term is jointly determined by the specular reflection weight to be solved and the micro-optical geometric feature factor controlled by the roughness parameter to be solved. The micro-optical geometric feature factor is used to characterize the micro-surface element normal distribution, Fresnel effect, and geometric occlusion effect.

[0050] For example, the preset micro-surface element reflection model can be concretized using diffuse reflection weights. Specular reflection weight and roughness parameters The mathematical model for the variables to be solved, where the roughness parameter... One of the parameters of the bidirectional reflection distribution function: Equation (1) In the formula, Indicates the direction of the incident light ray and observation direction The surface bidirectional reflection distribution function is below. To correspond to the direction of the incident light and observation direction a half-angle vector, This represents the macroscopic normal vector at the corresponding spatial location of each image sampling point on the plate surface, determined based on the three-dimensional geometric structure. For roughness parameter The distribution function of the normal to the micro-surface element is controlled. For Fresnel neck, This refers to geometric occlusion.

[0051] In the reflection estimation model as shown in Equation (1), the diffuse reflection term describes the relatively stable basic scattering response of the target smart blackboard surface under different viewing directions, while the specular reflection term describes the directional specular response influenced by the micro-roughness structure, the incident light direction, and the viewing direction. For the incident light direction and viewing direction within the visible hemisphere, the model can represent the reflection response at the same image sampling point as a combination of diffuse reflection and specular reflection contributions. Thus, the diffuse reflection weight can be used to reflect the basic scattering degree at the corresponding board surface position of the image sampling point, and the specular reflection weight and roughness parameter can be used to reflect the tendency and diffusion degree of directional specular highlights at that position.

[0052] Therefore, by dividing the reflection response of the blackboard surface into diffuse reflection and specular reflection terms, the reflection estimation process can simultaneously express the optical performance of chalk strokes, blackboard background, and board material under different viewing directions. Compared to methods that only determine highlight areas based on image grayscale or color thresholds, this model can characterize the reflection properties of image sampling points from the perspective of surface reflection structure.

[0053] In step S220, the discretized incident illumination parameters of multi-source ambient light are determined based on the ambient illumination data, and the light transmission relationship is established in conjunction with the reflection estimation model to calculate the predicted radiant intensity of each image sampling point under different viewing directions.

[0054] In practical implementation, ambient illuminance data can include illuminance measurements at different locations around the target smart blackboard, illuminance variation trends, or lighting information related to the light source location. The system can combine the locations of illuminance sensors, the positions of known lighting fixtures or projection equipment in the classroom, the direction of window light, and differences in illuminance distribution to approximately discretize the ambient lighting around the target smart blackboard into incident illuminance parameters along multiple incident directions. These discretized incident illuminance parameters can be used to characterize the illuminance intensity along the corresponding incident direction, and can also include the weight of that discrete direction in the illuminance integral approximation.

[0055] For example, by combining the aforementioned bidirectional reflectance distribution function parameters, a system can be constructed targeting the observation direction. Predicted radiation intensity The equation for the transmission of light: Equation (2) In the formula, This is the index number of the discretized incident direction. Indicates the first A discrete incident ray direction, This represents the discretized incident illumination parameters in the corresponding direction. In some examples, It can be set to parameters that already include the light intensity and discrete weights for the corresponding discrete directions, so as to make a discrete approximation of the actual continuous lighting environment.

[0056] Specifically, the light transmission equation, as shown in Equation (2), is used to calculate the predicted radiant intensity of the image sampling point under a specific viewing direction based on the given incident illumination conditions, the normal relationship of the plate surface, and the parameters of the bidirectional reflection distribution function. By accumulating the reflection contributions of multiple discrete incident directions, the relationship between ambient illumination, plate surface reflection characteristics, and viewing direction can be transformed into a calculable predicted radiant intensity result.

[0057] Here, ambient illuminance data is linked to the reflectance estimation model, enabling the light intensity at image sampling points under different viewing directions to be represented by calculable ray transmission relationships. As a result, the system can obtain predicted radiant intensity for comparison with actual light field observations, improving the physical consistency of reflectance parameter estimation.

[0058] In step S230, the observed radiation intensity of each image sampling point under different acquisition angles is extracted from the four-dimensional light field model, and a data consistency term is constructed based on the measurement residual between the predicted radiation intensity and the observed radiation intensity.

[0059] Here, the four-dimensional light field model records the radiation intensity information of the same spatial position on the target smart blackboard under different light directions. For a certain image sampling point, the observed radiation intensity of the image sampling point under different acquisition angles can be extracted from the four-dimensional light field model according to its spatial position on the board and the observation direction corresponding to different acquisition angles. This reflects the actual light intensity performance of the board position in the real acquisition image under different observation directions.

[0060] For example, for the first Each acquisition perspective can be used to extract the observation direction. The observed radiation intensity And combined with the predicted radiation intensity under the corresponding observation direction Constructing data consistency items : Equation (3) In the formula, This refers to the index number of the acquisition perspective. Indicates pointing to the first The observation direction from each collection perspective. This represents the square operation of the 2-norm.

[0061] Here, the data consistency term in equation (3) is used to measure the deviation between the predicted radiation intensity obtained under the current reflection parameters and the actual observed radiation intensity in the four-dimensional light field model. When the deviation is large, it indicates that the current reflection parameters cannot well explain the multi-view observation results; when the deviation decreases, it indicates that the current reflection parameters are more consistent with the actual collected light field distribution. Thus, by constraining the reflection parameter estimation through multi-view observation of radiation intensity, the parameter solution no longer depends on the brightness performance of a single viewpoint, thereby improving the stability of the reflection estimation results under complex lighting conditions.

[0062] In step S240, prior constraint terms for constraining diffuse reflection weights and specular reflection weights are generated based on the prior condition of chalk reflection, and a joint regularized fitting function is constructed based on the data consistency term and the prior constraint terms.

[0063] It should be noted that since both chalk strokes and specular glare can appear as highlighted areas in a single image, if parameter fitting is performed solely based on the data consistency term, some chalk stroke areas may be misinterpreted as areas with strong specular reflection. To mitigate this misjudgment risk, a chalk reflection a priori condition can be introduced. This chalk reflection a priori condition characterizes the material reflection differences between chalk areas and blackboard background areas. For example, chalk strokes typically exhibit strong diffuse reflection characteristics, while specular glare usually shows a more pronounced direction dependence.

[0064] Based on this prior condition for chalk reflection, prior constraint terms can be generated to constrain the diffuse reflection weights and specular reflection weights. These constraints can be used to penalize combinations of reflection parameters that do not conform to the characteristics of chalk reflection, making the parameter estimation results more consistent with the material properties of chalk strokes and blackboard backgrounds in a smart blackboard scenario. Furthermore, the data consistency term and prior constraint term can be combined to form a joint regularized fitting function, subjecting the reflection parameter estimation to both multi-view observation consistency and material reflection priors. Therefore, by introducing the prior condition for chalk reflection into the reflection parameter fitting, the probability of confusion between chalk strokes and specular highlights due to their similar brightness can be reduced, making the estimation results of diffuse reflection weights and specular reflection weights more consistent with the actual material differences in a smart blackboard scenario.

[0065] In step S250, the diffuse reflection weight, specular reflection weight, and roughness parameter corresponding to each image sampling point are determined by optimizing the joint regularization fitting function.

[0066] In some implementations, nonlinear least squares, gradient optimization, or other optimization algorithms suitable for solving nonlinear fitting problems can be used to iteratively solve the joint regularization fitting function. During the iteration process, diffuse reflection weights, specular reflection weights, and roughness parameters are updated step by step as variables to be solved, reducing the deviation between the predicted and observed radiation intensity and ensuring that the solution conforms to the material reflection characteristics defined by the a priori conditions of chalk reflection. The optimization process can stop when a preset number of iterations is reached, the parameter change is below a preset threshold, or the value of the joint regularization fitting function tends to stabilize.

[0067] Through the above solution process, the corresponding diffuse reflection weight, specular reflection weight, and roughness parameter can be determined for each image sampling point. For example, the diffuse reflection weight characterizes the degree of diffuse reflection contribution at the corresponding board surface location of the image sampling point, the specular reflection weight characterizes the degree of specular reflection contribution at the corresponding board surface location, and the roughness parameter characterizes the influence of the micro-roughness at that location on the specular distribution morphology. Thus, by obtaining the reflection parameters at the image sampling point level through optimization, the reflection differences between different regions of the target smart blackboard surface can be quantitatively expressed. This result can reflect the differences in reflection characteristics between chalk stroke areas, blackboard background areas, and panel areas prone to glare, improving the interpretability and applicability of the bidirectional reflectance distribution function parameter estimation.

[0068] Regarding the details of generating prior constraint terms, in some examples of embodiments of this application, an image semantic segmentation model is invoked to perform chalk region recognition on the main viewpoint image, outputting a chalk region confidence score representing the probability that each image sampling point belongs to a chalk stroke region. For example, a chalk region confidence score matrix corresponding to each image sampling point is obtained, and the elements in the chalk region confidence score matrix are used to represent the probability value that the corresponding image sampling point belongs to a chalk stroke region.

[0069] In practical implementation, the image semantic segmentation model can employ convolutional neural networks, visual transformer networks, or other image segmentation models capable of performing pixel-level classification. This model can be trained using sample images containing chalk writing, blackboard background, local glare, projected lighting, and occluded areas, or it can utilize existing segmentation models adapted for the smart blackboard scenario. The model input can be the main viewpoint image, and the output can be a confidence matrix of chalk regions corresponding to pixel positions in the main viewpoint image. For each image sampling point, the chalk region confidence score can be used to represent the probability that the sampling point belongs to a chalk stroke region.

[0070] Since chalk strokes typically possess image features such as elongated edges, granular texture, and localized highlights, and are easily visually confused with specular highlights under strong lighting or glare, obtaining confidence scores for chalk regions through image semantic segmentation models helps transform the image semantic information of chalk regions into numerical weights that can participate in the fitting of reflection parameters. This provides an image location-level confidence expression for the prior conditions of chalk reflection, allowing prior constraints to be applied differentially based on the probability that different image sampling points belong to chalk stroke regions.

[0071] Then, using the confidence level of the chalk region as a dynamically adjusted weight, a prior constraint term is constructed that includes a specular suppression component and a diffuse reflection approximation component. Here, the specular suppression component is used to characterize the energy scale of the specular reflection weight, and the diffuse reflection approximation component is used to characterize the degree of deviation between the diffuse reflection weight and the preset chalk diffuse reflection reference weight.

[0072] In a smart blackboard scenario, chalk strokes are typically formed by coarse powder particles adhering to the board surface, resulting in diffuse reflection. However, the blackboard panel, glass covering, or coating layer is more likely to exhibit directional specular reflection under specific incident light and viewing directions. Therefore, when estimating the diffuse and specular reflection weights of image sampling points, the confidence level of the chalk region can be used to constrain the parameter fitting results. This makes image sampling points with higher chalk confidence more likely to have lower specular reflection weights, and their diffuse reflection weights closer to reference values ​​that conform to the empirical characteristics of chalk materials. For blackboard background areas with lower chalk confidence, the effect of the above prior constraints is relatively weakened, thereby reducing interference with the estimation of the true reflection parameters of the background area.

[0073] In some implementations, for pixel spatial coordinates The image sampling points and their corresponding prior constraints It can be expressed by the following formula: Equation (4) In the formula, Represents the pixel space coordinates of the image sampling points. and These represent the specular reflection weight and diffuse reflection weight at the corresponding coordinates. The confidence matrix for the chalk area corresponds to the coordinates The probability value, To preset the reference weights for chalk diffuse reflection, The preset mirror suppression coefficient, This is a preset diffuse reflection constraint coefficient. For example, and Positive values ​​can be taken to control the influence of the specular suppression term and the diffuse reflection approach term on the prior constraint terms, respectively.

[0074] In the above formula (4), This is used to penalize the increase in specular reflection weight, so that image sampling points with high chalk confidence tend to reduce specular reflection contribution during parameter solving; This is used to measure the deviation between diffuse reflection weights and preset chalk diffuse reflection reference weights, so that the diffuse reflection weights of corresponding image sampling points tend to be closer to the reference reflection level of the chalk material. Chalk region confidence. As spatial adjustment weights, they are used to control the strength of the effects of the two components at different image sampling points. When the confidence level of the chalk region at a certain image sampling point is high, the constraint effect of the prior constraint term on that image sampling point is strengthened; when the confidence level is low, the influence of the prior constraint term on that image sampling point is correspondingly weakened. Therefore, for a given image sampling point, the higher the confidence level of the chalk region, the greater the penalty applied by the prior constraint term in the joint regularization fitting function, so that the specular reflection weight corresponding to the image sampling point within the chalk stroke region tends to decrease in the solution iteration, and the corresponding diffuse reflection weight tends to approach the preset chalk diffuse reflection reference weight.

[0075] By incorporating the confidence level of the chalk region into the prior constraint term through the embodiments of this application, the reflection parameter fitting process can be adaptively adjusted according to the probability of the chalk region at the image sampling points. For chalk stroke regions, this constraint can reduce the possibility of them being misfitted as specular highlights; for blackboard backgrounds or non-chalk regions, the impact of this constraint is relatively weakened, allowing their reflection parameters to still be estimated based on multi-view observation light intensity and ambient illuminance data. Therefore, the discriminative power of diffuse reflection weights and specular reflection weights between chalk regions and background regions can be improved, and the stability of parameter estimation for chalk writing regions in strong light interference scenarios can be enhanced.

[0076] Figure 3 A flowchart illustrating an example of determining the diffuse reflection component and specular reflection component corresponding to each image sampling point in a method according to an embodiment of this application is shown.

[0077] like Figure 3 As shown, in step S310, based on the physical characteristics of the viewpoint consistency of diffuse reflection, the diffuse reflection component to be solved is set as a common variable of diffuse reflection that is independent of the acquisition viewpoint. Based on the physical characteristics of the directional differences in specular reflection, the specular reflection components to be solved under each acquisition viewpoint are set as specular reflection-specific variables related to the acquisition viewpoint. .

[0078] In some implementations, for the same spatial location on the target smart blackboard, diffuse reflection components such as chalk lines and blackboard background typically exhibit relatively stable structural behavior and consistent radiative response under different viewing angles. In contrast, specular reflection components are closely related to the spatial relationship between the incident light direction, the board surface normal, and the viewing direction, and may show significant intensity or spatial distribution differences under different viewing angles. Therefore, the diffuse reflection component can be set as a common variable that does not change with the viewing angle, while the specular reflection component can be set as a unique variable that changes with the viewing angle.

[0079] By setting the variables described above, the observed light intensity in the main view image and other acquired view images can be represented as a combination of stable diffuse reflection components and view-dependent specular reflection components. Therefore, by utilizing the difference in view response between diffuse and specular reflection, the hybrid imaging result is decomposed into variables with different physical meanings, allowing diffuse reflection content and specular glare to be represented separately within the same optimization framework.

[0080] In step S320, the observed radiation intensity at each acquisition angle is extracted using a four-dimensional light field model, and the prior specular reflection estimate at each acquisition angle is predicted based on the bidirectional reflection distribution function parameters, ambient illuminance data, and corresponding incident light information.

[0081] In practical implementation, it can be based on pixel spatial coordinates as The image sampling points are extracted from the four-dimensional light field model based on their corresponding spatial positions on the board surface. Observed radiation intensity from a single acquisition perspective The observed radiation intensity reflects the light intensity of the target smart blackboard at its corresponding spatial position on the board surface in the actual acquisition environment, directed towards the acquisition viewpoint.

[0082] On the other hand, the bidirectional reflectance distribution function parameters, ambient illuminance data, and incident light information corresponding to the acquisition angle obtained in the aforementioned steps can be combined to determine the image sampling point at the th... The specular reflection contribution from each acquisition viewpoint is predicted to obtain the prior specular reflection estimate. This estimate characterizes the directional specular reflection intensity that an image sampling point may produce under current reflection parameters and illumination conditions. By simultaneously obtaining the observed radiant intensity and the prior specular reflection estimate, the inverse rendering solution process can take into account both the actual acquired data and the physical reflection estimation results.

[0083] In step S330, an inverse rendering optimization objective function is constructed, which includes a multi-view data consistency fidelity term, a specular reflection approximation term, a total variation regularization term, and an anisotropic chalk texture preservation term.

[0084] In some implementations, the inverse rendering optimization objective function may include a multi-view data consistency fidelity term. Specular reflection approximation term Total variational regularization term and anisotropic chalk texture retention item And expressed by the following formula: Equation (5) Among them, the multi-perspective data consistency and fidelity item It can be represented as: Equation (6) Specular reflection approximation term It can be represented as: Equation (7) Total variational regularization term It can be represented as: Equation (8) Anisotropic chalk texture retention It can be represented as: Equation (9) In the formula, This indicates that the summation is performed by iterating through and summing the data at each image sampling point in the main viewpoint image. For collecting perspective indexes, This indicates that the summation is performed across all data collection perspectives. For the diffuse reflection common variables, For the variable specific to the specular reflection, The observed radiation intensity, The prior specular reflection estimate is given. This represents the spatial gradient of the diffuse common variables. Represents the norm operation. These are the preset approximation weight coefficients. The preset total variation weighting coefficients, Maintain the weight coefficients for the preset texture.

[0085] In the above inverse rendering optimization objective function, the multi-view data consistency and fidelity term is used to constrain the reconstruction consistency between the observed radiant intensity and the common variables of diffuse reflection and the corresponding specular reflection specific variables. The specular reflection approximation term is used to constrain the specular reflection specific variables to approach the corresponding prior specular reflection estimates. The total variation regularization term is used to suppress spatial noise and abrupt changes in the common variables of diffuse reflection.

[0086] More specifically, the multi-view data consistency and fidelity term is used to constrain the sum of the diffuse reflection common variables and the specular reflection specific variables under the corresponding acquisition viewpoints to be close to the actual observed radiant intensity, so as to maintain the interpretability of the decomposition results for multi-view observation data. The specular reflection approximation term is used to constrain the specular reflection specific variables to be close to the prior specular reflection estimates obtained from the bidirectional reflectance distribution function parameters and illumination information, so as to reduce the uncertainty in the solution process of the specular components. The total variational regularization term is used to suppress local noise and unreasonable abrupt changes in the diffuse reflection common variables, so that the diffuse reflection components have better spatial continuity. The anisotropic chalk texture preservation term is used to preserve the structural features in the direction of chalk strokes while suppressing noise, so that the diffuse reflection components can better represent the blackboard texture.

[0087] Therefore, by incorporating observation data consistency, specular reflection prior, spatial smoothness constraints, and chalk texture preservation constraints into the same optimization objective, constrained joint estimation of diffuse and specular reflection components can be achieved. Based on the objective function provided in this embodiment, the ambiguity in single image decomposition can be reduced, ensuring that the separation results simultaneously conform to multi-view light intensity observations and plate surface reflection characteristics.

[0088] In step S340, the inverse rendering optimization objective function is solved by alternating iterative optimization, and the diffuse reflection common variables obtained by the solution are determined as the diffuse reflection component corresponding to the main view image, and the specular reflection specific variables obtained by the solution under the acquisition view corresponding to the main view are determined as the specular reflection component corresponding to the main view image.

[0089] In some implementations, diffuse reflection common variables can be used. and the unique variables of specular reflection from each acquisition perspective For the objective function to be solved, an alternating iterative optimization process is performed. For example, the diffuse reflection common variables can be updated while keeping the specular reflection-specific variables fixed, and the specular reflection-specific variables can also be updated while keeping the diffuse reflection common variables fixed. This update process can be implemented using gradient optimization, alternating minimization, block coordinate descent, or other numerical optimization methods suitable for this type of objective function.

[0090] During the iterative solution process, the convergence condition can be determined based on the changes in the objective function value, the changes in the variables to be solved, the number of iterations, or the changes in the residuals. If the preset convergence condition is met, the diffuse reflection common variables obtained from the solution can be... As the diffuse reflection component of the corresponding image sampling point in the main view image, and using the obtained main view corresponding acquisition view index as the solution. Specular reflection-specific variables Determine the specular reflection component corresponding to the main viewpoint image.

[0091] In this embodiment, by updating the common variables of diffuse reflection and the specific variables of specular reflection separately through alternating iterations, the computational difficulty of joint solution can be reduced, and various constraints can be gradually reflected in the component estimation results during the iteration process. Thus, the obtained diffuse reflection component can represent the relatively stable surface content in the main view image, and the obtained specular reflection component can represent the more significant highlight interference that changes with the viewing direction in the main view image, thereby improving the distinguishability of the two types of reflection components in the main view image.

[0092] Regarding the preservation of texture in anisotropic chalk In some examples of embodiments of this application, the construction details are as follows: the local structure tensor of the main view image at each image sampling point is calculated, and the local structure tensor is decomposed into features. The feature vector corresponding to the minimum feature value obtained by decomposition is determined as the tangential direction vector of the chalk stroke, and the feature vector corresponding to the maximum feature value is determined as the normal direction vector of the chalk stroke.

[0093] In some implementations, the spatial gradients of the main viewpoint image in the horizontal and vertical directions can be calculated first, and an initial tensor representing the direction of local gray-level changes can be constructed based on these spatial gradients. To reduce the impact of image acquisition noise, random fluctuations in chalk grain texture, or local glare residue on direction estimation, a smoothing kernel of a preset scale can be used to smooth the neighborhood of the initial tensor to obtain the local structure tensor at the corresponding image sampling point. For example, the local structure tensor... It can be obtained through the following expression:

[0094] Equation (10) In the formula, The scale parameter is represented as Gaussian smoothing kernel, This represents the spatial gradient vector at the corresponding location in the main view image. The symbol * indicates the outer product operation, and the symbol * indicates the convolution operation.

[0095] For local structure tensor After eigenvalue decomposition, we obtain maximal eigenvalues, minima, and their corresponding eigenvectors. Typically, the eigenvector corresponding to a maximal eigenvalue reflects the direction of strong local gray-level changes; this direction can be used as the normal direction across the chalk stroke edge. The eigenvector corresponding to a minima reflects the direction of weak local gray-level changes; this direction can be used as the tangential direction extending along the chalk stroke. Thus, the local texture structure at image sampling points can be transformed into mutually orthogonal tangential and normal direction vectors, providing directional guidance for the anisotropic chalk texture preservation term.

[0096] Then, the marginal confidence is calculated based on the numerical difference between the maximal and minimal eigenvalues, and an anisotropic adjustment coefficient for characterizing directional consistency is constructed based on the marginal confidence.

[0097] For example, edge confidence It can be determined based on the normalized difference between the maxima and minima, which can be calculated using the following formula: Equation (11) In the formula, and These represent the local structure tensor in coordinates. The corresponding maxima and minima, A preset positive number to avoid denominators that are zero or too small. Marginal confidence level. It can be used to characterize the strength of directionality in local regions. When the image sampling point is located at the edge of a chalk stroke or in a thin, elongated stroke area, the local grayscale changes usually have a clear main direction, the difference between the maximum and minimum eigenvalues ​​is large, and the corresponding edge confidence is high; when the image sampling point is located on a flat blackboard background or in an area with indistinct directionality, the difference between the two eigenvalues ​​is small, and the corresponding edge confidence is low.

[0098] In some examples, edge confidence can be... Directly used as anisotropy adjustment coefficient Alternatively, anisotropic adjustment coefficients can be obtained by performing linear mapping, threshold truncation, or smoothing nonlinear mapping on the marginal confidence scores. Its value range can be limited to between zero and one, and it is used to adjust the constraint strength of the chalk texture preservation term in the tangential and normal directions.

[0099] Furthermore, using the anisotropic adjustment coefficient and its complementary coefficient as weights, the tangential gradient components of the diffuse reflection common variables along the tangential direction vector and the normal gradient components along the normal direction vector are weighted and aggregated to construct the anisotropic chalk texture preservation term. Here, when an image sampling point is determined to be at the edge of a chalk stroke, its corresponding edge confidence increases, thereby driving the anisotropic chalk texture preservation term to adaptively enhance the continuity constraint along the tangential direction and reduce the smoothing penalty along the normal direction during the optimization process.

[0100] For example, diffuse reflection shared variables Anisotropic chalk texture preservation item It can be expressed by the following formula: Equation (12) In the formula, This represents the anisotropy adjustment coefficient determined by the marginal confidence level. This represents the tangential gradient component of the diffuse reflection common variable along the tangential direction vector. This represents the normal gradient component of the diffuse reflection common variable along the normal direction vector. This represents the norm operation used to measure the magnitude of the gradient components.

[0101] In the anisotropic chalk texture preservation term of equation (12) above, different local structural regions have different spatial constraint methods through the weighted combination of tangential gradient components and normal gradient components. When the image sampling point is in a chalk stroke region with strong directionality, the anisotropic adjustment coefficient increases, and the constraint weight corresponding to the tangential gradient component increases accordingly, thereby promoting the continuity of diffuse reflection common variables along the chalk stroke extension direction; at the same time, the constraint weight corresponding to the normal gradient component decreases relatively, thereby reducing the smoothing penalty across the chalk stroke edge direction, making the stroke edge less prone to over-smoothing. For the background region with weak directionality, the anisotropic adjustment coefficient is relatively small, and the emphasis of the chalk texture preservation term on directional differences is reduced, making the constraint of this region closer to the conventional spatial smoothing constraint.

[0102] This embodiment of the application utilizes local structure tensors to extract the directional information of chalk strokes and employs anisotropic adjustment coefficients to differentially constrain the tangential and normal gradients. This approach balances noise suppression and stroke edge preservation during the optimization of diffuse reflection common variables. Compared to methods that apply the same smoothing constraints to all directions, this embodiment reduces the risk of over-smoothing chalk stroke details and improves the preservation of chalk texture structure in diffuse reflection components.

[0103] Regarding the implementation details of determining the adaptive suppression factor for each image sampling point in the method of this application embodiment, in some examples of this application embodiment, the current ambient illuminance value is extracted from the ambient illuminance data, the ambient light incident direction is determined by combining the four-dimensional light field model, and based on the ambient light incident direction, the three-dimensional geometric structure of the target smart blackboard and the observation direction corresponding to the main view image, the spatial angle between the ideal reflection direction vector and the actual line-of-sight direction vector of the ambient light at each image sampling point is calculated, and the spatial angle is used as the observation deviation angle.

[0104] In practical implementation, to accurately capture the directional characteristics of specular glare, the system needs to perform geometric deduction based on the law of reflection in physical optics. First, utilizing the three-dimensional geometric structure of the target smart blackboard, the normal vector at the spatial position of the current image sampling point on the board surface is obtained. Subsequently, based on the incident direction vector of the ambient light... Calculate the ideal reflection direction vector of ambient light at that point:

[0105] Equation (13) Building upon this, to avoid numerical collapse caused by the inverse cosine function input exceeding its bounds due to floating-point calculation precision errors, the actual viewing direction vector of the main viewpoint image is calculated. With the ideal reflection direction vector When calculating the spatial angle, both are first normalized, and the dot product result is then clipped across intervals to obtain the corresponding observation deviation angle. : Equation (14) In the formula, and These represent the ideal reflection direction vector and the actual line-of-sight direction vector, respectively, after normalization. This indicates that the dot product result is constrained to... Within the interval. Through the above calculation process, the abstract three-dimensional spatial orientation is transformed into an angle scalar with numerical stability, providing a rigorous geometric measurement benchmark for determining whether the camera is in a highly reflective area.

[0106] Then, the specular reflection component corresponding to the main view image is normalized to obtain the normalized specular reflection intensity.

[0107] Here, the acquisition viewpoint index corresponding to the main viewpoint image is determined to be... The specular reflection component corresponding to the main view image Normalization is performed to obtain the normalized specular reflection intensity. In practice, the specular reflection component output from the aforementioned inverse rendering optimization solution is usually an absolute radiant energy value with a large dynamic range. Directly applying it to a nonlinear attenuation model may cause numerical overflow or underflow problems. Therefore, maximum value normalization and the introduction of a stabilization term are used to scale the amplitude of the main viewpoint specular reflection component. This is specifically achieved through the following operations:

[0108] Equation (15) Obtain the normalized specular reflection intensity mapped to the interval [0, 1]. Among them, the denominator contains This represents the global maximum value of the specular reflection component amplitude corresponding to all image sampling points within the main viewpoint image. To avoid a preset positive stable term where the denominator is zero when there is no global highlight, this operation not only eliminates the dimensional differences caused by different exposure conditions or light source power, but also ensures the numerical stability of the calculation model under different ambient illuminance.

[0109] Subsequently, the specular reflection weight, normalized specular reflection intensity, relative illuminance determined by the ratio of the current ambient illuminance value to the preset reference illuminance value, and non-negative directional consistency response determined by the observation deviation angle are synergistically coupled to construct a comprehensive suppression driving force. Then, using a preset exponential decay model, the comprehensive suppression driving force is mapped to an adaptive suppression factor.

[0110] In some implementations, adaptive inhibition factor It can be expressed by the following formula: Equation (16) In the formula, This represents the normalized specular reflection intensity at the corresponding image sampling point. This represents the current ambient illuminance value. The preset reference illuminance value, and These are preset dynamic attenuation control parameters, and , , The adaptive suppression factor is monotonically decreasing with respect to the comprehensive suppression driving amount, and decreases exponentially as the comprehensive suppression driving amount increases, so as to apply stronger attenuation suppression to the specular reflection component of the corresponding image sampling point.

[0111] In the above equation (16), a multi-dimensional conditionally driven specular attenuation function is constructed, where the product term constitutes the comprehensive suppression driving quantity: when the specular reflection property of the image sampling point is strong, the actual separated specular energy is high, and the viewing direction and the specular reflection direction are highly overlapping (i.e. The overall suppression driving force reaches its peak when the value approaches 1 and the ambient light is bright. Subsequently, through a negative exponential function... The driving quantity is smoothly mapped to an adaptive suppression factor between (0, 1). Through the above parameter configuration, this embodiment can apply a significant nonlinear attenuation to the direct strong glare area, while keeping the suppression factor close to 1 for the side light or diffuse reflection area, thereby achieving smooth and seamless highlight suppression processing.

[0112] This application's embodiments comprehensively utilize spatial geometric line-of-sight relationships, inherent material reflectivity, and real-time ambient light information to implement differentiated exponential attenuation processing for the specular reflection intensity at different image sampling points on the blackboard surface. This avoids image area distortion and visual texture fragmentation caused by a single, rigid global brightness cutoff. While effectively reducing local high-frequency glare interference, it fully preserves the natural gloss transition of the target smart blackboard surface, improving the visual clarity and physical fidelity of anti-glare rendering output under complex multi-source lighting environments.

[0113] Figure 4 A flowchart illustrating an example of obtaining an anti-glare rendered image of a target smart blackboard through fusion rendering in a method according to an embodiment of this application is shown.

[0114] like Figure 4 As shown, in step S410, the diffuse reflection component is normalized, and the normalized diffuse reflection component is enhanced by a preset piecewise nonlinear mapping function to obtain the enhanced diffuse reflection component.

[0115] Here, the piecewise nonlinear mapping function is configured as follows: for dark regions with brightness below the first brightness threshold, nonlinear stretching based on the first gamma coefficient is performed; for bright regions with brightness above the second brightness threshold, nonlinear compression based on the second gamma coefficient is performed; and for intermediate brightness regions between the first brightness threshold and the second brightness threshold, linear mapping is maintained.

[0116] In some implementations, although the diffuse component obtained through inverse rendering has largely eliminated specular glare interference, its brightness distribution may still be affected by uneven ambient illumination, local shadows on the panel surface, and camera exposure response, resulting in an overall dark appearance, insufficient local contrast, or unclear brightness levels. To facilitate uniform brightness adjustment of the diffuse component, it can be mapped to a preset normalization range, such as a normalized floating-point range from zero to one, to obtain a normalized diffuse component. .

[0117] After obtaining the normalized diffuse reflection component, a preset piecewise nonlinear mapping function can be used to perform brightness enhancement processing on different brightness ranges. Specifically, for dark areas with brightness below a first brightness threshold, a first gamma coefficient is used for nonlinear stretching to improve the brightness and legibility of the dark writing area; for intermediate brightness areas with brightness between the first and second brightness thresholds, the original brightness relationship remains unchanged; for bright areas with brightness above the second brightness threshold, a second gamma coefficient is used for nonlinear compression to avoid detail loss caused by excessive local brightness. For example, the piecewise nonlinear mapping function can be expressed by the following formula:

[0118] Equation (17) In the formula, coordinates Enhanced diffuse reflection component at that location, coordinates The normalized diffuse reflection component at that location, and These are the preset first brightness threshold and the second brightness threshold, respectively. , The first gamma factor is used to enhance the dark areas, and , This is the second gamma factor used to compress the highlighted areas, and .

[0119] In the piecewise nonlinear mapping function as shown in Equation (17), different adjustment methods are used in the dark area, the intermediate brightness area, and the bright area, respectively, so that the diffuse reflection component can improve the visibility of the dark blackboard writing without significantly destroying the continuity of brightness levels, and control the brightness growth of the bright area. As a result, the brightness distribution and local contrast of the diffuse reflection component are improved, so that the blackboard writing content has a more suitable visual appearance before fusion rendering.

[0120] In step S420, the intensity of the corresponding specular reflection component is attenuated by using an adaptive inhibition factor as an adjustment weight to obtain the attenuated specular reflection component.

[0121] Here, the adaptive inhibition factor The value of the adaptive suppression factor can range from zero to one, or be within a preset suppression interval containing one but not exceeding one. For image sampling points with strong specular reflection effects, a smaller adaptive suppression factor value results in a larger attenuation of the corresponding specular reflection component; conversely, for image sampling points with weak specular reflection effects, a larger adaptive suppression factor value results in a relatively higher retention of the corresponding specular reflection component. Based on this, the specular reflection component corresponding to the main viewpoint can be attenuated pixel-by-pixel according to the adaptive suppression factor, allowing areas with strong specular reflection effects to achieve a larger attenuation and areas with weak specular reflection effects to retain a relatively natural brightness transition.

[0122] Compared to applying a uniform constant to weaken the specular reflection component across the entire image, product attenuation processing can implement differentiated adjustments based on the glare intensity at different image sampling points. To ensure consistency between the specification and claims in the fusion rendering logic, and to avoid using the normalized specular reflection intensity as a scalar indicator... Misused as an image layer variable, this embodiment first uses the absolute specular reflection component corresponding to the main viewpoint. The entire image is linearly remapped to the [0, 1] normalized pixel range consistent with the enhanced diffuse reflection component to obtain the normalized specular reflection component corresponding to the main view image. Subsequently, an adaptive inhibition factor was used. As an adjustment weight, it is used in conjunction with the standardized specular reflection component. Perform pixel-by-pixel multiplication to obtain the attenuated specular reflection component.

[0123] In step S430, the enhanced diffuse reflection component and the attenuated specular reflection component are superimposed and fused for rendering. A preset clipping function is used to constrain the dynamic range of the fusion result to a legal standardized pixel range to obtain the rendering pixel values ​​of the anti-glare rendering image.

[0124] Considering that the imaging response of the target smart blackboard surface can be regarded as the superposition of diffuse reflection contribution and specular reflection contribution, the diffuse reflection component can be enhanced. The attenuated specular reflection component is then superimposed pixel-by-pixel to generate the fused rendering result. To ensure that the final output conforms to standard image bit-depth encoding, a cropping function is further used to constrain the dynamic range of the fused result to a legal standardized pixel range.

[0125] For example, the pixel values ​​of an anti-glare rendered image It can be expressed by the following formula: Equation (18) In the formula, This indicates a cropping function that constrains the overlay and fusion result to a preset normalized pixel range. As an adaptive inhibitory factor, The normalized specular reflection component corresponding to the main viewpoint image, and its relationship with the enhanced diffuse reflection component. They are within the same range of normalized pixels.

[0126] Through the above-described overlay, fusion, and dynamic range constraint processing, an anti-glare rendered image that balances readability of the blackboard and overall brightness continuity can be obtained. This step aims to recombine the enhanced diffuse reflection information with the suppressed specular reflection information into the output image, and control the output pixel values ​​within the displayable range to reduce the impact of overflow, saturation, or abnormal brightness truncation on the image display effect.

[0127] In step S440, the global specular residual energy index of the anti-glare rendered image is calculated, and when the global specular residual energy index exceeds the preset safety threshold, a feedback control command is generated and sent to the heterogeneous lighting control link to form a closed-loop adjustment of the dynamic lighting parameters of the target smart blackboard environment.

[0128] In practical implementation, under certain extreme lighting scenarios, such as strong direct sunlight from the side, high-power fill light at close range, or excessive brightness in the projection equipment, even after specular reflection suppression and blending rendering, significant specular residue may still be retained in the anti-glare rendered image. To quantify the degree of this residual specular highlight, a global specular residue energy assessment can be performed on the anti-glare rendered image.

[0129] For example, pixel energy exceeding a preset specular threshold can be used as residual specular energy, and its normalized ratio to the total energy of the entire image can be calculated. Global Specular Residual Energy Ratio It can be expressed, for example, by the following formula:

[0130] Equation (19) In the formula, To preset the highlight threshold, A preset positive number to avoid a denominator of zero. This indicates the operation of taking the non-negative part. Equation (19) above measures the proportion of residual energy in the part exceeding the specular threshold in the overall image energy of the anti-glare rendered image.

[0131] In some implementations, the global specular residual energy ratio can be determined by statistically analyzing the proportion of bright pixels, the proportion of pixels at saturation thresholds, or other quantitative methods that reflect the degree of specular residual. When the global specular residual energy ratio is determined to be greater than a preset safety threshold, a feedback control command can be generated and sent to the lighting adjustment device related to the target smart blackboard environment via a heterogeneous lighting control link. The lighting adjustment device may include local luminaires, dimmable fill lights, smart curtains, blackout curtains, electrochromic glass control units, or display panel backlight control units in a classroom lighting system. The feedback control command can be used to adjust the brightness, on / off state, illumination angle, or backlight intensity of the corresponding device, thereby reducing the impact of ambient lighting that causes specular residual. Thus, the degree of residual specular in the anti-glare rendering result is quantitatively evaluated, and when the residual specular exceeds the allowable range, the impact of specular on the board surface is further reduced through ambient lighting adjustment. This can improve the system's adaptability to changes in strong light environments and improve the stability and clarity of the target smart blackboard image under extreme lighting conditions.

[0132] Through the embodiments of this application, by performing piecewise nonlinear enhancement on the diffuse reflection component, adaptive attenuation on the specular reflection component, constrained fusion rendering of the two, and combining global specular residual energy ratio for illumination feedback adjustment, the output anti-glare rendering image achieves a more balanced processing result in terms of whiteboard readability, brightness level and specular control.

[0133] Figure 5 A schematic diagram illustrating the system operation mechanism of an example of a smart blackboard image anti-glare rendering method based on three-dimensional light field modeling according to an embodiment of this application is shown.

[0134] like Figure 5As shown, the system front-end can simultaneously acquire multi-view images of the target smart blackboard and ambient illuminance data through a multi-view camera array and an illuminance sensor. After the multi-view blackboard images and ambient illuminance data enter the processing flow as input data, geometric reconstruction and light field modeling, microscopic BRDF (Bidirectional Reflectance Distribution Function) estimation, specular and diffuse reflection separation, anti-glare rendering, and chalk content enhancement and fusion processing can be performed sequentially. Specifically, geometric reconstruction and light field modeling are used to uniformly map pixel data from different acquisition perspectives to the spatial position and light direction of the board surface; microscopic BRDF estimation is used to determine the reflection parameters corresponding to the image sampling points; and specular and diffuse reflection separation is used to distinguish stable diffuse reflection content from directional specular reflection components in the image.

[0135] During the separation of specular and diffuse reflection, the prior a priori information of chalk reflection can be used as constraint information in reflection parameter estimation or component separation processing to reduce the probability of chalk stroke areas being misidentified as specular reflection areas. After the separated diffuse and specular reflection components enter the anti-glare rendering module, adaptive suppression parameters are determined based on specular reflection weights, specular reflection components, ambient illuminance data, and the main viewing angle, and the specular reflection component is adaptively attenuated. Subsequently, the system enhances the diffuse reflection content and fuses it with the attenuated specular reflection component to obtain the anti-glare rendered image. The anti-glare rendered image can be used to generate high-quality anti-glare images or output to real-time interactive display devices.

[0136] In some implementations, the system can also include an adaptive control and linkage module. This module can output dynamic parameter adjustment information to the anti-glare rendering module based on the anti-glare rendering results, ambient illuminance data, or specular remnant status to adjust the specular reflection suppression intensity or blending rendering parameters. When it detects that the residual specular highlights caused by ambient lighting exceed a preset range, it can also send a closed-loop lighting control command to the classroom lighting system to adjust the dynamic lighting parameters of the environment where the target smart blackboard is located. Through the above processing, the system can jointly reduce the glare effect in the smart blackboard image on both the image processing side and the ambient lighting side, improving the readability and display stability of the blackboard content.

[0137] To verify the effectiveness and physical consistency of the anti-glare rendering method based on 3D light field modeling provided in this application, a multi-source lighting test platform was constructed in a standard smart classroom. For example, the target smart blackboard has dimensions of 3 m × 1.2 m. Image acquisition utilizes a 3×3 miniature industrial camera array with global shutter synchronization. The resolution of a single camera unit is 1920×1080, and the parallax baseline, after calibration, is approximately 5.0 ± 0.1 cm to reduce image distortion caused by line-by-line exposure during dynamic blackboard writing. Ambient lighting consists of a tunable full-spectrum LED array and a side-projection light source, and real-time ambient illuminance is acquired through illuminance sensors distributed along the edge of the blackboard. The illuminance sensors have a range of 0 to 5000 Lux.

[0138] In the experimental verification phase, three typical benchmark methods were selected and their performance compared with the method in the embodiments of this application. The specific benchmark comparison methods include: first, a brightness gradation restoration method, which mainly employs a threshold segmentation and texture synthesis algorithm based on local image contrast; second, a polarization modulation method, which, as a physical hardware solution, mainly suppresses specular reflection by dynamically rotating a polarizer using a stepper motor; and third, a single-image depth reflection separation method, which relies on an end-to-end deep learning network based on gradient sparsity priors for reflection separation. In contrast, the method provided in the embodiments of this application integrates microscopic BRDF estimation and four-dimensional light field inverse rendering processing to achieve anti-glare rendering for smart blackboard scenes.

[0139] In addition to conventional evaluation metrics such as Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index Measure (SSIM), this experiment further introduces Optical Character Recovery Rate (OCR-RR), Specular Residual Energy Ratio (SRE), and system end-to-end latency to comprehensively evaluate the engineering applicability of the proposed method from three aspects: usability of teaching content, glare suppression effect, and real-time processing performance. Specifically, the OCR-RR characterizes the recognizability of blackboard text after anti-glare rendering. Under a uniform OCR confidence threshold, it is calculated as the proportion of valid chalk characters successfully recognized and matching the actual blackboard content within the original glare coverage area to the total number of original characters in that area. A higher OCR-RR indicates a stronger ability to recover the underlying blackboard information, which is more beneficial for classroom recording, remote viewing, and subsequent text recognition processing.

[0140] The specular residual energy ratio (SRE) measures the degree of residual specular reflection interference after rendering. It can be defined as the ratio between the pixel luminance integral of the residual specular region in the rendered image after normalization and the luminance integral of the corresponding specular region in the original image. The lower the index, the smaller the residual glare energy and the more obvious the specular reflection suppression effect. The system end-to-end latency characterizes the real-time computational efficiency of the method in this application. Its statistical range can be from the start of synchronous triggering acquisition of multi-view image arrays to the final output of anti-glare rendered image, covering the processing processes such as light field data construction, reflection parameter estimation, inverse rendering optimization solution, and fusion rendering. The lower the index, the faster the system response speed, and the more suitable it is for application in real-time classroom interaction, online teaching, and recording display scenarios.

[0141] Figure 6 The diagram illustrates the comparative experimental simulation results of different methods in anti-glare processing of smart blackboards, showing a multidimensional performance comparison and an example of Pareto front distribution.

[0142] like Figure 6 The radar chart on the left shows the comprehensive performance of various methods across four dimensions: Peak Signal-to-Noise Ratio (PSNR), Structural Similarity (SSIM), Optical Character Reconstruction Rate (OCR-RR), and Inverted Spectral Residual Energy Ratio (1-SRE). Among these, the polarization modulation method has certain advantages in image fidelity, but its character reconstruction rate and spectral residual control are still affected by the physical adjustment method and scene adaptability. Image inpainting and depth reflection methods have high processing efficiency, but their character reconstruction rate and spectral residual suppression effects are relatively limited in complex chalk textures, locally saturated highlights, and blackboard edge regions. In contrast, the method provided in this application combines microscopic bidirectional reflectance distribution function estimation with four-dimensional light field inverse rendering processing, achieving a better overall balance between visual fidelity, character reconstruction effect, and spectral residual control.

[0143] like Figure 6 The scatter plot on the right further illustrates the distribution relationship between image fidelity PSNR and system latency for each method. The polarization modulation method achieves a high PSNR, but its system latency is relatively large due to its reliance on the mechanical adjustment of polarization devices. The image inpainting and depth reflection methods have lower latency, but relatively lower PSNR, indicating that their ability to recover physical information in complex glare areas remains insufficient. The method provided in this application embodiment achieves a system end-to-end latency of approximately 65ms and a PSNR of approximately 32.4dB under experimental conditions, demonstrating a better trade-off between image fidelity and processing latency compared to the benchmark method. Combined with the Pareto front distribution in the figure, it can be seen that the method provided in this application embodiment maintains high image fidelity while controlling system latency within a range close to real-time interaction requirements, making it suitable for applications such as smart blackboard recording, remote display, and classroom interaction.

[0144] Figure 7 The diagram illustrates the robustness test simulation effect of an example of the method provided in the embodiments of this application under different physical boundary conditions.

[0145] like Figure 7 As shown, this three-dimensional surface plot illustrates the changes in image rendering quality of the method provided in this application under conditions of varying ambient illuminance and light source incident angle. One horizontal axis represents the light source incident angle, another represents the ambient illuminance, and the vertical axis and surface color mapping are used to characterize the PSNR of the rendered image. By constructing a joint parameter space covering different ambient illuminance and different light source incident angles, the performance trends of the method in this application under normal lighting conditions and strong reflection conditions can be observed.

[0146] Depend on Figure 7 As can be seen, in typical teaching scenarios, such as when the ambient illuminance is below 1200 Lux and the incident angle of the light source is less than 60 degrees, the PSNR remains at a relatively high level and fluctuates little with parameter changes. This indicates that the method in this embodiment can maintain a relatively stable anti-glare rendering effect under common classroom lighting conditions. When the incident angle of the light source increases to near the grazing direction and the ambient illuminance further increases, the PSNR decreases significantly, indicating that under the combined effect of high illuminance and a large incident angle, the specular reflection of the board surface is enhanced, which will affect the image rendering quality to some extent.

[0147] From an optical imaging perspective, under conditions of large incident angles and high illumination, the directional reflectance component of the target smart blackboard surface will increase. Simultaneously, local areas may experience saturation, occlusion, or intensified changes in reflectance distribution, increasing the difficulty of specular reflection estimation and diffuse reflection recovery. Therefore, Figure 7 The nonlinear descent region in the diagram can be understood as the performance boundary of the method under extreme lighting conditions. Since the lighting layout of actual smart classrooms typically avoids strong light directly illuminating the blackboard at a large grazing angle, this type of condition is generally not part of regular teaching use. Therefore, under common classroom lighting conditions, the method provided in this embodiment can maintain good rendering stability and anti-glare processing effects.

[0148] In summary, to address the blackboard glare problem in multi-light source, complex texture teaching scenarios, the anti-glare rendering method for smart blackboard images based on 3D light field modeling proposed in this application can construct a four-dimensional light field model using multi-view images and combine it with micro-facet bidirectional reflectance distribution function estimation to parametrically characterize the reflective properties of the blackboard surface. Based on this, through inverse rendering optimization under physical constraints, diffuse reflection components and specular reflection components are separated, and combined with adaptive suppression and piecewise nonlinear enhancement processing, specular glare interference is reduced while maintaining the readability of chalk stroke texture and blackboard content. Therefore, this application can improve the visual stability of smart blackboard images, the readability of blackboard writing, and the applicability of subsequent image recognition processing without significantly altering the classroom's physical environment.

[0149] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of combined actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Secondly, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application. In the above embodiments, the descriptions of each embodiment have their own emphasis; for parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0150] Figure 8 A structural block diagram of an example of a smart blackboard image anti-glare rendering system based on three-dimensional light field modeling according to an embodiment of this application is shown.

[0151] like Figure 8 As shown, the smart blackboard image anti-glare rendering system 800 based on three-dimensional light field modeling includes a multi-source sampling unit 810, a board surface reconstruction unit 820, a light field modeling unit 830, a reflection parameter estimation unit 840, a reflection component decomposition unit 850, and an anti-glare fusion rendering unit 860.

[0152] The multi-source sampling unit 810 is used to acquire multi-view blackboard images, ambient illumination data and camera calibration parameters corresponding to each acquisition viewpoint, which are synchronously collected for the target smart blackboard. It also determines the main viewpoint image to be rendered from the multi-view blackboard images and uses the pixels in the main viewpoint image as image sampling points.

[0153] The board reconstruction unit 820 is used to perform three-dimensional reconstruction of the target smart blackboard based on the multi-view blackboard image and the camera calibration parameters, so as to obtain the three-dimensional geometric structure of the target smart blackboard and the board spatial position corresponding to each of the image sampling points.

[0154] The light field modeling unit 830 is used to remap the pixel data in the multi-view blackboard image to the corresponding board surface spatial position and light direction based on the three-dimensional geometric structure and the camera calibration parameters, and construct a four-dimensional light field model corresponding to the target smart blackboard; the four-dimensional light field model is used to characterize the radiation intensity information of each board surface spatial position under different light directions.

[0155] The reflection parameter estimation unit 840 is used to estimate the bidirectional reflectance distribution function parameters corresponding to each of the image sampling points based on the radiation intensity information, the spatial position of the board surface corresponding to each of the image sampling points, the ambient illuminance data, and the preset micro-facet reflectance model, and in combination with the chalk reflection a priori condition used to characterize the reflectance difference between the chalk area and the blackboard background area; the bidirectional reflectance distribution function parameters include diffuse reflection weights and specular reflection weights.

[0156] The reflection component decomposition unit 850 is used to construct an inverse rendering optimization objective function based on the main view image, the bidirectional reflection distribution function parameters and the four-dimensional light field model, and to determine the diffuse reflection component and specular reflection component corresponding to each of the image sampling points by solving the inverse rendering optimization objective function.

[0157] The anti-glare fusion rendering unit 860 is used to determine the adaptive suppression factor of each of the image sampling points based on the specular reflection weight, specular reflection component, ambient illuminance data, and the observation direction corresponding to the main view image determined based on the camera calibration parameters. The unit then uses the adaptive suppression factor to attenuate the specular reflection component and fuses the diffuse reflection component with the attenuated specular reflection component to obtain the anti-glare rendering image of the target smart blackboard.

[0158] In some embodiments, this application provides a non-volatile computer-readable storage medium storing one or more programs including execution instructions. The execution instructions can be read and executed by electronic devices (including but not limited to computers, servers, or network devices) to perform the steps of any of the above-described methods for anti-glare rendering of smart blackboard images based on three-dimensional light field modeling.

[0159] In some embodiments, this application also provides a computer program product, the computer program product including a computer program stored on a non-volatile computer-readable storage medium, the computer program including program instructions, which, when executed by a computer, cause the computer to perform the steps of any of the above-described methods for anti-glare rendering of smart blackboard images based on three-dimensional light field modeling.

[0160] In some embodiments, this application also provides an electronic device, comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the steps of a smart blackboard image anti-glare rendering method based on three-dimensional light field modeling.

[0161] The above-described product can perform the methods provided in the embodiments of this application, and has the corresponding functional modules and beneficial effects for performing the methods. Technical details not described in detail in this embodiment can be found in the methods provided in the embodiments of this application.

[0162] The electronic devices in this application can exist in various forms, including but not limited to: mobile communication devices, ultra-mobile personal computer devices, portable entertainment devices, or other airborne electronic devices with data interaction functions.

[0163] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0164] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented using software plus a general-purpose hardware platform, or of course, using hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0165] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for anti-glare rendering of smart blackboard images based on three-dimensional light field modeling, characterized in that, The method includes: Acquire multi-view blackboard images, ambient illumination data, and camera calibration parameters corresponding to each acquisition viewpoint, synchronously collected for the target smart blackboard, and determine the main view image to be rendered from the multi-view blackboard images, and use the pixels in the main view image as image sampling points. Based on the multi-view blackboard images and the camera calibration parameters, the target smart blackboard is reconstructed in three dimensions to obtain the three-dimensional geometric structure of the target smart blackboard and the spatial position of the blackboard surface corresponding to each of the image sampling points; Based on the three-dimensional geometric structure and the camera calibration parameters, the pixel data in the multi-view blackboard image is remapped to the corresponding board surface spatial position and light direction to construct a four-dimensional light field model corresponding to the target smart blackboard; the four-dimensional light field model is used to characterize the radiation intensity information of each board surface spatial position under different light directions; Based on the radiation intensity information, the spatial position of the board surface corresponding to each of the image sampling points, the ambient illuminance data, and the preset micro-facet reflection model, and combined with the chalk reflection a priori condition used to characterize the difference in reflectivity between the chalk area and the blackboard background area, the bidirectional reflectance distribution function parameters corresponding to each of the image sampling points are estimated; the bidirectional reflectance distribution function parameters include diffuse reflection weights and specular reflection weights. Based on the main view image, the bidirectional reflection distribution function parameters, and the four-dimensional light field model, an inverse rendering optimization objective function is constructed, and the diffuse reflection component and specular reflection component corresponding to each image sampling point are determined by solving the inverse rendering optimization objective function. Based on the specular reflection weight, specular reflection component, ambient illumination data, and observation direction corresponding to the main view image determined by the camera calibration parameters for each image sampling point, an adaptive suppression factor is determined for each image sampling point. The specular reflection component is attenuated using the adaptive suppression factor, and the diffuse reflection component is fused with the attenuated specular reflection component for rendering to obtain the anti-glare rendering image of the target smart blackboard.

2. The method according to claim 1, characterized in that, Based on the radiation intensity information, the spatial position of the board surface corresponding to each of the image sampling points, the ambient illuminance data, and the preset micro-area reflection model, and combined with the chalk reflection a priori condition used to characterize the reflectivity difference between the chalk area and the blackboard background area, the bidirectional reflectance distribution function parameters corresponding to each of the image sampling points are estimated, including: The preset micro-surface element reflection model is configured as a reflection estimation model consisting of diffuse reflection and specular reflection terms. The diffuse reflection term is controlled by the diffuse reflection weight to be solved, and the specular reflection term is jointly determined by the specular reflection weight to be solved and the micro-optical geometric feature factor controlled by the roughness parameter to be solved. The micro-optical geometric feature factor is used to characterize the micro-surface element normal distribution, Fresnel effect and geometric occlusion effect. Based on the ambient illuminance data, the discretized incident illumination parameters of the multi-source ambient light are determined, and the light transmission relationship is established in conjunction with the reflection estimation model to calculate the predicted radiant intensity of each image sampling point under different viewing directions. The observed radiation intensity of each image sampling point under different acquisition angles is extracted from the four-dimensional light field model, and a data consistency term is constructed based on the measurement residual between the predicted radiation intensity and the observed radiation intensity. Based on the prior condition of chalk reflection, a prior constraint term is generated to constrain the diffuse reflection weight and specular reflection weight, and a joint regularized fitting function is constructed based on the data consistency term and the prior constraint term. By optimizing the joint regularization fitting function, the diffuse reflection weight, specular reflection weight, and roughness parameter corresponding to each image sampling point are determined.

3. The method according to claim 2, characterized in that, The prior constraint terms are generated in the following manner: The image semantic segmentation model is called to identify chalk regions in the main view image, and the chalk region confidence score, which represents the probability that each image sampling point belongs to the chalk stroke region, is output. Using the confidence level of the chalk region as a dynamic adjustment weight, a prior constraint term including a specular suppression term and a diffuse reflection approximation term is constructed; wherein, the specular suppression term is used to characterize the energy scale of the specular reflection weight, and the diffuse reflection approximation term is used to characterize the degree of deviation between the diffuse reflection weight and the preset chalk diffuse reflection reference weight. For a given image sampling point, the higher the confidence level of the chalk region, the greater the penalty applied by the prior constraint term in the joint regularization fitting function, so that the specular reflection weight corresponding to the image sampling point within the chalk stroke region tends to decrease in the solution iteration, and the corresponding diffuse reflection weight tends to approach the preset chalk diffuse reflection reference weight.

4. The method according to claim 3, characterized in that, The process involves constructing an inverse rendering optimization objective function based on the main view image, the bidirectional reflectance distribution function parameters, and the four-dimensional light field model. By solving the inverse rendering optimization objective function, the diffuse reflection component and specular reflection component corresponding to each image sampling point are determined, including: Based on the physical characteristics of the consistent viewpoint of diffuse reflection, the diffuse reflection component to be solved is set as a common variable of diffuse reflection that is independent of the acquisition viewpoint. Based on the physical characteristics of the directional differences of specular reflection, the specular reflection component to be solved under each acquisition viewpoint is set as a specular reflection-specific variable that is related to the acquisition viewpoint. The observed radiation intensity at each acquisition angle is extracted using a four-dimensional light field model. Based on the bidirectional reflectance distribution function parameters, ambient illuminance data, and corresponding incident light information, the prior specular reflection estimate at each acquisition angle is predicted. An inverse rendering optimization objective function is constructed, comprising a multi-view data consistency and fidelity term, a specular reflection approximation term, a total variation regularization term, and an anisotropic chalk texture preservation term. The multi-view data consistency and fidelity term constrains the reconstruction consistency between the observed radiant intensity and the common variables of diffuse reflection and their corresponding specular reflection-specific variables. The specular reflection approximation term constrains the specular reflection-specific variables to approximate their corresponding prior specular reflection estimates. The total variation regularization term suppresses spatial noise and abrupt changes in the common variables of diffuse reflection. The inverse rendering optimization objective function is solved by alternating iterations, and the diffuse reflection common variables obtained from the solution are determined as the diffuse reflection component corresponding to the main view image, and the specular reflection specific variables obtained from the solution under the acquisition view corresponding to the main view are determined as the specular reflection component corresponding to the main view image.

5. The method according to claim 4, characterized in that, The anisotropic chalk texture preservation term is constructed in the following manner: Calculate the local structure tensor of the main view image at each image sampling point, and perform feature decomposition on the local structure tensor. Determine the eigenvector corresponding to the minimum eigenvalue obtained by decomposition as the tangential direction vector of the chalk stroke, and determine the eigenvector corresponding to the maximum eigenvalue as the normal direction vector of the chalk stroke. Marginal confidence is calculated based on the numerical difference between the maximum and minimum eigenvalues, and an anisotropic adjustment coefficient for characterizing directional consistency is constructed based on the marginal confidence. Using the anisotropic adjustment coefficient and its complementary coefficient as weights, the tangential gradient components of the diffuse common variable along the tangential direction vector and the normal gradient components along the normal direction vector are weighted and aggregated to construct the anisotropic chalk texture preservation term. Specifically, when an image sampling point is determined to be at the edge of a chalk stroke, the corresponding edge confidence increases, thereby driving the anisotropic chalk texture preservation term to adaptively enhance the continuity constraint along the tangential direction and reduce the smoothing penalty along the normal direction during the optimization process.

6. The method according to claim 4 or 5, characterized in that, The step of determining the adaptive suppression factor for each image sampling point based on the specular reflection weight, specular reflection component, ambient illumination data, and the viewing direction corresponding to the main view image determined based on the camera calibration parameters includes: The current ambient illuminance value is extracted from the ambient illuminance data, and the ambient light incident direction is determined by combining the four-dimensional light field model. Based on the ambient light incident direction, the three-dimensional geometric structure of the target smart blackboard, and the observation direction corresponding to the main view image, the spatial angle between the ideal reflection direction vector and the actual line of sight direction vector of the ambient light at each image sampling point is calculated, and the spatial angle is used as the observation deviation angle. Normalize the specular reflection component corresponding to the main view image to obtain the normalized specular reflection intensity. The specular reflection weight, the normalized specular reflection intensity, the relative illuminance determined based on the ratio of the current ambient illuminance value to the preset reference illuminance value, and the non-negative directional consistency response determined based on the observation deviation angle are synergistically coupled to construct a comprehensive suppression driving quantity. Using a preset exponential decay model, the comprehensive suppression driving quantity is mapped to an adaptive suppression factor; wherein, the adaptive suppression factor is monotonically decreasing with the comprehensive suppression driving quantity, and decreases exponentially as the comprehensive suppression driving quantity increases, so as to apply stronger attenuation suppression to the specular reflection component of the corresponding image sampling point.

7. The method according to claim 6, characterized in that, The process of attenuating the specular reflection component using the adaptive suppression factor and fusing the diffuse reflection component with the attenuated specular reflection component to obtain an anti-glare rendering image of the target smart blackboard includes: The diffuse reflection component is normalized, and the normalized diffuse reflection component is enhanced using a preset piecewise nonlinear mapping function to obtain the enhanced diffuse reflection component. The piecewise nonlinear mapping function is configured as follows: nonlinear stretching based on a first gamma coefficient is performed on dark areas with brightness below a first brightness threshold, nonlinear compression based on a second gamma coefficient is performed on bright areas with brightness above a second brightness threshold, and linear mapping is maintained for intermediate brightness areas between the first brightness threshold and the second brightness threshold. An adaptive inhibition factor is used as an adjustment weight to attenuate the intensity of the corresponding specular reflection component, resulting in the attenuated specular reflection component. The enhanced diffuse reflection component and the attenuated specular reflection component are superimposed and fused for rendering. The dynamic range of the fusion result is constrained to a legal standardized pixel range using a preset clipping function to obtain the rendering pixel values ​​of the anti-glare rendering image. The global specular residual energy index of the anti-glare rendered image is calculated, and when the global specular residual energy index exceeds a preset safety threshold, a feedback control command is generated and sent to the heterogeneous lighting control link to form a closed-loop adjustment of the dynamic lighting parameters of the target smart blackboard environment.

8. A smart blackboard image anti-glare rendering system based on three-dimensional light field modeling, characterized in that, The system includes: The multi-source sampling unit is used to acquire multi-view blackboard images, ambient illuminance data and camera calibration parameters corresponding to each acquisition viewpoint, which are synchronously collected for the target smart blackboard. It also determines the main view image to be rendered from the multi-view blackboard images and uses the pixels in the main view image as image sampling points. The board surface reconstruction unit is used to perform three-dimensional reconstruction of the target smart blackboard based on the multi-view blackboard image and the camera calibration parameters, so as to obtain the three-dimensional geometric structure of the target smart blackboard and the board surface spatial position corresponding to each of the image sampling points; The light field modeling unit is used to remap the pixel data in the multi-view blackboard image to the corresponding board surface spatial position and light direction based on the three-dimensional geometric structure and the camera calibration parameters, and construct a four-dimensional light field model corresponding to the target smart blackboard; the four-dimensional light field model is used to characterize the radiation intensity information of each of the board surface spatial positions under different light directions; The reflection parameter estimation unit is used to estimate the bidirectional reflectance distribution function parameters corresponding to each of the image sampling points based on the radiation intensity information, the spatial position of the board surface corresponding to each of the image sampling points, the ambient illuminance data, and the preset micro-facet reflectance model, and in combination with the chalk reflection a priori condition used to characterize the reflectance difference between the chalk area and the blackboard background area; the bidirectional reflectance distribution function parameters include diffuse reflection weights and specular reflection weights. The reflection component decomposition unit is used to construct an inverse rendering optimization objective function based on the main view image, the bidirectional reflection distribution function parameters and the four-dimensional light field model, and to determine the diffuse reflection component and specular reflection component corresponding to each of the image sampling points by solving the inverse rendering optimization objective function; The anti-glare fusion rendering unit is used to determine the adaptive suppression factor of each image sampling point based on the specular reflection weight, specular reflection component, ambient illuminance data, and the observation direction corresponding to the main view image determined based on the camera calibration parameters. The unit then uses the adaptive suppression factor to attenuate the specular reflection component and fuses the diffuse reflection component with the attenuated specular reflection component to obtain the anti-glare rendering image of the target smart blackboard.

9. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the steps of the method as described in any one of claims 1-7.

10. A storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the method as described in any one of claims 1-7.

Images