AI intelligent table lamp multi-scene learning assistance method and system

Abstract

The application discloses an AI intelligent table lamp multi-scene learning auxiliary method and system, relates to the technical field of image processing and lighting device control, and continuously collects desktop images through a wide-angle camera at the top of a lamp arm, performs illumination compensation on the desktop images according to known current lighting parameters of the table lamp to obtain compensated desktop images, identifies and learns the material category and the hand position of the compensated desktop images to determine the current learning scene of reading, writing, doing exercises or electronic screen auxiliary learning, adjusts the target lighting parameters of the table lamp according to the current learning scene, and then collects the target lighting parameters as known input for subsequent compensation, thereby forming a coupling loop of lighting output and image collection, and inverting the image change caused by the lighting from unknown interference to be eliminated into a separable known component explained by known control parameters.

Description

Technical Field

[0001] This invention relates to the field of image processing and lighting device control technology, specifically to an AI smart desk lamp multi-scene learning assistance method and system. Background Technology

[0002] Smart desk lamps have seen rapid development in recent years as a new generation of lighting products for children and students' learning scenarios. In four typical learning scenarios—reading, writing, doing exercises, and screen-assisted learning—students have significantly different needs regarding parameters such as color temperature, brightness, and illuminance uniformity: Warm color temperature, high color rendering index, and low glare are suitable for paper reading; cool color temperature and high uniformity are suitable for paper writing to suppress hand shadows obscuring the writing area; and low brightness and low color temperature ambient lighting are suitable for screen-assisted learning to reduce the brightness contrast between the screen and the environment. Traditional smart desk lamps often use a single, fixed working mode or rely on students to manually switch modes. This frequently interrupts the learning rhythm during focused study, and the limited self-adjustment ability of young students leads to insufficient actual utilization of the auxiliary functions.

[0003] To enable desk lamps to sense the status of the desktop and adjust the lighting accordingly, existing technologies have begun to incorporate image sensing methods. Chinese patent application CN105042399A discloses an image analysis-based adaptive brightness adjustment method for desk lamps. This method uses a camera positioned at the edge of the lampshade to capture comparative images of the illuminated and unilluminated areas of the lamp. It also combines this with a photosensor on the outer wall of the lampshade to obtain ambient brightness. By calculating the brightness difference between the illuminated and unilluminated areas in the comparative images and combining this with the ambient brightness, the current brightness of the illuminated area is calculated. Finally, the desk lamp brightness is adjusted according to a preset brightness threshold. However, this solution treats the desk lamp illumination as a single object of brightness measurement, and regards the desktop image captured by the camera as the basis for brightness measurement. The lighting output link and the image acquisition link are treated as two independent links. This solution neither performs content recognition on the desktop learning scene, nor uses the lighting parameters that the desk lamp is actively controlling and therefore fully known to compensate for the acquired image, nor maintains the recognition image quality as the lighting changes with the scene. Therefore, when the lighting is independently adjusted in pursuit of human eye comfort, it will inadvertently degrade the image quality used for desktop status recognition.

[0004] Chinese patent application CN107426894A discloses an automatic detection and control system and method for classroom lighting based on intelligent video recognition. This system uses an embedded device mounted diagonally at the front upper part of the room from top to bottom to estimate the number of people in the room by segmenting the area, and then intelligently turns the indoor lights on or off based on the number of people and the light intensity. However, this scheme identifies the number of people in the room and controls the turning the lights on or off, operating on a macroscopic scale within the classroom. It fails to address the crucial fact that there is a physical coupling between the lighting output of the desk lamps and the image acquisition of the same illuminated surface.

[0005] It is evident that existing technologies generally follow the processing paradigm of general machine vision, modeling lighting changes as unknown interference that needs to be eliminated, and treating lighting output that serves human eye comfort and image acquisition that serves recognition as two independent links. Their common core bottleneck lies in the fact that the color temperature, brightness, and direction of the desk lamp lighting output are physically coupled with the glare, shadows, and contrast of the image captured by the camera in the same desktop area and cannot be separated. Lighting is both an output that serves human eye comfort and a known and controllable lighting condition for machine vision imaging. However, existing technologies treat this known quantity as an unknown interference and discard the information that lighting can be calibrated, thus failing to maintain the scene recognition imaging quality of the desktop image during the adaptive switching of lighting with the learning scene. Summary of the Invention

[0006] Addressing the core technical bottleneck in existing intelligent desk lamp learning assistance technologies—namely, the physical coupling caused by the desk lamp's lighting output simultaneously serving as both human eye illumination and desktop image acquisition lighting conditions—leading to degraded scene recognition imaging quality when the lighting adaptively switches with the learning scene, this invention provides an AI intelligent desk lamp multi-scene learning assistance method and system. This method models desk lamp lighting as a controllable coupled variable simultaneously affecting human eye comfort and machine vision imaging. Based on the desk lamp's known current lighting parameters, it performs illumination compensation on the acquired image before recognizing the learning scene. The scene recognition result then drives lighting adjustment, and the adjusted lighting parameters are resampled as known input for subsequent compensation. Without altering the existing lighting hardware of the desk lamp, this method reverses the image changes caused by lighting from unknown interference to separable known components explained by known control parameters, achieving adaptive switching of lighting with the learning scene without degrading scene recognition imaging quality.

[0007] The technical solution of this invention is as follows:

[0008] The AI-powered smart desk lamp multi-scenario learning assistance method includes the following steps: Continuously capturing images of the desktop area below the lamp using a wide-angle camera mounted on the top of the lamp arm; obtaining the current lighting parameters of the lamp, including color temperature, brightness, and illuminance uniformity; performing illumination compensation on the desktop image based on the current lighting parameters to obtain a compensated desktop image; performing image content recognition on the compensated desktop image to determine the learning material category of the desktop objects and the student's hand position, where the learning material category includes paper books, paper exercise books, electronic screen devices, and no objects; determining the current learning scenario based on the learning material category and the student's hand position, where the current learning scenario is a reading scenario, a writing scenario, a problem-solving scenario, or an electronic screen-assisted learning scenario; adjusting the current lighting parameters to the target lighting parameters corresponding to the current learning scenario, and controlling the lamp to emit light according to the target lighting parameters; using the target lighting parameters as the current lighting parameters corresponding to the desktop image at the next moment, and cyclically performing illumination compensation, image content recognition, and target lighting parameter adjustment.

[0009] Further, the illumination compensation includes decomposing the desktop image into illumination components and scene content components based on the current illumination parameters, retaining the scene content components and normalizing the illumination components to obtain the compensated desktop image; when the current learning scene changes, the target illumination parameters are smoothly and gradually changed to the target illumination parameters after the change according to a preset transition time to obtain a transition illumination parameter sequence, and within the transition time, the illumination components of the desktop image at each moment are dynamically calibrated frame by frame according to the transition illumination parameter sequence to obtain the transition period scene content components; the total inter-frame change field is calculated for the image time sequence composed of the transition period scene content components, and an illumination-driven change field is constructed based on the transition illumination parameter sequence, and the total inter-frame change field is applied to the illumination-driven change field. The projected components on the changing field are removed to obtain the scene-driven changing field, and the current learning scene is updated according to the scene-driven changing field. Key points of the student's head contour and torso contour are extracted from the compensated desktop image. A subset of key points with stable changes is determined according to the scene-driven changing field. Based on the key point subset, the distance between the student's head and the desktop and the forward tilt angle of the torso are measured using image-based methods to obtain sitting posture measurement parameters. The scene recognition confidence of the compensated desktop image and the key point stability of the key point subset are calculated to obtain a recognition quality index. When the recognition quality index is lower than a preset recognition quality threshold, the target lighting parameters are fine-tuned within a preset human eye comfort constraint range to obtain fine-tuned lighting parameters. The desk lamp is then controlled to emit light according to the fine-tuned lighting parameters.

[0010] This invention also provides an AI smart desk lamp multi-scene learning assistance system, comprising: an image acquisition module, including a wide-angle camera mounted on the top of the lamp arm, for continuously acquiring desktop images of the desktop area below the lamp; a lighting compensation module, for acquiring the current lighting parameters of the lamp and performing lighting compensation on the desktop image based on the current lighting parameters to obtain a compensated desktop image; a learning scene recognition module, for performing image content recognition on the compensated desktop image to obtain the learning material category to which the desktop object belongs and the student's hand position, and determining the current learning scene based on the learning material category and the student's hand position; and a scene adaptive lighting adjustment module, for adjusting the current lighting parameters to target lighting parameters corresponding to the current learning scene, controlling the lamp to emit light according to the target lighting parameters, and providing the target lighting parameters as the current lighting parameters corresponding to the desktop image at the next moment to the lighting compensation module.

[0011] The beneficial effects of this invention are as follows:

[0012] First, by performing illumination compensation on the desktop image based on the current lighting parameters of the desk lamp before recognizing the learning scene, this invention achieves adaptive switching of lighting with the learning scene without degrading the scene recognition imaging quality. The mechanism is as follows: desk lamp illumination is actively controlled by the system itself and is therefore a completely knowable quantity. The image changes generated by the illumination in the desktop area can be explained forward from the current lighting parameters. Based on this, the invention explicitly decomposes the desktop image into an illumination component that can be explained by the lighting parameters and a scene content component generated by desktop objects and hands. The illumination component is normalized while retaining the scene content component, thus preventing the compensated desktop image sent for recognition from drifting with changes in lighting when the scene changes. Compared to treating lighting only as the object to be measured brightness and processing the lighting link and the acquisition link independently, this solution utilizes the knowable information of lighting, which has been discarded by existing technologies, making the scene recognition result insensitive to lighting changes.

[0013] Secondly, by sampling the target lighting parameters driven by scene recognition results back into the known input for illumination compensation in the next moment, a coupling loop is formed between lighting output and image acquisition, achieving mutual support between lighting adjustment and scene recognition. The mechanism is as follows: scene recognition drives lighting adjustment, and the adjusted lighting participates in the illumination compensation of the next frame as a known quantity. This allows recognition to remain accurate under the lighting changes it induces, while lighting can continuously obtain accurate scene input to perform appropriate adjustments. Compared to existing technologies that process the two links independently, this scheme creates a positive feedback loop. The more the lighting switches to a level comfortable for the human eye, the more synchronously the compensation follows the known lighting parameters, resulting in stable recognition accuracy without decreasing.

[0014] Third, by combining the comfortable lighting transitions designed for human eye use with the orthogonal decomposition of lighting-driven and scene-driven components of image changes based on a known sequence of transition lighting parameters, a synergistic effect exceeding the sum of the individual effects of the two is generated. The mechanism is as follows: a smooth lighting transition designed for human eye comfort causes continuous changes in lighting during the transition period. Conventional inter-frame differencing can misinterpret this lighting transition as scene motion, resulting in pseudo-motion. Neither a separate lighting transition nor a separate scene recognition can resolve this pseudo-motion. This invention constructs a lighting-driven change field based on a known sequence of transition lighting parameters during the transition period and removes its projection component from the total inter-frame change field to obtain the scene-driven change field. This accurately separates the pseudo-motion caused by the lighting transition, preserving both the comfortable smooth transition for human eye use and the stable recognition during the transition period. The combination of these two methods produces a highly stable recognition capability during the transition period that cannot be achieved by either individual means, thus achieving a dual-objective synergy between human eye comfort and machine recognition at the system level. Attached Figure Description

[0015] Figure 1 This is a flowchart of the multi-scene learning assistance method for the AI ​​smart desk lamp of the present invention;

[0016] Figure 2 This is a schematic diagram of the structure of the AI ​​smart desk lamp multi-scene learning assistance system of the present invention. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0018] In this embodiment, the desk lamp includes a base, a lamp arm extending upward from the base, and a lamp head located at the end of the lamp arm. The lamp head houses an array of light-emitting diodes (LEDs) with independently adjustable color temperature and brightness. A wide-angle camera is mounted at the top of the lamp arm, and the field of view of the wide-angle camera covers the illuminated area of ​​the desk lamp. The desktop area is the projection area of ​​the illuminated area onto the desktop. The desk lamp has a built-in main control chip. This main control chip outputs control signals for color temperature, brightness, and illuminance uniformity to drive the LED array, and simultaneously reads the desktop image captured by the wide-angle camera, thus enabling closed-loop control of lighting and image acquisition within the same main control chip. Figure 1 As shown, the AI ​​smart desk lamp multi-scene learning assistance method of the present invention includes steps S1 to S7.

[0019] Step S1: Continuous acquisition of desktop images. A wide-angle camera mounted on the top of the lamp arm continuously acquires images of the desktop area below the lamp, obtaining desktop images. In this embodiment, the wide-angle camera has a frame rate of 30fps and a resolution of 1920 pixels × 1080 pixels. The optical axis of the wide-angle camera points diagonally downwards from the top of the lamp arm towards the center of the desktop, with a field of view of not less than 100 degrees, ensuring that the acquisition range covers the projection area of ​​the lamp's illumination area onto the desktop, i.e., the desktop area. The desktop images continuously acquired by the wide-angle camera form a desktop image sequence in chronological order. The pixel coordinates of the desktop image acquired at the current moment are recorded. The gray value at that location is ,in and These are the x and y coordinates of a pixel, both in pixels. The desktop image undergoes denoising and geometric distortion correction preprocessing before being fed into subsequent steps to eliminate the barrel distortion inherent in wide-angle cameras. This denoising and geometric distortion correction is a well-known image preprocessing technique in the art.

[0020] The desktop area is adaptable to various application scenarios. In a child's home learning scenario, the desk lamp is placed on the desk, and the desktop area covers the area where children's books and workbooks are laid out. In a middle school student's study room, a university student's dormitory, or a shared study room, the desktop area covers the area where learning materials are laid out, respectively. The wide-angle camera only images the desktop area and does not collect environmental information other than the student's face, in order to protect privacy.

[0021] Step S2: Perform illumination compensation on the desktop image based on the current illumination parameters. Obtain the current illumination parameters of the desk lamp, including color temperature, brightness, and illuminance uniformity. In this embodiment, the current illumination parameters are denoted as a vector. ,in For color temperature, For brightness, The current illumination parameters are not estimated by sensors, but are directly read from the control quantities currently being output by the main control chip to drive the LED array. Therefore, they are completely known quantities to the system, which is the key premise that distinguishes this invention from the prior art.

[0022] The desktop image is subjected to illumination compensation based on the current lighting parameters to obtain a compensated desktop image. The illumination compensation includes decomposing the desktop image into an illumination component and a scene content component based on the current lighting parameters. The illumination component is the image component generated by the desk lamp illumination on the desktop area, and the scene content component is the image component generated by desktop objects and the student's hand. Subsequently, the scene content component is retained and the illumination component is normalized to obtain the compensated desktop image.

[0023] This step models the desktop image as the product of the scene's reflective properties and the illumination field generated by the desk lamp on the desktop area, and then separates the two in the logarithmic domain. First, the illumination field generated by the desk lamp on the desktop area is constructed based on the current lighting parameters, and then the logarithm is taken to obtain the illumination components; the formula for constructing the illumination field is:

[0024] ,

[0025] ,

[0026] in: The illumination field represents the pixel coordinates of the desk lamp on the desktop area. The illuminance produced at a given location is a scalar quantity, and its value range is... The unit is lx, derived from the current lighting parameters. The intensity of the illumination at that pixel is obtained by calculation using this formula; , is luminance, is one of the components of the current lighting parameter, is a scalar, and its value range is . The unit is lx, which is directly read from the control quantity currently output by the main control chip and represents the overall luminous intensity of the desk lamp. The normalized spatial illuminance distribution represents the illuminance attenuation ratio of each pixel relative to the center within the illuminated area. It is a scalar with a value range of [value range missing]. Dimensionless, obtained by calibrating and normalizing the illuminance distribution of the LED array on a standard whiteboard point by point before the lamp leaves the factory, characterizing the spatial non-uniformity of lighting on the desktop. Color temperature spectral gain, representing color temperature The spectral weighting coefficients of the grayscale response acquired by the camera are scalars, with values ​​ranging from 1 to 2. Dimensionless, determined by color temperature The differences in camera grayscale response at different color temperatures are characterized by looking up the pre-calibrated spectral response curves in a table. Color temperature is one of the components of the current lighting parameters; it is a scalar with a value range of [value range missing]. The unit is K, which is directly read from the control quantity currently output by the main control chip and represents the temperature of the lighting. Let be the illumination component, and be a scalar with a range of values. , dimensionless, is calculated by this formula and characterizes the contribution of illumination to the logarithmic field; To prevent zero constant, take , is a scalar, dimensionless, used to avoid zero values ​​in logarithmic operations that could cause numerical divergence; It is the natural logarithm function.

[0027] The lighting component is then subtracted from the desktop image in the logarithmic domain to obtain the scene content component. The lighting component is then normalized before reconstructing the compensated desktop image.

[0028] ,

[0029] ,

[0030] in: Let be the scene content component, and be a scalar with a value range of . Dimensionless, obtained by subtracting the illumination component from the logarithm of the desktop image, representing image content generated solely by desktop objects and student hands, independent of illumination; The desktop image in pixel coordinates The grayscale value at that location is a scalar, and its range is [value range missing]. , dimensionless, is acquired by step S1, and represents the brightness of the pixel; The definition is the same as the aforementioned formula for the illumination field; The definition is the same as the aforementioned formula for the illumination field; To compensate for the desktop image in pixel coordinates The grayscale value at that location is a scalar, and its range is [value range missing]. , dimensionless, is calculated by this formula, and represents the image content normalized to the reference illumination level after removing the influence of illumination; For reference lighting level, the rated intermediate illuminance of the LED array is taken as 500 lx, which is a scalar quantity with the unit lx. It is selected by the design and represents the reference lighting to which all frames are uniformly normalized. Its function is to make the compensated desktop images acquired under different lighting parameters have a consistent brightness reference. The function is the natural exponential function. Through the above decomposition and normalization, regardless of the current lighting parameters of the desk lamp, the resulting compensated desktop image is normalized to a uniform reference lighting level, thus ensuring that subsequent scene recognition does not drift with changes in lighting.

[0031] Step S3: Image Content Recognition and Current Learning Scene Determination. Image content recognition is performed on the compensated desktop image to obtain the learning material category to which the desktop objects belong and the student's hand position. The learning material categories include paper books, paper exercise books, electronic screen devices, and no objects. This step uses image classification and object detection methods known in the art to determine the category of desktop objects in the compensated desktop image, and uses hand keypoint detection methods known in the art to locate the student's hand position in the compensated desktop image. Since the input is the compensated desktop image normalized in step S2, the recognition remains stable under different lighting conditions. The identified desktop object category falls into one of four categories: paper books, paper exercise books, electronic screen devices, and no objects. The student's hand position is represented by the coordinates of the hand in the compensated desktop image and its positional relationship relative to the desktop objects.

[0032] The current learning scenario is determined based on the type of learning material and the student's hand position. The current learning scenario can be a reading scenario, a writing scenario, a problem-solving scenario, or an electronic screen-assisted learning scenario. In this embodiment, when the learning material is a paper book and the student's hand is far from the center of the page and not holding a pen, it is determined to be a reading scenario; when the learning material is a paper exercise book and the student's hand is in the writing area and holding a pen, it is determined to be a writing scenario; when the learning material is a paper book or paper exercise book and the student's hand moves alternately between the page and the draft, it is determined to be a problem-solving scenario; when the learning material is an electronic screen device, it is determined to be an electronic screen-assisted learning scenario. The determination of the current learning scenario serves as the basis for subsequent lighting adjustments.

[0033] It should be noted that all recognition outputs in this step are strictly limited to objective recognition results of image content, only distinguishing the type of learning material, hand position, and learning scene type. It does not involve judging the correctness of the knowledge of the learning content, nor does it involve assessing the student's learning ability, thus falling into the scope of objective technical parameter measurement in image processing.

[0034] Step S4: Scene Adaptive Lighting Adjustment and Coupled Backsampling. Based on the current learning scene, the current lighting parameters are adjusted to the target lighting parameters corresponding to the current learning scene, and the desk lamp is controlled to emit light according to the target lighting parameters. In this embodiment, the system pre-stores the target lighting parameters corresponding to each current learning scene: Reading scene corresponds to a warm, high color rendering index (CRI) and low glare lighting of 3000K color temperature, 500lx brightness, and 0.7 illuminance uniformity; writing scene corresponds to a cool, high uniformity lighting of 4500K color temperature, 600lx brightness, and 0.85 illuminance uniformity to suppress shadows cast by the hand on the writing area; problem-solving scene corresponds to neutral lighting of 4000K color temperature, 550lx brightness, and 0.8 illuminance uniformity; electronic screen-assisted learning scene corresponds to low brightness and low color temperature ambient lighting of 3500K color temperature, 300lx brightness, and 0.7 illuminance uniformity to reduce the brightness contrast between the screen and the environment. The main control chip outputs control signals according to the target lighting parameters to drive the LED array to emit light.

[0035] The target lighting parameters are used as the current lighting parameters corresponding to the desktop image at the next moment, and the illumination compensation, image content recognition, and adjustment of the target lighting parameters are executed cyclically. Specifically, while the main control chip emits light according to the target lighting parameters, it simultaneously writes the target lighting parameters into the current lighting parameter register read in step S2, so that the illumination compensation of the desktop image in the next frame directly uses the adjusted lighting parameters of this frame as a known quantity. Thus, scene recognition drives lighting adjustment, and lighting adjustment, in turn, serves as a known input to participate in the illumination compensation and scene recognition of the next frame, forming a coupling loop between lighting output and image acquisition. This loop enables recognition to remain accurate under the lighting changes it induces, while lighting can continuously obtain accurate scene input to perform appropriate adjustments; the two support each other rather than being independent of each other.

[0036] Step S5: Smooth Scene Transition and Frame-by-Frame Dynamic Calibration of Transition Illumination Components. When the current learning scene changes, the target illumination parameters are smoothly and gradually transitioned to the target illumination parameters after the change according to a preset transition duration, resulting in a transition illumination parameter sequence. Abrupt changes between two sets of target illumination parameters can cause visual discomfort for students; therefore, this step ensures that the illumination parameters gradually change along a smooth curve during scene transitions. The formula for constructing the transition illumination parameter sequence is:

[0037] ,

[0038] ,

[0039] in: For transition weights, is a scalar with a value range of . , dimensionless, is calculated by this formula, characterizing the transition process. It adopts a cubic smooth interpolation form to make the rate of change at the start and end times zero, thereby achieving a smooth and gradual change in lighting to avoid visual discomfort. Let be the elapsed time since the start of the transition, and be a scalar with a range of values. The unit is seconds (s), obtained from system timing. The preset transition time is set to 2 seconds in this embodiment. It is a scalar quantity and the unit is seconds. It is selected by the design. If it is too large, the lighting will lag behind the scene switching. If it is too small, the transition will not be smooth enough and will still cause visual discomfort. For the transition lighting parameter sequence at time The value of is a three-dimensional vector, and the range and unit of each component are the same as the current lighting parameters. It is calculated by this formula and represents the actual lighting parameters output by the table lamp at any moment during the transition period. The target illumination parameters before switching are three-dimensional vectors, determined by step S4. The target illumination parameters after switching are a three-dimensional vector, determined by step S4.

[0040] During the transition period, the lighting components of the desktop image at each moment are dynamically calibrated frame-by-frame according to the transition lighting parameter sequence to obtain the scene content components during the transition period. Since the transition lighting parameters change frame-by-frame, the lighting components calculated in step S2 using fixed current lighting parameters are no longer applicable. The lighting components must be recalculated using the values ​​of the transition lighting parameter sequence for that frame, as shown in the formula:

[0041] ,

[0042] ,

[0043] in: For the transition period The illumination component at time t is a scalar, dimensionless quantity, derived from the transition illumination parameters at that time using this formula. Substituting into the lighting field formula and taking the logarithm yields the lighting contribution of frame-by-frame dynamic calibration during the transition period; For Substituting the illumination field formula in step S2, the illumination field is obtained in lx; Let be the content component of the transition period scenario, be a scalar, dimensionless quantity, calculated by this formula, representing the th transition period. Image content that has been removed from the illumination contribution at any time; For the transition period Desktop images captured in real time at pixel coordinates The grayscale value at that location is a scalar, and its range is [value range missing]. , dimensionless, obtained from step S1; and The definition is the same as in step S2. Frame-by-frame dynamic calibration ensures that the illumination component of each frame during the transition period is based on the actual output illumination parameters of that frame, avoiding calibration drift caused by a fixed reference.

[0044] Step S6: Orthogonal decomposition of the lighting-driven change field and the scene-driven change field. Calculate the total inter-frame change field for the image time series composed of the scene content components during the transition period. Even after the frame-by-frame dynamic calibration in Step S5, the continuous gradual change of lighting during the transition period will still leave slight changes caused by lighting in the scene content components. Conventional inter-frame differencing will misjudge this lighting gradual change residue as the real movement of a tabletop object or hand, forming pseudo-motion. The formula for calculating the total inter-frame change field is:

[0045] ,

[0046] in: The total inter-frame change field in pixel coordinates The value at this position is a scalar, dimensionless, obtained by subtracting the content components of the transition scene from those of two adjacent frames, representing the value at position 1. Frame relative to the first The total image change of the frame, which includes both the changes caused by the residual lighting gradients and the changes caused by the actual movement of the tabletop objects and the hand; For the first The scene content components during the frame transition period are defined in the same way as in step S5. For the first The scene content component during the frame transition period is defined in the same way as in step S5.

[0047] A lighting-driven change field is constructed based on the aforementioned transition lighting parameter sequence. Since the transition lighting parameter sequence is fully known, the residual changes in lighting gradation within the scene content components can be predicted forward from the temporal changes of the lighting parameters, thus forming the lighting-driven change field.

[0048] ,

[0049] in: For lighting-driven changes in field at pixel coordinates The value at the position is a scalar, dimensionless, and is calculated by this formula. It represents the image change caused only by the gradual change in illumination, which can be explained by the changes in known illumination parameters. The gradient of the illumination component with respect to the illumination parameters in The value at the point is a three-dimensional vector, a dimensionless parameter per unit, obtained by taking the partial derivatives of the illumination component formula with respect to color temperature, brightness, and illuminance uniformity, respectively, and characterizes the change in illumination component caused by a change in a unit illumination parameter; An operator for calculating the gradient of the lighting parameter vector; The change in lighting parameters between two adjacent frames is a three-dimensional vector, obtained by subtracting adjacent terms from the transition lighting parameter sequence. This represents the vector dot product.

[0050] The projection component of the total inter-frame change field onto the lighting-driven change field is removed to obtain the scene-driven change field. That is, the component of the total change that can be explained by lighting-driven change is subtracted, and the remainder represents the change caused by the actual scene motion.

[0051] ,

[0052] in: Scene-driven change field at pixel coordinates The value at is a scalar, dimensionless, and is calculated by this formula. It represents the image changes caused only by the actual movement of the tabletop object and the hand after removing the contribution of lighting gradient. The total inter-frame change field is defined as before; For the lighting-driven changing field, the definition is the same as before; The set of pixels covered by the desktop area. Indicates to Sum all pixels within the range; To prevent zero constants, the same definition as in step S2 is used. The fraction in the formula is the projection coefficient of the total inter-frame change field in the direction of the lighting-driven change field. Multiplying this projection coefficient by the lighting-driven change field gives the component that can be explained by lighting. Subtracting this component from the total change field gives the scene-driven change field that is orthogonal to the lighting-driven change field.

[0053] The current learning scene is updated based on the scene-driven change field. The system aggregates the distribution of the scene-driven change field within the desktop area, re-determines the learning scene type based on the positional relationship between the real motion area and the desktop objects, and updates the current learning scene with this result, so that the scene determination during the transition period is not affected by pseudo-motion due to lighting gradients.

[0054] Step S7: Posture Image Measurement and Recognition Quality Feedback Closed Loop. Key points of the student's head and torso contours are extracted from the compensated desktop image. A subset of key points with stable changes is determined based on the scene-driven change field. The detectability of the student's head and torso contour key points during the transition period fluctuates with the illumination component. Since the scene-driven change field has eliminated the influence of illumination, key points with stable real-motion characteristics can be selected. The formulas for calculating key point stability and the key point subset are:

[0055] ,

[0056] ,

[0057] in: For the first The keypoint stability of each keypoint is a scalar with a value range of [value missing]. Dimensionless, calculated by this formula, characterizing the first... The proportion of real-world scene motion in the image changes within the neighborhood of a keypoint; the closer this value is to 1, the less affected the keypoint's changes are by lighting, and the more reliable they are. (Subscript) The key point is numbered, with a value of a positive integer, representing the first key point among the head contour key points and the torso contour key points. indivual; For the first The set of neighboring pixels of a key point is used in this embodiment, which takes an 11-pixel × 11-pixel window centered on the key point. This represents the summation of all pixels within the given neighborhood; Indicates taking the absolute value; For lighting-driven changing fields, The total inter-frame change field is defined in the same way as in step S6; To prevent zero constant, define the same step as S2; A subset of key points, and a set of key point numbers, consisting of all key points with a stability of not less than a threshold. The keypoint stability threshold is set to 0.6 in this embodiment, and is a scalar with a range of values ​​of [value missing]. Dimensionless, selected by the design; if too large, too few key points will be selected, leading to unstable posture measurement; if too small, key points affected by lighting will not be filtered out.

[0058] Based on the aforementioned keypoint subset, image-based measurements are performed on the distance between the student's head and the desktop, and the forward tilt angle of the torso, to obtain posture measurement parameters. In this embodiment, the distance between the student's head and the desktop is calculated by combining the imaging size of the head contour key points in the compensated desktop image with the calibration intrinsic parameters of the wide-angle camera. The forward tilt angle of the torso is obtained by the angle between the lines connecting the shoulder key points and the hip key points in the keypoint subset and the vertical direction. This angle is calculated using the inverse cosine of the vector angle, a geometric measurement method known in the art. The obtained posture measurement parameters include the distance between the head and the desktop and the forward tilt angle of the torso. When the posture measurement parameters exceed a preset posture range, for example, when the distance between the head and the desktop is less than 25cm or the forward tilt angle of the torso is greater than 30deg, the desk lamp is controlled to output a soft audio-visual prompt to guide the student to adjust their posture, thus avoiding a forced interruption to the learning rhythm in a gentle manner. The posture measurement parameters are objective geometric parameters obtained through image-based measurement and do not constitute a medical diagnosis of the student's health condition.

[0059] The scene recognition confidence score of the compensated desktop image and the keypoint stability score of the keypoint subset are calculated to obtain the recognition quality index. The recognition quality index characterizes the reliability of recognition and measurement under the current lighting conditions, and its calculation formula is as follows:

[0060] ,

[0061] ,

[0062] in: Let be the mean stability of key points, and be a scalar with a range of values ​​of . , dimensionless, is obtained by averaging the stability of each key point in the key point subset, and characterizes the overall reliability of sitting posture measurement; The number of key points in the key point subset is a positive integer, calculated from the preceding steps in step S7. This represents the summation of all key points within a subset of key points. For the first The keypoint stability of each keypoint is defined as before; To identify quality indicators, is a scalar, and its value range is . , dimensionless, is calculated by this formula and characterizes the overall reliability of scene recognition and posture measurement under the current lighting conditions; Let be the scene recognition confidence score, and let be a scalar with a value range of . Dimensionless, it is output by the image content recognition in step S3 when determining the current learning scene, and represents the degree of certainty in scene determination; and As weights, in this embodiment, 0.6 and 0.4 are used respectively, both of which are scalars, and their value range is [value range missing]. And satisfy Dimensionless, selected by design, used to balance the contribution of scene recognition confidence and key point stability to recognition quality indicators.

[0063] When the recognition quality index is lower than a preset recognition quality threshold, the target lighting parameters are fine-tuned within a preset human eye comfort constraint range to obtain the fine-tuned lighting parameters. When the lighting meets human eye comfort but causes a decrease in the recognition quality index, the system fine-tunes the lighting parameters in the direction of improving the recognition quality index without leaving the human eye comfort constraint range. The feedback fine-tuning formula is:

[0064] ,

[0065] in: The target lighting parameters to be fine-tuned are represented by a three-dimensional vector, with each component having the same dimensions as the current lighting parameters. The left side of the arrow represents the lighting parameters after fine-tuning, and the right side represents the target lighting parameters before fine-tuning. The projection operator indicates that the result within the parentheses is projected component by component onto the human eye comfort constraint region. If a component exceeds the range, the nearest boundary value of the range is taken. Its function is to ensure that the lighting parameters after fine-tuning never leave the range of human eye comfort constraints. To predefine the human eye comfort constraint range, which represents the allowable range of color temperature, brightness, and illuminance uniformity for each of the current learning scenarios, the allowable color temperature range for the reading scenario in this embodiment is: K, the allowable brightness range is lx; To fine-tune the step size, this embodiment uses 0.05, which is a scalar and has a range of values ​​of [value range missing]. Dimensionless, selected by the design; if too large, the fine adjustment of the illumination will be large, causing noticeable flicker; if too small, the recognition quality will recover slowly. To identify the gradient of the quality index with respect to the lighting parameters, a three-dimensional vector is generated by finite difference estimation of the quality index after applying a small perturbation to the lighting parameters, representing the change in the identification quality index caused by a unit change in the lighting parameters; For the operator that calculates the gradient of the lighting parameter vector, its finite difference estimate is obtained by perturbation on each component. The difference in the identification quality index before and after the perturbation is applied is divided by the amount of the perturbation.

[0066] The desk lamp is controlled to emit light according to the fine-tuned lighting parameters, causing the recognition quality index to rise above the preset recognition quality threshold. In this embodiment, the preset recognition quality threshold is 0.8. The main control chip outputs a control signal according to the fine-tuned lighting parameters to drive the LED array to emit light, and synchronously uses the fine-tuned lighting parameters as the current lighting parameters corresponding to the desktop image at the next moment. Thus, the decrease in the recognition quality index is transmitted in reverse to the fine-tuning of the lighting parameters. The lighting adaptively maintains the recognition imaging quality while ensuring human eye comfort, realizing a dual-objective collaborative closed loop of human eye comfort and machine recognition.

[0067] Building upon the aforementioned steps, this method can further perform text image recognition and formula image recognition on the compensated desktop image to obtain text recognition results and formula recognition results. These results represent objective recognition data of the image content. When a student's preset gesture is detected, such as when a student points to a word with their finger, the text recognition results and formula recognition results are pushed to a compatible mobile terminal for further querying by the student. The text image recognition and formula image recognition are well-known optical character recognition methods in the field, and their output is strictly limited to objective recognition results of the image content, without any judgment on the correctness of the learning content or the teaching method itself.

[0068] This method can also accumulate the duration of the current learning scenario (reading, writing, or problem-solving) to obtain the continuous learning time. When the continuous learning time exceeds a preset eye-use time threshold, such as 40 minutes, the target lighting parameters are controlled to change according to a preset rest prompt waveform, for example, the brightness slowly decreases and then increases within 3 seconds, gently prompting the student to take a break, which conforms to the visual health guidelines for teenagers. The accumulation of the continuous learning time is a well-known timing method in the art.

[0069] To illustrate the technical effects of this invention, this embodiment is compared with a control scheme that does not perform illumination compensation and coupled backsampling in the same desktop environment. The control scheme directly performs image content recognition on the uncompensated desktop image and causes abrupt changes in lighting parameters during scene switching. In tests involving repeated switching between reading and writing scenes, the control scheme's scene recognition confidence dropped sharply from 0.92 to 0.61 at the moment of switching due to the change in color temperature from 3000K to 4500K and brightness from 500lx to 600lx, accompanied by significant scene misjudgment. In contrast, this embodiment performs illumination compensation on the desktop image using the current lighting parameters of the desk lamp before and after switching, and uses the backsampling of the switched lighting parameters as known input for subsequent compensation. The compensated desktop image sent for recognition is always normalized to the reference lighting level of 500lx, and the scene recognition confidence remains above 0.88 during the switching process. During the 2-second smooth transition period of scene switching, the control scheme suffered from a false motion caused by continuous gradual changes in lighting, resulting in a scene misclassification rate of approximately 18%. In contrast, this embodiment reduces the scene misclassification rate to approximately 3% by constructing a lighting-driven change field based on the transition lighting parameter sequence and removing its projection component to obtain a scene-driven change field. Regarding posture measurement, the control scheme experienced approximately 12% loss of head contour key points during the transition period due to lighting fluctuations. However, this embodiment, after filtering a subset of key points based on the scene-driven change field, maintained stable key points during the transition period, and the inter-frame fluctuation in the head-to-table distance measurement decreased from approximately 2.4 cm in the control scheme to approximately 0.7 cm. These comparisons demonstrate that the present invention significantly maintains scene recognition imaging quality and posture measurement stability while adaptively switching lighting according to the learning scene.

[0070] like Figure 2 As shown, this invention also provides an AI smart desk lamp multi-scene learning assistance system, including an image acquisition module 1, a light compensation module 2, a learning scene recognition module 3, and a scene adaptive lighting adjustment module 4. The four modules correspond one-to-one with the steps of the aforementioned method, and the specific implementation of each module is consistent with the description of the corresponding steps in the aforementioned method embodiments, which will be briefly described below.

[0071] Image acquisition module 1 includes a wide-angle camera mounted on the top of the lamp arm, used to continuously acquire images of the desktop area below the lamp to obtain desktop images. The implementation of image acquisition module 1 is consistent with the aforementioned step S1. The field of view of the wide-angle camera covers the illumination area of ​​the lamp, and the acquired desktop images are sent to illumination compensation module 2.

[0072] The illumination compensation module 2 is used to acquire the current lighting parameters of the desk lamp and perform illumination compensation on the desktop image based on the current lighting parameters to obtain a compensated desktop image. The current lighting parameters include color temperature, brightness, and illuminance uniformity. The implementation of the illumination compensation module 2 is consistent with the aforementioned step S2. It decomposes the desktop image into lighting components and scene content components, retains the scene content components, and normalizes the lighting components to obtain the compensated desktop image. During the transition time of scene switching, the illumination compensation module 2 also performs frame-by-frame dynamic calibration of the lighting components of the desktop image at each moment based on the transition lighting parameter sequence provided by the scene adaptive lighting adjustment module 4 to obtain the transition period scene content components. It further calculates the total inter-frame change field, constructs the lighting-driven change field, and removes the projection components to obtain the scene-driven change field. Its implementation is consistent with the aforementioned steps S5 and S6. The compensated desktop image is sent to the learning scene recognition module 3.

[0073] The learning scene recognition module 3 is used to perform image content recognition on the compensated desktop image to obtain the learning material category to which the desktop objects belong and the position of the student's hands. Based on the learning material category and the student's hand position, it determines the current learning scene. The learning material category includes paper books, paper exercise books, electronic screen devices, and no objects. The current learning scene is a reading scene, a writing scene, a problem-solving scene, or an electronic screen-assisted learning scene. The implementation of the learning scene recognition module 3 is consistent with the aforementioned step S3. It also updates the current learning scene based on the scene-driven change field obtained by the illumination compensation module 2, extracts key points of the student's head contour and torso contour in the compensated desktop image, determines a stable subset of key points based on the scene-driven change field, obtains posture measurement parameters based on the key point subset, and calculates recognition quality indicators. Its implementation is consistent with the aforementioned step S7. The current learning scene is sent to the scene adaptive lighting adjustment module 4.

[0074] The scene-adaptive lighting adjustment module 4 is used to adjust the current lighting parameters to target lighting parameters corresponding to the current learning scene, control the desk lamp to emit light according to the target lighting parameters, and provide the target lighting parameters as the current lighting parameters corresponding to the desktop image at the next moment to the illumination compensation module 2. The implementation of the scene-adaptive lighting adjustment module 4 is consistent with the aforementioned step S4; during scene switching, it smoothly and gradually changes the target lighting parameters to a transition lighting parameter sequence according to a preset transition time and provides it to the illumination compensation module 2; when the recognition quality index is lower than the preset recognition quality threshold, it performs feedback fine-tuning of the target lighting parameters within a preset human eye comfort constraint range to obtain fine-tuned lighting parameters and controls the desk lamp to emit light, which is consistent with the aforementioned step S7. The scene-adaptive lighting adjustment module 4 samples the target lighting parameters back to the illumination compensation module 2, forming a coupling loop between lighting output and image acquisition.

[0075] The above modules can be implemented by software function modules carried by the main control chip, or by application-specific integrated circuits. The modules transmit desktop images, current lighting parameters, compensated desktop images, current learning scene and target lighting parameters through the data bus inside the main control chip.

[0076] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A multi-scenario learning assistance method for AI smart desk lamps, characterized in that, The method includes: A wide-angle camera mounted on the top of the lamp arm continuously captures images of the desktop area below the lamp to obtain desktop images. Obtain the current lighting parameters of the desk lamp, including color temperature, brightness, and illuminance uniformity; Based on the current lighting parameters, illumination compensation is performed on the desktop image to obtain a compensated desktop image; Image content recognition is performed on the compensated desktop image to obtain the learning material category to which the desktop object belongs and the position of the student's hand. The learning material category includes paper books, paper exercise books, electronic screen devices, and no object. The current learning scenario is determined based on the type of learning material and the position of the student's hand. The current learning scenario is a reading scenario, a writing scenario, a problem-solving scenario, or an electronic screen-assisted learning scenario. Based on the current learning scenario, the current lighting parameters are adjusted to target lighting parameters corresponding to the current learning scenario, and the desk lamp is controlled to emit light according to the target lighting parameters; The target lighting parameters are used as the current lighting parameters corresponding to the desktop image at the next moment, and the illumination compensation, image content recognition and adjustment of the target lighting parameters are performed cyclically.

2. The AI ​​smart desk lamp multi-scene learning assistance method according to claim 1, characterized in that, The step of performing illumination compensation on the desktop image based on the current lighting parameters to obtain a compensated desktop image includes: Based on the current lighting parameters, the desktop image is decomposed into a lighting component and a scene content component. The lighting component is the image component generated by the desk lamp illuminating the desktop area, and the scene content component is the image component generated by the desktop objects and the student's hand. The scene content component is preserved and the lighting component is normalized to obtain the compensated desktop image.

3. The AI ​​smart desk lamp multi-scene learning assistance method according to claim 2, characterized in that, The method further includes: When the current learning scene changes, the target lighting parameters are smoothly and gradually changed to the target lighting parameters after the change according to a preset transition time, so as to obtain a transition lighting parameter sequence; During the transition period, the lighting components of the desktop image at each moment are dynamically calibrated frame by frame according to the transition lighting parameter sequence to obtain the scene content components during the transition period.

4. The AI ​​smart desk lamp multi-scene learning assistance method according to claim 3, characterized in that, The method further includes: Calculate the total inter-frame change field for the image time series composed of the transition period scene content components within the transition duration; A lighting-driven change field is constructed based on the transition lighting parameter sequence. The projection component of the total inter-frame change field onto the lighting-driven change field is removed to obtain the scene-driven change field. The current learning scenario is updated based on the scenario-driven change field.

5. The AI ​​smart desk lamp multi-scene learning assistance method according to claim 4, characterized in that, The method further includes: Extract the key points of the student's head contour and the key points of the torso contour from the compensated desktop image. Based on the scene-driven change field, a subset of key points with stable changes is determined from the head contour key points and the torso contour key points; Based on the aforementioned subset of key points, the distance between the student's head and the desktop and the forward tilt angle of the torso are measured using image-based methods to obtain sitting posture measurement parameters. When the sitting posture measurement parameters exceed the preset sitting posture range, the desk lamp is controlled to output a soft sound and light prompt.

6. The AI ​​smart desk lamp multi-scene learning assistance method according to claim 5, characterized in that, The method further includes: Calculate the scene recognition confidence of the compensated desktop image and the keypoint stability of the keypoint subset to obtain the recognition quality index; When the recognition quality index is lower than the preset recognition quality threshold, the target lighting parameters are fine-tuned within the preset human eye comfort constraint range to obtain the fine-tuned lighting parameters. Control the desk lamp to emit light according to the fine-tuned lighting parameters, so that the recognition quality index rises back to above the preset recognition quality threshold; The preset human eye comfort constraint range is the allowable range of color temperature, brightness and illuminance uniformity corresponding to each current learning scene.

7. The AI ​​smart desk lamp multi-scene learning assistance method according to claim 1, characterized in that, The method further includes: The compensated desktop image is subjected to text image recognition and formula image recognition to obtain text recognition results and formula recognition results. The text recognition results and formula recognition results are objective recognition data of the image content. When a student's preset gesture is recognized, the text recognition result and the formula recognition result are pushed to the corresponding mobile terminal.

8. The AI ​​smart desk lamp multi-scene learning assistance method according to claim 1, characterized in that, The method further includes: The duration of the current learning scenario (reading, writing, or problem-solving) is accumulated to obtain the continuous learning duration. When the continuous learning time exceeds a preset eye use time threshold, the target lighting parameters are controlled to change according to a preset rest prompt waveform to prompt the student to take a break for their eyes.

9. The AI ​​smart desk lamp multi-scene learning assistance method according to claim 1, characterized in that, The field of view of the wide-angle camera covers the illumination area of ​​the desk lamp, and the desktop area is the projection area of ​​the illumination area on the desktop; the desktop area is suitable for children's home learning scenarios, middle school students' study room learning scenarios, college students' dormitory learning scenarios, or shared self-study room learning scenarios.

10. An AI smart desk lamp multi-scene learning assistance system, used to implement the AI ​​smart desk lamp multi-scene learning assistance method according to any one of claims 1-9, characterized in that, The system includes: The image acquisition module includes a wide-angle camera mounted on the top of the lamp arm, used to continuously acquire images of the desktop area below the lamp to obtain desktop images; The illumination compensation module is used to obtain the current illumination parameters of the desk lamp and perform illumination compensation on the desktop image based on the current illumination parameters to obtain a compensated desktop image. The current illumination parameters include color temperature, brightness and illuminance uniformity. The learning scene recognition module is used to perform image content recognition on the compensated desktop image to obtain the learning material category to which the desktop object belongs and the position of the student's hand, and to determine the current learning scene based on the learning material category and the position of the student's hand. The learning material category includes paper books, paper exercise books, electronic screen devices and no objects. The current learning scene is a reading scene, a writing scene, a problem-solving scene or an electronic screen-assisted learning scene. The scene adaptive lighting adjustment module is used to adjust the current lighting parameters to target lighting parameters corresponding to the current learning scene, control the desk lamp to emit light according to the target lighting parameters, and provide the target lighting parameters as the current lighting parameters corresponding to the desktop image at the next moment to the lighting compensation module.

Images