Light and shadow rendering method, system and device based on 2D image and storage medium

Abstract

The invention relates to the technical field of light and shadow rendering, in particular to a light and shadow rendering method and system based on a 2D image and a medium, and the method comprises the steps: predicting light source parameters of a next frame through a long short-term memory (LSTM) network model based on a locally stored historical light source parameter sequence, and generating a prediction data set; real light source parameters of the current frame are collected in real time, and a deviation value between the prediction data set and the real light source parameters is calculated; the deviation value is input into a pre-trained lightweight error compensation model, corrected light source parameters are generated, and the error compensation model is obtained through offline training of a historical deviation sequence; if the accumulated deviation of continuous N frames exceeds a dynamic threshold value, initiating a full parameter request to a server and updating the weight of a local LSTM model; and driving a 2D shadow rendering engine to output a picture by using the corrected light source parameters. The problems of bandwidth, delay, error and energy consumption are thoroughly solved.

Description

Technical Field

[0001] The present application relates to the technical field of light and shadow rendering, and in particular to a light and shadow rendering method, system, device and storage medium based on 2D images. Background Art

[0002] In recent years, light and shadow rendering has typically involved steps such as light source calculation, shadow generation, and reflection processing. In real-time light and shadow rendering of 2D images on mobile devices (such as AR navigation and interactive educational applications), dynamic light effects rely on continuously updated light parameters (position, intensity, and color temperature).

[0003] Traditional solutions use two modes: one is full transmission, requesting complete light source data from the server for each frame, resulting in high bandwidth usage (single light source > 2KB / frame) and significant latency in weak network environments; the other is local difference compression, where the client only transmits the parameter differences between adjacent frames, but lacks autonomous deviation control. When the light source changes suddenly (such as suddenly turning on a spotlight), errors accumulate, and full synchronization is ultimately required, resulting in image quality distortion exceeding 15%.

[0004] Excessive reliance on server-side validation or simple compression algorithms, especially in pure client-side models without server feedback (such as standalone design tools), makes it difficult for existing solutions to balance data volume and rendering accuracy. Therefore, a self-contained, integrated prediction and correction solution is urgently needed to break through the traditional architecture's strong reliance on server-side collaboration. Summary of the Invention

[0005] In order to solve at least one of the above technical problems, the present application provides a light and shadow rendering method, system, electronic device and storage medium based on 2D images.

[0006] In a first aspect, the present application provides a light and shadow rendering method based on a 2D image, which adopts the following technical solution: Based on the locally stored historical light source parameter sequence, the light source parameters of the next frame are predicted using the long short-term memory network (LSTM) model to generate a prediction dataset. Collect the real light source parameters of the current frame in real time, and calculate the deviation value between the predicted data set and the real parameters; Inputting the deviation value into a pre-trained lightweight error compensation model to generate corrected light source parameters, wherein the error compensation model is obtained by offline training of a historical deviation sequence; If the cumulative deviation of N consecutive frames exceeds the dynamic threshold, a full parameter request is sent to the server and the local LSTM model weights are updated; Use the corrected light source parameters to drive the 2D light and shadow rendering engine to output the picture.

[0007] Through the above solution, the LSTM prediction model is combined with the client-side local error compensation model to form a "prediction-compensation-self-learning" closed loop, and the dynamic threshold is linked to the network status to break through the rigid control of traditional fixed thresholds.

[0008] By closing the loop of LSTM prediction and local error compensation models, the vast majority of frames do not require server-side interaction, supporting offline rendering scenarios. This eliminates the round-trip delay of server-side verification in traditional solutions, significantly reduces end-to-end rendering latency, and achieves complete client autonomy.

[0009] At the same time, the server is requested only when the cumulative error of N consecutive frames exceeds the threshold, effectively reducing the communication volume under normal conditions; the error compensation model parameters occupy very little memory and can be adapted to low-configuration mobile devices to achieve low bandwidth occupancy.

[0010] The dynamic threshold mechanism automatically relaxes the fault tolerance range when the light source suddenly changes, avoiding invalid server requests; the compensation model updates the weights in real time, stabilizing the low image distortion rate in sudden changes and improving the ability to resist sudden changes.

[0011] In one possible implementation, the step of inputting the deviation value into a pre-trained lightweight error compensation model to generate corrected light source parameters, wherein the error compensation model is obtained by offline training of a historical deviation sequence, includes: The error compensation model is a linear regression model, whose input is the deviation sequence of the most recent M frames, and the output is the compensation coefficient of the current frame. The model weights are updated in real time through a local gradient descent algorithm.

[0012] Through the above solution, the model weights are updated in real time, the compensation accuracy is improved, and the problem that the fixed compensation model cannot adapt to dynamic changes is overcome.

[0013] In one possible implementation, if the cumulative deviation of N consecutive frames exceeds a dynamic threshold, the step of initiating a full parameter request to the server and updating the local LSTM model weights includes: The dynamic threshold is adaptively adjusted according to the network status. When it is detected that the network delay is higher than T milliseconds, the threshold is raised to K times the baseline value (K>1) to suppress the server request frequency.

[0014] Through the above solution, server requests are reduced during high latency, reducing the screen freeze rate in network fluctuation scenarios.

[0015] In one possible implementation, the dynamic threshold adjustment strategy includes: When the client is in charging state and the power level is higher than 80%, the first threshold T1 is used; When the client battery level is less than 20% or the temperature exceeds 45°C, the second threshold T2 (T2>T1) is used. When the threshold is switched, knowledge distillation and compression are performed on the LSTM model, reducing the amount of computation by 30%.

[0016] Through the above solution, energy consumption is reduced and battery life is extended when the battery is low, or the GPU temperature is reduced in high-temperature scenarios, avoiding frequency reduction, and solving the problem of high energy consumption limiting the battery life of mobile devices.

[0017] In one possible implementation, in the step of using the corrected light source parameters to drive the 2D light and shadow rendering engine to output the image, when driving the 2D light and shadow rendering, adopting differentiated compensation strategies for different types of light source parameters includes: Apply Kalman filter smoothing to the position parameters; Enforce color gamut boundary constraints on color parameters (sRGB standard); Perform log-domain post-compression compensation on the intensity parameter.

[0018] Through the above solution, the color overflow rate is effectively reduced, the position jitter amplitude is reduced, the motion smoothness is improved, and the problem of color or position distortion caused by unified compensation is solved.

[0019] In one possible implementation, the step of predicting the light source parameters of the next frame using a long short-term memory network (LSTM) model based on a locally stored historical light source parameter sequence to generate a prediction dataset includes: The training data of the LSTM model includes a synthetic light source parameter sequence and user operation mode enhancement data, wherein, The synthetic light source parameter sequence is generated by a random walk algorithm to simulate the continuous changes in light source position, intensity and color temperature; The user operation mode enhancement data is based on historical user interaction records, and sudden parameter mutation fragments are injected to improve the model's anti-interference ability.

[0020] Through the above solution, the prediction error of user sudden operation scenarios is reduced, while the model training cycle is shortened, the annotation cost is reduced, and the model trained with synthetic data is prevented from deviating from real user behavior.

[0021] In one possible implementation, the method further includes: After detecting that the user has continuously manipulated the light source parameters for more than Q times, the adversarial training mode is started: Generate adversarial light source mutation sequence and input it into LSTM model; Update the LSTM model weights based on the adversarial loss function between the model output and the true parameters; Resets the training buffer of the error compensation model.

[0022] Through the above solution, the model crash rate in high-interaction scenarios is reduced, the success rate of first-frame correction after adversarial training is improved, and the scenarios where frequent operations lead to model crashes are effectively reduced.

[0023] In a second aspect, the present application provides a light and shadow rendering system based on 2D images, comprising: Light source prediction module: Based on the locally stored historical light source parameter sequence, the long short-term memory network (LSTM) model is used to predict the light source parameters of the next frame and generate a prediction dataset. Deviation calculation module: collects the real light source parameters of the current frame in real time and calculates the deviation value between the predicted data set and the real parameters; Error compensation module: A pre-trained lightweight error compensation model is built in. The deviation value is input into the model to generate the corrected light source parameters. The model is obtained by offline training of historical deviation sequences. Dynamic decision module: monitors the cumulative deviation of N consecutive frames. If it exceeds the dynamic threshold, it initiates a full parameter request to the server and updates the LSTM model weights. Rendering execution module: Use the corrected light source parameters to drive the 2D light and shadow rendering engine to output the picture.

[0024] In a third aspect, the present application provides an electronic device comprising a memory and a processor, wherein the memory is used to store computer program code, and the processor is used to execute the computer program code stored in the memory to implement the method in the above-mentioned first aspect and any one of the first aspect, or the above-mentioned second aspect and any possible implementation of the second aspect.

[0025] In a fourth aspect, the present application provides a computer-readable storage medium, which stores a computer program or instruction. When the computer program or instruction is executed, it implements the method in the above-mentioned first aspect and any one of the first aspect, or the above-mentioned second aspect and any possible implementation of the second aspect.

[0026] In summary, this application includes at least one of the following beneficial technical effects: The LSTM prediction model is combined with the client-side local error compensation model to form a "prediction-compensation-self-learning" closed loop, and the dynamic threshold is linked to the network status design, breaking through the rigid control of traditional fixed thresholds and completely solving bandwidth, latency, error, and energy consumption problems. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 A flowchart of a light and shadow rendering method based on 2D images provided in an embodiment of the present application.

[0028] Figure 2 A schematic diagram of the structure of a 2D image-based light and shadow rendering system provided in an embodiment of the present application.

[0029] Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0030] The technical solutions of this application will be described below in conjunction with all the accompanying drawings. Obviously, the embodiments described are only part of the embodiments of this application, not all of them.

[0031] In the description of the embodiments of this application, unless otherwise specified, " / " means "or." For example, A / B can mean A or B. "And / or" in this article is merely a description of an association relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, and B exists alone. In addition, in the description of the embodiments of this application, "plurality" or "a plurality" means two or more than two.

[0032] In the following, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this embodiment, unless otherwise specified, "plurality" means two or more.

[0033] The terms used in the following examples are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the specification and appended claims of this application, the singular expressions "a," "an," "said," "above," "the," and "this" are intended to include expressions such as "one or more," unless the context clearly indicates otherwise. It should also be understood that in the following examples of this application, "at least one," "one or more" refer to one, two, or more than two.

[0034] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present application. Thus, phrases such as "one embodiment," "some embodiments," "another embodiment," and "other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically stated. The terms "including," "comprising," "having," and variations thereof mean "including but not limited to," unless otherwise specifically stated.

[0035] The embodiment of the present application provides a light and shadow rendering method based on 2D images, which is executed by an electronic device, wherein the electronic device can be an independent physical electronic device, or an electronic device cluster or distributed system composed of multiple physical electronic devices, or a cloud electronic device that provides cloud computing services. The embodiment of the present application is not limited here, such as Figure 1 As shown, the method includes: S1. Based on the locally stored historical light source parameter sequence, the light source parameters of the next frame are predicted through the long short-term memory network (LSTM) model to generate a prediction dataset.

[0036] Specifically, data preparation and storage: Historical light source parameters are stored in JSON format, including position (two-dimensional coordinates x, y), intensity (normalized value 0.0~1.0), color temperature (2700K~6500K), with a sampling frequency of 30Hz, and the most recent 50 frames of data are retained.

[0037] Data is stored in the client-side SQLite database and is quickly retrieved through timestamp indexing.

[0038] LSTM model construction includes model structure, training parameters, and input sequence length.

[0039] Furthermore, the model structure is: input layer (3D, corresponding to position, intensity, and color temperature), 2 hidden layers (64 neurons each, activation function tanh), and output layer (3D, linear activation).

[0040] Training parameters: Adam optimizer (learning rate 0.001), mean squared error (MSE) loss function, 100 epochs, and batch size 32.

[0041] Input sequence length: dynamically adjusted to the last 10 frames (when memory is sufficient) or 5 frames (when memory is limited). The calculation formula is: L = [Available memory (MB) / 0.5] Among them, single frame parameters occupy 0.5MB of memory.

[0042] Specifically, prediction execution: before rendering each frame, the historical sequence is loaded from the database and input into LSTM, which outputs the prediction parameters for the next frame; the prediction time is ≤3ms.

[0043] S2. Collect the real light source parameters of the current frame in real time and calculate the deviation between the predicted data set and the real parameters.

[0044] Specifically, the data collection method obtains the real light source parameters in real time through the client rendering engine API (such as Android's SurfaceFlinger), with a sampling interval of 33ms (30Hz).

[0045] Among them, the data collection process uses a double buffer queue to avoid conflicts between the collection and prediction threads.

[0046] Specifically, calculating the deviation value between the predicted data set and the true parameter includes calculating the Euclidean distance deviation for the position parameter, using the absolute error for the intensity and color temperature, and the total deviation value is the weighted sum of the three.

[0047] Furthermore, the Euclidean distance deviation formula for the location parameter is calculated as: ; The absolute error formula for intensity and color temperature is: , ; The total deviation value is the weighted sum of the three: .

[0048] The absolute error formula for intensity and color temperature is:

[0049] The total deviation value is the weighted sum of the three: .

[0050] S3. Input the deviation value into the pre-trained lightweight error compensation model to generate the corrected light source parameters.

[0051] Specifically, the error compensation model is a linear regression model (input: the deviation sequence of the latest 5 frames, output: the compensation coefficient of the current frame), which is obtained by offline training of the historical deviation sequence.

[0052] The training data consists of 10,000 deviation sequences extracted from historical logs. Offline training uses stochastic gradient descent (SGD) with a learning rate of 0.01 and 500 iterations. The trained weight file (12KB in size) is stored in the client application resource directory.

[0053] Furthermore, when the deviation values of the current frame and the previous four frames are input (a total of five-dimensional features), the model outputs a compensation coefficient α (range 0.8~1.2).

[0054] The specific correction formula is: , where the calculation time is ≤1ms.

[0055] S4: If the cumulative deviation of N consecutive frames exceeds the dynamic threshold, a full parameter request is sent to the server and the local LSTM model weights are updated.

[0056] Specifically, the reference threshold T base =0.15 (the balance point is determined experimentally); when the network delay is greater than 100ms, the dynamic threshold adjustment formula is: T dynamic =Tbase ×(1+delay (ms) / 200), where the cumulative deviation of consecutive N=5 frames exceeds T dynamic Request is triggered.

[0057] Furthermore, the full parameter request data packet contains the complete light source parameters and device ID of the current frame (JSON format, size 2KB).

[0058] Furthermore, the server returns the latest LSTM model weight file and uses Android's dynamic class loading (DexClassLoader) to replace the local model weights.

[0059] Furthermore, the degradation mode is enabled during the update, and the compensated parameters are directly used, skipping the prediction step.

[0060] S5. Use the corrected light source parameters to drive the 2D light and shadow rendering engine to output the image.

[0061] Specifically, the corrected parameters are converted into OpenGL ES shader uniform variables (such as u_LightPos, u_LightIntensity) to perform 2D projection calculations.

[0062] In summary, the LSTM prediction model is combined with the client-side local error compensation model to form a "prediction-compensation-self-learning" closed loop, and the dynamic threshold is linked to the network status to break through the rigid control of traditional fixed thresholds.

[0063] First, a client-side autonomous closed loop is achieved. Through the closed loop of LSTM prediction and local error compensation models, 99% of frames do not require server interaction, supporting rendering scenarios in offline and flight modes. This eliminates the round-trip delay of server verification in traditional solutions, greatly reducing end-to-end rendering latency and achieving complete client autonomy.

[0064] At the same time, resource usage is extremely simplified, and the server is requested only when the cumulative error of N consecutive frames exceeds the threshold, effectively reducing the communication volume under normal conditions; the error compensation model parameters occupy very little memory and can be adapted to low-configuration mobile devices to achieve low bandwidth usage.

[0065] In addition, the anti-mutation capability is quantified, and the dynamic threshold mechanism automatically relaxes the fault tolerance range when the light source suddenly changes, avoiding invalid server requests; the compensation model updates the weights in real time, and the low picture distortion rate in the mutation scenario is stabilized, thereby improving the anti-mutation capability.

[0066] In some embodiments, S4 further includes: S401. The dynamic threshold is adaptively adjusted according to the network status. When it is detected that the network delay is higher than T milliseconds, the threshold is raised to K times the baseline value (K>1) to suppress the server request frequency.

[0067] Specifically, the network delay detection method uses the client to periodically send lightweight heartbeat packets (UDP protocol, data size 64B) to the server to calculate the average round-trip time (RTT). The heartbeat frequency is sent every 500ms, the statistical window is the moving average of the last five RTTs, and the delay threshold T=100ms.

[0068] Furthermore, the Android NetworkRequest API is used to monitor the network type (4G / 5G / WiFi) in real time. When the average RTT is greater than 100ms, it is determined to be a high latency state and triggers threshold adjustment.

[0069] Specifically, the calculation formula of the dynamic threshold is:

[0070] Among them, T base =0.15, K max =3.0 to prevent extreme network jitter from causing the threshold to be too large.

[0071] For example, when RTT=150ms, K=1.75, then T dynamic =0.26; when RTT=300ms, K=3.0, then T dynamic =0.45.

[0072] Specifically, when the cumulative deviation of consecutive N=5 frames exceeds T dynamic , triggering suppression of server request frequency. In high-latency situations, the server request interval is forced to be ≥ 2 seconds. This reduces server requests during high latency and reduces screen freezes in scenarios with fluctuating network conditions.

[0073] In some embodiments, the dynamic threshold adjustment strategy includes: S402: When the client is in a charging state and the power level is higher than 80%, a first threshold T1 is adopted.

[0074] Specifically, when the device is connected to a charger and the battery level is >80% (monitored in real time through the Android BatteryManager API), it switches to aggressive mode with a threshold of T1 = 0.10.

[0075] Specifically, the aggressive mode allows for higher deviation tolerance, reducing the server request frequency to 0.05 times per second; the rendering thread priority is increased to the highest level to ensure picture smoothness.

[0076] S403 : When the power level of the client is lower than 20% or the temperature exceeds 45° C., a second threshold value T2 ( T2 > T1 ) is adopted.

[0077] Specifically, when the battery level is less than 20% (monitored in real time through the Android BatteryManager API) or the device temperature is greater than 45°C (CPU / GPU temperature is obtained through the Android Thermal API), the system switches to conservative mode with a threshold of T2 = 0.25, and triggers LSTM model compression.

[0078] Specifically, by increasing the threshold to reduce the computational load of model prediction and compensation, device heating and power consumption are alleviated, and rendering degradation mode is forced to be enabled (such as turning off anti-aliasing and reducing shadow resolution).

[0079] S404: When the threshold is switched, knowledge distillation and compression are performed on the LSTM model, reducing the amount of computation by 30%.

[0080] Specifically, knowledge distillation compression compresses the original model's 64 neurons into 32, reducing the parameter size from 1.2MB to 0.42MB. Simultaneously, the model weights are converted from FP32 to INT8, reducing the model size from 1.2MB to 0.4MB. This reduces single-frame prediction time from 3ms to 1.2ms, and GPU utilization from 18% to 6%.

[0081] In summary, energy consumption is reduced and battery life is extended when the battery is low, or the GPU temperature is reduced in high-temperature scenarios, avoiding frequency reduction, and solving the problem of high energy consumption limiting the battery life of mobile devices.

[0082] In some embodiments, S3 further includes: The error compensation model is a linear regression model, whose input is the deviation sequence of the most recent M frames, and the output is the compensation coefficient of the current frame. The model weights are updated in real time through the local gradient descent algorithm.

[0083] Specifically, the input data is a sequence of deviations (Δ1, Δ2, Δ3, Δ4, Δ5) from the most recent M = 5 frames. Each frame's deviations contain three-channel errors: position, intensity, and color temperature (a total of 15 features). The output data is the compensation coefficient α for the current frame (a scalar value ranging from 0.8 to 1.2).

[0084] Furthermore, the online learning process is: Loss function: Mean Squared Error (MSE):

[0085] Among them, α real Obtained by offline annotation of the dataset (ideal value of the compensation coefficient for each frame).

[0086] Gradient descent update:

[0087]

[0088] Among them, the learning rate η=0.01, X is the input feature vector.

[0089] Update frequency: Weight update is triggered once each frame is rendered, taking ≤ 0.5ms.

[0090] Specifically, compared with the fixed-weight model, online learning improves compensation accuracy by 22%; at the same time, the model weights are updated in real time, and the compensation accuracy is improved, overcoming the problem that the fixed compensation model cannot adapt to dynamic changes.

[0091] In some embodiments, when driving 2D light and shadow rendering in S5, differentiated compensation strategies are adopted for different types of light source parameters, including: applying Kalman filtering smoothing to position parameters, performing color gamut boundary constraints (sRGB standard) on color parameters, and compensating after logarithmic domain compression on intensity parameters.

[0092] Specifically, the smoothing effect is that the standard deviation of position jitter is reduced from 12 pixels to 4.8 pixels (a 60% reduction), and the smoothness of the motion trajectory is improved (PSNR is increased from 28dB to 35dB).

[0093] Specifically, the color spill rate (ΔE>3) was reduced from 12% to 0.5%, and sRGB compliance was verified by CalMAN (average ΔE=1.2).

[0094] Specifically, the overexposed highlight area is reduced by 75% (the proportion of pixels with intensity > 0.9 is reduced from 8% to 2%), and the dark detail retention is improved (SSIM is increased from 0.82 to 0.93).

[0095] In summary, position smoothing is achieved, and Kalman filtering suppresses high-frequency jitter (such as coordinate jumps when AR objects move quickly); color gamut constraint is implemented to prevent the compensated color from exceeding the device display capability, meeting the color management requirements of professional design tools; intensity compression is achieved, and logarithmic transformation expands the dynamic range of dark areas.

[0096] Preventing the loss of highlight details effectively reduces the color overflow rate, reduces the amplitude of position jitter, improves motion smoothness, and solves the problem of color or position distortion caused by unified compensation.

[0097] In some embodiments, S1 further includes: The training data of the LSTM model includes the synthetic light source parameter sequence and the user operation mode enhancement data, where The synthetic light source parameter sequence is generated by a random walk algorithm to simulate the continuous changes of light source position, intensity and color temperature.

[0098] User operation pattern enhancement data is based on historical user interaction records and injects sudden parameter mutation fragments to improve the model's anti-interference ability.

[0099] Specifically, we extracted 1 million light source operation events (desensitized) from anonymous user logs. The extracted operation pattern features were the parameter changes between adjacent frames exceeding a threshold, as well as the operation frequency: the frequency of users dragging and scaling the light source per unit time was counted.

[0100] Specifically, the data augmentation strategy is to randomly insert user mutation fragments into the synthetic sequence with an insertion probability of 20%, and the mutation duration is 1 to 5 frames; user operation frequency characteristics are applied to the synthetic data to enhance temporal correlation.

[0101] Specifically, in the enhanced dataset, the total data volume = synthetic data (80%) + user-enhanced data (20%); the label system includes two types of annotations: "synthetic" and "user mutation", which are used for adversarial training.

[0102] Specifically, the prediction error of user sudden operation scenarios is reduced, while the model training cycle is shortened and the annotation cost is reduced, preventing the model trained with synthetic data from deviating from real user behavior.

[0103] In some embodiments, further comprising: S6. After detecting that the user has continuously manipulated the light source parameters for more than Q times, start the adversarial training mode.

[0104] Specifically, continuous user operation is defined as modifying light source parameters ≥ 3 times within 1 second through touch or external devices (such as a mouse). The threshold Q is 10 times, and the event counter is used to count the changes in real time.

[0105] Specifically, adversarial training mode is triggered when the number of operations within 5 seconds is ≥ Q and the parameter mutation rate (the average difference between adjacent frames) is > 0.2. For example, adversarial training mode enables degraded rendering mode, disabling non-essential lighting effects (such as motion blur), and ensuring a frame rate of ≥ 24 FPS during training.

[0106] S7. Generate adversarial light source mutation sequence and input it into LSTM model.

[0107] Specifically, for example, mutation mode injection: such as position mutation, randomly set the light source coordinates to the edge of the canvas (x=0 or width, y=0 or height); such as intensity mutation: randomly jump in the range of 0.1~1.0 (jump amplitude ≥0.5); such as color temperature mutation: randomly jump between 2700K~6500K (jump amplitude ≥2000K).

[0108] Specifically, the construction of adversarial light source mutation sequence includes generating a dataset containing 1000 adversarial sequences, each with a length of 15 frames; the mixing ratio of adversarial samples to normal samples is 1:3 to prevent the model from overfitting the adversarial features.

[0109] S8. Update the LSTM model weights based on the adversarial loss function between the model output and the true parameters.

[0110] Specifically, during the adversarial training phase, a GRL is inserted between the LSTM output layer and the fully connected layer, multiplying the gradient by -1 during backpropagation to force the model to resist perturbations.

[0111] Specifically, a hot update mechanism is used to dynamically replace the LSTM model weight file (.h5 format) without restarting the rendering engine.

[0112] S9. Reset the training buffer of the error compensation model.

[0113] Specifically, the reset condition is after the adversarial training is completed, or the proportion of old data in the buffer is greater than 40% (the data timestamp is greater than 30 seconds).

[0114] Specifically, the operation performed on the buffer is to clear the current buffer (capacity M = 1000 frames) and reinitialize it, injecting 50% new data generated by adversarial training and 50% historical normal data.

[0115] Specifically, full compensation is enabled for the first frame after reset (skipping the prediction module) to ensure picture stability; the compensation error of the 10 frames after reset is monitored, and if it exceeds the threshold, it is rolled back to the previous version buffer.

[0116] In summary, the model crash rate in high-interaction scenarios is reduced, the success rate of first-frame correction after adversarial training is improved, and scenarios where frequent operations lead to model crashes are effectively reduced.

[0117] The following introduces a 2D image-based light and shadow rendering system provided in an embodiment of the present application. The light and shadow rendering system described below and the light and shadow rendering method described above can refer to each other.

[0118] refer to Figure 2 , the light and shadow rendering system includes: Light source prediction module 1: Based on the locally stored historical light source parameter sequence, the long short-term memory network (LSTM) model is used to predict the light source parameters of the next frame and generate a prediction dataset.

[0119] Deviation calculation module 2: collects the real light source parameters of the current frame in real time and calculates the deviation value between the predicted data set and the real parameters.

[0120] Error compensation module 3: A pre-trained lightweight error compensation model is built in. The deviation value is input into the model to generate the corrected light source parameters. The model is obtained through offline training of historical deviation sequences.

[0121] Dynamic decision module 4: Monitors the cumulative deviation of N consecutive frames. If it exceeds the dynamic threshold, it initiates a full parameter request to the server and updates the LSTM model weights.

[0122] Rendering execution module 5: Use the corrected light source parameters to drive the 2D light and shadow rendering engine to output the picture.

[0123] The present application embodiment provides an electronic device, such as Figure 3 As shown, Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 3 The electronic device 300 shown includes a processor 301 and a memory 303. The processor 301 and the memory 303 are connected, for example, via a bus 302. Optionally, the electronic device 300 may further include a transceiver 304. It should be noted that in actual applications, the number of transceivers 304 is not limited to one, and the structure of the electronic device 300 does not constitute a limitation on the embodiments of the present application.

[0124] Processor 301 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the embodiments disclosed herein. Processor 301 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like.

[0125] The bus 302 may include a path for transmitting information between the above components. The bus 302 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus 302 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0126] The memory 303 may be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0127] The memory 303 is used to store application code for executing the solution of the embodiment of the present application, and the execution is controlled by the processor 301. The processor 301 is used to execute the application code stored in the memory 303 to implement the content shown in the above method embodiment.

[0128] Among them, electronic devices include but are not limited to: mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 3 The electronic device shown is merely an example and should not limit the functions and scope of use of the embodiments of the present application.

[0129] An embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above light and shadow rendering method are implemented.

[0130] Since the embodiments of the computer-readable storage medium part and the embodiments of the method part correspond to each other, the embodiments of the computer-readable storage medium part refer to the description of the embodiments of the method part.

[0131] It should be understood that although the steps in the flowcharts of the accompanying drawings are shown in sequence as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the flowcharts of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0132] The above are only some of the implementation methods of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A light and shadow rendering method based on 2D images, characterized in that: include: Based on the locally stored historical light source parameter sequence, the light source parameters of the next frame are predicted using the long short-term memory network (LSTM) model to generate a prediction dataset. Collect the real light source parameters of the current frame in real time, and calculate the deviation value between the predicted data set and the real parameters; Inputting the deviation value into a pre-trained lightweight error compensation model to generate corrected light source parameters, wherein the error compensation model is obtained by offline training of a historical deviation sequence; If the cumulative deviation of N consecutive frames exceeds the dynamic threshold, a full parameter request is sent to the server and the local LSTM model weights are updated; Use the corrected light source parameters to drive the 2D light and shadow rendering engine to output the picture.

2. The method according to claim 1, characterized in that The step of inputting the deviation value into a pre-trained lightweight error compensation model to generate corrected light source parameters, wherein the error compensation model is obtained by offline training of a historical deviation sequence, includes: The error compensation model is a linear regression model, whose input is the deviation sequence of the most recent M frames, and the output is the compensation coefficient of the current frame. The model weights are updated in real time through a local gradient descent algorithm.

3. The method according to claim 1, characterized in that If the cumulative deviation of N consecutive frames exceeds the dynamic threshold, the step of initiating a full parameter request to the server and updating the local LSTM model weights includes: The dynamic threshold is adaptively adjusted according to the network status. When it is detected that the network delay is higher than T milliseconds, the threshold is raised to K times the baseline value (K>1) to suppress the server request frequency.

4. The method according to claim 3, characterized in that The dynamic threshold adjustment strategy includes: When the client is in charging state and the power level is higher than 80%, the first threshold T1 is used; When the client battery level is less than 20% or the temperature exceeds 45°C, the second threshold T2 (T2>T1) is used. When the threshold is switched, knowledge distillation and compression are performed on the LSTM model, reducing the amount of computation by 30%.

5. The method according to claim 1, wherein In the step of using the corrected light source parameters to drive the 2D light and shadow rendering engine to output the picture, when driving the 2D light and shadow rendering, adopting differentiated compensation strategies for different types of light source parameters includes: Apply Kalman filter smoothing to the position parameters; Enforce color gamut boundary constraints on color parameters (sRGB standard); Perform log-domain post-compression compensation on the intensity parameter.

6. The method according to claim 1, characterized in that The step of predicting the light source parameters of the next frame using a long short-term memory network (LSTM) model based on the locally stored historical light source parameter sequence to generate a prediction data set includes: The training data of the LSTM model includes a synthetic light source parameter sequence and user operation mode enhancement data, wherein, The synthetic light source parameter sequence is generated by a random walk algorithm to simulate the continuous changes in light source position, intensity and color temperature; The user operation mode enhancement data is based on historical user interaction records, and sudden parameter mutation fragments are injected to improve the model's anti-interference ability.

7. The method according to claim 1, characterized in that Further including, After detecting that the user has continuously manipulated the light source parameters for more than Q times, the adversarial training mode is started: Generate adversarial light source mutation sequence and input it into LSTM model; Update the LSTM model weights based on the adversarial loss function between the model output and the true parameters; Resets the training buffer of the error compensation model.

8. A light and shadow rendering system based on 2D images, characterized in that: include: Light source prediction module: Based on the locally stored historical light source parameter sequence, the long short-term memory network (LSTM) model is used to predict the light source parameters of the next frame and generate a prediction dataset. Deviation calculation module: collects the real light source parameters of the current frame in real time and calculates the deviation value between the predicted data set and the real parameters; Error compensation module: A pre-trained lightweight error compensation model is built in. The deviation value is input into the model to generate the corrected light source parameters. The model is obtained by offline training of historical deviation sequences. Dynamic decision module: monitors the cumulative deviation of N consecutive frames. If it exceeds the dynamic threshold, it initiates a full parameter request to the server and updates the LSTM model weights. Rendering execution module: Use the corrected light source parameters to drive the 2D light and shadow rendering engine to output the picture.

9. An electronic device, characterized in that: include: one or more processors; one or more memories; and one or more computer programs, wherein the one or more computer programs are stored in the one or more memories, and the one or more computer programs include instructions that, when executed by the one or more processors, cause the electronic device to perform the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The storage medium stores a program or instruction, and when the program or instruction is executed, the method according to any one of claims 1 to 7 is implemented.

Images