A spectrum adjustment method and device, a storage medium and an electronic device

Abstract

The present application provides a spectrum adjustment method and device, a storage medium and an electronic device, and is applied to the technical field of virtual shooting. The present application solves the spectral reflectivity of the foreground object in real time by the synchronously acquired spectral data and depth data, generates an overflow sensitivity atlas, reversely optimizes the spectral distribution of the light source device according to the overflow sensitivity atlas, converts the spectral distribution into a time sequence light mixing driving signal synchronized with the exposure window of the shooting device, and dynamically reconstructs the output spectrum of the light source device, so that the overflow is eliminated from the physical properties of the light source, and the spectral fusion of the foreground and the background is realized.

Description

Technical Field

[0001] This invention relates to the field of virtual photography technology, and in particular to a spectral adjustment method, apparatus, storage medium, and electronic device. Background Technology

[0002] In current virtual filming, the lighting equipment simultaneously provides both the background image and the main foreground light source. Existing lighting equipment uses RGB (Red, Green, Blue) primary color mixing, and its spectral energy distribution has narrow-band peak characteristics, especially in the green and blue light bands. This characteristic leads to physical color bleeding problems: when highly saturated narrow-band light shines on the actor's skin or semi-transparent clothing, spectral bleeding occurs, which is difficult to separate using traditional post-production color bleeding removal processes, resulting in residual green or blue edges at the keying edges, as well as abnormal skin tones.

[0003] To address the aforementioned issues, existing solutions have significant limitations: physical shading flags are cumbersome to operate and easily reveal flaws; global color temperature adjustment sacrifices background color performance; post-processing digital color correction can easily damage the normal color details of foreground objects and lacks real-time feedback on the spectral dimension, making it impossible to provide targeted compensation from the light source.

[0004] Therefore, how to perceive and dynamically reconstruct the output spectrum of light source devices in real time, eliminate color overflow from the physical properties of the light source, and optimize light and shadow fusion has become a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] In view of the above problems, the present invention provides a spectral adjustment method, apparatus, storage medium, and electronic device to overcome or at least partially solve the above problems, the technical solution of which is as follows:

[0006] A spectral adjustment method, comprising:

[0007] Obtain spectral and depth data that are triggered synchronously with the imaging equipment;

[0008] Using the spectral data and the depth data, the spectral reflectance of the foreground object is determined;

[0009] Using the spectral reflectance, an overflow sensitivity map is generated;

[0010] Using the aforementioned color overflow sensitivity map, the output spectrum of the light source device is inversely optimized to obtain the optimized spectral distribution;

[0011] The optimized spectral distribution is decomposed into driving signals for driving the light source device, and a timing mixing control signal synchronized with the exposure window of the shooting device is generated.

[0012] Based on the timing mixing control signal, the driving signal is injected into the light source device to reconstruct the output spectrum of the light source device.

[0013] A spectral adjustment device includes: a synchronous data acquisition unit for an imaging device, a spectral reflectance determination unit, a color overflow sensitivity spectrum generation unit, a spectral distribution acquisition unit, a signal acquisition unit, and a light source control unit;

[0014] The camera device synchronization data acquisition unit is used to acquire spectral data and depth data that are triggered synchronously with the camera device.

[0015] The spectral reflectance determination unit is used to determine the spectral reflectance of the foreground object using the spectral data and the depth data;

[0016] The color spill sensitivity spectrum generation unit is used to generate a color spill sensitivity spectrum using the spectral reflectance;

[0017] The spectral distribution acquisition unit is used to reverse optimize the output spectrum of the light source device using the color overflow sensitivity map to obtain the optimized spectral distribution;

[0018] The signal acquisition unit is used to decompose the optimized spectral distribution into a driving signal for driving the light source device, and generate a timing mixing control signal synchronized with the exposure window of the shooting device.

[0019] The light source control unit is used to inject the driving signal into the light source device according to the timing mixing control signal, so as to reconstruct the output spectrum of the light source device.

[0020] A computer-readable storage medium having a program stored thereon that, when executed by a processor, implements the spectral adjustment method.

[0021] An electronic device includes at least one processor, at least one memory connected to the processor, and a bus; wherein the processor and the memory communicate with each other via the bus; the processor is used to call program instructions in the memory to execute the spectral adjustment method.

[0022] By employing the aforementioned technical solution, this invention provides a spectral adjustment method, device, storage medium, and electronic device. By synchronously triggering spectral and depth data, it calculates the spectral reflectance of foreground objects in real time, overcoming the limitation of existing RGB three-channel systems that cannot infer the physical causes of color bleeding. The resulting color bleeding sensitivity map can accurately locate high-risk areas, avoiding the indiscriminate sacrifice of background color by traditional global adjustments. Furthermore, based on this map, the output spectrum of the light source device is inversely optimized and decomposed into a temporal mixing drive signal synchronized with the exposure window of the shooting device, completing spectral reconstruction within a single frame exposure time. This process directly compensates for the narrowband peak components that cause color bleeding from the light source end, without relying on post-processing color bleeding removal algorithms, thus avoiding damage to the normal color details of foreground objects and solving the problem of physical light-blocking flags being unable to adapt to dynamic movement. Ultimately, it achieves the technical effect of eliminating color bleeding from the physical properties of the light source and optimizing the spectral fusion of the foreground and virtual background.

[0023] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0024] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0025] Figure 1 A schematic diagram of the composition structure of the virtual shooting system provided in an embodiment of the present invention is shown;

[0026] Figure 2 A flowchart illustrating one embodiment of the spectral adjustment method provided by this invention is shown.

[0027] Figure 3 The diagram shows a specific implementation of step S210 in the spectral adjustment method provided by the present invention.

[0028] Figure 4 The diagram shows a specific implementation of step S220 in the spectral adjustment method provided by the present invention.

[0029] Figure 5 The diagram shows a specific implementation of step S230 in the spectral adjustment method provided by the present invention.

[0030] Figure 6 The diagram shows a specific implementation of step S240 in the spectral adjustment method provided by the present invention.

[0031] Figure 7 A schematic diagram of the spectral adjustment device provided in an embodiment of the present invention is shown;

[0032] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present invention is shown. Detailed Implementation

[0033] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this invention will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

[0034] In virtual filming technology based on LED (Light Emitting Diode) walls, the LED screen is used not only to display the virtual background image but also as the main light source to illuminate the actors and props in the foreground. However, existing LED display devices generally use an RGB (Red, Green, Blue) three-primary-color mixing scheme, and their output spectral energy distribution exhibits significant narrow-band peak characteristics, especially in the green and blue bands. This discontinuous spectral characteristic causes physical color bleeding problems during actual filming. Specifically, when highly saturated narrow-band green or blue light shines on the actor's skin, semi-transparent clothing, or highly reflective prop surfaces, physical spectral penetration occurs, causing the color bleeding to deeply blend with the surface texture of the object. Traditional post-production software, based on chroma keying principles, cannot completely separate these deeply coupled color bleeding components when performing color bleeding removal, resulting in obvious "green edges" or "blue edges" remaining at the edges of the keyed image, or skin tones appearing unnaturally sallow or bluish-gray under certain lighting conditions.

[0035] To address the aforementioned issues, several solutions exist in existing technologies, but all have significant limitations: 1. The physical flag method involves manually placing black flags to block side light, which is cumbersome, unsuitable for actors' dynamic movements, and prone to revealing errors. 2. Global color temperature adjustment alleviates color bleeding by reducing overall saturation or adjusting white balance, but this sacrifices the color expressiveness and immersion of the virtual scene, resulting in a dark background. 3. While post-production digital color bleeding removal is widely used, the algorithms are typically based on chroma keying, which can easily damage the normal color details of foreground objects (such as green eyes or blue clothing), causing color banding and noise. More fundamentally, all of the above solutions lack a real-time feedback mechanism for the spectral dimension: existing technologies mostly rely on RGB image signals for feedback processing, ignoring the physical spectral properties of light itself. Different object surfaces have vastly different reflectivities for different wavelengths of light; RGB three-channel data alone cannot accurately infer the physical causes of color bleeding, let alone provide targeted spectral compensation from the light source.

[0036] Based on this, this embodiment of the invention provides a spectral adjustment method, which calculates the spectral reflectance of the foreground object in real time by synchronously acquiring spectral data and depth data and generates a color overflow sensitivity map. Based on this, the spectral distribution of the light source device is optimized in reverse and converted into a time-series mixing drive signal synchronized with the exposure window of the shooting device. The LED output spectrum is dynamically reconstructed, thereby eliminating color overflow from the physical properties of the light source and realizing the spectral fusion of the foreground and background.

[0037] The spectral adjustment method provided in this invention can be deployed in a virtual imaging system. (Reference) Figure 1 The diagram shown illustrates the structural composition of the virtual shooting system. Figure 1The diagram demonstrates the connection between the shooting equipment, the light source, and the control unit. The light source can be composed of RGBW four-color LED modules, supporting independent control of the current and pulse width of each sub-pixel, with a refresh rate set to 3840Hz. The shooting equipment uses a cinematic camera with a frame rate of 24fps and a shutter angle of 180°. A snapshot-type hyperspectral camera (covering 400nm-700nm, spectral resolution of 5nm) and a polarization depth camera are rigidly connected to the hot shoe on the top of the shooting equipment. The control unit employs a dual-server architecture: one is a GPU workstation responsible for image processing and AI inference; the other is a real-time control card equipped with an FPGA, responsible for millisecond-level driving of the light source signals. Use a BNC (Bayonet Neill-Concelman) cable to connect the Genlock signal output of the imaging device to the synchronization input of the hyperspectral camera and the FPGA (Field-Programmable Gate Array) control card. Set the hyperspectral camera to external trigger mode and set the trigger frequency to 1 / 2 of the imaging device's frame rate, that is, capture spectral data once every 2 frames (12Hz).

[0038] Based on the above description, the spectral adjustment method in the embodiments of the present invention will be described below. Please refer to [link / reference]. Figure 2 A flowchart illustrating one embodiment of the spectral adjustment method provided in this invention may include:

[0039] S200: Acquires spectral and depth data that are triggered synchronously with the imaging device.

[0040] The shooting equipment refers to the camera that captures the foreground object and the virtual background image, and its frame rate, shutter angle and synchronization signal serve as the timing reference for spectral adjustment.

[0041] Among them, spectral data refers to spectral cube data collected by a snapshot hyperspectral camera, covering the visible light band from 400nm to 700nm and with a spectral resolution better than 5nm, containing light intensity information of more than 31 bands for each pixel.

[0042] Depth data refers to the distance information of each point in the field of view relative to the camera, obtained by the polarization depth camera, which is used to calculate the incident angle of the object surface and distinguish between specular reflection and diffuse reflection areas.

[0043] Specifically, in this embodiment of the invention, a snapshot-type hyperspectral camera and a polarization depth camera can be synchronously connected to the shooting device via a hardware interface in a virtual shooting environment, and hardware-level timing synchronization is achieved through an FPGA. The shooting device operates at 24fps, and the hyperspectral camera is set to sample at 1 / 2 or 1 / 4 of the shooting device's frequency. Each time synchronization is triggered, the hyperspectral camera acquires cubic data of 31 spectral bands within the 400-700nm range, while the depth camera acquires the corresponding depth map. All data is transmitted to the GPU workstation in real time via a high-speed network, and motion compensation is used to perform inter-frame interpolation on the spectral data to meet the frame rate requirements of the shooting device.

[0044] As examples, embodiments of the present invention can integrate a snapshot-type hyperspectral camera (covering 400nm-700nm, spectral resolution <5nm) and a polarization depth camera into the pan-tilt unit of the imaging device. The hyperspectral camera, imaging device, and LED controller are locked via FPGA hardware triggering. A BNC cable connects the Genlock signal output of the imaging device to the synchronization input of the hyperspectral camera and the FPGA control card. The hyperspectral camera is set to external trigger mode, with the trigger frequency set to half the frame rate of the imaging device (24fps), i.e., spectral data is captured once every two frames (12Hz). At the start of imaging, the imaging device outputs an RGB image at 24fps. The synchronization signal triggers the hyperspectral camera to acquire spectral cube data of 31 bands (400-700nm) within the current field of view, and the depth camera simultaneously acquires a depth map. Since the spectral acquisition frequency is lower than the frame rate, the system uses optical flow to estimate the motion vector of the previous frame's spectral data and interpolates to generate pseudo-spectral data for the intermediate frame to match the real-time requirement of 24fps.

[0045] S210. Using spectral and depth data, determine the spectral reflectance of the foreground object.

[0046] In this context, foreground objects refer to actors, props, and other subjects in front of the light source in a virtual shooting scene. Specifically, these can include areas prone to physical color bleeding, such as skin, hair, semi-transparent clothing, and highly reflective props.

[0047] Among them, spectral reflectance refers to the proportion of incident light energy reflected by the surface of an object at different wavelengths. It is calculated from spectral data, the reference spectral power distribution of the light source device, the cosine value of the incident angle, and the camera response function, and serves as the "spectral fingerprint" of the object.

[0048] Specifically, in embodiments of the present invention, the spectral reflectance of the region of interest can be calculated using spectral data based on the region of interest of the foreground object extracted from the current scene image output by the shooting device, combined with the incident angle determined by depth data.

[0049] S220. Using spectral reflectance, generate a color overflow sensitivity map.

[0050] Among them, the color spill sensitivity map refers to a two-dimensional mapping map generated based on spectral reflectance, which identifies high color spill risk areas in the foreground object where the reflection intensity is abnormally high at the narrow band peak wavelength of the light source device, and records them in the form of a set of pixel coordinates.

[0051] Specifically, in embodiments of the present invention, the predicted reflection intensity can be calculated based on the spectral power and spectral reflectance of the light source device at the narrowband peak wavelength, and compared with the expected intensity of the preset spectral reflectance model at the same wavelength to mark high color bleeding risk areas. A color bleeding sensitivity map is then generated using the set of pixel coordinates of the high color bleeding risk areas in the current scene.

[0052] Optionally, in the above Figure 2 Based on one or more embodiments shown, in another optional embodiment provided by the present invention, before step S230, the method further includes:

[0053] Visual error prediction is performed using spectral reflectance and color spill sensitivity maps, and a system delay prediction mechanism is introduced.

[0054] Among them, visual error prediction refers to simulating the human visual system (CIE-XYZ) and the color science of the shooting equipment, predicting the color difference value of the foreground object in the final image based on the current spectral reflectance and the output spectrum of the light source equipment. .

[0055] The delay prediction mechanism refers to using pseudo-spectral data obtained by motion compensation interpolation to predict the spectral reflectance of foreground objects in the next or future frame in order to compensate for the system delay (approximately 16-33ms) between spectral capture and the response of the light source device, and to adjust and optimize the target in advance accordingly.

[0056] Specifically, in this embodiment of the invention, after generating the color overflow sensitivity map, based on the spectral data of the current frame and the real-time output spectrum of the light source device, the CIE-XYZ colorimetric system is used to simulate human visual perception, and combined with the color science of the shooting device (such as LogC curves or lookup tables), the predicted color difference value of the foreground object under the current illumination is calculated. Meanwhile, to address the inherent latency issue of the system, pseudo-spectral data obtained through motion compensation interpolation using optical flow is used to estimate the spectral reflectance variation trend of the foreground object in the next frame. This predicted value is then used as the input for subsequent inverse optimization, ensuring that the optimized output spectrum of the light source device precisely matches the actual position and reflectance characteristics of the object at the exposure time.

[0057] S230. Using the color overflow sensitivity map, the output spectrum of the light source device is optimized in reverse to obtain the optimized spectral distribution.

[0058] Among them, the light source equipment refers to the light-emitting diode display wall that serves as the background display and main light source for virtual shooting. It can be composed of RGBW or multi-primary-color LED modules (such as LED screens), supports independent control of the current and pulse width of each sub-pixel, and has a refresh rate of over 3840Hz.

[0059] The output spectrum refers to the distribution of light energy actually emitted by the light source device at different wavelengths, i.e., the spectral power distribution.

[0060] Among them, the spectral distribution refers to the target spectral power distribution that the light source device should output after inverse optimization. It is composed of multiple primary color spectral substrates in a linear combination and is used to decompose it into driving signals for each channel.

[0061] Specifically, embodiments of the present invention can utilize the reflectance data of high color spill risk areas in spectral reflectance to construct a multi-objective optimization function with the goals of minimizing color spill and minimizing chromaticity deviation. Under the constraint that the output spectrum of the light source device to be optimized is composed of a linear combination of multiple primary color spectral bases and the total luminous flux fluctuation is limited, the optimal combination of driving values ​​is solved, and the optimized spectral distribution is synthesized accordingly.

[0062] S240: Decompose the optimized spectral distribution into driving signals for driving the light source device, and generate a timing mixing control signal synchronized with the exposure window of the shooting device.

[0063] Among them, the driving signal refers to the numerical driving instruction that decomposes the optimized spectral distribution into each color channel (such as RGBW) of the light source device, which is then converted into a timing duty cycle after pulse width modulation and used to control the luminous intensity of the sub-pixels.

[0064] The exposure window refers to the time interval during the exposure of each frame by the shooting device, indicated by the synchronization signal (Genlock), including the timing position of rolling shutter line-by-line exposure or global shutter full-frame exposure.

[0065] Among them, the timing mixing control signal refers to the multi-channel pulse width modulation signal that is precisely synchronized with the exposure window of the shooting device. It is injected into the light source device in the shutter scanning order (rolling shutter) or the overall synchronization mode (global shutter) within a single frame exposure time to achieve microsecond-level multi-primary color spectrum superposition to reconstruct the output spectrum.

[0066] Specifically, in embodiments of the present invention, the optimized spectral distribution can be decomposed into driving values ​​of each color channel and mapped to the timing duty cycle of a high-frequency modulation signal according to the timing position of the exposure window of the shooting device. By synchronizing with the shutter exposure line of the shooting device, a timing mixing control signal is generated.

[0067] S250: Based on the timing mixing control signal, the drive signal is injected into the light source device to reconstruct the output spectrum of the light source device.

[0068] Specifically, in this embodiment of the invention, a timing mixing control signal can be injected into the light source device receiver card in real time at a microsecond-level high frequency via an FPGA, driving multi-color LED sub-pixels such as RGBW, so that the light source device dynamically outputs the optimized spectral distribution within the single-frame exposure time of the shooting device. After the next frame of spectral acquisition arrives, the system detects the color difference in the foreground ROI area. If it does not meet the standard, the optimization parameters are corrected in a timely manner and the relevant spectral features are recorded in the material database, realizing closed-loop adjustment and adaptive learning, and continuously improving the performance of suppressing color bleeding and color reproduction.

[0069] As examples, embodiments of the present invention can use an FPGA to write the generated timing-based color mixing control signal into the receiving card of the LED screen. Spectral lookup table mapping is directly executed at the FPGA layer of the receiving card, converting the traditional video signal into multi-primary-color driving signals in real time with a delay controlled within one frame. Based on the timing-based color mixing control signal, the driving signal is injected into the LED screen within a single frame exposure time, controlling the LED screen to display a spectrally corrected background image. Because the injected signal is precisely synchronized with the exposure window of the shooting device, the accumulated spectral energy received by the sensor conforms to the optimized spectral distribution, thereby completing the dynamic reconstruction of the output spectrum within a single frame exposure time, achieving real-time closed-loop control from spectral perception to physical light source correction.

[0070] This invention provides a spectral adjustment method that, by synchronously triggering spectral and depth data, calculates the spectral reflectance of foreground objects in real time, overcoming the limitation of existing RGB three-channel methods that cannot infer the physical causes of color bleeding. The resulting color bleeding sensitivity map can accurately locate high-risk areas, avoiding the indiscriminate sacrifice of background color by traditional global adjustments. Furthermore, based on this map, the output spectrum of the light source is optimized in reverse and decomposed into a temporal mixing drive signal synchronized with the exposure window of the shooting device, completing spectral reconstruction within a single frame exposure time. This process directly compensates for the narrowband peak components that cause color bleeding at the light source end, without relying on post-processing color bleeding removal algorithms, thus avoiding damage to the normal color details of foreground objects and solving the problem of physical light-blocking flags being unable to adapt to dynamic movement. Ultimately, this achieves the technical effect of eliminating color bleeding from the physical properties of the light source and optimizing the spectral fusion of the foreground and virtual background.

[0071] Optional, based on Figure 2 The method shown is as follows: Figure 3 The diagram shows a specific implementation of step S210 in the spectral adjustment method provided by this invention. Step S210 may specifically include:

[0072] S300: Extract at least one region of interest corresponding to the foreground object from the current scene output by the shooting device.

[0073] The current scene image refers to the complete RGB image captured by the camera at the current frame, which includes foreground objects and the virtual background displayed by the light source device. It serves as a visual reference benchmark for semantic segmentation, region of interest extraction, and subsequent color overflow analysis.

[0074] The region of interest refers to a local area of ​​a foreground object that is prone to physical color bleeding, which is locked in real time from the current scene through a semantic segmentation network. This includes actors' skin, hair, semi-transparent clothing, and highly reflective props.

[0075] Specifically, embodiments of the present invention can utilize a pre-trained lightweight convolutional neural network to perform real-time semantic segmentation on the current scene output by the imaging device, identifying areas prone to physical color bleeding, such as "faces," "skin," "hair," and "semi-transparent gauze clothing." Simultaneously, by combining polarization information provided by a depth camera, the Fresnel reflectance of the object's surface is calculated to distinguish between specular and diffuse reflection regions, further refining the segmentation results and ultimately identifying these regions as regions of interest. Subsequent high-precision spectral analysis is performed only on these regions to reduce the system's computational load.

[0076] S310. Using depth data, determine the angle of incidence at each point within the region of interest.

[0077] The incident angle refers to the angle between the surface normal of a point on the foreground object and the direction of the incident ray when the light emitted from the light source illuminates that point. It is calculated based on the spatial coordinates of the object's surface provided by depth data and the geometric position of the light source, and is used as the cosine correction term in the spectral reflectance inversion formula.

[0078] Specifically, in this embodiment of the invention, the normal direction of the object's surface at each point can be calculated based on the fixed geometric position of the light source device in the photography studio and the coordinates of each point within the region of interest in the depth map. Then, the direction vector of the light emitted from the light source device pointing to that point is determined, and the angle between this direction vector and the surface normal is calculated as the incident angle at that point, which is used for cosine correction in subsequent spectral reflectance inversion.

[0079] S320. Using spectral data and the incident angles at points within the region of interest, determine the spectral reflectance of the region of interest.

[0080] Specifically, embodiments of the present invention can read the reference spectral power distribution of a light source device that has been pre-calibrated and stored in a lookup table. and the spectral response function of the hyperspectral camera For each point within the region of interest, acquire the corresponding spectral data. and the angle of incidence calculated at that point Substitute into the formula: The spectral reflectance of this point in each wavelength band from 400nm to 700nm was calculated. The calculated spectral reflectance is stored as a "spectral fingerprint" of the scene for subsequent color spill sensitivity analysis.

[0081] This invention extracts the region of interest corresponding to the foreground object, accurately calculates the incident angle by combining depth data, and then uses spectral data to invert the spectral reflectance of the region. This reduces the computational load while achieving accurate quantification of the reflectance characteristics of key areas, providing a reliable data foundation for subsequent color spill sensitivity analysis and spectral inverse optimization.

[0082] Optional, based on Figure 3 The method shown is as follows: Figure 4 The diagram shows a specific implementation of step S220 in the spectral adjustment method provided by this invention. Step S220 may specifically include:

[0083] S400: Obtain the spectral power value of the light source device at at least one narrowband peak wavelength.

[0084] Narrowband peak wavelength refers to the peak position where the spectral energy of a primary color sub-pixel (such as green and blue) is concentrated in a light source device. For example, the peak wavelength of the green sub-pixel is about 525 nm, and that of the blue sub-pixel is about 450 nm. The spectral energy of these wavelengths is significantly higher than that of natural light and is the main band that causes physical color bleeding.

[0085] Among them, the spectral power value refers to the radiant power of the light source device within a unit wavelength interval at a specific narrowband peak wavelength, that is, the specific value of the spectral power distribution at that wavelength, which is used to characterize the intensity of the incident light in that band.

[0086] Specifically, in this embodiment of the invention, each primary color sub-pixel of the light source device can be pre-calibrated using a spectroradiometer to obtain the spectral power distribution values ​​at the peak wavelength of the green sub-pixel (525nm) and the peak wavelength of the blue sub-pixel (450nm). During actual shooting, based on the current driving current and pulse width settings of the light source device, the spectral power value of the corresponding narrowband peak wavelength is read in real time and used as the incident light intensity parameter for subsequent calculation and prediction of the reflection intensity.

[0087] S410. For narrowband peak wavelengths, extract the reflectance value corresponding to the wavelength from the spectral reflectance.

[0088] The reflectance value refers to the reflectance value extracted from the calculated spectral reflectance of the object's surface, corresponding to the narrowband peak wavelength, representing the object's ability to reflect that wavelength.

[0089] Specifically, in this embodiment of the invention, the spectral reflectance of the region of interest can be used as a "spectral fingerprint". For the narrowband peak wavelength of the light source device, the reflectance values ​​at the corresponding wavelengths are extracted from the spectral reflectance curve to obtain the characterization of the region's ability to reflect green and blue narrowband light.

[0090] S420. Using the spectral power value and reflectance value, the predicted reflection intensity of the region of interest is obtained.

[0091] The predicted reflectance intensity refers to the product obtained by multiplying the spectral power value of the light source device at the narrowband peak wavelength by the reflectance value of the object at that wavelength. It represents the actual light energy intensity reflected by the object's surface at that wavelength and is used for comparison with natural material standards.

[0092] Specifically, in this embodiment of the invention, the spectral power value (incident light intensity) of the light source device at the narrowband peak wavelength can be multiplied by the reflectance value of the region of interest at that wavelength to obtain the predicted reflection intensity of that region at that wavelength. This predicted reflection intensity represents the light energy actually reflected back to the shooting direction from the object surface under the current narrowband illumination of the LED, and directly determines the degree of color bleeding.

[0093] S430. Compare the predicted reflection intensity with the expected intensity. If the predicted reflection intensity exceeds the expected intensity, the corresponding region of interest is identified as a high-risk area for color bleeding.

[0094] The expected intensity refers to the predicted reflectance intensity of the natural material calculated based on a preset spectral reflectance model at the same narrowband peak wavelength (i.e., the product of the model reflectance value and the standard light source spectral power), which serves as the threshold for comparison. The high-risk area for color bleeding refers to the region of interest where the predicted reflectance intensity of the foreground object surface significantly exceeds the expected intensity of the natural material. In these areas, the LED narrowband light is excessively reflected, resulting in noticeable color pollution in the final image (such as green fringing, blue fringing, or abnormal skin tones), making them critical areas requiring spectral compensation.

[0095] Specifically, embodiments of the present invention can read a preset spectral reflectance model (e.g., the Otsu natural skin reflectance model), calculate the expected intensity obtained by multiplying the model by the spectral power value of a standard natural light source at the same narrowband peak wavelength, where the preset spectral reflectance model refers to a pre-established reference model of the spectral reflectance of natural materials (such as natural skin, conventional fabrics) under standard daylight illumination, used as a benchmark to determine whether the reflectance is abnormal. The predicted reflectance intensity is compared with the expected intensity; if the predicted reflectance intensity exceeds the expected intensity, it can be determined that the region of interest has a significant risk of color bleeding, and it is marked as a high-risk area for color bleeding.

[0096] Optionally, in specific embodiments of the present invention, when the predicted reflection intensity exceeds the expected intensity and reaches a preset multiple, the corresponding region of interest is identified as a high-risk area for color bleeding. The preset multiple refers to a threshold coefficient used to determine the risk of color bleeding, such as 1.5 times. When the predicted reflection intensity exceeds the expected intensity by this multiple, the region is considered to have a high risk of color bleeding.

[0097] S440. Record the set of pixel coordinates of the high-risk area for color bleeding in the current scene as a color bleeding sensitivity map.

[0098] Specifically, in this embodiment of the invention, the two-dimensional coordinates of all pixels identified as high-risk areas for color bleeding are recorded in their respective current scene frames to form a color bleeding sensitivity map of that frame. This map is stored in the form of a set of pixel coordinates and is used to guide subsequent spectral inverse optimization steps, clarifying the spatial range in which local spectral modulation needs to be performed at the corresponding location of the light source device.

[0099] This invention obtains the predicted reflection intensity by multiplying the spectral power value of the light source device at the narrowband peak wavelength with the reflectance value of the foreground object at the corresponding wavelength, and compares it with the expected intensity of the natural material model. This allows for the precise location of high-risk areas for color bleeding where the predicted reflection intensity exceeds the expected intensity, and the coordinates of these areas are recorded as a color bleeding sensitivity map. This achieves fine spatial marking of color bleeding areas, providing clear spatial guidance for subsequent local spectral inverse optimization targeting only the risk areas. It avoids interference from global spectral adjustments to non-risk areas, improving optimization efficiency and targeting.

[0100] Optional, based on Figure 4 The method shown is as follows: Figure 5 The diagram shows a specific implementation of step S230 in the spectral adjustment method provided by this invention. Step S230 may specifically include:

[0101] S500: Obtain reflectance data in spectral reflectance for areas with high risk of color bleeding.

[0102] Among them, reflectance data refers to the set of reflectance values ​​in the spectral reflectance of high-risk areas for color overflow corresponding to the narrowband peak wavelength of the light source device. It is used to quantify the reflectance capability of the region to incident light from different wavelengths of light sources and is the basic input for calculating the color overflow term in the multi-objective optimization function.

[0103] Specifically, in this embodiment of the invention, the spectral reflectance curve of the corresponding region can be extracted from the calculated spectral reflectance based on the set of pixel coordinates of the high-risk area for color overflow. The reflectance value of the region is then extracted for the narrowband peak wavelength of the light source device and used as the basic data for calculating the color overflow term in the subsequent optimization function.

[0104] S510. Construct a multi-objective optimization function, wherein the first term of the multi-objective optimization function is the cumulative value of the product of reflectance data and the output spectrum of the light source device in the high color overflow risk region, and the second term of the multi-objective optimization function is the deviation between the chromaticity coordinates of the output spectrum of the light source device and the target chromaticity coordinates.

[0105] The multi-objective optimization function contains two mutually constraining objective terms. The first term is the cumulative value of the product of reflectance data and the output spectrum of the light source device in the high-risk area of ​​color bleeding, which is used to minimize the degree of color bleeding on the foreground object. The second term is the deviation between the chromaticity coordinates of the output spectrum of the light source device and the target chromaticity coordinates, which is used to maintain the visual consistency of the background image.

[0106] Chromaticity coordinates refer to the two-dimensional coordinate values ​​representing color in a color space (such as CIE-XYZ). They are used to quantify the color perception of the output spectrum of a light source device, are independent of luminance information, and facilitate the adjustment of the spectral composition while keeping the color unchanged.

[0107] The target chromaticity coordinates refer to the chromaticity coordinates that the output spectrum of the desired light source device should reach. They usually correspond to the color of the original background image or the reference color specified by the director, and serve as the benchmark reference value in the second term of the optimization function.

[0108] Specifically, embodiments of the present invention can construct a multi-objective optimization function containing two mutually constraining objectives: the first term is the reflectance data of each point within the high-bleeding-risk area. With the output spectrum of the light source device The first term is the sum of the values ​​across all pixels in the region after multiplication; this sum quantifies the degree of color bleeding on the foreground object. The second term is the chromaticity coordinates of the output spectrum. With the preset target chromaticity coordinates The Euclidean distance deviation between them is used to maintain visual consistency of the background image. Weighting coefficients are set during system initialization. (Focus on eliminating color overflow) (Focusing on maintaining the background color). The multi-objective optimization function is: .

[0109] S520. Set constraints, including that the output spectrum of the light source device is composed of a linear combination of multiple primary color spectral bases, and the total luminous flux fluctuation of the output spectrum does not exceed a preset range.

[0110] Among them, the primary color spectral base refers to the basic spectral power distribution corresponding to each color channel that constitutes the output spectrum of the light source device. Any achievable output spectrum is a linear combination of these bases, and different driving values ​​correspond to different combination coefficients.

[0111] The total luminous flux refers to the integral value of the radiant power of the output spectrum of the light source in the visible light band, which characterizes the overall brightness of the light source. In reverse optimization, its fluctuation needs to be controlled to avoid visual flicker or exposure changes.

[0112] The preset range refers to the pre-defined allowable fluctuation range of the total luminous flux (e.g., ±5%), which serves as a constraint condition for the optimization problem. It is used to ensure the brightness stability of the light source device during dynamic dimming and to prevent screen flickering or exposure jumps caused by drastic changes in luminous flux.

[0113] Specifically, embodiments of the present invention can specify the output spectrum of the light source device. It must be able to be represented as a linear combination of the four primary color spectral bases RGBW (i.e. ,in, The red primary color spectral base is the normalized spectral power distribution of the red sub-pixels of the light source device under a unit drive value. The green primary color spectral base is the normalized spectral power distribution of the green sub-pixels of the light source device under a unit driving value. The blue primary color spectral base is the normalized spectral power distribution of the blue sub-pixels of the light source device under a unit driving value. The white primary color spectrum base represents the normalized spectral power distribution of the white sub-pixels (or additional auxiliary primary colors such as amber and cyan) of the light source device under a unit drive value; a, b, c, and d are the combination coefficients corresponding to the drive values. Simultaneously, a total luminous flux fluctuation constraint is set: the change in total luminous flux before and after optimization must not exceed a preset range (e.g., ±5%) to ensure stable brightness of the light source device during dynamic dimming and prevent screen flickering or exposure jumps caused by drastic changes in luminous flux.

[0114] S530. Using the driving values ​​corresponding to multiple primary color spectral bases as variables, obtain a combination of driving values ​​that satisfies the constraints and minimizes the multi-objective optimization function.

[0115] Among them, the driving value refers to the numerical value that controls the luminous intensity of each primary color channel of the light source device. After mapping, it is converted into the timing duty cycle of the high-frequency pulse width modulation signal, which is used to drive the sub-pixel to emit light and synthesize the target spectral distribution.

[0116] Specifically, in this embodiment of the invention, the driving values ​​(i.e., combination coefficients a, b, c, d) corresponding to the four primary color spectral bases of RGBW can be used as variables to be solved. Using the non-negative least squares method, under the conditions of satisfying the total luminous flux fluctuation constraint and the non-negativity of the driving values ​​of each channel, the algorithm searches for driving value combinations that minimize the multi-objective optimization function. If a high-risk area for color bleeding is detected corresponding to a specific primary color (such as green), the algorithm will automatically prioritize reducing the driving value of that primary color channel, while appropriately increasing the driving values ​​of other primary colors (such as amber and white) to reduce narrowband peaks while maintaining visual chromaticity.

[0117] S540. Based on the combination of driving values, synthesize the optimized spectral distribution.

[0118] Specifically, in this embodiment of the invention, the optimal combination of driving values ​​obtained by solving the problem can be linearly weighted and synthesized with a pre-calibrated RGBW primary color spectral base to generate an optimized output spectrum of the light source device. This synthesized spectrum is consistent with the target background color in chromaticity coordinates, while effectively filling in narrow band peaks (such as the 520-540nm green light band), making the output spectrum closer to natural light or blackbody radiation curves, thereby reducing the excessive reflection energy that causes color bleeding at the physical light source level.

[0119] This invention accurately locates high-risk areas for color bleeding and extracts their reflectance data. It constructs a multi-objective optimization function with the goals of minimizing color bleeding and minimizing chromaticity deviation, which is constrained by the linear combination of the primary color spectrum base and the fluctuation of total luminous flux. The optimal combination of driving values ​​is solved to synthesize the optimized spectral distribution, thereby achieving targeted spectral compensation of the color bleeding area from the perspective of the physical properties of the light source. This effectively reduces narrowband color bleeding energy while maintaining the visual consistency of the background.

[0120] Optionally, in addition to the global spectral reconstruction optimization as described in steps S500-S540, embodiments of the present invention can perform local spectral modulation on the output spectrum of the light source device. Local spectral modulation refers to adjusting the output spectrum of the light source device's background area, independent of other areas, specifically for areas with high color bleeding risk. For example, if an actor is wearing a green semi-transparent costume, the driving value of the pure green component is reduced at the LED pixel location corresponding to the risk area, while the driving values ​​of the amber, cyan, or white components are increased to maintain visual brightness but alter the spectral composition.

[0121] Specifically, embodiments of the present invention can locate the backlight area corresponding to the high-risk area of ​​color overflow on the light source device based on the pixel coordinate set in the color overflow sensitivity map. For these local areas, an additional round of local spectral optimization will be performed: based on the global optimization results, with the main goal of reducing the degree of color overflow in the risk area, the total luminous flux is allowed to be adjusted more significantly locally within a preset range, such as prioritizing the reduction of the drive value at the narrowband peak wavelength and appropriately supplementing auxiliary primary colors (such as amber, white, or cyan). The optimized local spectral distribution is injected into the receiver card of the light source device through the block-by-block or pixel-by-pixel drive capability of the FPGA. Non-risk areas retain the globally optimized spectral distribution, thereby eliminating color overflow while maximizing the color consistency and brightness uniformity of the background image.

[0122] Optionally, in the above Figure 5 Based on one or more embodiments shown, in another optional embodiment provided by the present invention, after step S250, the method further includes:

[0123] Obtain the color difference between the displayed color of the foreground object under the output spectrum illumination of the light source device and the preset expected color; determine whether the color difference value converges to within the preset threshold; if not, increase the weight of the first term in the multi-objective optimization function and re-execute step S200.

[0124] Display color refers to the color of a foreground object in the final captured image after being illuminated by the output spectrum of a light source and simulated by the human visual system or the shooting device. This can be achieved by multiplying the spectral reflectance of the foreground object by the output spectrum of the current light source, and then performing a colorimetric conversion to obtain the corresponding chromaticity coordinates or color values ​​used for color difference calculations.

[0125] The preset expected color refers to the target color that a foreground object should appear under ideal, color-free lighting conditions (such as standard white light or natural sunlight). This color is calculated based on the inherent spectral reflectance of the object's material and a standard light source (such as a D65 light source), or as a reference skin tone, clothing color, etc., specified by the director or colorist, serving as a benchmark for color difference assessment.

[0126] The preset threshold refers to the critical value of color difference used to determine whether color bleeding has been eliminated to an acceptable level, such as the color difference value. A value less than 2.0 indicates a color deviation range that is imperceptible or negligible to the human eye. When the calculated color difference value is less than this threshold, it is considered that color overflow has been effectively suppressed, and the current optimization strategy is maintained; otherwise, the optimization weights need to be adjusted and the spectral optimization needs to be re-executed.

[0127] Specifically, in the embodiments of the present invention, the spectral data of the foreground object can be acquired again in the next frame spectral acquisition cycle. Combined with the output spectrum of the current light source device (optimized or the current actual spectrum), the display color of the foreground object in the final image is calculated by simulating the human visual system (such as CIE-XYZ) and the color science of the shooting device. Simultaneously, the expected color of the material (e.g., skin tone standard value) under preset ideal lighting without color overflow (such as a standard D65 light source) is read. After converting the displayed color and the expected color to the CIE Lab color space, the color difference value between the two is calculated. This value quantifies the degree of perceived color shift caused by the current color overflow. (Judging the color difference value) Is it less than a preset threshold (e.g., ΔE < 2.0, indicating a color difference range imperceptible to the human eye)? If If the color overflow is less than the threshold, it is determined that the color overflow has been effectively suppressed, and the current optimization strategy and weight coefficients are maintained; if If the value is greater than or equal to the threshold, it is determined that the color overflow has not met the standard, and the weight of the first term (color overflow elimination term) in the multi-objective optimization function is automatically adjusted. Increase (e.g., from 0.7 to 0.8), while correspondingly decreasing the weight of the second item (background color preservation item). This is done to strengthen the suppression of color overflow. Then, the process restarts from step S200, which involves re-acquiring spectral and depth data, recalculating the spectral reflectance of the foreground object, generating a color overflow sensitivity map, and performing spectral inverse optimization again until the color difference value converges to within the preset threshold.

[0128] This invention provides a closed-loop feedback mechanism by real-time monitoring of the color difference between the displayed color of the foreground object under the current light source's output spectrum illumination and the expected color. When the color difference fails to converge to a preset threshold, the weight of the color overflow term in the multi-objective optimization function is adaptively increased and the spectrum optimization is re-executed. This ensures that the color overflow suppression effect continues to converge to a range imperceptible to the human eye, significantly improving the system's spectral adaptability to dynamic shooting scenes and different materials.

[0129] Optionally, in this embodiment of the invention, when the color difference value converges to within a preset threshold, the spectral reflectance of the foreground object is recorded as a spectral fingerprint, and the spectral fingerprint is associated with the corresponding color overflow suppression strategy and stored in the local spectral material library; in subsequent shooting, when the same spectral fingerprint is identified again, the color overflow suppression strategy is directly called as the initial condition for reverse optimization of the output spectrum of the light source device.

[0130] Among them, spectral fingerprint refers to a unique identifier obtained by inverting spectral data to characterize the spectral reflectance properties of a specific material surface (such as skin, silk, or metal), that is, the calculated spectral reflectance. Curve. Spectral fingerprints record the reflectivity of a material at different wavelengths, which is used for subsequent material identification and matching with spillover suppression strategies.

[0131] Among them, the color bleeding suppression strategy refers to the optimal output spectrum of the light source device and its corresponding driving value combination, weight coefficient settings, and other parameter set obtained by inverse optimization for the spectral fingerprint of a specific material. The color bleeding suppression strategy records the specific spectral compensation schemes that successfully eliminate color bleeding for this material, which can be directly reused.

[0132] The local spectral material library refers to a database stored locally on the system, used to record the spectral fingerprints of different materials and their associated color overflow suppression strategies. As the shooting process progresses, the system automatically accumulates processing experience for various materials. When the same spectral fingerprint is identified again in subsequent shots, the corresponding strategy can be directly called as the initial condition, improving the system's response speed and prediction accuracy.

[0133] Specifically, embodiments of the present invention can be implemented in color difference values. Converging to a preset threshold (e.g.) When the value is less than 2.0, it indicates that the current optimization strategy has successfully eliminated color bleeding. At this point, the calculated spectral reflectance of the material will be... As the unique spectral fingerprint of this material, the optimal combination of driving values ​​and weight coefficients obtained from this reverse optimization (such as...) =0.7、 The parameters, such as (=0.3) and the optimized spectral distribution, are used as the corresponding "optimal color bleeding suppression strategy." These data are associated and stored in a local spectral material library, categorized and indexed by material type (e.g., skin, silk, metal) for rapid reuse later. During subsequent shooting, when the spectral reflectance curve of the foreground object is acquired again by the hyperspectral camera, this curve is compared with the spectral fingerprints stored in the local spectral material library. If the spectral fingerprints are determined to be identical or highly similar, the optimal color bleeding suppression strategy associated with that fingerprint is directly read from the material library, and the combination of driving values ​​or the optimized spectral distribution is used as the initial condition for the current reverse optimization. This skips the initial multi-round iterative calculations, quickly obtaining a suitable spectral compensation scheme, significantly improving the system response speed and the prediction accuracy of color bleeding suppression.

[0134] This invention, through its embodiment, stores the spectral reflectance at which color difference convergence successfully eliminates color overflow as a spectral fingerprint, associated with the corresponding color overflow suppression strategy, in a local spectral material library. When the same spectral fingerprint is encountered in subsequent shooting, the strategy is directly invoked as the initial condition, achieving adaptive learning and rapid reuse of historical successful experiences. This significantly improves the system response speed and the accuracy of predicting color overflow suppression for similar materials.

[0135] Optional, based on Figure 1 The method shown is as follows: Figure 6 The diagram shows a specific implementation of step S240 in the spectral adjustment method provided by this invention. Step S240 may specifically include:

[0136] S600: Determine the timing position of the exposure window of the shooting device.

[0137] The timing position refers to the start and end times and scanning sequence of the rolling shutter line-by-line exposure or the global shutter full-frame exposure during each frame exposure. This timing position can be accurately determined by reading the camera's sync signal (Genlock), which is then used for subsequent synchronization with the drive signals of the light source equipment.

[0138] Specifically, in this embodiment of the invention, an FPGA control card can receive the Genlock synchronization signal output by the shooting device via a BNC line, and parse the frame synchronization and line synchronization information in the signal to determine the exposure start time, exposure duration, and line scanning order of the current frame. For a rolling shutter, the exposure start time and end time of each line also need to be obtained; for a global shutter, the exposure window range of the entire frame is obtained.

[0139] S610, Decompose the optimized spectral distribution into the driving values ​​of each color channel of the light source device.

[0140] Specifically, in this embodiment of the invention, the optimized spectral distribution can be used as the target spectrum. Using a pre-calibrated RGBW four primary color spectral base, the driving value combination corresponding to each channel is solved by non-negative least squares method. These driving values ​​correspond to the luminous intensity coefficients of the red, green, blue, and white sub-pixels, respectively.

[0141] S620: Maps the driving value to the timing duty cycle of a high-frequency modulation signal.

[0142] The high-frequency modulation signal refers to the driving signal generated using high-frequency pulse width modulation technology. Its refresh rate is greater than 3840Hz, enabling microsecond-level multi-primary color timing mixing. This signal converts the driving values ​​of each color channel into pulse sequences with different duty cycles, used to control the emission duration of sub-pixels in the light source device.

[0143] The timing duty cycle refers to the ratio of the high-level (driving LED light emission) time to the total cycle time within one period of the high-frequency modulation signal. For each color channel, the optimized driving value is mapped to the corresponding timing duty cycle, and the spectral energy output of that channel can be linearly controlled by adjusting the duty cycle.

[0144] Specifically, in this embodiment of the invention, an FPGA can be used to convert the drive values ​​of each channel into the timing duty cycle of the corresponding high-frequency pulse width modulation signal. Since the refresh rate of the light source device is set to 3840Hz, each modulation cycle is approximately 260 microseconds. Based on the magnitude of the drive value, the FPGA calculates the duration of the high level (i.e., the duty cycle) within each cycle. For example, drive value 0 corresponds to 0% duty cycle (fully off), drive value 1 corresponds to 100% duty cycle (fully on), and intermediate values ​​are linearly mapped to their corresponding duty cycles.

[0145] S630: Based on the timing position, synchronize the timing duty cycle with the shutter exposure line of the shooting device to generate a timing mixing control signal.

[0146] In this context, "shutter exposure line" refers to the process where, for cameras using rolling shutters, the image sensor exposes line by line, with each line constituting a shutter exposure line. Based on the timing position, the timing duty cycle of each color channel is precisely synchronized with the exposure window of that line, ensuring that the cumulative spectral energy received by the pixels in that line during exposure matches the optimized spectral distribution.

[0147] Specifically, in this embodiment of the invention, an FPGA can precisely align the timing duty cycle signals of each color channel with the shutter exposure lines of the imaging device based on the timing position of the exposure window. For rolling shutters, the FPGA generates an independent set of RGBW duty cycle signals for each row of pixels according to the row scanning order and injects them into the driving circuit of the light source device to ensure that the cumulative spectral energy received by that row when exposed to the sensor conforms to the optimized spectral distribution. For global shutters, the FPGA simultaneously outputs the duty cycle signals of each channel within the entire frame exposure window, generating the final timing mixing control signal, which is written to the receiving card of the light source device to drive the light source device to display the spectrally corrected background image.

[0148] As examples, during the timing-based color mixing control signal generation stage, the FPGA control card generates corresponding drive value mapping tables for different spatial regions of the light source device based on the coordinate information provided by the color bleeding sensitivity map. For regions marked as high color bleeding risk areas, locally optimized drive value combinations are used; for non-risk areas, globally optimized drive value combinations are used. All drive values ​​are converted into high-frequency pulse width modulation signals and synchronized with the shutter exposure line of the imaging device to achieve local spectral reconstruction in the spatial dimension.

[0149] This invention decomposes the optimized spectral distribution into the driving values ​​of each color channel and maps them to the timing duty cycle of a high-frequency modulation signal. Then, it precisely synchronizes the optimized spectral distribution with the shutter exposure line of the shooting device. This ensures that the cumulative spectral energy received by the sensor within a single frame exposure time is consistent with the optimized spectral distribution. Thus, it achieves precise spectral reconstruction from the light source to the imaging end at the physical level, avoiding color overflow compensation failure caused by timing misalignment.

[0150] Although the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous.

[0151] It should be understood that the various steps described in the method embodiments of the present invention may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present invention is not limited in this respect.

[0152] Corresponding to the above method embodiments, this invention also provides a spectral adjustment device, the structure of which is as follows: Figure 7 As shown, it may include: a camera synchronous data acquisition unit 10, a spectral reflectance determination unit 20, a color overflow sensitivity spectrum generation unit 30, a spectral distribution acquisition unit 40, a signal acquisition unit 50, and a light source control unit 60.

[0153] The shooting device synchronization data acquisition unit 10 is used to acquire spectral data and depth data that are triggered synchronously with the shooting device.

[0154] The spectral reflectance determination unit 20 is used to determine the spectral reflectance of a foreground object using spectral data and depth data.

[0155] The color spill sensitivity spectrum generation unit 30 is used to generate a color spill sensitivity spectrum using spectral reflectance.

[0156] The spectral distribution acquisition unit 40 is used to reverse optimize the output spectrum of the light source device using the color overflow sensitivity map to obtain the optimized spectral distribution.

[0157] The signal acquisition unit 50 is used to decompose the optimized spectral distribution into driving signals for driving the light source device, and generate a timing mixing control signal synchronized with the exposure window of the shooting device.

[0158] The light source control unit 60 is used to inject a driving signal into the light source device within a single frame exposure time according to the timing mixing control signal, so as to reconstruct the output spectrum of the light source device.

[0159] Optionally, the spectral reflectance determination unit 20 can be used to extract at least one region of interest corresponding to the foreground object in the current scene image output by the shooting device; determine the incident angle of each point in the region of interest using depth data; and determine the spectral reflectance of the region of interest using spectral data and the incident angle of each point in the region of interest.

[0160] Optionally, the color bleeding sensitivity map generation unit 30 can be used to obtain the spectral power value of the light source device at at least one narrowband peak wavelength; extract the reflectance value of the corresponding wavelength from the spectral reflectance for the narrowband peak wavelength; obtain the predicted reflection intensity of the region of interest using the spectral power value and the reflectance value; compare the predicted reflection intensity with the expected intensity, and determine the corresponding region of interest as a high color bleeding risk area if the predicted reflection intensity exceeds the expected intensity; and record the set of pixel coordinates of the high color bleeding risk area in the current scene as a color bleeding sensitivity map.

[0161] Optionally, the spectral distribution acquisition unit 40 can be used to obtain reflectance data in the spectral reflectance of high-risk areas for color bleeding; construct a multi-objective optimization function, wherein the first term of the multi-objective optimization function is the cumulative value of the product of reflectance data and the output spectrum of the light source device in the high-risk areas for color bleeding, and the second term of the multi-objective optimization function is the deviation between the chromaticity coordinates of the output spectrum of the light source device and the target chromaticity coordinates; set constraints, including that the output spectrum of the light source device is linearly combined from multiple primary color spectral bases, and the total luminous flux fluctuation of the output spectrum does not exceed a preset range; use the driving values ​​corresponding to the multiple primary color spectral bases as variables to obtain a combination of driving values ​​that satisfies the constraints and minimizes the multi-objective optimization function; and synthesize the optimized spectral distribution based on the combination of driving values.

[0162] Optionally, the spectral adjustment device may further include: a color difference value acquisition unit and a color difference convergence determination unit.

[0163] Color difference value acquisition unit. Used to obtain the color difference value between the displayed color of the foreground object under the output spectrum illumination of the light source device and the preset expected color.

[0164] The color difference convergence determination unit is used to determine whether the color difference value has converged to within the preset threshold. If not, the weight of the first term in the multi-objective optimization function is increased, and the shooting device synchronous data acquisition unit 10 is triggered.

[0165] Optionally, the spectral adjustment device may also include a spectral fingerprint application unit.

[0166] The spectral fingerprint application unit is used to record the spectral reflectance of the foreground object as a spectral fingerprint when the color difference convergence determination unit determines that the color difference value has converged to within a preset threshold. The spectral fingerprint is then associated with the corresponding color overflow suppression strategy and stored in the local spectral material library. In subsequent shooting, when the same spectral fingerprint is identified again, the color overflow suppression strategy is directly called as the initial condition for reverse optimization of the output spectrum of the light source device.

[0167] Optionally, the signal acquisition unit 50 can be used to determine the timing position of the exposure window of the shooting device; decompose the optimized spectral distribution into the driving values ​​of each color channel of the light source device; map the driving values ​​into the timing duty cycle of the high-frequency modulation signal; and synchronize the timing duty cycle with the shutter exposure line of the shooting device according to the timing position to generate a timing mixing control signal.

[0168] This invention provides a spectral adjustment device that, by synchronously triggering spectral and depth data, calculates the spectral reflectance of foreground objects in real time, overcoming the limitation of existing RGB three-channel systems that cannot infer the physical causes of color bleeding. The resulting color bleeding sensitivity map can accurately locate high-risk areas, avoiding the indiscriminate sacrifice of background color by traditional global adjustments. Furthermore, based on this map, the output spectrum of the light source is optimized in reverse and decomposed into a temporal mixing drive signal synchronized with the exposure window of the shooting device, completing spectral reconstruction within a single frame exposure time. This process directly compensates for the narrowband peak components that cause color bleeding at the light source end, without relying on post-processing color bleeding removal algorithms, thus avoiding damage to the normal color details of foreground objects and solving the problem of physical light-blocking flags being unable to adapt to dynamic movement. Ultimately, it achieves the technical effect of eliminating color bleeding from the physical properties of the light source and optimizing the spectral fusion of the foreground and virtual background.

[0169] Regarding the apparatus in the above embodiments, the specific manner in which each unit performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0170] The spectral adjustment device includes a processor and a memory. The aforementioned synchronous data acquisition unit 10, spectral reflectance determination unit 20, color overflow sensitivity spectrum generation unit 30, spectral distribution acquisition unit 40, signal acquisition unit 50, and light source control unit 60 are all stored as program units in the memory. The processor executes the aforementioned program units stored in the memory to realize the corresponding functions.

[0171] The processor contains a kernel that retrieves the corresponding program units from memory. One or more kernels can be configured. By adjusting kernel parameters, the spectral reflectance of foreground objects is calculated in real time using synchronously acquired spectral and depth data, generating a color bleeding sensitivity map. Based on this, the spectral distribution of the light source is inversely optimized and converted into a temporal mixing drive signal synchronized with the exposure window of the shooting device. Within a single frame exposure time, the output spectrum of the light source is dynamically reconstructed, thereby eliminating color bleeding from the physical properties of the light source and achieving spectral fusion between the foreground and background.

[0172] This invention provides a computer-readable storage medium storing a program that, when executed by a processor, implements the spectral adjustment method.

[0173] This invention provides a processor for running a program, wherein the program executes the spectral adjustment method during runtime.

[0174] like Figure 8 As shown, this embodiment of the invention provides an electronic device 1000, which includes at least one processor 1001, at least one memory 1002 connected to the processor 1001, and a bus 1003. The processor 1001 and the memory 1002 communicate with each other via the bus 1003. The processor 1001 is used to call program instructions in the memory 1002 to execute the aforementioned spectral adjustment method. The electronic device described herein can be a server, PC, PAD, mobile phone, etc.

[0175] The present invention also provides a computer program product that, when executed on an electronic device, is suitable for executing a program that initializes a spectral adjustment method step.

[0176] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatuses, electronic devices (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0177] In a typical configuration, an electronic device includes one or more processors (CPUs), memory, and a bus. The electronic device may also include input / output interfaces, network interfaces, etc.

[0178] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, like read-only memory (ROM) or flash RAM, and memory includes at least one memory chip. Memory is an example of computer-readable media.

[0179] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0180] In the description of this invention, it should be understood that if the terms "upper", "lower", "front", "rear", "left" and "right" are used to indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the position or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this invention.

[0181] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0182] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0183] The above are merely embodiments of the present invention and are not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of the present invention should be included within the scope of the present invention.

Claims

1. A spectral adjustment method, characterized in that, include: Obtain spectral and depth data that are triggered synchronously with the imaging equipment; Using the spectral data and the depth data, the spectral reflectance of the foreground object is determined; Using the spectral reflectance, an overflow sensitivity map is generated; Using the aforementioned color overflow sensitivity map, the output spectrum of the light source device is inversely optimized to obtain the optimized spectral distribution; The optimized spectral distribution is decomposed into driving signals for driving the light source device, and a timing mixing control signal synchronized with the exposure window of the shooting device is generated. Based on the timing mixing control signal, the driving signal is injected into the light source device to reconstruct the output spectrum of the light source device.

2. The method according to claim 1, characterized in that, Determining the spectral reflectance of a foreground object using the spectral data and the depth data includes: Extract at least one region of interest corresponding to the foreground object from the current scene image output by the shooting device; Using the depth data, the angle of incidence at each point within the region of interest is determined; The spectral reflectance of the region of interest is determined using the spectral data and the incident angle at each point within the region of interest.

3. The method according to claim 2, characterized in that, The step of generating a color bleeding sensitivity map using the spectral reflectance includes: Obtain the spectral power value of the light source device at at least one narrowband peak wavelength; For the narrowband peak wavelength, the reflectance value corresponding to the wavelength is extracted from the spectral reflectance; The predicted reflection intensity of the region of interest is obtained using the spectral power value and the reflectance value. The predicted reflection intensity is compared with the expected intensity. If the predicted reflection intensity exceeds the expected intensity, the corresponding region of interest is identified as a high-risk area for color bleeding. The set of pixel coordinates of the high-risk area for color bleeding in the current scene is recorded as the color bleeding sensitivity map.

4. The method according to claim 3, characterized in that, The step of using the spillover sensitivity map to inversely optimize the output spectrum of the light source device to obtain an optimized spectral distribution includes: Obtain the reflectance data of the high-risk area for color bleeding in the spectral reflectance; A multi-objective optimization function is constructed, wherein the first term of the multi-objective optimization function is the cumulative value of the product of the reflectance data and the output spectrum of the light source device in the high color bleeding risk region, and the second term of the multi-objective optimization function is the deviation between the chromaticity coordinates of the output spectrum of the light source device and the target chromaticity coordinates; Set constraints, wherein the constraints include that the output spectrum of the light source device is composed of a linear combination of multiple primary color spectral bases, and the total luminous flux fluctuation of the output spectrum does not exceed a preset range; Using the driving values ​​corresponding to the multiple primary color spectral bases as variables, a combination of driving values ​​that satisfies the constraints and minimizes the multi-objective optimization function is obtained; Based on the aforementioned combination of driving values, an optimized spectral distribution is synthesized.

5. The method according to claim 4, characterized in that, After injecting the driving signal into the light source device according to the timing mixing control signal to reconstruct the output spectrum of the light source device, the method further includes: The color difference value between the displayed color of the foreground object under the output spectrum illumination of the light source device and the preset expected color is obtained; Determine whether the color difference value converges to within a preset threshold. If not, increase the weight of the first term in the multi-objective optimization function and re-execute the step of obtaining spectral data and depth data that are synchronously triggered with the shooting device.

6. The method according to claim 5, characterized in that, Also includes: If the color difference value converges to within a preset threshold, the spectral reflectance of the foreground object is recorded as a spectral fingerprint, and the spectral fingerprint is associated with the corresponding color overflow suppression strategy and stored in the local spectral material library. In subsequent shooting, when the same spectral fingerprint is identified again, the overflow suppression strategy is directly invoked as the initial condition for reverse optimization of the output spectrum of the light source device.

7. The method according to any one of claims 1 to 6, characterized in that, The step of decomposing the optimized spectral distribution into driving signals for driving the light source device and generating a timing mixing control signal synchronized with the exposure window of the imaging device includes: Determine the timing position of the exposure window of the shooting device; The optimized spectral distribution is decomposed into the driving values ​​of each color channel of the light source device; The driving value is mapped to the timing duty cycle of a high-frequency modulation signal; Based on the timing position, the timing duty cycle is synchronized with the shutter exposure line of the shooting device to generate a timing mixing control signal.

8. A spectral adjustment device, characterized in that, include: The system includes a synchronous data acquisition unit for the shooting equipment, a spectral reflectance determination unit, a color overflow sensitivity map generation unit, a spectral distribution acquisition unit, a signal acquisition unit, and a light source control unit. The camera device synchronization data acquisition unit is used to acquire spectral data and depth data that are triggered synchronously with the camera device. The spectral reflectance determination unit is used to determine the spectral reflectance of the foreground object using the spectral data and the depth data; The color spill sensitivity spectrum generation unit is used to generate a color spill sensitivity spectrum using the spectral reflectance; The spectral distribution acquisition unit is used to reverse optimize the output spectrum of the light source device using the color overflow sensitivity map to obtain the optimized spectral distribution; The signal acquisition unit is used to decompose the optimized spectral distribution into a driving signal for driving the light source device, and generate a timing mixing control signal synchronized with the exposure window of the shooting device. The light source control unit is used to inject the driving signal into the light source device according to the timing mixing control signal, so as to reconstruct the output spectrum of the light source device.

9. A computer-readable storage medium having a program stored thereon, characterized in that, When the program is executed by the processor, it implements the spectral adjustment method as described in any one of claims 1 to 7.

10. An electronic device, characterized in that, The electronic device includes at least one processor, at least one memory connected to the processor, and a bus; wherein the processor and the memory communicate with each other through the bus; the processor is used to call program instructions in the memory to execute the spectral adjustment method as described in any one of claims 1 to 7.

Images