Method and device for determining alignment area in virtual and real light alignment, virtual and real light alignment method and device and medium

Abstract

The invention relates to an alignment area determination method and device in virtual and real light alignment and a virtual and real light alignment method and device and a medium, and the method comprises the steps: obtaining a camera image of a virtual shooting scene, the virtual shooting scene comprises a screen used for displaying a virtual scene and a real scene located in front of the screen; extracting a real scene setting area and a virtual scene setting area in the camera picture, and respectively dividing the real scene setting area and the virtual scene setting area into a plurality of real blocks and a plurality of virtual blocks; performing material and color matching on the plurality of virtual blocks and the plurality of real blocks to obtain a plurality of groups of candidate alignment regions; and selecting at least one group of target alignment areas from the plurality of groups of candidate alignment areas so as to perform virtual and real light alignment on the virtual shooting scene by using the at least one group of target alignment areas. Therefore, the alignment area can be more accurately, objectively and automatically determined, and the virtual and real light alignment efficiency and the alignment effect are improved.

Description

Technical Field

[0001] This disclosure relates to the field of virtual shooting, and more particularly to a method, apparatus and medium for determining the alignment area and aligning virtual and real lights in the process of aligning virtual and real lights. Background Technology

[0002] Virtual shooting technology is widely used in film, television, games, advertising, and other fields. It presents virtual sets on LED screens, combining them with real sets in front of the screen to create a highly realistic scene that blends virtual and real elements. To achieve an integrated expression of virtual and real light and shadow, the physical lighting in the real set needs to be aligned with the virtual lighting of the virtual set displayed on the LED screen to ensure visual consistency.

[0003] However, in existing technologies, aligning virtual and real lighting typically requires lighting technicians to visually select reference points with similar colors and materials in both the virtual and real sets. This allows them to adjust the lighting effect of the physical lighting system in the virtual scene on the real set, ensuring consistency between the lighting effects in the virtual and real sets. This manual intervention not only increases operational complexity and reduces efficiency but also relies heavily on human experience and is highly subjective, thus affecting the alignment of virtual and real lighting and consequently impacting the automation and efficiency improvement of the entire virtual shooting process. Summary of the Invention

[0004] In view of this, this disclosure proposes a method, apparatus and medium for determining the alignment area and performing virtual and real light alignment in virtual and real light alignment, which can more accurately and objectively determine the alignment area automatically to perform virtual and real light alignment, thereby reducing manual intervention and improving the efficiency and effect of virtual and real light alignment.

[0005] According to one aspect of this disclosure, a method for determining an alignment region in virtual-real lighting alignment is provided, comprising: acquiring a camera view of a virtual shooting scene, the virtual shooting scene including: a screen for displaying a virtual scene and a real scene located in front of the screen; extracting the real scene area and the virtual scene area from the camera view, and dividing the real scene area and the virtual scene area into multiple real blocks and multiple virtual blocks respectively; obtaining multiple sets of candidate alignment regions by performing material and color matching between the multiple virtual blocks and the multiple real blocks, each set of candidate alignment regions including a set of target real blocks and target virtual blocks with matching materials and colors; selecting at least one set of target alignment regions from the multiple sets of candidate alignment regions, so as to use the at least one set of target alignment regions to perform virtual-real lighting alignment on the virtual shooting scene.

[0006] In one possible implementation, the step of obtaining multiple sets of candidate alignment regions by matching the material and color of the multiple virtual blocks with the multiple real blocks includes: determining the color uniformity of each virtual block and selecting multiple target virtual blocks from the multiple virtual blocks whose color uniformity is greater than a preset threshold; determining the color uniformity of each real block and selecting multiple target real blocks from the multiple real blocks whose color uniformity is greater than the preset threshold; determining the color features and texture features of each target virtual block and each target real block respectively, and determining multiple sets of candidate alignment regions based on the color features and texture features of each target virtual block and each target real block, wherein the texture features represent the material and the color features represent the dominant color tone.

[0007] In one possible implementation, the multiple sets of candidate alignment regions are determined based on the texture similarity and color similarity between the target real block and the target virtual block, wherein the texture similarity represents the similarity between the texture features of the target real block and the texture features of the target virtual block, and the color similarity represents the similarity between the color features of the target real block and the color features of the target virtual block.

[0008] In one possible implementation, the multiple sets of candidate alignment regions are determined based on the comprehensive matching degree between the target real block and the target virtual block, wherein the comprehensive matching degree is a weighted sum of texture similarity and color similarity.

[0009] In one possible implementation, selecting at least one target alignment region from the plurality of candidate alignment regions includes: obtaining the comprehensive matching degree of each candidate alignment region, wherein the comprehensive matching degree is determined based on the texture similarity and color similarity between the target real block and the target virtual block in each set of material and color matching; and selecting at least one candidate alignment region from the plurality of candidate alignment regions whose comprehensive matching degree is greater than a preset matching threshold as the target alignment region.

[0010] In one possible implementation, the step of extracting the real and virtual scene areas from the camera frame and dividing the real and virtual scene areas into multiple real blocks and multiple virtual blocks respectively includes: extracting the real and virtual scene areas from the camera frame using a mask image, wherein the mask image is determined by the difference between the camera frame captured when the screen is completely black and the camera frame captured when the screen is fully bright; and extracting multiple real blocks and multiple virtual blocks of the same size from the real and virtual scene areas respectively using a preset window with a preset step size.

[0011] In one possible implementation, acquiring the camera image of the virtual shooting scene includes: acquiring the original camera image captured by the camera of the virtual shooting scene, wherein the colors in the original camera image are in the RGB color space; converting the original camera image from the RGB color space to the LAB color space, so as to perform the method using the camera image in the LAB color space.

[0012] According to another aspect of this disclosure, a method for aligning virtual and real lighting is provided, including the method for determining an alignment region in the alignment of virtual and real lighting. The alignment method further includes: adjusting the lighting control parameters of the physical lighting system in the virtual shooting scene based on the difference in color values ​​between the same group of real target blocks and virtual target blocks in the at least one group of target alignment regions, until the color values ​​between the same group of real target blocks and virtual target blocks in the camera image are consistent, wherein the physical lighting system is used to illuminate the real scene.

[0013] According to another aspect of this disclosure, an apparatus for determining an alignment region in virtual-real lighting alignment is provided, comprising: an acquisition module for acquiring a camera view of a virtual shooting scene, the virtual shooting scene including: a screen for displaying a virtual scene and a real scene located in front of the screen; an extraction module for extracting a real scene area and a virtual scene area from the camera view, and dividing the real scene area and the virtual scene area into multiple real blocks and multiple virtual blocks respectively; a matching module for obtaining multiple sets of candidate alignment regions by performing material and color matching between the multiple virtual blocks and the multiple real blocks, each set of candidate alignment regions including a set of target real blocks and target virtual blocks with matching materials and colors; and a selection module for selecting at least one set of target alignment regions from the multiple sets of candidate alignment regions, so as to use the at least one set of target alignment regions to perform virtual-real lighting alignment on the virtual shooting scene.

[0014] According to another aspect of this disclosure, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the above-described method.

[0015] According to another aspect of this disclosure, a non-volatile computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps of the above-described method.

[0016] According to another aspect of this disclosure, a computer program product is provided, including a computer program or a non-volatile computer-readable storage medium carrying the computer program, wherein the computer program, when executed by a processor, implements the steps of the above-described method.

[0017] According to various aspects of this disclosure, by extracting the real and virtual scene areas from the camera image and dividing them into multiple real blocks and multiple virtual blocks respectively, and then matching the colors and materials between each block to obtain multiple sets of target real blocks and target virtual blocks with matching materials and colors, and then selecting at least one set of target alignment areas from multiple sets of candidate alignment areas, it is possible to automatically select more accurate and objective target alignment areas to perform virtual and real lighting alignment, reduce manual intervention, improve the efficiency and effect of virtual and real lighting alignment, thereby facilitating the automation and efficiency improvement of the entire virtual shooting process.

[0018] Other features and aspects of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0019] The accompanying drawings, which are included in and form part of this specification, illustrate exemplary embodiments, features, and aspects of this disclosure together with the specification and serve to explain the principles of this disclosure.

[0020] Figure 1 A schematic diagram of a virtual shooting scene according to an embodiment of the present disclosure is shown.

[0021] Figure 2 A flowchart illustrating a method for determining an alignment region in aligning virtual and real lights according to an embodiment of the present disclosure is shown.

[0022] Figure 3 A block diagram of an apparatus for determining an alignment region in virtual and real light alignment according to an embodiment of the present disclosure is shown.

[0023] Figure 4 A block diagram of an electronic device according to an embodiment of the present disclosure is shown. Detailed Implementation

[0024] Various exemplary embodiments, features, and aspects of this disclosure will now be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of the embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.

[0025] As used herein, the terms “comprising,” “including,” “having,” or variations thereof are open-ended and include one or more of the stated features, integrals, elements, steps, components, or functions, but do not exclude the presence or addition of one or more other features, integrals, elements, steps, components, functions, or groups thereof.

[0026] When an element is referred to as “connected,” “coupled,” “responding,” or a variation thereof relative to another element, it may be directly connected, coupled, or responding to another element, or there may be an intermediate element present.

[0027] Although the terms first, second, third, etc., may be used herein to describe various elements / operations, these elements / operations should not be limited by these terms. These terms are only used to distinguish one element / operation from another. Therefore, without departing from the teachings of the inventive concept, a first element / operation in some embodiments may be referred to as a second element / operation in other embodiments.

[0028] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments.

[0029] Furthermore, to better illustrate this disclosure, numerous specific details are set forth in the following detailed description. Those skilled in the art will understand that this disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art have not been described in detail in order to highlight the main points of this disclosure.

[0030] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, data stored, data displayed, etc.) and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant regions.

[0031] Figure 1 The diagram illustrates a virtual shooting scene according to an embodiment of the present disclosure, such as... Figure 1 As shown, the virtual shooting scene includes a physical camera 01, a physical screen 02, a physical lighting system 03, a main control device 04, and a real set 05.

[0032] Depending on the lens used, the physical camera 01 can be a telephoto camera, a wide-angle camera, etc.; depending on the shutter used, the physical camera 01 can be a rolling shutter camera, a global shutter camera, etc., and this embodiment does not impose any limitations on this. The physical screen 02 can be an LED screen, which can be a curved screen or a flat screen, etc. The type, number, size, resolution, etc. of the physical screen 02 in the virtual shooting system can be customized according to actual needs, and this embodiment does not impose any limitations on this. The physical lighting system 03 can be any known lighting system for photography. It should be understood that those skilled in the art can customize the type, number, position, etc. of the lights in the physical lighting system 03 according to actual needs, and this embodiment does not impose any limitations on this.

[0033] The main control device 04 can be an electronic device with computing and processing control capabilities, such as a desktop computer or laptop computer. The main control device 04 can establish communication connections with the physical camera 01, the physical screen 02, and the physical lighting system 03, respectively. It should be understood that this embodiment does not limit the communication connection method between devices. For example, the main control device 04 and the physical lighting system 03 can use the DMX (Digital Multiplex) protocol to establish a communication connection. The DMX protocol is a digital communication protocol widely used in stage lighting, performance venues, and architectural lighting, mainly used to control the brightness, color, and other effects of lighting equipment.

[0034] The main control device 04 can be equipped with Unreal Engine (UE), which can construct a virtual environment consistent with the real shooting environment. The real shooting environment includes the physical camera 01, physical screen 02, and physical lighting system 03. The virtual environment includes a virtual camera, virtual screen, and virtual scene. The virtual scene includes a virtual lighting system and virtual scenery (such as various virtual objects). The virtual lighting system is used to simulate lighting effects in the real world, such as virtual light sources to simulate the sunlight rising from the east, virtual light sources to simulate indoor lighting, and virtual light sources to simulate candlelight. These virtual light sources will cause objects in the virtual scenery to produce light and shadow effects. The camera intrinsics of the virtual camera in the virtual shooting system are consistent with the camera intrinsics of the physical camera 01. The relative positional relationship between the virtual camera and the virtual screen is consistent with the relative positional relationship between the physical camera 01 and the physical screen 02. The virtual screen can be understood as a 1:1 replica of the screen model of the physical screen 02. The shape, size, etc. of the virtual screen are consistent with the physical screen 02.

[0035] During the virtual shooting process using the aforementioned virtual shooting scene, the main control device 04 can drive the virtual camera in Unreal Engine to render and generate a rendered image of the virtual scene. After the rendered image is transformed into a 3D projection, it is projected onto the screen model and then mapped onto the physical screen 02. That is, the physical screen 02 can display the pre-made virtual scene. At the same time, the actors can perform in front of the physical screen 02, and the main control device 04 can control the physical camera 01 to shoot. In the shooting image, the actors and the real scene 05 around the actors are used as the foreground, and the virtual scene in the physical screen 02 is used as the background. The shooting image can also be synchronously transmitted to the director's monitor, where the visual effect of the real actors' performance and the virtual scene being integrated can be presented in real time, thereby realizing virtual shooting.

[0036] It should be understood that Figure 1The real-scene setup 05 shown is merely an exemplary implementation. In actual virtual shooting scenarios, the real-scene setup in front of the screen can be set according to actual shooting needs. For example, the real-scene setup includes physical props, environmental backgrounds, or people arranged on the shooting location. In virtual shooting, the lighting of the real-scene setup needs to be aligned with that of the virtual-scene setup to achieve a better blending effect between the real-scene setup area and the virtual-scene setup area in the same camera frame.

[0037] To achieve alignment between real and virtual lighting, the control parameters (such as brightness, color temperature, and color cast) of the physical lighting system 03 can be adjusted to align the lighting effects of the physical lighting system 03 on the real set with the lighting effects of the virtual set displayed on the LED screen at the shooting location, thus achieving a realistic and vivid fusion of real and virtual lighting. However, as mentioned above, real-virtual lighting alignment typically requires the lighting technician to visually select reference points with similar colors and materials in both the virtual and real sets. These reference points are then used as a benchmark to adjust the lighting effects of the physical lighting system on the real set in the virtual shooting scene, ensuring consistency between the lighting effects of the real and virtual sets. This method increases operational complexity, is less efficient, and relies heavily on human experience, making it highly subjective and thus affecting the alignment effect and efficiency of real-virtual lighting.

[0038] Therefore, this disclosure addresses the need for virtual and real lighting alignment in virtual shooting operations by providing a method for automatically selecting alignment areas with similar materials and uniform colors from both virtual and real set areas. This method can assist lighting technicians in selecting alignment areas more accurately and objectively to achieve precise and efficient virtual and real lighting alignment. It can also serve as input for similar lighting control algorithms that require manual intervention in area selection, thereby facilitating the complete automation of virtual and real lighting alignment in virtual shooting, reducing manual intervention, and improving operational efficiency.

[0039] It should be noted that the methods of this disclosure can be deployed on various terminal devices (such as the aforementioned main control device 04) through software or hardware modifications. The terminal devices involved in this disclosure can refer to devices with wireless and / or wired connection functions. Wireless connection means that they can connect to other devices via Wi-Fi, Bluetooth, or other wireless connection methods. The terminal devices involved in this disclosure can also communicate with other devices via wired connection functions. The terminal devices involved in this disclosure can be touchscreen, non-touchscreen, or screenless. Touchscreen devices can be controlled by clicking or swiping on the display screen using fingers, styluses, etc. Non-touchscreen devices can connect to input devices such as mice, keyboards, and touch panels to control the terminal device. Screenless devices can be, for example, screenless Bluetooth speakers. For example, the terminal devices in this application can include, but are not limited to, user equipment (UE), mobile devices, user terminals, terminals, handheld devices, tablet computers, laptops, PDAs, computing devices, etc.

[0040] The method of this disclosure can also be deployed on a server, which can be located in the cloud or locally, and can be a physical device or a virtual device, such as a virtual machine or container, with wireless communication capabilities. These wireless communication capabilities can be configured in the server's chip (system) or other components. This can refer to a device with wireless connectivity, meaning it can connect to other servers or terminal devices via wireless connections such as Wi-Fi or Bluetooth. The server involved in this disclosure can also have wired communication capabilities. For example, the server in this disclosure can be located in the cloud, communicate with terminal devices, receive camera images sent by the terminal devices, and use the method deployed on the server to determine a target alignment area based on the camera images, returning it to the terminal device so that the terminal device can perform a virtual-real lighting alignment process based on the target alignment area.

[0041] Figure 2 A flowchart illustrating a method for determining an alignment region in aligning virtual and real lights according to an embodiment of the present disclosure is shown. Figure 2 As shown, the method includes steps S11 to S14.

[0042] In step S11, the camera view of the virtual shooting scene is acquired. The virtual shooting scene includes a screen for displaying the virtual scene and a real scene located in front of the screen.

[0043] In practical applications, after setting up the real-world set for virtual shooting and controlling the screen to display the virtual set, a camera (such as the physical camera mentioned above) can be used to capture images of the screen displaying the virtual set and the real-world set in front of the screen, thus obtaining a camera view. It should be understood that this camera view includes the real-world set area (such as the physical props, background environment, or people set up on the shooting location) and the virtual set area (i.e., the area where the virtual set is displayed on the screen in the camera view, which is also the area where the screen is located in the camera view).

[0044] Considering that the original camera footage captured by the camera is usually in the RGB color space (i.e., a color model that describes colors using three color channels: Red, Green, and Blue), and that subsequent color processing performed in the RGB color space has low accuracy and consistency, in some embodiments, acquiring the camera footage of the virtual shooting scene may include: acquiring the original camera footage captured by the camera of the virtual shooting scene, wherein the colors in the original camera footage are in the RGB color space; converting the original camera footage from the RGB color space to the LAB color space (i.e., a color model designed based on human visual perception that describes colors using a luminance component L, a red-green component A, and a blue-yellow component B), so as to use the camera footage in the LAB color space to perform the method of the embodiments of this disclosure. Therefore, unless otherwise specified, the camera footage in the embodiments of this disclosure is all in the LAB color space, and the pixel colors in the candidate regions extracted from the camera footage and the blocks divided from the candidate regions are also in the LAB color space.

[0045] The LAB color space is a uniform color space defined by the International Commission on Illumination (ICI). The luminance component represents the brightness of a color, reflecting its lightness or darkness; the red-green component represents the color component along the red-green axis; and the blue-yellow component represents the color component along the yellow-blue axis. It should be understood that those skilled in the art can use relevant color space conversion methods to convert the original camera image from the RGB color space to the LAB color space, that is, to convert the RGB color values ​​of each pixel in the original camera image to LAB color values. This disclosure does not limit this approach. Because the LAB color space has a luminance and color separation structure that better conforms to the characteristics of human visual perception, it can more accurately represent the color differences between the real and virtual scene areas in the camera image. This is beneficial for enhancing the accuracy of subsequent color processing and improving the consistency of color perception, thereby improving the accuracy of subsequent matching and alignment areas.

[0046] In step S12, the real scene area and the virtual scene area in the camera image are extracted, and the real scene area and the virtual scene area are divided into multiple real blocks and multiple virtual blocks respectively.

[0047] In practical applications, those skilled in the art can employ known image processing techniques or artificial intelligence techniques to extract the real and virtual scene areas from the camera image, and divide the real and virtual scene areas into multiple real blocks and multiple virtual blocks, respectively. This disclosure does not limit this approach. For example, edge detection algorithms or semantic segmentation networks can be used to perform region recognition on the camera image, separating the real and virtual scene areas at the pixel level.

[0048] In some embodiments, considering that film and television shooting is usually done in scenes, the position and layout of the real set in the same scene are usually unchanged, and the position of the screen displaying the virtual set is fixed. Therefore, in the camera footage captured by a camera with a fixed orientation, the positions of the real set area and the virtual set area are also unchanged. Thus, in some embodiments, in order to improve the extraction efficiency of the real set area and the virtual set area, a mask image for distinguishing the real set area and the virtual set area can be obtained in advance, and the real set area and the virtual set area in the camera footage (that is, the pixel set that distinguishes the real set area and the virtual set area in the camera footage) can be directly extracted using this mask image. Specifically, extracting the real set area and the virtual set area in the camera footage may include: extracting the real set area and the virtual set area in the camera footage using a mask image, wherein the mask image is determined using the difference between the camera footage captured when the screen is completely black and the camera footage captured when the screen is completely bright.

[0049] In this context, "screen completely black" means setting the entire screen to its lowest brightness, or displaying all black; "screen completely bright" means setting the entire screen to its highest brightness, or displaying all white. After setting up the actual scene in front of the screen, a camera can capture two images: one with the screen completely black and the other with the screen completely bright. Subtracting the two images yields a mask that distinguishes the virtual scene area from the real scene area. It should be understood that since the real scene remains unchanged in both images, but the colors in the screen area (i.e., the virtual scene area) change, subtracting the two images (i.e., subtracting the color values ​​of the pixels in the two images) reveals that areas with smaller pixel values ​​(e.g., 0) in the mask represent the real scene area, while areas with larger pixel values ​​represent the virtual scene area.

[0050] In some embodiments, to efficiently divide the image into blocks for subsequent block matching, a sliding window method (e.g., setting the window size to 100×100 pixels and the window movement step size to 100 pixels) can be used to extract multiple real blocks and multiple virtual blocks in both the virtual and real scene areas. Thus, the above-mentioned division of the real and virtual scene areas into multiple real blocks and multiple virtual blocks can include: using a preset window with a preset step size to extract multiple real blocks and multiple virtual blocks of the same size from both the real and virtual scene areas. The sliding window method refers to using a fixed-size window in an image, sliding it gradually from left to right and from top to bottom with a specified step size to extract a local region. Therefore, a preset window (e.g., 100×100 pixels) can be used in both the real and virtual scene areas, sliding gradually from left to right and from top to bottom with a preset step size (e.g., 100 pixels) to obtain multiple real blocks and multiple virtual blocks of the same size.

[0051] In practical applications, a frame of the same size as the preset window can be used to identify the divided blocks, and the pixel positions of each block (i.e., each frame) in the camera frame can be used to identify different blocks. Thus, a set of aligned regions determined later can be identified by the pixel positions of two blocks in the set of aligned regions. This disclosure does not limit this aspect.

[0052] In step S13, multiple sets of candidate alignment regions are obtained by matching the material and color of multiple virtual blocks with multiple real blocks. Each set of candidate alignment regions includes a set of target real blocks and target virtual blocks that are matched in material and color.

[0053] In this context, a set of target real and target virtual blocks with matching materials and colors can be understood as a set of target real and target virtual blocks with similar materials, uniform color distribution, and the same dominant color tone. Similar materials between two blocks can be represented by similar textures within the two blocks. Uniform color distribution within any block can be understood as the color values ​​of the pixels within that block being evenly distributed (i.e., small color differences or minimal color fluctuations between pixels within the block). Consistent dominant color tone between two blocks can be understood as the main color tone being consistent within the two blocks (e.g., both blocks are predominantly red). Similar materials ensure that a set of matched target real and target virtual blocks are as similar as possible (e.g., both being flooring). Uniform color distribution avoids multiple significantly different colors within the same block (e.g., a block containing white walls and gray floors has uneven color distribution, while a block containing only gray floors has uniform color distribution). The same dominant color tone ensures that a set of target real and target virtual blocks are as similar as possible.

[0054] By matching real and virtual target blocks with similar materials, uniform color distribution, and the same main color tone, the accuracy and reliability of virtual and real light alignment can be improved when using the target alignment area for virtual and real light alignment in the future.

[0055] In some embodiments, step S13 above, which involves matching the material and color of multiple virtual blocks with multiple real blocks to obtain multiple sets of candidate alignment regions, may include:

[0056] Step S131: Determine the color uniformity of each virtual block, and select multiple target virtual blocks from multiple virtual blocks whose color uniformity is greater than a preset threshold.

[0057] Step S132: Determine the color uniformity of each real block, and select multiple target real blocks from multiple real blocks whose color uniformity is greater than a preset threshold.

[0058] Step S133: Determine the color features and texture features of each target virtual block and each target real block respectively, and determine multiple sets of candidate alignment regions based on the color features and texture features of multiple target virtual blocks and multiple target real blocks respectively. The texture features represent the material and the color features represent the main color tone.

[0059] In steps S131 and S132, the color uniformity of each virtual block and the color uniformity of each real block can be obtained, for example, by calculating the color standard deviation within each virtual block (i.e., the standard deviation of the color values ​​of each pixel within the block) and the color standard deviation within each real block. It should be understood that the smaller the color standard deviation within a block, the more uniform the color distribution within that block, and thus the greater the color uniformity of that block. A certain mapping relationship can be established between color standard deviation and color uniformity. For example, color uniformity can be a negative value of the color standard deviation, or it can be a more complex mapping relationship; this embodiment of the present disclosure does not limit this. Therefore, calculating the color standard deviation within any block yields the corresponding color uniformity. Of course, those skilled in the art can use other techniques in the art to determine the color uniformity of each block. For example, the color uniformity of each block can be characterized by calculating the standard deviation of the global entropy value or local entropy value of the color within the block; this embodiment of the present disclosure does not limit this.

[0060] Furthermore, multiple target virtual blocks and multiple target real blocks with color uniformity greater than a preset threshold can be filtered out. Therefore, it can be assumed that the color distribution within each block of the multiple target virtual blocks and multiple target real blocks is uniform. It should be understood that those skilled in the art can customize the specific value of the aforementioned preset threshold according to actual circumstances, and this disclosure does not limit this.

[0061] In step S133, for example, the color features of each target virtual block and each target real block can be obtained by calculating the average color value within each target virtual block and the average color value within each target real block. That is, the average color value of each pixel within the block (i.e., the average color value) can be used to characterize the dominant hue of the color within the block. Alternatively, the number of pixels belonging to each color value within the block can be counted, and the color value with the largest number of pixels can be selected as the color feature to characterize the dominant hue of the color within the block. That is, the color value with the most color distribution within the block can be found as the color feature. This embodiment of the present disclosure does not limit this approach.

[0062] In step S133, texture features of each target virtual block and each target real block can be extracted using a texture feature extraction model. Texture features can be represented as feature vectors. In practical applications, texture feature extraction models known in the art can be used to extract the texture features of the blocks. For example, a Gabor filter (a two-dimensional linear filter) can be used to extract the texture direction and frequency features (which can characterize the direction and density of linear textures on the surface of an object in the block). The Local Binary Patterns (LBP) algorithm can be used to extract the local texture features of the block. Thus, the texture features of any block can specifically include, for example, local texture features describing the material within the block, as well as texture direction and frequency features. This embodiment of the invention does not limit the method of extracting texture features within blocks.

[0063] In step S133, multiple sets of candidate alignment regions can be determined based on the texture similarity and color similarity between the target real block and the target virtual block. Texture similarity represents the similarity between the texture features of the target real block and the texture features of the target virtual block, and color similarity represents the similarity between the color features of the target real block and the color features of the target virtual block. For example, a matching target virtual block can be found for any target real block among multiple target real blocks. That is, the texture similarity and color similarity between the target real block and each target virtual block can be determined based on the color and texture features of the target real block, as well as the color and texture features of each target virtual block. Then, based on the texture and color similarity between the target real block and each target virtual block, a target virtual block matching the material and color of the target real block is determined from among the multiple target virtual blocks to obtain a set of candidate alignment regions.

[0064] For example, the texture similarity between any real target block and each virtual target block can be calculated by measuring the cosine similarity of texture features. Alternatively, the texture similarity between real and virtual target blocks can be calculated, and the texture characteristics between blocks can be compared using cosine similarity to obtain a texture similarity score (i.e., texture similarity) for each pair of blocks. Of course, other similarity algorithms in the art, such as mean squared error, can also be used to calculate the texture similarity, and this disclosure does not limit this approach.

[0065] For example, the color similarity between any target real block and each target virtual block can be obtained by calculating the Euclidean distance (i.e., the Euclidean distance between color features in the LAB color space) of color features (such as average color values) between the target real block and each target virtual block. This is equivalent to calculating the color similarity between the target real block and the target virtual block. Of course, other similarity algorithms in the art, such as cosine similarity, can also be used to calculate the above color similarity, and this disclosure does not limit this approach.

[0066] In step S133, for example, the target virtual blocks can be sorted according to their texture similarity and color similarity. Then, the target virtual block with the highest color similarity and texture similarity can be selected from the sorted results as the target virtual block whose material and color match the target real block, thus obtaining a set of candidate alignment regions. It should be understood that for each target real block among multiple target real blocks, the target virtual blocks matching the material and color of each target real block can be determined separately using the above method, thereby obtaining multiple sets of candidate alignment regions.

[0067] Considering that the similarity of materials and colors between real and virtual scenes may have different importance for the alignment of virtual and real lighting—for example, users may pay more attention to color consistency—in some embodiments, in step S133 above, the multiple sets of candidate alignment regions can also be determined based on the comprehensive matching degree between the target real block and the target virtual block. This comprehensive matching degree is a weighted sum of texture similarity and color similarity. For example, the comprehensive matching degree (also called comprehensive similarity score) between the target real block and each target virtual block can be obtained by weighted summing of the texture similarity and color similarity between any target real block and each target virtual block. Then, the target virtual block with the highest comprehensive matching degree can be selected from multiple target virtual blocks as the target virtual block that matches the material and color of the target real block. Alternatively, all target virtual blocks and target real blocks can be divided into pairs without repetition, and then multiple pairs of target virtual blocks and target real blocks with comprehensive matching degrees exceeding a threshold can be selected to obtain multiple sets of candidate alignment regions.

[0068] It should be understood that users can set the weights corresponding to texture similarity and color similarity according to their actual needs. Then, based on the weights set by the user, the texture similarity and color similarity between any target real block and each target virtual block can be weighted and summed to obtain the comprehensive matching degree between the target real block and each target virtual block. Then, according to the comprehensive matching degree determined by the user's actual needs for aligning virtual and real lights, the target virtual block that best matches the material and color of each target real block can be selected. The highest comprehensive matching degree means that the material and color are best matched on the basis of uniform color distribution.

[0069] The above example demonstrates finding matching target virtual blocks based on target real blocks. Conversely, it's possible to find matching target real blocks based on target virtual blocks. For instance, for any target virtual block among multiple target virtual blocks, based on its color and texture features, as well as the color and texture features of each target real block, the texture similarity and color similarity between the target virtual block and each target real block can be determined. Then, based on these similarities, target real blocks matching the material and color of the target virtual block can be identified from the multiple target real blocks, resulting in a set of candidate alignment regions. Finally, the process can be performed separately based on the texture and color similarity between a specific target virtual block and each target real block. The virtual blocks are sorted, and the target real blocks with the highest color similarity and texture similarity can be selected from the sorting results as the target real blocks that match the material and color of the target virtual block, thus obtaining a set of candidate alignment regions; or, the texture similarity and color similarity between any target virtual block and each target real block can be weighted and summed to obtain the comprehensive matching degree between the target virtual block and each target virtual block, and then the target real block with the highest comprehensive matching degree can be selected from multiple target real blocks as the target real block that matches the material and color of the target virtual block. This disclosure does not limit the implementation of this embodiment.

[0070] For example, suppose that m target virtual blocks are obtained through step S131 above, and n target real blocks are obtained through step S132 above, which means there are m×n combinations of blocks, where... , Represents the i-th target virtual block out of m target virtual blocks, using , Let j represent the j-th target real block out of n target real blocks. Then for each of the m×n combinations (that is, for each...) and (combinations), can calculate each kind and The cosine similarity (i.e., texture similarity, denoted as S) of texture features between them texture The Euclidean distance (i.e., color similarity, denoted as S) between the average color value in the LAB color space and the average color value in the LAB color space color Then you can calculate each one. and The overall matching degree between them, i.e., S ij =w t ×S texture + w c ×S color , where w t and w c These represent the weights of texture similarity and color similarity, respectively. For example, both can be set to 1 (meaning the weights of the two similarities are equal). Of course, w t and w c Different values ​​can also be set; furthermore, based on the overall matching degree, the value can be selected that matches each The best match This results in n sets of candidate alignment regions that correspond one-to-one.

[0071] In step S14, at least one set of target alignment regions is selected from multiple sets of candidate alignment regions to perform virtual and real lighting alignment on the virtual shooting scene using at least one set of target alignment regions.

[0072] In some embodiments, at least one set of candidate alignment regions with the highest texture similarity and the highest color similarity can be selected from multiple sets of candidate alignment regions as the target alignment region to perform virtual and real lighting alignment on the virtual shooting scene.

[0073] As described above, multiple sets of candidate alignment regions can be determined by calculating the comprehensive matching degree between real blocks and virtual blocks. Therefore, in some embodiments, at least one target alignment region can be selected from multiple sets of candidate alignment regions based on the comprehensive matching degree corresponding to each set of candidate alignment regions. Specifically, selecting at least one target alignment region from multiple sets of candidate alignment regions includes:

[0074] Obtain the overall matching degree of each group of candidate alignment regions. The overall matching degree is determined based on the texture similarity and color similarity between a set of target real blocks and target virtual blocks with matching materials and colors. For details, please refer to the above method for determining the overall matching degree, which will not be repeated here. Select at least one group of alignment regions with an overall matching degree greater than a preset matching threshold from multiple groups of candidate alignment regions as the target alignment region.

[0075] Users can set preset matching thresholds according to their actual needs, and this embodiment does not impose any restrictions on this. This method ensures that the target real block and the target virtual block in the final selected target alignment area have the best color and material matching based on uniform color distribution, thereby facilitating more accurate alignment of virtual and real lights using the target alignment area.

[0076] The target alignment regions selected according to the preset matching threshold can be real target blocks and virtual target blocks that meet the requirements for virtual and real light alignment. If there are many target alignment regions selected according to the preset matching threshold, in some embodiments, after selecting multiple groups of target alignment regions according to the preset matching threshold, each group of target alignment regions can be further sorted according to the comprehensive matching degree, and then the top x target alignment regions can be selected to perform virtual and real light alignment on the virtual shooting scene. This disclosure does not limit this aspect.

[0077] In practical applications, known methods for aligning real and virtual lighting can be used. The target real blocks and target virtual blocks in each group of target alignment regions are used as inputs to the method, thereby aligning the lighting effects of the real and virtual sets in a virtual shooting scene. Exemplarily, this disclosure provides a method for aligning real and virtual lighting, including the above-described method. Figure 1The method for determining an alignment area in virtual-real lighting alignment is shown. The alignment method further includes: adjusting the lighting control parameters of the physical lighting system in the virtual shooting scene based on the difference in color values ​​between the same set of real target blocks and virtual target blocks in at least one set of target alignment areas, until the color values ​​between the same set of real target blocks and virtual target blocks in the camera image are consistent, wherein the physical lighting system is used to light the real scene. Specifically, the brightness, color temperature, and color difference of the physical lighting system can be adjusted based on the color value difference between each set of target real blocks and target virtual blocks (such as the difference between one or more components in the LAB color space: luminance component L, red-green component A, and blue-yellow component B). These lighting control parameters can be sent to the physical lighting system in real-time via DMX signals to adjust the lighting effect on the real scene. Changes in the lighting effect on the real scene are captured in real-time by the camera, and the alignment method can be repeated based on the camera's re-captured image. This alignment process is iterated until the physical lighting system adjusts the color values ​​of each set of target real blocks and target virtual blocks in the camera image to match, thus completing the alignment of real and virtual lighting. The specific correspondence between the color value difference and the lighting control parameters such as brightness, color temperature, and color difference, or their adjustment amounts, can be set as needed and is not limited in this application. The color value of any block is the average color value of each pixel in that block (i.e., the average value of each pixel in that block across the luminance component L, red-green component A, and blue-yellow component B). Therefore, the difference in color values ​​between each set of target real blocks and target virtual blocks can be understood as the difference between the average color value in each set of target real blocks and the average color value in each set of target virtual blocks.

[0078] According to the method of this disclosure, by extracting the real scene area and virtual scene area in the camera image and dividing them into multiple real blocks and multiple virtual blocks respectively, and then matching the colors and materials between each block to obtain multiple sets of target real blocks and target virtual blocks with matching materials and colors, and then selecting at least one set of target alignment areas from multiple sets of candidate alignment areas, it is possible to automatically select more accurate and objective target alignment areas to perform virtual and real lighting alignment, reduce manual intervention, improve the efficiency and effect of virtual and real lighting alignment, thereby facilitating the automation and efficiency improvement of the entire virtual shooting process.

[0079] According to the method of this disclosure, using color and texture analysis technology, a method is provided to automatically select target alignment areas with similar materials, uniform colors, and consistent main colors from virtual and real scene areas, addressing the need for virtual-real lighting alignment in virtual shooting operations. This method can assist lighting technicians in selecting alignment areas more accurately and objectively. It can also serve as input for lighting control algorithms or virtual-real alignment algorithms that require manual intervention in area selection, thereby achieving complete automation of lighting alignment in virtual shooting, reducing manual intervention, and improving operational efficiency.

[0080] Figure 3 A block diagram of an apparatus for determining an alignment region in virtual and real light alignment according to an embodiment of the present disclosure is shown, as follows: Figure 3 As shown, the device includes:

[0081] The acquisition module 301 is used to acquire camera images of a virtual shooting scene, wherein the virtual shooting scene includes: a screen for displaying the virtual scene and a real scene located in front of the screen;

[0082] The extraction module 302 is used to extract the real scene area and the virtual scene area in the camera image, and divide the real scene area and the virtual scene area into multiple real blocks and multiple virtual blocks respectively;

[0083] Matching module 303 is used to obtain multiple sets of candidate alignment regions by matching the material and color of the multiple virtual blocks with the multiple real blocks. Each set of candidate alignment regions includes a set of target real blocks and target virtual blocks that are matched in material and color.

[0084] The selection module 304 is used to select at least one set of target alignment regions from the multiple sets of candidate alignment regions, so as to use the at least one set of target alignment regions to perform virtual and real lighting alignment on the virtual shooting scene.

[0085] In one possible implementation, the step of obtaining multiple sets of candidate alignment regions by matching the material and color of the multiple virtual blocks with the multiple real blocks includes: determining the color uniformity of each virtual block and selecting multiple target virtual blocks from the multiple virtual blocks whose color uniformity is greater than a preset threshold; determining the color uniformity of each real block and selecting multiple target real blocks from the multiple real blocks whose color uniformity is greater than the preset threshold; determining the color features and texture features of each target virtual block and each target real block respectively, and determining multiple sets of candidate alignment regions based on the color features and texture features of each target virtual block and each target real block, wherein the texture features represent the material and the color features represent the dominant color tone.

[0086] In one possible implementation, the multiple sets of candidate alignment regions are determined based on the texture similarity and color similarity between the target real block and the target virtual block, wherein the texture similarity represents the similarity between the texture features of the target real block and the texture features of the target virtual block, and the color similarity represents the similarity between the color features of the target real block and the color features of the target virtual block.

[0087] In one possible implementation, the multiple sets of candidate alignment regions are determined based on the comprehensive matching degree between the target real block and the target virtual block, wherein the comprehensive matching degree is a weighted sum of texture similarity and color similarity.

[0088] In one possible implementation, selecting at least one target alignment region from the plurality of candidate alignment regions includes: obtaining the comprehensive matching degree of each candidate alignment region, wherein the comprehensive matching degree is determined based on the texture similarity and color similarity between the target real block and the target virtual block in each set of material and color matching; and selecting at least one candidate alignment region from the plurality of candidate alignment regions whose comprehensive matching degree is greater than a preset matching threshold as the target alignment region.

[0089] In one possible implementation, the step of extracting the real and virtual scene areas from the camera frame and dividing the real and virtual scene areas into multiple real blocks and multiple virtual blocks respectively includes: extracting the real and virtual scene areas from the camera frame using a mask image, wherein the mask image is determined by the difference between the camera frame captured when the screen is completely black and the camera frame captured when the screen is fully bright; and extracting multiple real blocks and multiple virtual blocks of the same size from the real and virtual scene areas respectively using a preset window with a preset step size.

[0090] In one possible implementation, acquiring the camera image of the virtual shooting scene includes: acquiring the original camera image captured by the camera of the virtual shooting scene, wherein the colors in the original camera image are in the RGB color space; converting the original camera image from the RGB color space to the LAB color space, so as to perform the method using the camera image in the LAB color space.

[0091] Based on the above-described apparatus for determining the alignment area in virtual-real lighting alignment, this disclosure also provides a virtual-real lighting alignment apparatus, including the above-described apparatus for determining the alignment area in virtual-real lighting alignment, and further including: a lighting alignment module, used to adjust the lighting control parameters of the physical lighting system in the virtual shooting scene based on the difference in color values ​​between the same group of real target blocks and virtual target blocks in the at least one group of target alignment areas, until the color values ​​between the same group of real target blocks and virtual target blocks in the camera image are consistent, wherein the physical lighting system is used to illuminate the real scene.

[0092] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0093] This disclosure also provides an electronic device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the above method.

[0094] This disclosure also provides a non-volatile computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the above-described method.

[0095] This disclosure also provides a computer program product, including a computer program or a non-volatile computer-readable storage medium carrying the computer program, wherein the computer program, when executed by a processor, implements the steps of the above method.

[0096] Figure 4 A block diagram of an electronic device 1900 according to an embodiment of the present disclosure is shown. For example, the electronic device 1900 may be provided as a server or a terminal device. (Refer to...) Figure 4 The electronic device 1900 includes a processing component 1922, which further includes one or more processors, and memory resources represented by memory 1932 for storing instructions, such as application programs, that can be executed by the processing component 1922. The application programs stored in memory 1932 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 1922 is configured to execute instructions to perform the methods described above.

[0097] Electronic device 1900 may also include a power supply component 1926 configured to perform power management of electronic device 1900, a wired or wireless network interface 1950 configured to connect electronic device 1900 to a network, and an input / output interface 1958 (I / O interface). Electronic device 1900 can operate on an operating system, such as Windows Server, stored in memory 1932. TM Mac OS X TM Unix TM Linux TM FreeBSD TM Or similar.

[0098] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions that can be executed by a processing component 1922 of an electronic device 1900 to perform the above-described method.

[0099] Computer-readable storage media can be tangible devices capable of holding and storing programs / instructions used by instruction execution devices. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination of the foregoing. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0100] The computer program (or computer-readable program instructions) described herein can be downloaded from a computer-readable storage medium to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage medium in the respective computing / processing device.

[0101] The computer program (or computer program instructions) used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing state information from the computer-readable program instructions to implement various aspects of this disclosure.

[0102] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0103] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0104] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0105] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0106] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method of determining an alignment area in a virtual-real light alignment, characterized by, The method comprises: acquiring a camera picture of a virtual shooting scene, the virtual shooting scene comprising a screen for displaying a virtual setting and a real setting in front of the screen; extracting a real setting area and a virtual setting area in the camera picture, and dividing the real setting area and the virtual setting area into a plurality of real blocks and a plurality of virtual blocks respectively; obtaining a plurality of groups of candidate alignment areas by performing material and color matching on the plurality of virtual blocks and the plurality of real blocks, any group of candidate alignment areas comprising a group of target real blocks and target virtual blocks matched in material and color; selecting at least one group of target alignment areas from the plurality of groups of candidate alignment areas to perform virtual-real light alignment on the virtual shooting scene using the at least one group of target alignment areas.

2. The method of claim 1, wherein, The method of obtaining a plurality of groups of candidate alignment areas by performing material and color matching on the plurality of virtual blocks and the plurality of real blocks comprises: determining color uniformity of each virtual block, and selecting a plurality of target virtual blocks with color uniformity greater than a preset threshold from the plurality of virtual blocks; determining color uniformity of each real block, and selecting a plurality of target real blocks with color uniformity greater than the preset threshold from the plurality of real blocks; determining color features and texture features of each target virtual block and each target real block respectively, and determining a plurality of groups of candidate alignment areas according to the color features and the texture features of the plurality of target virtual blocks and the color features and the texture features of the plurality of target real blocks, wherein the texture features represent material, and the color features represent a dominant color tone.

3. The method of claim 2, wherein, The plurality of groups of candidate alignment areas are determined according to texture similarity and color similarity between target real blocks and target virtual blocks, wherein the texture similarity represents similarity between texture features of a target real block and texture features of a target virtual block, and the color similarity represents similarity between color features of the target real block and color features of the target virtual block.

4. The method of claim 3, wherein, The plurality of groups of candidate alignment areas are determined according to comprehensive matching degrees between target real blocks and target virtual blocks, wherein the comprehensive matching degrees are weighted sums of the texture similarity and the color similarity.

5. The method according to any one of claims 1 to 4, characterized in that, The method of selecting at least one group of target alignment areas from the plurality of groups of candidate alignment areas comprises: acquiring a comprehensive matching degree of each group of candidate alignment areas, wherein the comprehensive matching degree is determined based on the texture similarity and the color similarity between each group of target real blocks and target virtual blocks matched in material and color; selecting at least one group of candidate alignment areas with a comprehensive matching degree greater than a preset matching threshold from the plurality of groups of candidate alignment areas as target alignment areas.

6. The method according to any one of claims 1 to 4, characterized in that, The method of extracting a real setting area and a virtual setting area in the camera picture, and dividing the real setting area and the virtual setting area into a plurality of real blocks and a plurality of virtual blocks comprises: extracting the real setting area and the virtual setting area in the camera picture by using a mask image, wherein the mask image is determined by using a difference between a camera picture taken when the screen is completely dark and a camera picture taken when the screen is completely bright; A plurality of real blocks and a plurality of virtual blocks with the same size are extracted from the real setting area and the virtual setting area respectively by using a preset window according to a preset step length.

7. The method according to any one of claims 1 to 4, characterized in that, The method further comprises: The camera picture of the virtual shooting scene is obtained, and the virtual shooting scene comprises a screen for displaying a virtual setting and a real setting in front of the screen. The method further comprises:

8. A virtual real light alignment method, comprising the method of any one of claims 1 to 7, characterized in that, The camera picture of the virtual shooting scene is obtained, and the virtual shooting scene comprises a screen for displaying a virtual setting and a real setting in front of the screen. The method further comprises:

9. An apparatus for determining an alignment area in aligning virtual and real lights, characterized in that, The camera picture of the virtual shooting scene is obtained, and the virtual shooting scene comprises a screen for displaying a virtual setting and a real setting in front of the screen. The method further comprises: The camera picture of the virtual shooting scene is obtained, and the virtual shooting scene comprises a screen for displaying a virtual setting and a real setting in front of the screen. The method further comprises: The camera picture of the virtual shooting scene is obtained, and the virtual shooting scene comprises a screen for displaying a virtual setting and a real setting in front of the screen.

10. An electronic device comprising a memory, a processor, and a computer program stored on the memory, wherein the computer program, when executed by the processor, is arranged to perform the method of any one of claims 1 to 9. The processor executes the computer program to implement the steps of the method in any one of claims 1 to 8.

11. A non-transitory computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 8.

12. A computer program product comprising a computer program or a non-transitory computer readable storage medium bearing a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 8.

Images