A lamp synchronization method, apparatus, device and storage medium

Abstract

Embodiments of the present application disclose a lamp synchronization method, device and equipment, and a storage medium, which are used to improve the accuracy of calibrating virtual lamps and improve the synchronization effect of virtual lamps and real lamps. The method of the embodiments of the present application comprises: assigning each lamp in a target scene a unique corresponding lamp detection identifier, each lamp detection identifier corresponding to a lamp model and lamp information; collecting an identifier detection image corresponding to each lamp detection identifier; based on each identifier detection image, obtaining first pose information of each lamp detection identifier in the target scene; converting the first pose information into second pose information of each lamp detection identifier corresponding to a three-dimensional virtual space coordinate system; based on the second pose information of each lamp detection identifier, the lamp model and the lamp information, calibrating with a virtual lamp in a virtual scene to obtain a calibrated lamp calibration pose; and based on the lamp calibration pose, synchronously adjusting the lamps in the target scene.

Description

Technical Field

[0001] This application relates to the field of lighting control technology, and in particular to a lighting synchronization method, apparatus, device, and storage medium. Background Technology

[0002] With the development of technology, in the process of filming or stage performance, it is often necessary to have the same lighting atmosphere in virtual and real scenes. In the current technology, it is usually done by some professional staff to manually adjust the real lights so that the shooting effect or stage effect can be perfectly presented.

[0003] However, lamps differ from other ordinary objects. Light has the characteristic of self-illumination, and there are many types of light. From the perspective of the shape that produces the light effect, they can be divided into parallel light, point light, laser, panel light, and so on. Light has multiple attributes, such as brightness, chromaticity, and color temperature. Point light sources also have opening angles, etc.

[0004] In existing technologies, most lighting matching work typically involves matching common parameters such as brightness, chromaticity, and color temperature of the lights. However, in practical applications, it has been found that the position and orientation of the light fixtures have a significant impact on the lighting and shadow effects produced when the light hits an object. It's understandable that the more concentrated the light source, the greater the impact of the fixture's position and orientation on the lighting and shadow effects and on the camera. However, because manual installation and measurement of the light fixtures are inaccurate, inefficient, time-consuming, and prone to errors, and because dozens of lights are often working in the same space during filming or stage performances, it's difficult to quickly adjust the lighting effects by manually observing the atmosphere after lighting in real time. This increases the difficulty of synchronizing the dynamic lighting effects of virtual and real lights. Summary of the Invention

[0005] This application provides a lighting synchronization method, apparatus, device, and storage medium. Before lighting is applied to a target scene, the method acquires the true pose information (first pose information) of each lighting detection marker in the target scene and converts this true pose information into second pose information corresponding to each lighting detection marker in a three-dimensional virtual space coordinate system. This allows for the calibration of virtual lighting, adjusting it to match the pose of the lighting in the target scene. Based on this calibrated pose, the method enables synchronized adjustment between the lighting in the target scene and the virtual lighting when the lighting is applied to the target scene, thereby improving the synchronization effect between the virtual and real lighting.

[0006] One embodiment of this application provides a lighting synchronization method, including:

[0007] Assign a unique lamp detection identifier to each lamp in the target scene, where each lamp detection identifier corresponds to a lamp model and lamp information;

[0008] Collect the detection image of the corresponding indicator for each lamp;

[0009] Based on each detected image, obtain the first pose information of each detected light fixture in the target scene;

[0010] The first pose information is converted into the second pose information corresponding to each lamp detection mark in the three-dimensional virtual space coordinate system;

[0011] Based on the second pose information, lamp model and lamp information corresponding to each lamp detection mark, the lamp is calibrated with the virtual lamp in the virtual scene to obtain the calibrated lamp pose.

[0012] Based on the luminaire calibration pose, the luminaires in the target scene are adjusted synchronously.

[0013] Another aspect of this application provides a lighting synchronization method apparatus, comprising:

[0014] The processing unit is used to assign a unique lamp detection identifier to each lamp in the target scene, wherein each lamp detection identifier corresponds to a lamp model and lamp information;

[0015] The acquisition unit is used to acquire the identification code detection image corresponding to each lamp detection mark;

[0016] The acquisition unit is also used to acquire the first pose information of each light fixture detection mark in the target scene based on each mark detection image;

[0017] The processing unit is also used to convert the first pose information into the second pose information corresponding to each lamp detection mark in the three-dimensional virtual space coordinate system;

[0018] The processing unit is also used to calibrate the virtual lamps in the virtual scene based on the second pose information, lamp model and lamp information corresponding to each lamp detection mark, so as to obtain the calibrated lamp pose.

[0019] The control unit is used to synchronously adjust the lighting fixtures in the target scene based on the lighting fixture calibration posture.

[0020] In one possible design, in another implementation of the embodiments of this application, the processing unit may specifically be used for:

[0021] Based on the lamp model and lamp information, obtain the initial pose information corresponding to the virtual lamp;

[0022] Based on the second pose information, the initial pose information corresponding to the virtual lamp is calibrated to obtain the lamp calibration pose.

[0023] In one possible design, in another implementation of the embodiments of this application, the processing unit may specifically be used for:

[0024] The second pose information is compared with the initial pose information corresponding to the virtual lamp to obtain the comparison result;

[0025] If the comparison results show that the information is consistent, then the initial pose information corresponding to the virtual lamp will be used as the lamp calibration pose corresponding to the virtual lamp.

[0026] If the comparison result shows that the information is inconsistent, the initial pose information corresponding to the virtual lamp is replaced with the second pose information to obtain the lamp calibration pose corresponding to the virtual lamp.

[0027] In one possible design, in another implementation of the embodiments of this application, the processing unit may specifically be used for:

[0028] Obtain the total number of lights in the target scene, and retrieve the set of QR codes corresponding to the total number of lights from the QR code dictionary;

[0029] Based on the QR code set, each lamp is assigned a unique corresponding QR code and a QR code identifier for each QR code.

[0030] Specifically, the acquisition unit can be used to: acquire the QR code detection image corresponding to each of the QR codes;

[0031] The acquisition unit can be specifically used to: acquire the first pose information of each QR code in the target scene based on each QR code detection image;

[0032] Specifically, the processing unit can be used to convert the first pose information into second pose information corresponding to each of the QR codes in a three-dimensional virtual space coordinate system.

[0033] In one possible design, in another implementation of the embodiments of this application, the processing unit may specifically be used for:

[0034] Based on the lamp model and lamp information, the lamps are classified to obtain P lamp categories and the lamp set corresponding to each lamp category, where P is an integer greater than or equal to 1;

[0035] Based on P lighting categories and the set of lighting fixtures corresponding to each category, the QR code set is divided into P QR code subsets;

[0036] Assign each QR code and its corresponding QR code identifier from the P subsets of QR codes to the lamps in the P sets of lamps.

[0037] In one possible design, in another implementation of the embodiments of this application, the acquisition unit may specifically be used for:

[0038] Install the QR code corresponding to each lamp on the corresponding lamp in the target scene to obtain the QR code detection point corresponding to each lamp;

[0039] At the time of data acquisition, the QR code at each QR code detection point in the target scene is captured, and the QR code detection image corresponding to each QR code is obtained.

[0040] In one possible design, in another implementation of the embodiments of this application, the acquisition unit may specifically be used for:

[0041] Obtain the first coordinate position of the QR code in each QR code detection image;

[0042] Based on the size of the QR code installed at each QR code detection point, calculate the first pose corresponding to each QR code.

[0043] In one possible design, in another implementation of the embodiments of this application, the processing unit may specifically be used for:

[0044] Obtain the coordinates of the shooting origin in the target scene, and the coordinates of the virtual origin in the three-dimensional virtual space coordinate system;

[0045] Based on the three-dimensional virtual space coordinate system, the virtual origin coordinate position, and the shooting origin coordinate position, the first coordinate position and the first pose are converted into the second coordinate position and the second pose.

[0046] In one possible design, in another implementation of the embodiments of this application, the control unit may specifically be used for:

[0047] If an adjustment command for a virtual light fixture is received, the virtual light fixture in the virtual scene is adjusted based on the adjustment command and the light fixture calibration pose, and a corresponding light fixture adjustment signal is generated.

[0048] Based on the lighting adjustment signal, the virtual lighting fixtures in the target scene are controlled to adjust synchronously.

[0049] This application also provides a computer device, including: a memory, a processor, and a bus system;

[0050] The memory is used to store programs;

[0051] The processor implements the methods described above when executing a program in memory;

[0052] Bus systems are used to connect memory and processor to enable communication between them.

[0053] Another aspect of this application provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the methods described above.

[0054] As can be seen from the above technical solutions, the embodiments of this application have the following beneficial effects:

[0055] By assigning a unique lamp detection tag to each lamp in the target scene, and then acquiring the tag detection image corresponding to each lamp detection tag, the first pose information of each lamp detection tag in the target scene is obtained based on each tag detection image. The first pose information is then converted into the second pose information corresponding to each lamp detection tag in the three-dimensional virtual space coordinate system. Then, based on the second pose information corresponding to each lamp detection tag, the lamp model and lamp information, calibration can be performed with the virtual lamps in the virtual scene to obtain the calibrated lamp calibration pose. Based on the calibrated lamp pose, the lamps in the target scene are synchronously adjusted. By employing the above method, before the actual lighting of the target scene is used, a unique corresponding lamp detection tag can be assigned to each lamp in the target scene. Based on the collected tag detection images, the true pose information of each lamp detection tag in the target scene, i.e., the first pose information, can be obtained. Then, by converting the true pose information into the second pose information corresponding to each lamp detection tag in the three-dimensional virtual space coordinate system, the virtual lamps can be calibrated. That is, the virtual lamps are adjusted to be consistent with the pose state of the lamps in the target scene. Based on the lamp calibration pose, when the actual lighting of the target scene is used, the lamps in the target scene and the virtual lamps can be synchronously adjusted, thereby improving the synchronization effect between the virtual lamps and the real lamps. Attached Figure Description

[0056] Figure 1 This is a schematic diagram of the architecture of a lighting control system in an embodiment of this application;

[0057] Figure 2 This is a flowchart of one embodiment of the lighting synchronization method in this application;

[0058] Figure 3 This is a flowchart of another embodiment of the lighting synchronization method in this application;

[0059] Figure 4 This is a flowchart of another embodiment of the lighting synchronization method in this application;

[0060] Figure 5 This is a flowchart of another embodiment of the lighting synchronization method in this application;

[0061] Figure 6 This is a flowchart of another embodiment of the lighting synchronization method in this application;

[0062] Figure 7 This is a flowchart of another embodiment of the lighting synchronization method in this application;

[0063] Figure 8 This is a flowchart of another embodiment of the lighting synchronization method in this application;

[0064] Figure 9 This is a flowchart of another embodiment of the lighting synchronization method in this application;

[0065] Figure 10 This is a flowchart of another embodiment of the lighting synchronization method in this application;

[0066] Figure 11 This is a schematic diagram illustrating the principle of the lighting synchronization method in the embodiments of this application;

[0067] Figure 12 This is a schematic diagram illustrating another principle and flow of the lighting synchronization method in this application embodiment;

[0068] Figure 13 This is a schematic diagram illustrating the effect of the lamp positions in the lamp synchronization method in the embodiments of this application;

[0069] Figure 14 This is a schematic diagram of one embodiment of the lamp synchronization method apparatus in this application;

[0070] Figure 15 This is a schematic diagram of one embodiment of the computer device described in this application. Detailed Implementation

[0071] This application provides a lighting synchronization method, apparatus, device, and storage medium. Before lighting is applied to a target scene, the method acquires the true pose information (first pose information) of each lighting detection marker in the target scene and converts this true pose information into second pose information corresponding to each lighting detection marker in a three-dimensional virtual space coordinate system. This allows for the calibration of virtual lighting, adjusting it to match the pose of the lighting in the target scene. Based on this calibrated pose, the method enables synchronized adjustment between the lighting in the target scene and the virtual lighting when the lighting is applied to the target scene, thereby improving the synchronization effect between the virtual and real lighting.

[0072] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and drawings of this application are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “corresponding to,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0073] To facilitate understanding, some terms or concepts involved in the embodiments of this application will be explained first.

[0074] 1. Virtual Studio: A virtual studio is a modern digital photography studio that integrates a green screen or LED background wall with a virtual camera system, a spatial positioning system, and a real-time rendering system to achieve the effect of pre-production processing.

[0075] 2. Camera data: Footage captured by the main camera during film and television production, primarily presented in video format.

[0076] 3. Virtual scene data: Digital scenes created in digital content generation tools based on the artist's needs or real-world scenes, including two-dimensional, three-dimensional, and panoramic content.

[0077] 4. Camera synchronization data: The position and attitude data of the camera in real-world space, usually represented by six degrees of freedom (6DOF), are calculated in real time or offline during film and television production and can be reproduced in three-dimensional virtual space.

[0078] 5. Art-Net, sACN: A data distribution protocol that allows DMX512 and RDM lighting data to be transmitted over Ethernet. It uses a simple UDP-based packet structure and is designed to provide efficient and low-overhead data streaming.

[0079] 6. Digital Multiplex (DMX): DMX512 is a standard for digital communication networks, commonly used to control lighting and effects.

[0080] It is understood that in the specific embodiments of this application, data related to QR code detection images and first pose information are involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0081] It should be understood that the lighting synchronization method provided in this application can be applied to various scenarios, including but not limited to artificial intelligence, cloud technology, maps, and intelligent transportation. It is used to obtain the true pose information of the QR code corresponding to each real lighting fixture, convert the true pose information into the second pose information corresponding to each QR code in the three-dimensional virtual space coordinate system, and calibrate the virtual lighting fixture to complete the synchronous control of the virtual lighting fixture on the real lighting fixture. It can be applied to scenarios such as virtual film production scenes, stage performance scenes, movie shooting scenes, TV series or advertising shooting scenes, and live press conferences.

[0082] To address the aforementioned problems, this application proposes a lighting synchronization method, which is applied to... Figure 1 Please refer to the lighting control system shown. Figure 1 , Figure 1 This is a schematic diagram of the architecture of the lighting control system in an embodiment of this application, as shown below. Figure 1 As shown, the server assigns a unique lamp detection identifier to each lamp in the target scene. Then, it acquires the identifier detection image corresponding to each lamp detection identifier collected by the terminal device. Based on each identifier detection image, it obtains the first pose information of each lamp detection identifier in the target scene and converts the first pose information into the second pose information corresponding to each lamp detection identifier in the three-dimensional virtual space coordinate system. Then, based on the second pose information corresponding to each lamp detection identifier, the lamp model and lamp information, it can be calibrated with the virtual lamps in the virtual scene to obtain the calibrated lamp calibration pose. Based on the lamp calibration pose, the lamps in the target scene are synchronously adjusted. By employing the above method, before the actual lighting of the target scene is used, a unique corresponding lamp detection tag can be assigned to each lamp in the target scene. Based on the collected tag detection images, the true pose information of each lamp detection tag in the target scene, i.e., the first pose information, can be obtained. Then, by converting the true pose information into the second pose information corresponding to each lamp detection tag in the three-dimensional virtual space coordinate system, the virtual lamps can be calibrated. That is, the virtual lamps are adjusted to be consistent with the pose state of the lamps in the target scene. Based on the lamp calibration pose, when the actual lighting of the target scene is used, the lamps in the target scene and the virtual lamps can be synchronously adjusted, thereby improving the synchronization effect between the virtual lamps and the real lamps.

[0083] Understandable, Figure 1 Only one type of terminal device is shown in the diagram. In real-world scenarios, many more types of terminal devices can participate in the data processing. These include, but are not limited to, mobile phones, computers, smart voice interaction devices, smart home appliances, and in-vehicle terminals. The specific number and types depend on the actual scenario and are not limited here. Furthermore, Figure 1 The diagram shows one server, but in real-world scenarios, multiple servers can be involved, especially in scenarios involving multi-model training and interaction. The number of servers depends on the specific scenario and is not limited here.

[0084] It should be noted that in this embodiment, the server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. Terminal devices and servers can be directly or indirectly connected via wired or wireless communication, and terminal devices and servers can be connected to form a blockchain network; this application does not impose any limitations on this.

[0085] Based on the above introduction, the lighting synchronization method in this application will be described below. Please refer to [link / reference]. Figure 2 One embodiment of the lighting synchronization method in this application includes:

[0086] In step S101, a unique corresponding lamp detection identifier is assigned to each lamp in the target scene, wherein each lamp detection identifier corresponds to a lamp model and lamp information;

[0087] In this embodiment, when there is a target scene that requires lighting, in order to better obtain the real pose information of each real lamp in the target scene, so that the virtual lamps in the virtual scene can be better calibrated based on the real pose information of each real lamp, thereby improving the accuracy of virtual lamp calibration and the synchronization effect between virtual lamps and real lamps, this embodiment assigns a unique lamp detection identifier to each lamp in the target scene, and uses the lamp detection identifier to accurately obtain the real pose information of each real lamp.

[0088] The target scene refers to a real-world scene where various real lighting fixtures can be used for lighting, such as film and television shooting (movies, TV series, or short videos), stage speeches, or performances.

[0089] Each luminaire in the target scene has a unique luminaire model and information. Therefore, after assigning a unique corresponding luminaire inspection identifier (such as a QR code, barcode, or special number) to each luminaire in the target scene, it can be seen that each luminaire inspection identifier corresponds to a luminaire model and information. The luminaire information includes the luminaire name (e.g., spotlight, softlight, or beam light), luminaire description, and luminaire power, etc. The luminaire model indicates the specific product model of the luminaire, such as PAR46 downlight, PAR46 downlight, or S-shaped luminaire, etc.

[0090] Specifically, such as Figure 11 As shown, by first obtaining the total number of lights required in the target scene, a set of identifiers (such as QR code set, barcode set, and special label set) corresponding to the total number of lights can be obtained from a pre-built identifier dictionary (such as QR code dictionary, barcode dictionary, and special label dictionary). Then, the set of identifiers (such as QR code set, barcode set, and special label set) corresponding to the total number of lights can be used to assign a unique corresponding light fixture detection identifier to each light fixture in the target scene. For example, assuming a target scene is a performance that requires 20 S-type lights from a certain manufacturer, 20 different light fixture detection identifiers can be obtained from the identifier dictionary to obtain the corresponding identifier set. These 20 different light fixture detection identifiers can be encoded, such as assigning S-numbers S_0-S_19, and the light fixture detection identifiers with IDs 0-19 can be assigned to the 20 lights.

[0091] In step S102, the identification detection image corresponding to each lamp detection identification is acquired;

[0092] In this embodiment of the application, after assigning a unique corresponding lamp detection tag to each lamp in the target scene, the assigned lamp detection tag can be installed at the lamp in the corresponding target scene. Then, the lamp detection tags corresponding to the lamps in the target scene are photographed to collect the tag detection image corresponding to each lamp detection tag, so that the real pose information of each lamp detection tag in the target scene can be accurately obtained based on each tag detection image.

[0093] Specifically, such as Figure 11 As shown, before officially using the lights in the target scene, the assigned light fixture inspection labels (such as QR codes, barcodes, and special markings) can be installed on the corresponding lights in the target scene. Then, use a camera to photograph the light fixture inspection labels (such as QR codes, barcodes, and special markings) installed on the lights in the target scene. For example, use a camera to photograph... Figure 11The image shows the lamp detection identifier corresponding to marker ID001 for Class A lamp 001. The image is then captured using a data acquisition card (e.g.,...). Figure 11 A capture card (e.g., an HDMI capture card) is used to convert the captured sign detection images into digital signals and transmit them to the server software (e.g., image processing software). This allows the software (e.g., image processing software) to export the digital signals as position files for each light fixture detection sign, thereby accurately obtaining the true pose information of each light fixture detection sign in the target scene.

[0094] In step S103, based on each identifier detection image, the first pose information of each light fixture detection identifier in the target scene is obtained;

[0095] In this embodiment of the application, after obtaining the identification detection image corresponding to each lamp detection identification, the first pose information of each lamp detection identification in the target scene can be obtained based on each identification detection image. This allows the first pose information to be converted into the second pose information corresponding to each lamp detection identification in the three-dimensional virtual space coordinate system, thereby enabling the virtual lamps to be calibrated based on the real pose information, which can improve the accuracy of calibrating the virtual lamps to a certain extent.

[0096] Specifically, after obtaining the identification image corresponding to each lighting fixture identification mark, the key point coordinates of the lighting fixture identification mark can be obtained through a deep learning-based model, such as the three-dimensional coordinates of the upper left corner of the lighting fixture identification mark (e.g., QR code, barcode, and special markings), or through physical segmentation algorithms, edge detection algorithms, or other algorithms (no specific restrictions are placed here), to obtain the true coordinate position of the lighting fixture identification mark in the target scene in each identification image (e.g., the position coordinates of the lighting fixture, position(x,y,z), represent the position of the lighting fixture in the world coordinate system). That is, the first coordinate position corresponding to the lamp detection mark. Then, based on the size of the lamp detection mark installed at each real lamp, according to visual algorithms and deep learning models, or other pose algorithms (no specific restrictions are imposed here), the real pose corresponding to each lamp detection mark (such as the rotation amount orientation(x,y,z,w) represents the rotational pose of the lamp in the world coordinate system) can be calculated. That is, the first pose corresponding to the lamp detection mark. Thus, based on the first coordinate position and the first pose corresponding to the lamp detection mark, the first pose information of each lamp detection mark in the target scene can be obtained.

[0097] In step S104, the first pose information is converted into the second pose information corresponding to each lamp detection identifier in the three-dimensional virtual space coordinate system;

[0098] In this embodiment of the application, after obtaining the first pose information, the first pose information can be converted into the second pose information corresponding to each lamp detection identifier in the three-dimensional virtual space coordinate system, so that the initial pose information of the virtual lamps in the virtual scene can be calibrated using the second pose information in the same coordinate system, thereby improving the accuracy of calibrating the virtual lamps to a certain extent.

[0099] Specifically, first obtain the shooting origin coordinates of the shooting device (such as a camera) in the target scene (such as a performance stage), and then obtain the virtual origin coordinates of the virtual shooting device (such as a virtual camera) in the three-dimensional virtual space coordinate system of the virtual scene constructed based on the target scene.

[0100] Furthermore, after obtaining the coordinates of the virtual origin and the shooting origin, the first pose information (such as the position of the lamp in the world coordinate system and the rotational attitude of the lamp in the world coordinate system) can be converted into the second pose information in the three-dimensional virtual space coordinate system based on the coordinates of the three-dimensional virtual space coordinate system, the coordinates of the virtual origin, and the coordinates of the shooting origin, according to the coordinate transformation algorithm.

[0101] In step S105, based on the second pose information, lamp model and lamp information corresponding to each lamp detection identifier, the lamp is calibrated with the virtual lamp in the virtual scene to obtain the calibrated lamp pose.

[0102] In this embodiment, after obtaining the second pose information corresponding to each lamp detection identifier, the initial pose information of the virtual lamps in the virtual scene can be calibrated based on the second pose information corresponding to each lamp detection identifier, the lamp model, and the lamp information to obtain the calibrated lamp calibration pose. This allows for the subsequent synchronous adjustment of the lamps in the target scene and the virtual lamps based on the calibrated lamp calibration pose, thereby improving the accuracy of calibrating the virtual lamps and enhancing the synchronization effect between the virtual lamps and the real lamps to a certain extent.

[0103] Virtual scenes are digital scenes created using digital content generation tools, representing a 1:1 replica of the real world (i.e., the target scene). The lighting fixtures in the target scene correspond one-to-one with the virtual lighting fixtures in the virtual scene, and the filming equipment also corresponds one-to-one. The target scene corresponds to the real-world coordinate system, while the virtual scene corresponds to a three-dimensional virtual space coordinate system. Virtual scenes can specifically represent virtual production scenes, 3D stage animation scenes, and other types of scenes; no specific limitations are made here. Virtual production refers to various digital workflows and methods utilizing computer-aided production and film visualization. Driven by real-time rendering technology, its applications can include pre-production virtual pre-visualization, real-time motion capture of virtual characters, green screen virtual production, and LED virtual production.

[0104] It is understandable that, in practical applications (such as the target scenario), such as Figure 13 As shown, the position and orientation of the lighting fixtures have a significant impact on the light and shadow effects produced when light shines on an object. For example, the more concentrated the light source, the greater the influence of the fixture's position and orientation on the presentation of light and shadow effects and on the camera. Alternatively, changes in the fixture's position and orientation also affect the shaping of light and shadow. Figure 13 The illustration in the upper left corner can highlight the lighting and shadow effects on the legs of the performer, or, as shown... Figure 13 The illustration in the lower left corner can highlight the lighting and shadow effects on the performer's upper body, such as the waist.

[0105] Therefore, to avoid the inaccuracy and inefficiency of manual installation and measurement of lighting fixtures, especially in scenarios where dozens of lights often operate in the same space, leading to discrepancies in the positional information of the lights between the virtual and real scenes, and the inability to synchronize or accurately control changes in the positional information of the lights in the real scene through the virtual scene, therefore, as... Figure 12 As shown, after obtaining the second pose information of each lamp detection mark in the three-dimensional virtual space coordinate system, the virtual light can be matched with the real light position (that is, the initial pose information of the virtual lamp is matched with the second pose information of the real lamp). Specifically, the initial pose information of the corresponding virtual lamp in the three-dimensional virtual space coordinate system can be obtained according to the lamp model and lamp information. Then, based on the second pose information corresponding to each lamp detection mark, the initial pose information of the virtual lamp can be calibrated. Specifically, the initial pose information of the virtual lamp can be compared with the second pose information of the real lamp in the same three-dimensional virtual space coordinate system to obtain the corresponding comparison result. Then, based on the corresponding comparison result, the pose information of the virtual lamp and the real lamp can be adjusted to be consistent, so as to achieve synchronous adjustment of the lamps in the target scene and the virtual lamps.

[0106] For example, assuming a target scene is a stage performance at a festival gala, a 1:1 virtual production scene is constructed. Before the actual lighting is applied to the target scene, each light fixture in the target scene can be assigned a unique corresponding light fixture detection identifier. Based on the second pose information, light fixture model, and light fixture information corresponding to each light fixture detection identifier, the virtual lights in the virtual production scene can be calibrated in advance. That is, the virtual lights are adjusted to match the pose state of the lights in the target scene. This allows the virtual lights to be calibrated and, when the actual lighting is applied to the target scene, the pose of the real lights in the target scene can be synchronously controlled and adjusted by real-time motion capture of the virtual lights and corresponding light fixture pose adjustments, thereby creating the lighting and shadow effects required for the target scene.

[0107] In step S106, the lamps in the target scene are synchronously adjusted based on the lamp calibration pose.

[0108] In this embodiment of the application, after adjusting the poses of the virtual lights in the virtual scene to the calibrated poses, the pose information of the virtual lights and the real lights can be adjusted to be consistent. Thus, based on the adjustment of the virtual lights, the virtual lights can synchronously control the lighting of the real lights in the target scene, thereby improving the synchronization effect when the virtual lights and the real lights are lighting.

[0109] Specifically, the poses of the virtual lights in the virtual scene are adjusted to the light calibration poses (e.g., ... Figure 11 After initializing and calibrating the virtual light pose as shown, the pose information of the virtual lights and the real lights can be adjusted to be consistent, that is, maintaining the consistency and synchronization between the virtual lights and the real lights. Then, for example, assuming that the lighting manager, based on the lighting requirements of the target scene, can input the corresponding lighting operation through the display interface of the virtual scene, so that the terminal device can respond to the lighting operation command of the lighting manager, generate the corresponding adjustment command and send it to the server, so that the server can receive the adjustment command about the virtual lights. Then, the server can adjust or control the virtual lights in the virtual scene based on the adjustment command and the light calibration pose, and generate the corresponding light adjustment signal. Then, the lighting control engine can transmit the light adjustment signal to the corresponding real lights to control or adjust the lights in the target scene corresponding to the virtual lights to make the corresponding lighting or direction adjustments.

[0110] In this application embodiment, a lighting synchronization method is provided. Through this method, before the actual lighting of the target scene is used, a unique corresponding lighting detection identifier is assigned to each lighting fixture in the target scene. Based on the acquired identifier detection image, the true pose information of each lighting detection identifier in the target scene, i.e., the first pose information, is obtained. Then, by converting the true pose information into the second pose information corresponding to each lighting detection identifier in a three-dimensional virtual space coordinate system, the virtual lighting fixtures are calibrated. This means adjusting the virtual lighting fixtures to match the pose state of the lighting fixtures in the target scene. Based on the calibrated pose, when the lighting fixtures in the target scene are used for actual lighting, the lighting fixtures in the target scene and the virtual lighting fixtures are synchronized, thereby improving the synchronization effect between the virtual and real lighting fixtures.

[0111] Optionally, in the above Figure 2 Based on the corresponding embodiments, in another optional embodiment of the lighting synchronization method provided in this application, such as... Figure 3 As shown, step S105 calibrates the virtual lights in the virtual scene based on the second pose information, light model, and light information corresponding to each light fixture detection identifier, to obtain the calibrated light fixture pose, including:

[0112] In step S301, the initial pose information corresponding to the virtual lamp is obtained according to the lamp model and lamp information;

[0113] In step S302, based on the second pose information, the initial pose information corresponding to the virtual lamp is calibrated to obtain the lamp calibration pose.

[0114] In this embodiment, after obtaining the second pose information corresponding to each lamp detection identifier, the initial pose information corresponding to the virtual lamp can be obtained first according to the lamp model and lamp information. Then, the initial pose information corresponding to the virtual lamp can be calibrated based on the second pose information to obtain the lamp calibration pose. This enables the lamps in the target scene to be synchronously adjusted with the virtual lamps based on the calibrated lamp calibration pose, thereby improving the accuracy of calibrating the virtual lamps and the synchronization effect between the virtual lamps and the real lamps to a certain extent.

[0115] Specifically, such as Figure 12As shown, after obtaining the second pose information of each lamp detection identifier in the three-dimensional virtual space coordinate system, the virtual light real light position matching can be performed (that is, matching the initial pose information of the virtual lamp with the second pose information of the real lamp). Specifically, it can be based on the lamp model and lamp information, such as Class A lamp 001, Class B lamp 001, etc. Based on the one-to-one correspondence between real lamps and virtual lamps, the virtual lamp corresponding to each real lamp in the virtual scene can be accurately obtained, as well as the position and posture of each virtual lamp initialized when constructing the virtual scene, so as to obtain the initial pose information corresponding to each virtual lamp.

[0116] Furthermore, after obtaining the initial pose information of the virtual lamp corresponding to each real lamp in the three-dimensional virtual space coordinate system, the second pose information can be compared with the initial pose information corresponding to the virtual lamp to obtain the corresponding comparison result. If the comparison result is consistent, it can be understood that the position and posture of the virtual lamp relative to the real lamp in the target scene have not shifted, that is, no calibration is required. In this case, the initial pose information corresponding to the virtual lamp can be used as the lamp calibration pose corresponding to the virtual lamp. If the comparison result is inconsistent, it can be understood that the position and posture of the virtual lamp relative to the real lamp in the target scene have shifted, that is, calibration is required. In this case, the initial pose information corresponding to the virtual lamp can be replaced with the second pose information, and the second pose information can be used as the lamp calibration pose corresponding to the virtual lamp. Then, based on the lamp calibration pose corresponding to the virtual lamp, the pose information of the virtual lamp and the real lamp can be adjusted to be consistent, so as to achieve synchronous adjustment of the lamps in the target scene and the virtual lamps.

[0117] Optionally, in the above Figure 3 Based on the corresponding embodiments, in another optional embodiment of the lighting synchronization method provided in this application, such as... Figure 4 As shown, step S302 calibrates the initial pose information corresponding to the virtual lamp based on the second pose information to obtain the lamp calibration pose, including:

[0118] In step S401, the second pose information is compared with the initial pose information corresponding to the virtual lamp to obtain the comparison result;

[0119] In step S402, if the comparison result is consistent, the initial pose information corresponding to the virtual lamp is used as the lamp calibration pose corresponding to the virtual lamp.

[0120] In step S403, if the comparison result is inconsistent, the initial pose information corresponding to the virtual lamp is replaced with the second pose information to obtain the lamp calibration pose corresponding to the virtual lamp.

[0121] In this embodiment, after obtaining the second pose information corresponding to each lamp detection identifier, the initial pose information corresponding to the virtual lamp can be obtained first according to the lamp model and lamp information. Then, the second pose information can be compared with the initial pose information corresponding to the virtual lamp to obtain the comparison result. If the comparison result is consistent, the initial pose information corresponding to the virtual lamp is used as the lamp calibration pose corresponding to the virtual lamp. If the comparison result is inconsistent, the initial pose information corresponding to the virtual lamp can be replaced with the second pose information to obtain the lamp calibration pose corresponding to the virtual lamp. This allows the lamps in the target scene to be synchronously adjusted with the virtual lamps based on the calibrated lamp calibration pose, thereby improving the accuracy of calibrating the virtual lamps and the synchronization effect between the virtual lamps and the real lamps to a certain extent.

[0122] Specifically, such as Figure 12 As shown, after obtaining the second pose information of each lamp detection identifier in the three-dimensional virtual space coordinate system, the virtual light real light position matching can be performed (that is, matching the initial pose information of the virtual lamp with the second pose information of the real lamp). Specifically, it can be based on the lamp model and lamp information, such as Class A lamp 001, Class B lamp 001, etc. Based on the one-to-one correspondence between real lamps and virtual lamps, the virtual lamp corresponding to each real lamp in the virtual scene (virtual Class A lamp 001) can be accurately obtained, as well as the position and posture of each virtual lamp initialized when constructing the virtual scene, so as to obtain the initial pose information corresponding to each virtual lamp.

[0123] Furthermore, after obtaining the initial pose information of the virtual lamp corresponding to each real lamp in the three-dimensional virtual space coordinate system, the second pose information can be compared with the initial pose information corresponding to the virtual lamp to obtain the corresponding comparison result.

[0124] Furthermore, (such as) Figure 12 (As shown, the effect of the source file is restored). For example, if the comparison result is consistent, it can be understood that the position (such as the position of the virtual lamp in the three-dimensional virtual space coordinate system) and posture (such as the rotation posture of the virtual lamp in the three-dimensional virtual space coordinate system) of the virtual lamp relative to the real lamp in the target scene have not shifted. That is, no calibration is required. Then the initial pose information corresponding to the virtual lamp can be used as the calibration pose of the virtual lamp.

[0125] Conversely, if the comparison results show inconsistencies, it can be understood that the position (e.g., the virtual lamp's position in the three-dimensional virtual space coordinate system) and posture (e.g., the virtual lamp's rotational posture in the three-dimensional virtual space coordinate system) of the virtual lamp relative to the real lamp in the target scene have shifted, which means that calibration is required. In this case, the initial pose information corresponding to the virtual lamp can be replaced or reassigned to the second pose information, so that the second pose information can be used as the lamp calibration pose corresponding to the virtual lamp.

[0126] Furthermore, after the initial pose information of all virtual lights in the virtual scene is calibrated to the light calibration pose, that is, the pose information of the virtual lights and the real lights has been adjusted to be consistent, it is possible to subsequently adjust the lights in the target scene and the virtual lights synchronously based on the light calibration pose of the virtual lights.

[0127] Optionally, in the above Figure 2 Based on the corresponding embodiments, in another optional embodiment of the lighting synchronization method provided in this application, such as... Figure 5 As shown, the lamp detection identifier includes a QR code; step S101 assigns a unique lamp detection identifier to each lamp in the target scene, including: steps S501 to S502; and step S102 includes step S503; step S103 includes step S504; step S104 includes step S505.

[0128] In step S501, the total number of lights in the target scene is obtained, and the set of QR codes corresponding to the total number of lights is obtained from the QR code dictionary;

[0129] In step S502, based on the QR code set, a unique corresponding QR code and a QR code identifier are assigned to each lamp;

[0130] In this embodiment, when there is a target scene that requires lighting, in order to better obtain the real pose information of each real lamp in the target scene, so that the virtual lamps in the virtual scene can be better calibrated based on the real pose information of each real lamp, thereby improving the accuracy of virtual lamp calibration and the synchronization effect between virtual lamps and real lamps, this embodiment can first obtain the total number of lamps in the target scene, and obtain the QR code set corresponding to the total number of lamps from the QR code dictionary. Then, based on the QR code set, a unique QR code and a QR code identifier corresponding to each QR code can be assigned to each lamp to accurately obtain the real pose information of each real lamp through the QR code.

[0131] Specifically, such as Figure 11As shown, before assigning a unique QR code to each light fixture in the target scene, this embodiment pre-constructs a QR code dictionary (such as...). Figure 11 The QR code dictionary shown is, for example, containing 1000 different QR codes.

[0132] Furthermore, for example, assuming the target scene is a performance using a total of 200 lights, that is, the total number of lights in the target scene, we can randomly obtain 200 QR codes from the QR code dictionary to obtain a set of QR codes corresponding to the total number of lights. The corresponding QR code identifier can be represented as an ID number from 0 to 199.

[0133] Furthermore, after obtaining the QR code set, a unique QR code and its corresponding identifier can be assigned to each lamp. For example, suppose a target scenario is a performance using 200 lamps, of which 20 are S-type lamps from a certain manufacturer. Then, 20 different QR codes with ID numbers from 0 to 19 can be obtained from the QR code set and assigned to the 20 S-type lamps, such as numbering them S_0-S_19.

[0134] In step S503, the QR code detection image corresponding to each QR code is acquired;

[0135] In this embodiment of the application, after assigning a unique corresponding QR code to each lamp in the target scene, the assigned QR code can be installed on the lamp in the corresponding target scene. Then, the QR code corresponding to the lamp in the target scene is photographed to collect the QR code detection image corresponding to each QR code, so that the real pose information of each QR code in the target scene can be accurately obtained based on each QR code detection image.

[0136] Specifically, such as Figure 11 As shown, before using the lighting fixtures in the target scene, the assigned QR codes can be installed on the corresponding lighting fixtures. Then, use a camera to photograph the QR codes installed on the lighting fixtures in the target scene. For example, use a camera to photograph... Figure 11 The QR code corresponding to marker point ID001 of the Class A lighting fixture 001 shown is used to acquire the QR code detection image, and then the image is captured by a data acquisition card (such as...). Figure 11 A capture card (e.g., an HDMI capture card) is used to convert the captured QR code detection image into a digital signal and transmit it to the server software (e.g., image processing software). This allows the software (e.g., image processing software) to export the digital signal as a location file for each QR code, thereby accurately obtaining the true pose information of each QR code in the target scene.

[0137] In step S504, based on each QR code detection image, the first pose information of each QR code in the target scene is obtained;

[0138] In this embodiment of the application, after obtaining the QR code detection image corresponding to each QR code, the first pose information of each QR code in the target scene can be obtained based on each QR code detection image. This allows the first pose information to be converted into the second pose information corresponding to each QR code in the three-dimensional virtual space coordinate system. This enables the virtual lamps to be calibrated based on the real pose information, thereby improving the accuracy of calibrating the virtual lamps to a certain extent.

[0139] Specifically, after obtaining the QR code detection image corresponding to each QR code, the coordinates of key points of the QR code can be obtained through a deep learning-based model, such as the three-dimensional coordinates of the upper left corner of the QR code, or through physical segmentation algorithms, edge detection algorithms, or other algorithms (no specific restrictions here), to obtain the real coordinate position of the QR code in the target scene in each QR code detection image (e.g., the position coordinates of the lamp (position(x,y,z) represent the position of the lamp in the world coordinate system), that is, the first coordinate position corresponding to the QR code. Then, based on the size of the QR code installed at each real lamp, according to visual algorithms and deep learning models, or other pose algorithms (no specific restrictions here), the real pose corresponding to each QR code (e.g., the rotation amount orientation(x,y,z,w) represents the rotational pose of the lamp in the world coordinate system), that is, the first pose corresponding to the QR code, can be calculated, based on the first coordinate position and the first pose corresponding to the QR code.

[0140] In step S505, the first pose information is converted into the second pose information corresponding to each QR code in the three-dimensional virtual space coordinate system;

[0141] In this embodiment of the application, after obtaining the first pose information, the first pose information can be converted into the second pose information corresponding to each QR code in the three-dimensional virtual space coordinate system, so that the initial pose information of the virtual lamps in the virtual scene can be calibrated by the second pose information in the same coordinate system, thereby improving the accuracy of calibrating the virtual lamps to a certain extent.

[0142] Specifically, first obtain the shooting origin coordinates of the shooting device (such as a camera) in the target scene (such as a performance stage), and then obtain the virtual origin coordinates of the virtual shooting device (such as a virtual camera) in the three-dimensional virtual space coordinate system of the virtual scene constructed based on the target scene.

[0143] Furthermore, after obtaining the coordinates of the virtual origin and the shooting origin, the first pose information (such as the position of the lamp in the world coordinate system and the rotational attitude of the lamp in the world coordinate system) can be converted into the second pose information in the three-dimensional virtual space coordinate system based on the coordinates of the three-dimensional virtual space coordinate system, the coordinates of the virtual origin, and the coordinates of the shooting origin, according to the coordinate transformation algorithm.

[0144] Optionally, in the above Figure 5 Based on the corresponding embodiments, in another optional embodiment of the lighting synchronization method provided in this application, such as... Figure 6 As shown, step S502 assigns a unique QR code and a corresponding QR code identifier to each lamp based on the QR code set, including:

[0145] In step S601, the lamps are classified based on the lamp model and lamp information to obtain P lamp categories and a set of lamps corresponding to each lamp category, where P is an integer greater than or equal to 1;

[0146] In step S602, based on P lighting categories and the lighting set corresponding to each lighting category, the QR code set is divided into P QR code subsets;

[0147] In step S603, each QR code in the P QR code subsets and the QR code identifier corresponding to each QR code are assigned to the lamps in the P lamp sets.

[0148] In this embodiment, when there is a target scene requiring lighting, in order to better obtain the real pose information of each real lamp in the target scene, so that the virtual lamps in the virtual scene can be better calibrated based on the obtained real pose information of each real lamp, thereby improving the accuracy of virtual lamp calibration and the synchronization effect between virtual and real lamps, this embodiment can first obtain the total number of lamps in the target scene and obtain the QR code set corresponding to the total number of lamps from the QR code dictionary. Then, the lamps can be classified based on the lamp model and lamp information to obtain P lamp categories and the lamp set corresponding to each lamp category. Then, based on the P lamp categories and the lamp set corresponding to each lamp category, the QR code set is divided into P QR code subsets, and each QR code in the P QR code subsets and the QR code identifier corresponding to each QR code are assigned to the lamps in the P lamp sets, so as to accurately obtain the real pose information of each real lamp through the QR code.

[0149] like Figure 11 As shown, before assigning a unique QR code to each light fixture in the target scene, this embodiment pre-constructs a QR code dictionary (such as...). Figure 11The QR code dictionary shown is, for example, containing 1000 different QR codes.

[0150] Furthermore, for example, assuming the target scene is a performance using a total of 200 lights, that is, the total number of lights in the target scene, we can randomly obtain 200 QR codes from the QR code dictionary to obtain a set of QR codes corresponding to the total number of lights. The corresponding QR code identifier can be represented as an ID number from 0 to 199.

[0151] Furthermore, lighting fixtures can be classified based on their model and information. For example, if the target scenario is a performance using 200 lights, they can be classified according to the manufacturer and model of the lights. Suppose we can obtain P (e.g., 10) lighting fixture categories (e.g., A to J categories) and a set of lights corresponding to each category, where each set of lights contains at least one light.

[0152] Furthermore, after obtaining the QR code set, based on the P lighting fixture categories and the corresponding lighting fixture set for each category, the QR code set is divided into P QR code subsets. For example, the QR code set can be divided into 10 QR code subsets of categories A to J. For instance, assuming there are 10 lighting fixtures in category A, the corresponding QR code subset also contains 10 different QR codes with ID numbers from 20 to 29. Then, the 10 different QR codes with ID numbers from 20 to 29 can be assigned to 10 category A lighting fixtures, such as numbering A_20-A_29. Similarly, each QR code in the P QR code subsets and its corresponding QR code identifier can be assigned to the lighting fixtures in the P lighting fixture sets.

[0153] Optionally, in the above Figure 5 Based on the corresponding embodiments, in another optional embodiment of the lighting synchronization method provided in this application, such as... Figure 7 As shown, in step S503, the QR code detection image corresponding to each QR code is acquired, including:

[0154] In step S701, the QR code corresponding to each lamp is installed at the corresponding lamp in the target scene to obtain the QR code detection point corresponding to each lamp;

[0155] In step S702, at the acquisition time, the QR code at each QR code detection point in the target scene is captured to obtain the QR code detection image corresponding to each QR code.

[0156] In this embodiment of the application, after assigning a unique corresponding QR code to each lamp in the target scene, the QR code corresponding to each lamp can be installed at the corresponding lamp in the target scene to obtain the QR code detection point corresponding to each lamp. Then, at the acquisition time, the QR code of each QR code detection point in the target scene is photographed to obtain the QR code detection image corresponding to each QR code, so that the real pose information of each QR code in the target scene can be accurately obtained based on each QR code detection image.

[0157] Specifically, such as Figure 11 As shown, before using the lights in the target scene for illumination, the assigned QR codes can be installed on the corresponding lights in the target scene. Specifically, each QR code can be customized using lightweight and non-deformable materials such as acrylic sheets. The size of the QR code sheet can be set according to the current usage scene. The QR code sheet with the corresponding number is fixed on the light base, and then the lights are installed to obtain the QR code detection point for each light.

[0158] Furthermore, at the time of data collection (i.e., earlier than the time when the lighting fixtures in the target scene are actually used), a camera can be used to capture the QR code at each QR code detection point in the target scene. For example, a camera can be used to capture... Figure 11 The QR code corresponding to marker point ID001 of the Class A lighting fixture 001 shown is used to acquire the QR code detection image, and then the image is captured by a data acquisition card (such as...). Figure 11 The acquisition card shown (e.g., HDMI acquisition card) is used to convert the acquired QR code detection image into a digital signal and transmit it to the server software (e.g., image processing software). This allows the software (e.g., image processing software) to export the digital signal as a position file for each QR code, thereby accurately obtaining the true pose information of each QR code in the target scene, i.e., the first coordinate position and first pose of the QR code in the image coordinate system centered on the camera.

[0159] Optionally, in the above Figure 7 Based on the corresponding embodiments, in another optional embodiment of the lighting synchronization method provided in this application, such as... Figure 8 As shown, the first pose information includes the first coordinate position and the first pose; step S504, based on each QR code detection image, obtains the first pose information of each QR code in the target scene, including:

[0160] In step S801, the first coordinate position corresponding to the QR code in each QR code detection image is obtained;

[0161] In step S802, the first pose corresponding to each QR code is calculated based on the size of the QR code installed at each QR code detection point.

[0162] In this embodiment, after obtaining the QR code detection image corresponding to each QR code, the first coordinate position of the QR code in each QR code detection image can be obtained first. Then, based on the size of the QR code installed at each QR code detection point, the first pose corresponding to each QR code is calculated to obtain the first pose information corresponding to each QR code. This allows the first pose information to be converted into the second pose information corresponding to each QR code in the three-dimensional virtual space coordinate system, thereby enabling the virtual lamp to be calibrated based on the real pose information, which can improve the accuracy of calibrating the virtual lamp to a certain extent.

[0163] Specifically, after obtaining the QR code detection image corresponding to each QR code, the coordinates of the key points of the QR code can be obtained through a deep learning-based model, such as the three-dimensional coordinates of the upper left corner of the QR code, or through physical segmentation algorithms, edge detection algorithms, etc., or other algorithms, without specific restrictions here, to obtain the real coordinate position of the QR code in the target scene in each QR code detection image (such as the position coordinates of the lamp, position(x,y,z), which represents the position of the lamp in the world coordinate system, that is, the position of the QR code in the image coordinate system centered on the camera), that is, the first coordinate position corresponding to the QR code.

[0164] Furthermore, based on the size of the QR code at each QR code detection point in the target scene (e.g., customizing the size of the QR code board corresponding to each QR code using lightweight and non-deformable materials such as acrylic sheets), and according to visual algorithms and deep learning models, or other pose algorithms (no specific restrictions are imposed here), the true pose corresponding to each QR code (e.g., the rotation amount orientation(x,y,z,w) represents the rotational posture of the lamp in the world coordinate system, that is, the pose of the QR code in the image coordinate system centered on the camera) can be calculated. This is the first pose corresponding to the QR code. Thus, based on the first coordinate position and the first pose corresponding to the QR code, the first pose information of each QR code in the target scene can be obtained.

[0165] Optionally, in the above Figure 8 Based on the corresponding embodiments, in another optional embodiment of the lighting synchronization method provided in this application, such as... Figure 9 As shown, the second pose information includes the second coordinate position and the second orientation; step S505 converts the first pose information into the second pose information corresponding to each QR code in the three-dimensional virtual space coordinate system, including:

[0166] In step S901, the coordinates of the shooting origin in the target scene and the coordinates of the virtual origin in the three-dimensional virtual space coordinate system are obtained;

[0167] In step S902, based on the three-dimensional virtual space coordinate system, the virtual origin coordinate position, and the shooting origin coordinate position, the first coordinate position and the first attitude are converted into the second coordinate position and the second attitude.

[0168] In this embodiment, after obtaining the first coordinate position and the first pose, the coordinate position of the shooting origin in the target scene and the coordinate position of the virtual origin in the three-dimensional virtual space coordinate system can be obtained first. Then, based on the three-dimensional virtual space coordinate system, the coordinate position of the virtual origin and the coordinate position of the shooting origin, the first coordinate position and the first pose are converted into the second coordinate position and the second pose, so that the initial pose information of the virtual lamp in the virtual scene can be calibrated using the second pose information in the same coordinate system, thereby improving the accuracy of calibrating the virtual lamp to a certain extent.

[0169] Specifically, first obtain the shooting origin coordinates of the shooting device (such as a camera) in the target scene (such as a performance stage) (i.e., the position in the image coordinate system centered on the camera), and then obtain the virtual origin coordinates of the virtual shooting device (such as a virtual camera) in the three-dimensional virtual space coordinate system of the virtual scene constructed based on the target scene (i.e., the position in the three-dimensional virtual space coordinate system centered on the virtual camera).

[0170] Furthermore, after obtaining the coordinates of the virtual origin and the shooting origin, the first pose information (such as the position of the lamp in the world coordinate system and the rotational attitude of the lamp in the world coordinate system) can be converted into the second pose information in the three-dimensional virtual space coordinate system based on the three-dimensional virtual space coordinate system, the coordinates of the virtual origin, and the coordinates of the shooting origin, according to the coordinate transformation algorithm (such as translation, rotation, and matrix multiplication algorithms).

[0171] Optionally, in the above Figure 2 Based on the corresponding embodiments, in another optional embodiment of the lighting synchronization method provided in this application, such as... Figure 10 As shown, step S106 involves synchronously adjusting the lighting fixtures in the target scene based on the lighting fixture calibration pose, including:

[0172] In step S1001, if an adjustment command for the virtual lighting fixture is received, the virtual lighting fixture in the virtual scene is adjusted based on the adjustment command and the lighting fixture calibration pose, and a corresponding lighting fixture adjustment signal is generated.

[0173] In step S1002, based on the lighting adjustment signal, the lighting fixtures in the target scene corresponding to the virtual lighting fixtures are controlled to be adjusted synchronously.

[0174] In this embodiment, after adjusting the poses of all virtual lights in the virtual scene to the calibrated poses, the pose information of the virtual lights and the real lights can be made consistent. If an adjustment command for the virtual lights is received, the virtual lights in the virtual scene can be adjusted based on the adjustment command and the calibrated poses, and a corresponding light adjustment signal can be generated. This allows the lights in the target scene corresponding to the virtual lights to be adjusted synchronously based on the light adjustment signal, thereby achieving synchronous control of the virtual lights to illuminate the real lights in the target scene and improving the synchronization effect when the virtual lights and the real lights illuminate.

[0175] Specifically, such as Figure 11 As shown, the poses of all virtual lights in the virtual scene are adjusted to the light calibration poses (e.g., ...). Figure 11 After the virtual light pose initialization calibration is completed, the coordinate position and posture of the virtual light fixture and the real light fixture are adjusted to be consistent, thereby maintaining the consistency and synchronization between the virtual light fixture and the real light fixture.

[0176] Furthermore, once the pose information of each virtual lamp in the virtual scene matches the pose information of the corresponding real lamp in the target scene, for example, if a lighting manager needs to adjust or rotate one or more lamps in the target scene based on the lighting requirements of the target scene, they can first input the corresponding lighting operation (e.g., adjust the brightness of lamp 001 of type A, such as increase it) on the virtual scene display interface of the terminal device, so that the terminal device can respond to the lighting operation of the lighting manager, generate the adjustment command (increase the brightness of virtual lamp 001 of type A) and send it to the server.

[0177] Furthermore, when the server receives an adjustment command for the virtual lighting fixture (increasing the brightness of virtual Class A lighting fixture 001), the server can, based on the adjustment command, first determine the virtual lighting fixture (such as virtual Class A lighting fixture 001) corresponding to the calibrated lighting fixture's calibration pose in the virtual scene. Then, based on the adjustment command (increasing the brightness of virtual Class A lighting fixture 001), the server can adjust or control the virtual lighting fixture corresponding to the calibrated lighting fixture's calibration pose. Furthermore, based on the consistency and synchronization between the virtual lighting fixture and the real lighting fixture, it can... This generates corresponding lighting adjustment signals for real lighting fixtures (such as engine-sent signals or DMX signals, for example, increasing the brightness of Class A lighting fixture 001). Then, the lighting adjustment signals (such as engine-sent signals or DMX signals, for example, increasing the brightness of Class A lighting fixture 001) can be transmitted to the corresponding real lighting fixtures in the target scene through the real engine control console, so as to control or adjust the lighting effect of the corresponding real lighting fixtures in the target scene and the virtual lighting fixtures in the virtual scene to synchronize (such as increasing the brightness of Class A lighting fixture 001).

[0178] The lighting synchronization method and apparatus in this application are described in detail below. Please refer to [link / reference]. Figure 14 , Figure 14 This is a schematic diagram of one embodiment of the lamp synchronization method apparatus in this application. The lamp synchronization method apparatus 20 includes:

[0179] The processing unit 201 is used to assign a unique corresponding lamp detection identifier to each lamp in the target scene, wherein each lamp detection identifier corresponds to a lamp model and lamp information;

[0180] The acquisition unit 202 is used to acquire the identification image corresponding to each lamp detection identification;

[0181] The acquisition unit 202 is also used to acquire the first pose information of each light fixture detection mark in the target scene based on each mark detection image;

[0182] The processing unit 201 is also used to convert the first pose information into the second pose information corresponding to each lamp detection mark in the three-dimensional virtual space coordinate system;

[0183] The processing unit 201 is also used to calibrate the virtual lamps in the virtual scene based on the second pose information, lamp model and lamp information corresponding to each lamp detection mark, so as to obtain the calibrated lamp pose.

[0184] The control unit 203 is used to synchronously adjust the lighting fixtures in the target scene based on the lighting fixture calibration posture.

[0185] Optionally, in the above Figure 14Based on the corresponding embodiments, in another embodiment of the lighting synchronization device provided in this application, the processing unit 201 can specifically be used for:

[0186] Based on the lamp model and lamp information, obtain the initial pose information corresponding to the virtual lamp;

[0187] Based on the second pose information, the initial pose information corresponding to the virtual lamp is calibrated to obtain the lamp calibration pose.

[0188] Optionally, in the above Figure 14 Based on the corresponding embodiments, in another embodiment of the lighting synchronization device provided in this application, the processing unit 201 can specifically be used for:

[0189] The second pose information is compared with the initial pose information corresponding to the virtual lamp to obtain the comparison result;

[0190] If the comparison results show that the information is consistent, then the initial pose information corresponding to the virtual lamp will be used as the lamp calibration pose corresponding to the virtual lamp.

[0191] If the comparison result shows that the information is inconsistent, the initial pose information corresponding to the virtual lamp is replaced with the second pose information to obtain the lamp calibration pose corresponding to the virtual lamp.

[0192] Optionally, in the above Figure 14 Based on the corresponding embodiments, in another embodiment of the lighting synchronization device provided in this application, the processing unit 201 can specifically be used for:

[0193] Obtain the total number of lights in the target scene, and retrieve the set of QR codes corresponding to the total number of lights from the QR code dictionary;

[0194] Based on the QR code set, each lamp is assigned a unique corresponding QR code and a QR code identifier for each QR code.

[0195] The acquisition unit 202 can be specifically used to: acquire the QR code detection image corresponding to each of the QR codes;

[0196] The acquisition unit 202 can be specifically used to: acquire the first pose information of each QR code in the target scene based on each QR code detection image;

[0197] The processing unit 201 can be specifically used to: convert the first pose information into second pose information corresponding to each of the QR codes in a three-dimensional virtual space coordinate system.

[0198] Optionally, in the above Figure 14 Based on the corresponding embodiments, in another embodiment of the lighting synchronization device provided in this application, the processing unit 201 can specifically be used for:

[0199] Based on the lamp model and lamp information, the lamps are classified to obtain P lamp categories and the lamp set corresponding to each lamp category, where P is an integer greater than or equal to 1;

[0200] Based on P lighting categories and the set of lighting fixtures corresponding to each category, the QR code set is divided into P QR code subsets;

[0201] Assign each QR code and its corresponding QR code identifier from the P subsets of QR codes to the lamps in the P sets of lamps.

[0202] Optionally, in the above Figure 14 Based on the corresponding embodiments, in another embodiment of the lighting synchronization device provided in this application, the acquisition unit 202 can specifically be used for:

[0203] Install the QR code corresponding to each lamp on the corresponding lamp in the target scene to obtain the QR code detection point corresponding to each lamp;

[0204] At the time of data acquisition, the QR code at each QR code detection point in the target scene is captured, and the QR code detection image corresponding to each QR code is obtained.

[0205] Optionally, in the above Figure 14 Based on the corresponding embodiments, in another embodiment of the lighting synchronization device provided in this application, the acquisition unit 202 can specifically be used for:

[0206] Obtain the first coordinate position of the QR code in each QR code detection image;

[0207] Based on the size of the QR code installed at each QR code detection point, calculate the first pose corresponding to each QR code.

[0208] Optionally, in the above Figure 14 Based on the corresponding embodiments, in another embodiment of the lighting synchronization device provided in this application, the processing unit 201 can specifically be used for:

[0209] Obtain the coordinates of the shooting origin in the target scene, and the coordinates of the virtual origin in the three-dimensional virtual space coordinate system;

[0210] Based on the three-dimensional virtual space coordinate system, the virtual origin coordinate position, and the shooting origin coordinate position, the first coordinate position and the first pose are converted into the second coordinate position and the second pose.

[0211] Optionally, in the above Figure 14 Based on the corresponding embodiments, in another embodiment of the lighting synchronization device provided in this application, the control unit 203 can specifically be used for:

[0212] If an adjustment command for a virtual light fixture is received, the virtual light fixture in the virtual scene is adjusted based on the adjustment command and the light fixture calibration pose, and a corresponding light fixture adjustment signal is generated.

[0213] Based on the lighting adjustment signal, the virtual lighting fixtures in the target scene are controlled to adjust synchronously.

[0214] This application also provides a schematic diagram of another computer device, such as... Figure 15 As shown, Figure 15 This is a schematic diagram of a computer device structure provided in an embodiment of this application. The computer device 300 can vary significantly due to different configurations or performance. It may include one or more central processing units (CPUs) 310 (e.g., one or more processors) and a memory 320, and one or more storage media 330 (e.g., one or more mass storage devices) for storing application programs 331 or data 332. The memory 320 and storage media 330 can be temporary or persistent storage. The program stored in the storage media 330 may include one or more modules (not shown in the diagram), each module including a series of instruction operations on the computer device 300. Furthermore, the CPU 310 may be configured to communicate with the storage media 330 and execute the series of instruction operations in the storage media 330 on the computer device 300.

[0215] Computer device 300 may also include one or more power supplies 340, one or more wired or wireless network interfaces 350, one or more input / output interfaces 360, and / or one or more operating systems 333, such as Windows Server. TM Mac OS X TM Unix TM Linux TM FreeBSD TM etc.

[0216] The aforementioned computer device 300 is also used to perform, for example Figures 2 to 10 The steps in the corresponding embodiments.

[0217] Another aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements... Figures 2 to 10 The steps in the method described in the illustrated embodiment.

[0218] Another aspect of this application provides a computer program product comprising a computer program, which, when executed by a processor, implements as follows: Figures 2 to 10 The steps in the method described in the illustrated embodiment.

[0219] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0220] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.

[0221] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0222] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0223] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A method for synchronizing lighting fixtures, characterized in that, include: Assign a unique lamp detection identifier to each lamp in the target scene, wherein each lamp detection identifier corresponds to a lamp model and lamp information; Collect the identification image corresponding to each of the lamp detection marks; Based on each of the detected images, the first pose information of each of the detected lamps in the target scene is obtained. The first pose information includes the position of the lamp in the world coordinate system and the rotational attitude of the lamp in the world coordinate system. The first pose information is converted into second pose information corresponding to each of the lamp detection identifiers in the three-dimensional virtual space coordinate system. The second pose information includes the position of the lamp in the three-dimensional virtual space coordinate system and the rotation attitude of the lamp in the three-dimensional virtual space coordinate system. Based on the second pose information corresponding to each of the lamp detection identifiers, the lamp model and the lamp information, the lamps are calibrated with the virtual lamps in the virtual scene to obtain the calibrated lamp pose; If an adjustment instruction for a virtual light fixture is received, the virtual light fixture in the virtual scene is adjusted based on the adjustment instruction and the light fixture calibration pose, and a corresponding light fixture adjustment signal is generated. Based on the lighting adjustment signal, the lighting fixtures in the target scene corresponding to the virtual lighting fixture are controlled to be adjusted synchronously.

2. The method according to claim 1, characterized in that, The step of calibrating the virtual lamps in the virtual scene based on the second pose information corresponding to each lamp detection identifier, the lamp model, and the lamp information to obtain the calibrated lamp pose includes: Based on the lamp model and the lamp information, obtain the initial pose information corresponding to the virtual lamp; Based on the second pose information, the initial pose information corresponding to the virtual lamp is calibrated to obtain the calibrated pose of the lamp.

3. The method according to claim 2, characterized in that, The step of calibrating the initial pose information corresponding to the virtual lamp based on the second pose information to obtain the calibrated pose of the lamp includes: The second pose information is compared with the initial pose information corresponding to the virtual lamp to obtain the comparison result; If the comparison result is consistent, then the initial pose information corresponding to the virtual lamp is used as the lamp calibration pose corresponding to the virtual lamp. If the comparison result is inconsistent, the initial pose information corresponding to the virtual lamp is replaced with the second pose information to obtain the lamp calibration pose corresponding to the virtual lamp.

4. The method according to claim 1, characterized in that, The lighting fixture detection identifier includes a QR code and a corresponding QR code identifier for each QR code; assigning a unique lighting fixture detection identifier to each lighting fixture in the target scene includes: Obtain the total number of lights in the target scene, and obtain the set of QR codes corresponding to the total number of lights from the QR code dictionary; Based on the QR code set, a unique corresponding QR code and a QR code identifier are assigned to each of the lamps; The acquisition of the identification detection image corresponding to each of the lamp detection identification marks includes: Collect the QR code detection image corresponding to each of the QR codes; The step of obtaining the first pose information of each light fixture detection icon in the target scene based on each icon detection image includes: Based on each of the QR code detection images, obtain the first pose information of each QR code in the target scene; The step of converting the first pose information into second pose information corresponding to each of the lamp detection identifiers in a three-dimensional virtual space coordinate system includes: The first pose information is converted into the second pose information corresponding to each of the QR codes in a three-dimensional virtual space coordinate system.

5. The method according to claim 4, characterized in that, The step of assigning a unique QR code to each lamp based on the QR code set and a QR code identifier corresponding to each QR code includes: Based on the lamp model and the lamp information, the lamps are classified to obtain P lamp categories and a set of lamps corresponding to each lamp category, where P is an integer greater than or equal to 1; Based on the P categories of lighting fixtures and the set of lighting fixtures corresponding to each category, the QR code set is divided into P subsets of QR codes. Each QR code in the P subsets of QR codes, along with its corresponding QR code identifier, is assigned to one of the P sets of lamps.

6. The method according to claim 4, characterized in that, The process of acquiring the QR code detection image corresponding to each QR code includes: Install the QR code corresponding to each of the lamps onto the corresponding lamps in the target scene to obtain the QR code detection point corresponding to each of the lamps; At the time of acquisition, the QR code at each QR code detection point in the target scene is captured to obtain the QR code detection image corresponding to each QR code.

7. The method according to claim 6, characterized in that, The first pose information includes a first coordinate position and a first pose; the step of obtaining the first pose information of each QR code in the target scene based on each QR code detection image includes: Obtain the first coordinate position corresponding to the QR code in each of the QR code detection images; Based on the size of the QR code installed at each QR code detection point, the first pose corresponding to each QR code is calculated.

8. The method according to claim 7, characterized in that, The second pose information includes a second coordinate position and a second orientation; the step of converting the first pose information into second pose information corresponding to each of the QR codes in a three-dimensional virtual space coordinate system includes: Obtain the coordinates of the shooting origin in the target scene and the coordinates of the virtual origin in the three-dimensional virtual space coordinate system; Based on the three-dimensional virtual space coordinate system, the virtual origin coordinate position, and the shooting origin coordinate position, the first coordinate position is converted into the second coordinate position and the second posture using the first posture.

9. A lighting synchronization method device, characterized in that, include: The processing unit is used to assign a unique lamp detection identifier to each lamp in the target scene, wherein each lamp detection identifier corresponds to a lamp model and lamp information; The acquisition unit is used to acquire the identification detection image corresponding to each of the lamp detection identification marks; The acquisition unit is further configured to acquire, based on each of the identification detection images, the first pose information of each of the lighting detection identifiers in the target scene, the first pose information including the position of the lighting fixture in the world coordinate system and the rotational attitude of the lighting fixture in the world coordinate system; The processing unit is further configured to convert the first pose information into second pose information corresponding to each of the lamp detection identifiers in the three-dimensional virtual space coordinate system. The second pose information includes the position of the lamp in the three-dimensional virtual space coordinate system and the rotational attitude of the lamp in the three-dimensional virtual space coordinate system. The processing unit is further configured to calibrate with the virtual lamps in the virtual scene based on the second pose information corresponding to each lamp detection identifier, the lamp model and the lamp information, to obtain the calibrated lamp pose; The control unit is configured to, upon receiving an adjustment instruction for a virtual lamp, adjust the virtual lamp in the virtual scene based on the adjustment instruction and the lamp calibration pose, and generate a corresponding lamp adjustment signal; and based on the lamp adjustment signal, control the lamp in the target scene corresponding to the virtual lamp to perform synchronous adjustment.

10. The apparatus according to claim 9, characterized in that, The processing unit is specifically used for: Based on the lamp model and the lamp information, obtain the initial pose information corresponding to the virtual lamp; Based on the second pose information, the initial pose information corresponding to the virtual lamp is calibrated to obtain the calibrated pose of the lamp.

11. The apparatus according to claim 10, characterized in that, The processing unit is specifically used for: The second pose information is compared with the initial pose information corresponding to the virtual lamp to obtain the comparison result; If the comparison result is consistent, then the initial pose information corresponding to the virtual lamp is used as the lamp calibration pose corresponding to the virtual lamp. If the comparison result is inconsistent, the initial pose information corresponding to the virtual lamp is replaced with the second pose information to obtain the lamp calibration pose corresponding to the virtual lamp.

12. The apparatus according to claim 9, characterized in that, The lamp detection mark includes a QR code and a QR code identifier corresponding to each QR code; the processing unit is specifically used for: Obtain the total number of lights in the target scene, and obtain the set of QR codes corresponding to the total number of lights from the QR code dictionary; Based on the set of QR codes, a unique QR code and a QR code identifier are assigned to each of the lamps; The acquisition unit is specifically used to acquire the QR code detection image corresponding to each QR code; The acquisition unit is specifically used to acquire the first pose information of each QR code in the target scene based on each QR code detection image; The processing unit is specifically used to convert the first pose information into second pose information corresponding to each of the QR codes in a three-dimensional virtual space coordinate system.

13. The apparatus according to claim 12, characterized in that, The processing unit is specifically used for: Based on the lamp model and the lamp information, the lamps are classified to obtain P lamp categories and a set of lamps corresponding to each lamp category, where P is an integer greater than or equal to 1; Based on the P categories of lighting fixtures and the set of lighting fixtures corresponding to each category, the QR code set is divided into P subsets of QR codes. Each QR code in the P subsets of QR codes, along with its corresponding QR code identifier, is assigned to one of the P sets of lamps.

14. The apparatus according to claim 12, characterized in that, The acquisition unit is specifically used for: Install the QR code corresponding to each of the lamps onto the corresponding lamps in the target scene to obtain the QR code detection point corresponding to each of the lamps; At the time of acquisition, the QR code at each QR code detection point in the target scene is captured to obtain the QR code detection image corresponding to each QR code.

15. The apparatus according to claim 14, characterized in that, The first pose information includes a first coordinate position and a first orientation; the acquisition unit is specifically used for: Obtain the first coordinate position corresponding to the QR code in each of the QR code detection images; Based on the size of the QR code installed at each QR code detection point, the first pose corresponding to each QR code is calculated.

16. The apparatus according to claim 15, characterized in that, The second pose information includes a second coordinate position and a second orientation; the processing unit is specifically used for: Obtain the coordinates of the shooting origin in the target scene and the coordinates of the virtual origin in the three-dimensional virtual space coordinate system; Based on the three-dimensional virtual space coordinate system, the virtual origin coordinate position, and the shooting origin coordinate position, the first coordinate position is converted into the second coordinate position and the second posture using the first posture.

17. A computer device comprising a memory, a processor, and a bus system, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 8; The bus system is used to connect the memory and the processor to enable communication between the memory and the processor.

18. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 8.

19. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 8.

Images