LED screen display control method and device, electronic device, storage medium and program product

Abstract

The present disclosure relates to a display control method and device for an LED screen, an electronic device, a storage medium, and a program product. The method comprises: obtaining position information of a rolling camera during a virtual shooting process; determining the inner cone image and the pixel position information of multiple target LED lamp beads based on the position information and the screen model of the LED screen; determining the scanning timing of the target LED lamp beads based on the starting shooting time of the rolling camera, the pixel position information, and the rolling parameters of the rolling camera; determining the display timing of the multiple target LED lamp beads based on the scanning timing of the target LED lamp beads, the shutter duration, and the continuous display duration of the LED lamp beads; and controlling the color display of the multiple target LED lamp beads in the LED screen based on the display timing of the multiple target LED lamp beads. As a result, the rolling camera can be applied to virtual shooting scenes with higher frame rate image refresh requirements, greatly improving the display acquisition ratio of the LED screen.

Description

Technical Field

[0001] The present disclosure relates to the field of virtual shooting, and in particular to a display control method and device for an LED screen, an electronic device, a storage medium, and a program product. Background Art

[0002] Virtual Production is an innovative filming method that combines real-time computer graphics technology with traditional film and television production techniques. By integrating virtual sets, special effects, and live-action content in real time, it enables directors, cinematographers, and other creative staff to visualize the final result during filming, without having to wait for post-production. This technology significantly improves production efficiency while providing greater flexibility for creators. In recent years, virtual production technology has become an increasingly popular solution in the industry, aiming to improve the quality of film and television productions while reducing both the time and financial costs of production.

[0003] A rolling shutter camera is an imaging device that uses progressive scanning exposure. Instead of capturing the entire image all at once, its sensor reads image data line by line from top to bottom. Because its sensor uses a rolling shutter design, it eliminates the need for a separate storage unit for each pixel (as required with a global shutter), resulting in lower hardware costs. Therefore, applying rolling shutter cameras to virtual photography has become a growing trend.

[0004] However, due to the rolling shutter characteristics of the rolling camera's line-by-line scanning exposure, the time it takes to capture a frame is longer than that of a global camera, resulting in a relatively low display acquisition rate on the LED screen. Therefore, it may not be able to meet the virtual shooting scenarios that require higher frame rate image refresh. Summary of the Invention

[0005] In view of this, the present disclosure proposes a display control method and device, electronic device, storage medium and program product for an LED screen, which can apply a rolling camera to virtual shooting scenes with higher frame rate image refresh requirements without being troubled by the rolling time, and greatly improve the display acquisition ratio of the LED screen.

[0006] According to one aspect of the present disclosure, a display control method for an LED screen is provided, which is applied to shooting a virtual shooting scene of the LED screen using a rolling camera, the method comprising: obtaining the posture information of the rolling camera during the virtual shooting process; determining, based on the posture information of the rolling camera and the screen model of the LED screen, an inner cone image to be displayed on the LED screen within the inner cone range of the rolling camera and pixel position information of a plurality of target LED lamp beads in the LED screen within the inner cone range of the rolling camera, wherein the pixel position information includes the pixel position of the target LED lamp bead projected onto the imaging plane of the rolling camera; determining, based on the starting shooting moment of the rolling camera, the pixel positions of the plurality of target LED lamp beads, The information and the rolling parameters of the rolling camera are used to determine the scanning timing of each target LED lamp bead; wherein, the rolling parameters represent the rolling duration of the rolling camera, and the scanning timing includes the start time when each target LED lamp bead is scanned by the rolling camera; according to the scanning timing of each target LED lamp bead, the shutter duration of the rolling camera and the continuous display duration of a single LED lamp bead, the display timing of the multiple target LED lamp beads displaying the inner cone picture is determined, wherein, the display timing represents the start display time and the end display time of each target LED lamp bead; according to the display timing of the multiple target LED lamp beads displaying the inner cone picture, the multiple target LED lamp beads in the LED screen are controlled to display colors.

[0007] In one possible implementation, the determining of the scanning timing of each target LED lamp bead according to the starting shooting moment of the rolling camera, the pixel position information of the multiple target LED lamp beads, and the rolling parameters of the rolling camera includes: for the i-th row of pixels on the imaging plane of the rolling camera, determining the i-th group of lamp beads corresponding to the i-th row of pixels according to the pixel position information of the multiple target LED lamp beads, the i-th group of lamp beads including a row or multiple target LED lamp beads projected onto the i-th row of pixels, 1≤i≤N, N is The total pixel height of the imaging plane; when i is 1, the starting time for the rolling camera to start scanning the first group of lamp beads corresponding to the first row of pixels is determined according to the starting shooting time of the rolling camera and the rolling time and shutter time of the rolling camera; when i is greater than 1, the starting time for the rolling camera to start scanning the i-th group of lamp beads is determined according to the pixel height of the i-th row of pixels, the rolling parameters, the total pixel height of the imaging plane and the starting time when the rolling camera is to start scanning the first group of lamp beads corresponding to the first row of pixels.

[0008] In one possible implementation, the display timing of the multiple target LED lamp beads displaying the inner cone image is determined based on the scanning timing of each target LED lamp bead, the shutter duration of the rolling camera, and the continuous display duration of a single LED lamp bead, including: taking the difference between the shutter duration of the rolling camera and the continuous display duration of a single LED lamp bead as a redundant duration; for the i-th group of lamp beads corresponding to the i-th row of pixels, determining the start display time of the i-th group of lamp beads based on the start time of the rolling camera to start scanning the i-th group of lamp beads and the redundant duration, wherein the start display time of the i-th group of lamp beads is earlier than the start time when the rolling camera starts scanning; and taking the sum of the start display time of the i-th group of lamp beads and the continuous display duration of the single LED lamp bead as the end display time of the i-th group of lamp beads.

[0009] In a possible implementation, the rolling shutter duration represents the time difference between the start exposure time of the top row of pixels of the rolling shutter camera and the start exposure time of the last row of pixels, wherein the calibration process of the rolling shutter parameters of the rolling shutter camera includes: in the process of controlling the LED screen to alternately display the first image frame and the second image frame and continuously adjusting the display phase of the LED lamp beads in the LED screen, using the rolling shutter camera to shoot the LED screen at a preset acquisition frame rate to obtain a captured video; wherein the picture contents of the first image frame and the second image frame are different; by dividing the video picture of the captured video The method comprises the steps of: analyzing the first moment when the first mixed picture just changes to the picture content of the second image frame, and obtaining a second moment when the picture content of the second image frame just does not reveal the picture content of the second mixed picture; wherein the first mixed picture includes a top area of ​​the video picture as the picture content of the first image frame and other areas as the picture content of the second image frame, and the second mixed picture includes a bottom area of ​​the video picture as the picture content of the first image frame and other areas as the picture content of the second image frame; and determining the rolling parameters of the rolling camera based on the first moment, the second moment, the single-frame display duration of the second image frame, and the shutter duration of the rolling camera.

[0010] In a possible implementation, the method of determining, based on the posture information of the rolling camera and the screen model of the LED screen, an inner cone image to be displayed on the LED screen and pixel position information of multiple target LED lamp beads in the LED screen and within the inner cone of the rolling camera includes: determining, based on the posture information of the rolling camera, a relative posture of a virtual camera and the screen model in a virtual shooting environment and the inner cone image to be displayed on the LED screen, wherein the posture and internal parameters of the virtual camera and the rolling camera are the same; determining an inner cone area in the screen model and within the inner cone of the virtual camera based on the relative posture, and using multiple LED lamp beads in the inner cone area as target LED lamp beads; and projecting each target LED lamp bead in the screen model onto the imaging plane of the virtual camera based on the spatial position of each target LED lamp bead in the screen model to obtain pixel position information of the multiple target LED lamp beads in the inner cone of the rolling camera.

[0011] In one possible implementation, the inner cone picture to be displayed is any one of at least two pictures to be displayed alternately, wherein the display timing of displaying the inner cone picture according to the multiple target LED lamp beads controls the multiple target LED lamp beads in the LED screen to display colors, including: determining the target display timing of the multiple target LED lamp beads alternately displaying at least two pictures according to the display timing of displaying any one picture and the continuous display duration of a single LED lamp bead; controlling the individual target LED lamp beads in the LED screen to display colors according to the target display timing of the multiple target LED lamp beads alternately displaying at least two pictures.

[0012] According to another aspect of the present disclosure, a display control device for an LED screen is provided, which is used for shooting a virtual shooting scene of the LED screen using a rolling camera, and the device includes: an acquisition module for acquiring the posture information of the rolling camera during the virtual shooting process; a first determination module for determining, based on the posture information of the rolling camera and the screen model of the LED screen, an inner cone picture to be displayed on the LED screen within the inner cone range of the rolling camera and pixel position information of a plurality of target LED lamp beads in the LED screen within the inner cone range of the rolling camera, wherein the pixel position information includes the pixel position of the target LED lamp bead projected onto the imaging plane of the rolling camera; a second determination module for determining, based on the starting shooting moment of the rolling camera, the posture information of the plurality of target LED lamp beads The pixel position information and the rolling parameters of the rolling camera are used to determine the scanning timing of each target LED lamp bead; wherein, the rolling parameters represent the rolling duration of the rolling camera, and the scanning timing includes the start time when each target LED lamp bead is scanned by the rolling camera; a third determination module is used to determine the display timing of the multiple target LED lamp beads displaying the inner cone image according to the scanning timing of each target LED lamp bead, the shutter duration of the rolling camera and the continuous display duration of a single LED lamp bead, wherein the display timing represents the start display time and the end display time of each target LED lamp bead; a display control module is used to control the multiple target LED lamp beads in the LED screen to display colors according to the display timing of the multiple target LED lamp beads displaying the inner cone image.

[0013] According to another aspect of the present disclosure, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the above method.

[0014] According to another aspect of the present disclosure, a non-volatile computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above method are implemented.

[0015] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, or a non-volatile computer-readable storage medium carrying the computer program, wherein the computer program implements the steps of the above method when executed by a processor.

[0016] According to various aspects of the present disclosure, by obtaining the posture information of the rolling camera, the inner cone image and the pixel positions of multiple target LED lamp beads within the inner cone range can be determined. Then, according to the pixel positions of each target LED lamp bead and the rolling parameters of the rolling camera, the scanning timing of each target LED lamp bead scanned by the rolling camera can be obtained. The scanning timing is related to the position of each target LED lamp bead. The target LED lamp bead closer to the bottom starts to be scanned later by the rolling camera. Then, the continuous display time of a single LED lamp bead can be combined to obtain the line-by-line scanning time of the rolling camera. The display timing of each target LED lamp bead adapted to the scanning characteristics, and then the color display of each target LED lamp bead is controlled according to the display timing, which is equivalent to matching the display of the inner cone image on the screen with the working timing of the progressive scanning exposure of the rolling camera. Therefore, the rolling camera can be used in virtual shooting scenes with higher frame rate image refresh requirements (that is, lower continuous display time of LED lamp beads) without being troubled by the rolling time (that is, rolling time), and greatly improves the display acquisition ratio of the LED lamp beads in the LED screen, that is, improves the utilization rate of the LED screen.

[0017] Further features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the disclosure and, together with the description, serve to explain the principles of the disclosure.

[0019] Figure 1 A structural diagram of a virtual shooting system according to an embodiment of the present disclosure is shown.

[0020] Figure 2 Schematic diagram showing the relationship between the pixel start exposure time and the pixel position within the viewing cone of a rolling camera.

[0021] Figure 3a A schematic diagram showing scanning results when the dynamic phase scheme of the embodiment of the present disclosure is not adopted.

[0022] Figure 3b A schematic diagram showing scanning results when the dynamic phase scheme according to an embodiment of the present disclosure is adopted.

[0023] Figure 4 A flow chart of a display control method for an LED screen according to an embodiment of the present disclosure is shown.

[0024] Figure 5 A schematic diagram illustrating a method for determining a target LED lamp bead according to an embodiment of the present disclosure is shown.

[0025] Figure 6aA schematic diagram of a rolling shutter parameter calibration system according to an embodiment of the present disclosure is shown.

[0026] Figure 6b A schematic diagram illustrating a rolling shutter duration analysis process according to an embodiment of the present disclosure is shown.

[0027] Figure 7 A schematic diagram showing a camera scanning situation when the lamp beads display simultaneously according to the related art.

[0028] Figure 8 A schematic diagram illustrating a camera scanning situation when the lamp beads are controlled to display in sequence based on a display timing according to an embodiment of the present disclosure.

[0029] Figure 9 A block diagram of another virtual shooting system proposed in an embodiment of the present disclosure is shown.

[0030] Figure 10 A block diagram of a display control device for an LED screen according to an embodiment of the present disclosure is shown.

[0031] Figure 11 A block diagram of an electronic device 1900 according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0032] Various exemplary embodiments, features, and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise indicated.

[0033] As used herein, the terms "comprises," "comprising," "having," or variations thereof are open ended and include one or more stated features, integers, elements, steps, parts, or functions, but do not preclude the presence or addition of one or more other features, integers, elements, steps, parts, functions, or groups thereof.

[0034] When an element is referred to as being "connected," "coupled," "responsive" or variations thereof to another element, it can be directly connected, coupled or responsive to the other element or intervening elements may be present.

[0035] Although the terms first, second, third, etc. may be used herein to describe various elements / operations, these elements / operations should not be limited by these terms. These terms are only used to distinguish one element / operation from another element / operation. Therefore, without departing from the teachings of the present invention, the first element / operation in some embodiments may be referred to as the second element / operation in other embodiments.

[0036] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0037] In addition, numerous specific details are provided in the following detailed description to better illustrate the present disclosure. Those skilled in the art will appreciate that the present disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art are not described in detail in order to highlight the main points of the present disclosure.

[0038] Figure 1 FIG. 1 shows a structural diagram of a virtual shooting system according to an embodiment of the present disclosure; FIG. Figure 1 As shown, the virtual shooting system may include: a main control device 101, a virtual rendering device 102, a broadcast control device 103, an LED screen 104, a shooting device 105 and a motion capture device 106; wherein, the number of each type of equipment can be flexibly configured according to actual needs and is not limited to this; in addition, in actual applications, other devices may be configured in the virtual shooting system as needed, such as mobile terminals or network devices, etc.; one or more devices in the virtual shooting system may also be integrated into one device as needed, for example, the main control device 101, the virtual rendering device 102, and the broadcast control device 103 may be integrated into one device.

[0039] Among them, the main control device 101 can be connected to the virtual rendering device 102, and the motion capture device 106 can be connected to the main control device 101. The specific connection method can be selected according to actual needs and device compatibility. For example, a wired or wireless connection can be made through a local area network or the Internet, and a network transmission protocol can be used for communication. The motion capture device 106 can capture the posture information of the shooting device 105 and send it to the main control device 101. The main control device 101 can send a control instruction to the virtual rendering device 102 connected thereto based on the posture information of the shooting device 105 to control the virtual rendering device 102 to render the picture to be displayed on the LED screen 104; alternatively, the motion capture device 106 can also be connected to the virtual rendering device 102 to directly send the captured posture information to the virtual rendering device, so that the virtual rendering device 102 can render the picture based on the posture information of the shooting device 105.

[0040] Among them, the virtual rendering device 102 can be deployed with a rendering engine, such as a UE (Unreal Engine) engine, and a virtual shooting environment (including a virtual camera, a screen model, etc.) consistent with the real shooting environment can be constructed in the rendering engine, wherein the position and posture of the camera intrinsic parameters of the virtual camera are consistent with the camera intrinsic parameters and posture of the shooting device 105, and the positional relationship between the virtual camera and the screen model is consistent with the positional relationship between the shooting device 105 and the LED screen 104; the screen model can be understood as a screen model that restores the real LED screen 104 in a 1:1 manner, and the shape, size, number and layout of the screen model, the number and layout of the lamp beads on each box, etc. are all consistent with the real LED screen 104.

[0041] In practical applications, those skilled in the art may calibrate the intrinsic camera parameters of the camera device 105 using camera calibration techniques known in the art, and may also use model building techniques known in the art to build a screen model of the real LED screen 104, which is not limited in the present embodiment. Furthermore, spatial calibration techniques known in the art may be used to calibrate the positional relationship between the virtual camera and the screen model to be consistent with the positional relationship between the real camera device 105 and the real LED screen 104.

[0042] The virtual rendering device 102 can be connected to the broadcast control device 103, and the broadcast control device 103 is connected to the LED screen 104. The specific connection method can be selected according to actual needs and device compatibility. The broadcast control device 103 can be used for data transmission and decoding. For example, the broadcast control device 103 can receive the rendering image from the virtual rendering device 102 and decode it into a format suitable for display on the LED screen 104. It can also send control instructions to the LED screen 104 to control and manage the LED screen 104 connected to it, such as overall control and scheduling of the LED screen 104, including brightness adjustment, color correction, grayscale control, etc. It can also be used to partition the LED screen 104, dividing the LED screen 104 into multiple independent areas, each of which can display different content. It can also control the display color and display duration of any LED lamp beads in the LED screen.

[0043] The LED screen 104 can be a curved screen or a flat screen, and the type, quantity, size, resolution, etc. of the LED screen 104 in the virtual shooting system can be customized according to actual needs, and there is no limitation on this. It can be seen that an LED screen is generally composed of multiple LED boxes, each of which contains LED lamp beads arranged at equal intervals. Each LED lamp bead can be used for color display. By controlling the color and display duration of each LED lamp bead, it is possible to control the corresponding image displayed by the LED screen and the display duration of each frame.

[0044] Among them, the main control device 101 can also be connected to the shooting device 105; the specific connection method can be selected according to actual needs and the compatibility of the device. The shooting device 105 can transmit the shot video to the main control device 101 to display the shooting results to the user in the main control device 101. The embodiment of the present disclosure does not limit the type of shooting device 105; as an example, according to the different lenses used, the shooting device 105 can be a telephoto camera, a wide-angle camera, etc.; as another example, according to the different shutters used, the shooting device 105 can be a rolling camera, a global camera, etc.; wherein, the rolling camera is equipped with a rolling shutter, and the rolling shutter scans the exposed image data line by line in a sequential manner; the global camera is equipped with a global shutter, and the global shutter allows all pixels on the image to be exposed at the same time.

[0045] During the virtual shooting process, the main control device 101 in the virtual shooting system controls the virtual rendering device 102 to render the picture, that is, it can drive the virtual camera in the rendering engine to render and generate a virtual picture and project the virtual picture into a three-dimensional projection onto the screen model, and then map it to the LED screen 104, that is, the rendered picture is projected onto the LED screen 104 for display through the broadcast control device 103 (that is, the picture is put on the screen). At the same time, the actor can perform in front of the LED screen 104, and the main control device 101 can control the shooting device 105 (such as a rolling camera) to shoot. The shot video uses the actor as the foreground and the picture on the LED screen as the background, thereby completing the virtual shooting.

[0046] In actual applications, a synchronization signal generator can be configured in the virtual shooting system to achieve signal synchronization between the shooting device 105 and the broadcast control device 103, that is, to synchronize the picture display of the LED screen with the video shooting of the shooting device 105, so that when the broadcast control device 103 controls the LED screen 104 to display the picture, the shooting device 105 can capture the complete picture.

[0047] Based on the above virtual shooting system, when the shooting device 105 adopts a rolling camera, as shown in FIG. Figure 2As shown, a rolling camera shoots an LED screen. Due to the rolling characteristics of the camera, on the inner viewing cone of the rolling camera, the lower the vertical position (that is, the lower the camera screen row coordinates), the later the camera starts exposing each row of pixels, and the exposure is the latest on the last row of pixels (that is, the pixels corresponding to the maximum number of rows H on the screen). The starting exposure time of the last row of pixels, T_shutter_start(last_row), is equal to the sum of the starting exposure time of the top row of pixels, T_shutter_start(first_row), and a rolling duration, T_roll. That is, T_shutter_start(last_row) = T_shutter_start(first_row) + T_roll. The difference between the starting exposure time of the top and last rows of pixels is a rolling time (T_roll, i.e., the rolling duration). In this case, for a rolling shutter camera to capture effectively, the duration of the LED's continuous display, or the duration of a single frame (T_slice_led), must satisfy the following requirement: T_slice_led ≥ T_roll + t_exp, where t_exp is the shutter duration, the time interval between the camera's shutter opening and closing. This controls how long light remains on the image sensor, affecting image brightness and dynamic performance. Assuming T_roll is 10ms and t_exp is 5ms, then T_slice_led must be above 15ms. This is unsuitable for virtual shooting scenarios requiring higher frame rates (i.e., high screen refresh rates). In this case, other solutions, such as a global camera, may be considered.

[0048] Take a single-camera green-screen shooting as an example (a typical scene in virtual shooting. In order to take into account the flexibility of later production, it is necessary to shoot both the picture with background and the picture with green background. In this way, the LED screen displays the content and the green screen in an alternating manner. After the shooting is completed, the entire video is divided into two parts: content background and green screen background by odd-even frame splitting). Assuming that the acquisition frame rate of the rolling camera is 100fps (equivalent to requiring the screen refresh rate to be 100fps), then the display frame rates of the green screen and the content are both 50fps. Assuming the shutter time is 5ms and the rolling parameter is 10ms, if the display control method proposed in the embodiment of the present disclosure is not adopted, each frame of the picture requires the LED lamp beads to display for 15ms, but the acquisition frame rate of 100fps requires that each frame of content lasts for a maximum of 10ms.

[0049] To address the aforementioned issues and eliminate the need for rolling camera shutter duration, enabling the capture of more high-refresh LED screens, the present disclosure proposes a display control method for rolling camera capture of LED screens. This method overcomes the impact of rolling camera shutter duration, significantly improving the display acquisition ratio of LED screens and thus meeting the advanced capture requirements of various virtual capture scenarios. The core of the method is to adjust the display timing of each LED on the LED screen to the exposure time of the rolling camera. This can be described as a dynamic adjustment scheme for the LED lamp phase, referred to as a dynamic phase scheme. Using the dynamic phase scheme proposed in the present disclosure, in the single-camera green-interrupted capture example above, each frame can be displayed by only a single LED lamp for 5ms. Considering some redundant time, a single LED lamp can be displayed for 8ms, achieving a display frame rate of 125fps, enabling normal capture using the rolling camera. In other words, using the display control method of the present disclosure, virtual capture requires only real-time calculation of the display timing of each row of target LED lamps. Controlling the color display of each target LED lamp according to this display timing allows the use of a rolling camera to achieve virtual capture at high screen refresh rates.

[0050] For example, Figure 3a A schematic diagram showing the scanning results when the dynamic phase scheme of the embodiment of the present disclosure is not adopted, Figure 3b A schematic diagram showing the scanning results when the dynamic phase scheme of the embodiment of the present disclosure is used is shown, wherein the gray area represents the display time of the lamp bead being lit, the small horizontal line represents the time when the lamp bead is scanned by the rolling shutter camera, the time occupied by a row of small horizontal lines represents the shutter time (i.e., 5ms), and the red vertical line represents the lamp bead that cannot be normally photographed by the rolling shutter camera. Figure 3a As shown in the figure, when the dynamic phase scheme is not adopted, all LED lamp beads in the inner viewing cone of the rolling camera light up at the same time and display continuously for 10ms. The LED lamp beads closer to the bottom of the viewing cone are scanned later by the rolling camera. Because the delay of the rolling camera's line-by-line scanning (that is, it takes 14ms to complete the scan) is greater than the 10ms continuous display time of the LED lamp beads, the LED lamp beads near the bottom of the viewing cone are extinguished before the rolling camera completes the scan, indicating that the current shooting of the rolling camera cannot meet the 10ms continuous display time of the LED lamp beads. Figure 3b As shown in the figure, when the dynamic phase solution is adopted, the scanning time of each LED lamp bead is within the LED lamp bead display time. In this way, by arranging different display times of different LED lamp beads, it is possible to use a rolling camera to achieve 5ms shutter shooting under the condition of 8ms continuous display time of the LED lamp bead.

[0051] The following Figures 4 to 9 The display control method of the LED screen (i.e., the dynamic phase solution) proposed in the embodiment of the present disclosure is introduced in detail.

[0052] Figure 4 The flowchart of the display control method of the LED screen according to one embodiment of the present disclosure is shown. The method is applied to a virtual shooting scene of an LED screen using a rolling shutter camera. In the virtual shooting scene, the method can be executed by the main control device 101 in the above-mentioned virtual shooting system, or by other terminal devices or servers, which is not limited by the embodiment of the present disclosure. Figure 4 As shown, the method includes: steps S11 to S15.

[0053] In step S11 , during the virtual shooting process, the position and posture information of the rolling camera is obtained.

[0054] As described above, the virtual shooting system may include a motion capture device, which may be a motion capture system such as OptiTrack. The motion capture device may be used to collect the posture information of the rolling camera (i.e., the position and posture of the rolling camera). Of course, other known posture capture technologies in the art may also be used, such as wheel odometers and inertial measurement units, etc., which are not limited to this embodiment of the present disclosure.

[0055] In step S12, based on the posture information of the rolling camera and the screen model of the LED screen, the inner cone image to be displayed on the LED screen and the pixel position information of multiple target LED lamp beads in the LED screen within the inner cone range of the rolling camera are determined, wherein the pixel position information includes the pixel position of the target LED lamp beads projected onto the imaging plane of the rolling camera, and the inner cone image to be displayed by the multiple target LED lamp beads.

[0056] As described above, a virtual shooting environment consistent with a real shooting environment can be constructed in the Unreal rendering engine. The virtual shooting environment includes a virtual camera and a screen model. The camera internal parameters (such as lens, focal length, etc.) of the virtual camera and the camera pose of the rolling camera are consistent. The positional relationship between the virtual camera and the screen model is consistent with the positional relationship between the rolling camera and the LED screen. Therefore, the above-mentioned determination of the inner cone image within the inner cone range of the rolling camera to be displayed on the LED screen and the pixel position information of multiple target LED lamp beads within the inner cone range of the rolling camera on the LED screen based on the pose information of the rolling camera and the screen model of the LED screen includes:

[0057] According to the position information of the rolling shutter camera, the relative position of the virtual camera and the screen model in the virtual shooting environment and the inner cone image to be displayed on the LED screen are determined;

[0058] Determine an inner viewing cone area of ​​the screen model within the inner viewing cone range of the virtual camera based on the relative posture, and use multiple LED lamp beads within the inner viewing cone area as target LED lamp beads;

[0059] Based on the spatial position of each target LED lamp bead in the screen model, each target LED lamp bead in the screen model is projected onto the imaging plane of the virtual camera to obtain the pixel position information of multiple target LED lamp beads within the inner viewing cone of the rolling camera.

[0060] It should be understood that the posture information of the rolling camera is known, that is, the posture of the virtual camera in the virtual shooting environment is known, and the position of the screen model in the virtual shooting environment remains unchanged. Therefore, the relative posture of the virtual camera and the screen model is also known. At the same time, the picture can be rendered based on the inner cone range of the virtual camera under the relative posture, that is, the inner cone picture to be displayed on the LED screen is obtained. At the same time, the coordinates of the inner cone area of ​​the screen model that is within the inner cone range of the virtual camera (that is, the regional position of the inner cone of the rolling camera on the LED screen) can also be obtained. Then, the multiple LED lamp beads in the inner cone area of ​​the screen model can be used as target LED lamp beads, that is, the multiple target LED lamp beads on the LED screen that are within the inner cone range of the rolling camera for displaying the inner cone picture are obtained.

[0061] Projecting each target LED in the screen model onto the virtual camera's imaging plane is equivalent to using a rolling camera to project the target LEDs in three-dimensional space onto the rolling camera's imaging plane, thereby obtaining the pixel positions of each target LED on the imaging plane. For example, based on the virtual camera's position and internal parameters, the imaging mapping relationship from three-dimensional space to a two-dimensional plane can be determined. Based on this mapping relationship, the spatial positions of each target LED can be converted into pixel positions. If the rolling camera's resolution is lower than the screen's resolution, multiple LEDs may be projected onto the same pixel.

[0062] For example, Figure 5 A schematic diagram showing a method for determining a target LED lamp bead is shown as follows: Figure 5As shown, the LED screen can be spatially calibrated in advance to obtain the position information of the LED screen (that is, the position of the screen model in the virtual shooting environment). During the shooting process, the spatial tracking device (that is, a motion capture device) can be used to track the position information of the rolling camera (that is, tracking data) and send it to the computing device (equivalent to the main control device and the rendering device). The computing device can determine the relative position of the virtual camera and the screen model in the virtual shooting environment based on the position information of the rolling camera and the position information of the LED screen, and then determine the coordinates of the inner cone area of ​​the LED screen within the inner cone range of the rolling camera (that is, the cone image area coordinates). Then, the coordinates of the box within the inner cone area of ​​the LED screen can be located (that is, box positioning), and the spatial coordinates of each target LED lamp bead on each box within the inner cone area can be obtained. Then, based on the mapping relationship between the LED screen projection and the imaging plane of the rolling camera, the pixel coordinates of each target LED lamp bead can be obtained through coordinate mapping. At the same time, the computing device can also perform cone rendering (that is, render the inner cone image) based on the position information of the rolling camera.

[0063] In step S13, the scanning timing of each target LED lamp bead is determined according to the starting shooting time of the rolling camera, the pixel position information of multiple target LED lamp beads, and the rolling parameters of the rolling camera; wherein the rolling parameters represent the rolling duration of the rolling camera, and the scanning timing includes the starting time when each target LED lamp bead is scanned by the rolling camera.

[0064] The rolling camera's starting shooting time can be understood as the time when the rolling camera begins shooting, and it should be understood that the starting shooting time is known. The rolling shutter duration represents the time difference between the start exposure time of the top row of pixels and the start exposure time of the last row of pixels of the rolling camera, or in other words, the time difference between the start exposure of the top row of pixels and the start exposure of the last row of pixels of the rolling camera. The rolling shutter parameters can be pre-calibrated, for example, using the high-speed moving object method (i.e., using the deformation caused by an object of known speed (such as a rotating disk or a fast-moving LED light) in the image to calculate the rolling shutter duration through geometric relationships), the synchronous flashing light source method (i.e., controlling the flashing frequency of the light source to be asynchronous with the rolling shutter's line scanning and observing the width of the light band in the image to calculate the rolling shutter duration), or the high-speed camera direct measurement method (i.e., using a high-speed camera with a higher frame rate to synchronously capture the rolling shutter's operating process, directly recording the start and end times of each line of exposure, and then calculating the rolling shutter duration).

[0065] In one possible implementation, the calibration process of the rolling shutter parameters of the rolling shutter camera includes:

[0066] In the process of controlling the LED screen to alternately display the first image frame and the second image frame and continuously adjusting the display phase of the LED lamp beads in the LED screen, the LED screen is photographed using a rolling camera at a preset acquisition frame rate to obtain a captured video; wherein the image content of the first image frame is different from that of the second image frame;

[0067] By analyzing the video screen of the captured video, a first moment at which the first mixed screen just changes to the screen content of the second image frame and a second moment at which the screen content of the second image frame just does not reveal the second mixed screen are obtained; wherein the first mixed screen includes a top area of ​​the video screen that is the screen content of the first image frame and the rest of the area that is the screen content of the second image frame, and the second mixed screen includes a bottom area of ​​the video screen that is the screen content of the first image frame and the rest of the area that is the screen content of the second image frame;

[0068] The rolling parameters of the rolling camera are determined according to the first moment, the second moment, the single-frame display duration of the second image frame, and the shutter duration of the rolling camera.

[0069] For example, Figure 6aIn the rolling shutter parameter calibration system shown in FIG, an LED screen alternately displays the contents of frame A (i.e., the first image frame) and frame B (i.e., the second image frame). The contents of frame A and frame B need to be significantly different. For example, frame A is a picture filled with diagonal lines, and frame B is a picture filled with monochrome. A synchronization signal generator is used for time synchronization between the rolling shutter camera and the screen controller (i.e., the broadcast control device). After the rolling shutter camera starts shooting, the screen controller can continuously adjust the display phase of the LED lamp beads in the LED screen (equivalent to continuously adjusting the start display time of the first image frame and the second image frame). For example, assuming that the preset acquisition frame rate of the rolling shutter camera is fixed at 50fps, and it is assumed that frame A and frame B are alternately displayed for 20ms respectively. After the rolling camera starts shooting, the acquisition frame rate of the rolling camera is kept unchanged. The screen controller can adjust the display phase of the LED lamp beads in the LED screen (that is, adjust the offset of the LED lamp bead lighting time) so that the start display time of the A frame and the B frame is continuously offset relative to the shutter time of the rolling camera (the offset can be advanced or delayed). During this process, the boundary between the A frame and the B frame in the video image of the captured video shot by the rolling camera is constantly moving, and the AB mixed area will appear at the bottom of the B frame or the AB mixed area at the bottom will disappear, and the AB mixed area will appear at the top of the B frame or the AB mixed area at the top will disappear. Therefore, by analyzing the video screen of the captured video, we can obtain the first moment when the first mixed screen just becomes the screen content of the second image frame, that is, the moment when the top AB mixing just disappears, and the second moment when the screen content of the second image frame just does not reveal the second mixed screen, that is, the moment when the bottom AB mixing area just does not appear (that is, the moment before the bottom AB mixing area just appears); in other words, assuming that in the process of continuously adjusting the display phase of the LED lamp beads, the content of the B frame screen in the video screen is continuously moving downward, then the moment when the lower boundary of the B frame screen just touches the bottom can be recorded as t1. As the display phase of the LED lamp beads is continuously adjusted, the content of the B frame screen can move downward at a uniform speed, then the moment when the upper boundary of the B frame screen just touches the top can be recorded as t2.

[0070] More vividly, Figure 6b As shown, fixing the acquisition frame rate of the rolling camera and adjusting the display phase of the LED lamp beads is equivalent to fixing the trapezoidal shadow in the figure and moving the position of the rectangle in the figure left and right. This can obtain the first moment t1 when the top AB mixing just disappears and the second moment t2 when the bottom AB mixing just does not appear. Combined with the single-frame display duration of the second image frame (i.e., frame duration t_pic) and the shutter duration of the rolling camera (i.e., t_exp), the rolling duration t_roll of the rolling camera can be calculated as: t_roll = t_pic - t_exp - |t2 - t1|.

[0071] Based on the rolling shutter parameters obtained by the calibration, in one possible implementation, step S13 determines the scanning sequence of each target LED lamp bead according to the start shooting time of the rolling shutter camera, the pixel position information of the multiple target LED lamp beads, and the rolling shutter parameters of the rolling shutter camera, which may include:

[0072] Step S131: For the i-th row of pixels on the imaging plane of the rolling camera, determine the i-th group of LEDs corresponding to the i-th row of pixels based on the pixel position information of each target LED lamp bead. The i-th group of LEDs includes multiple target LED lamp beads projected onto the i-th row of pixels, where 1≤i≤N, where N is the total pixel height of the imaging plane.

[0073] Step S132: When i is 1, the start time for the rolling camera to start scanning the first group of lamp beads corresponding to the first row of pixels is determined based on the start shooting time of the rolling camera and the rolling time and shutter time of the rolling camera;

[0074] Step S133, when i is greater than 1, determine the starting time when the rolling camera starts scanning the i-th group of lamp beads corresponding to the i-th row of pixels based on the pixel height of the i-th row of pixels, the rolling shutter parameters, the total pixel height of the imaging plane, and the starting time when the rolling camera starts scanning the first group of lamp beads corresponding to the first row of pixels.

[0075] The i-th row of pixels on the imaging plane can be understood as the i-th row of pixels in the rolling camera's sensor array. The total pixel height of the imaging plane represents the total number of rows of sensor units in the rolling camera's sensor array. One sensor unit can represent one pixel. Knowing the pixel position of each target LED lamp bead is equivalent to knowing which row of pixels (i.e., sensor unit) each target LED lamp bead is projected onto on the imaging plane. Therefore, the correspondence between each target LED lamp bead and each pixel in the imaging plane is also known, and any group of lamp beads corresponding to any row of pixels can be obtained. This group of lamp beads can be located in different rows and columns on the screen. As long as they are all projected onto the same row of pixels on the imaging plane, they are the group of lamp beads corresponding to that row of pixels.

[0076] It should be understood that the acquisition frame rate of the rolling camera is fixed, and the starting shooting time is known. It is possible to deduce the starting time when the rolling camera starts scanning the first group of lamp beads corresponding to the first row of pixels during the display of the cone image in any frame on the LED screen. For example, assuming that the starting shooting time of the rolling camera is 0ms, the starting time when the rolling camera starts scanning the first group of lamp beads in the first shooting cycle during the display of the cone image in the first frame can be 0ms. If the rolling time of the rolling camera is 10ms and the shutter time is 5ms, then the shooting cycle of the rolling camera is 0ms. The rolling shutter duration + shutter duration is 15ms (which means that the rolling shutter camera takes 15ms to capture one frame). During the second shooting cycle of the second frame's intra-cone image display, the rolling shutter camera starts scanning the first group of lamp beads at 15ms, that is, it takes a shot every 15ms. Similarly, based on the shooting cycle corresponding to the frame sequence of the intra-cone image in the current frame, the rolling shutter camera starts scanning the first group of lamp beads corresponding to the first row of pixels in the shooting cycle of the intra-cone image in the current frame. In other words, the time difference between the start times of scanning the same group of lamp beads between two adjacent frames is at least the rolling shutter duration + shutter duration to ensure a complete shooting process.

[0077] As mentioned above, the lamp beads of the LED screen box are evenly arranged at equal intervals. Assuming that all target LED lamp beads of the LED screen display the inner cone image at time 0, and the first group of lamp beads are also scanned by the rolling shutter camera at time 0, then after the rolling shutter time t_roll of the rolling shutter camera is known, the starting time t_shutter_start_i when the i-th group of lamp beads corresponding to the i-th row of pixels are scanned can be expressed as: t_shutter_start_i=[t_roll / (N-1)]×(i-1). For example, the rolling shutter time is 3ms, there are 3 rows of pixels, the starting time of the first group of lamp beads being scanned is 0, the starting time of the second group of lamp beads in the second row of pixels being scanned is [3 / (3-1)]×(2-1)=1.5ms, and the starting time of the third group of lamp beads in the third row of pixels being scanned is [3 / (3-1)]×(3-1)=3m s; if the starting time when the rolling camera starts scanning the first group of lamp beads is expressed as t_shutter_start_0 (that is, not 0), then the starting time when the i-th group of lamp beads is scanned by the rolling camera can be calculated as: t_shutter_start_i=t_shutter_start_0+[t_roll / (N-1)]×(i-1), where "i" represents the pixel height of the i-th row of pixels, N is the total pixel height of the imaging plane, t_roll is the rolling parameter (that is, the rolling duration), and t_shutter_start_0 represents the starting time when the first group of lamp beads corresponding to the first row of pixels are scanned; wherein, the starting time of the i-th group of lamp beads can be synchronized with the clock signal of the LED screen, that is, it can correspond to the screen timestamp of the LED screen, so that the LED screen display and camera shooting can be performed under the same clock.

[0078] In step S14, the display timing of the inner cone image displayed by multiple target LED lamp beads is determined based on the scanning timing of each target LED lamp bead, the shutter duration of the rolling camera, and the continuous display duration of a single LED lamp bead, wherein the display timing represents the start display time and the end display time of each target LED lamp bead.

[0079] In practical applications, if the start time of the rolling shutter camera to scan each target LED lamp bead is known, the end time of the scanning of each target LED lamp bead can be expressed as the start time of each target LED lamp bead plus the shutter duration t_exp of the rolling shutter camera. For example, based on the start time of the scanning of the i-th group of lamp beads being t_shutter_start_i, the end time of the scanning of the i-th group of lamp beads is t_shutter_start_i+t_exp, which means that the scanning time of the i-th group of lamp beads is the time range from the start time t_shutter_start_i to the end time t_shutter_start_i+t_exp. Therefore, the continuous display time t_led_a_frame of the content of the cone image in each frame on each row of lamp beads only needs to satisfy: t_led_a_frame≥t_exp; and if Figure 7 As shown, if the display time of all target LED lamp beads is the same, that is, the time t_led_start when N groups of lamp beads (i.e., lamp bead 1, lamp bead 2, ..., lamp bead i, ... lamp bead N) start to display frame A (inner cone picture) is the same, in order for the rolling shutter camera to capture the complete inner cone picture, the display duration t_led_a_frame of each frame must at least satisfy: t_led_a_frame ≥ t_exp + t_roll.

[0080] In actual applications, if t_led_a_frame = t_exp, the start display time of the i-th group of lamp beads can be the start time t_shutter_start_i when the rolling shutter camera starts scanning, and the end display time can be the start time t_shutter_start_i + the continuous display time t_led_a_frame equal to t_shutter_start_i + t_exp. For example, taking the rolling shutter time as 10ms and the shutter time as 5ms, the starting scanning time is time 0. If it is calculated that the start time of the rolling shutter camera starting to scan the first group of lamp beads in multiple shooting cycles is 0ms, 15ms, 30ms, 45ms..., and the start time of scanning the second group of lamp beads is 0.1ms, 15.1ms, 30.1ms, 45.1ms..., if t_led_a_frame = t_exp = 5ms, then the display timing of the first group of lamp beads can include: 0ms-5ms s, 15ms-20ms, 30ms-35ms, 45ms-50ms…, the display timing of the second group of lamp beads may include: 0.1ms-5.1ms, 15.1ms-20.1ms, 30.1ms-35.1ms, 45.1ms-50.1ms…, and so on for other groups of lamp beads, that is, the display timing of each target LED lamp bead within the inner visual cone is obtained, and then each group of lamp beads can be controlled in turn to display the contents of the first frame, the second frame, the third frame, the fourth frame, etc. according to the display timing.

[0081] If t_led_a_frame-t_exp>0, due to the presence of error factors such as camera pose tracking error and lamp bead positioning error, the redundant time difference between t_led_a_frame and t_exp can be used to enhance the fault tolerance of the lamp bead scanning time, that is, the display time of each group of lamp beads can be greater than the scanning time of each group of lamp beads. For example, the start display time of each group of lamp beads can be advanced relative to the start time of each group of lamp beads being scanned. The advance time can be set as needed, as long as the end display time can be later than the end time of each group of lamp beads being scanned. Therefore, in a possible implementation method, the above step S14 determines the display timing of the multiple target LED lamp beads when displaying the inner cone image according to the scanning timing of each target LED lamp bead, the shutter duration of the rolling camera, and the continuous display duration of a single LED lamp bead, including:

[0082] Step S141, taking the difference between the shutter duration of the rolling camera and the continuous display duration of a single LED lamp bead as the redundant duration;

[0083] Step S142: For the i-th group of lamp beads corresponding to the i-th row of pixels, determine the display start time of the i-th group of lamp beads based on the start time of the rolling camera to start scanning the i-th group of lamp beads and the redundant time length, wherein the display start time of the i-th group of lamp beads is earlier than the start time of the rolling camera to start scanning;

[0084] Step S143: The sum of the start display time of the i-th group of lamp beads and the continuous display time of a single LED lamp bead is used as the end display time of the i-th group of lamp beads.

[0085] In step S141, the redundancy time can be expressed as t_redudancy = t_led_a_frame - t_exp. Then, in step S142, the start display time of the i-th group of lamp beads corresponding to the i-th row of pixels can be optimized as: t_led_start_i_better = t_shutter_start_i - t_redudancy / 2 =

[0086] t_shutter_start_0+[t_roll / (N-1)]×(i-1)-t_redudancy / 2 is equivalent to advancing the start display time of the i-th group of lamp beads relative to the start time of scanning of the i-th group of lamp beads by 0.5 times the redundant time. Of course, it can also be advanced to 0.4 or 0.6 times the redundant time, which is not limited in the embodiment of the present disclosure; then in step S143, the end display time of the i-th group of lamp beads corresponding to the i-th row of pixels can be expressed as: t_led_end_i_better=t_led_ start_i_better+t_led_a_frame, that is, when the i-th group of lamp beads displays the inner cone picture, it starts displaying from t_led_start_i_better to t_led_end_i_better. The rolling shutter camera starts scanning the i-th group of lamp beads from t_shutter_start_i to t_shutter_start_i+t_exp. After the i-th group of lamp beads finishes displaying the color of the current frame, it can continue to display the color required to display the inner cone picture of the next frame. For example, taking the rolling shutter time as 10ms and the shutter time t_exp as 5ms, the starting scanning time is time 0. If it is calculated that the starting time of the rolling shutter camera to start scanning the first group of lamp beads in multiple shooting cycles is 0ms, 15ms, 30ms, 45ms..., and the starting time of scanning the second group of lamp beads is 0.1ms, 15.1ms, 30.1ms, 45.1ms..., if the continuous display time of a single LED lamp bead t_led_a_frame = 10ms, it means that the redundant time is t_redudancy = t_led_a_frame-t_exp = 5ms, t_redudancy / 2 = 2.5ms s, then if the LED screen also starts displaying from time 0, the display timing of the first group of lamp beads can include: 0ms-10ms, 12.5ms-22.5ms, 27.5ms-37.5ms, 42.5ms-52.5ms..., the display timing of the second group of lamp beads can include: 0.1ms-10.1ms, 12.6ms-22.6ms, 27.6ms-37.6ms, 42.6ms-52.6ms..., and so on for other groups of lamp beads, that is, the display timing of each target LED lamp bead within the inner visual cone is obtained, and then each group of lamp beads can be controlled in turn to display the contents of the first frame, the second frame, the third frame, the fourth frame, etc. according to the display timing.

[0087] For example, Figure 8 The display timing of each group of lamp beads determined by the method of the embodiment of the present disclosure is shown. Figure 7In terms of the display start time of each group of lamp beads, it is different and related to the position of each group of lamp beads. The lower the lamp beads are, the later they start to display. In this way, the display time of each group of lamp beads can be matched with the line-by-line scanning timing of the rolling camera. In other words, the display follows the line-by-line scanning timing of the rolling camera, so that the rolling camera is not troubled by the length of the rolling shutter. It can also improve the display acquisition ratio of the lamp beads in the LED screen, which is equivalent to improving the utilization rate of the LED lamp beads.

[0088] In step S15, the multiple target LED lamp beads in the LED screen are controlled to display colors according to the display timing when the multiple target LED lamp beads display the inner cone image.

[0089] In practical applications, after calculating the display timing (also known as display phase) of multiple target LED lamp beads when displaying the inner visual cone image, the display timing can be sent to the broadcast control device (or screen controller) in the virtual shooting system, so that the broadcast control device can control the color display of each target LED lamp bead on the LED screen according to the display timing, that is, dynamically light up each target LED lamp bead, or control each target LED lamp bead to display the desired display content within its respective display time. It should be understood that the embodiments of the present disclosure do not limit the control method of the LED lamp beads, as long as the display is performed according to the specified display timing, the embodiments of the present disclosure are not limited to this.

[0090] It should be understood that if the position of the rolling camera does not change during the shooting process, the target LED lamp beads within the inner viewing cone of the rolling camera are also fixed. In this case, the target LED lamp beads can be controlled to display colors according to the display timing determined above. If the position of the rolling camera changes during the shooting process, the multiple target LED lamp beads within the inner viewing cone of the rolling camera in the current position can be re-determined according to the above steps S11 to S14, and the display timing of each target LED lamp bead can be determined. The newly determined multiple target LED lamp beads can then be controlled to display colors according to the newly determined display timing of each target LED lamp bead.

[0091] It should be understood that the inner viewing cone of a rolling camera may be smaller than the LED screen. Each target LED within the inner viewing cone can be controlled according to a display sequence, while other LEDs outside the inner viewing cone can simultaneously display a fixed outer viewing cone image. This outer viewing cone image is not captured by the rolling camera and primarily serves as illumination. Therefore, there are no restrictions on the content and display method of this outer viewing cone image. Furthermore, while controlling the color display of multiple target LEDs within the LED screen, the rolling camera can be simultaneously controlled to capture the LED screen and the actors and scenery in front of it, thereby achieving virtual photography that combines both the real and the virtual.

[0092] In practical applications, virtual shooting may require shooting with multiple backgrounds, for example, a single-camera green-screen shooting requirement, which is a typical scene in virtual shooting. In order to take into account the flexibility of later production, it is necessary to shoot both the picture with content background and the picture with green screen background. In this way, the LED screen needs to alternately display the virtual content background picture and the green screen background picture, so that after the shooting is completed, the entire video can be divided into two videos, the content background and the green screen background, by odd-even frame splitting. Thus, the inner cone picture to be displayed can be any one of the at least two pictures to be displayed alternately (for example, it can be a virtual content background picture). Therefore, the display timing of any one picture can be determined first, and then the other pictures can be displayed in sequence based on the continuous display time of a single LED lamp bead, that is, the display timing of various other pictures can be obtained. Therefore, the display timing of the inner cone picture displayed by multiple target LED lamp beads is controlled to display colors of the multiple target LED lamp beads in the LED screen, including:

[0093] According to the display timing of multiple target LED lamp beads displaying any one picture, determine the target display timing of multiple target LED lamp beads alternately displaying at least two pictures; according to the target display timing of multiple target LED lamp beads alternately displaying at least two pictures, control each target LED lamp bead in the LED screen to display color.

[0094] For example, assuming that the virtual shooting requires the LED screen to display two pictures alternately, which are divided into a first picture (such as a background picture with content) and a second picture (such as a green screen background picture), the display timing of any picture can be calculated first. For example, if the display timing of the first group of lamp beads to display the first picture is calculated to include: 0ms-10ms, 12.5ms-22.5ms, 27.5ms-37.5ms, 42.5ms-52.5ms..., and the display timing of the second group of lamp beads includes: 0.1ms-10.1ms, 12.6ms-22.6ms, 27.6ms-37.6ms, 42.6ms-52.6ms..., etc., then based on the display order of the two pictures (such as the first picture is displayed first and the second picture is displayed later), the target display timing of each group of lamp beads to alternately display the two pictures can be determined, for example, the first picture and the second picture can be displayed alternately according to the above display timing. For example, the first group of lamp beads displays the first picture at 0-10ms, the second picture at 12.5ms-22.5ms, the first picture at 27.5ms-37.5ms, and the second picture at 42.5ms-52.5ms, and so on. The second group of lamp beads displays the first picture at 0.1ms-10.1ms, the second picture at 12.6ms-22.6ms, the first picture at 27.6ms-37.6ms, and the second picture at 42.6ms-52.6ms, and so on. Of course, the second picture can also be displayed first and then the first picture. This depends on user needs and is not limited to this embodiment of the present disclosure. It should be understood that if the position of the rolling camera is fixed, the display control of each target LED lamp bead can be performed according to the initially determined target display timing. If the position of the rolling camera changes, the multiple target LED lamp beads within the inner field of view of the rolling camera in the current position can be re-determined, and the target display timing of each target LED lamp bead alternatingly displaying the first image and the second image can be re-determined, and the multiple target LED lamp beads newly determined can be controlled to display colors according to the newly determined target display timing. The embodiments of the present disclosure do not limit this.

[0095] According to the display control method of the embodiment of the present disclosure, the inner cone image and the pixel positions of multiple target LED lamp beads within the inner cone range can be determined by obtaining the posture information of the rolling shutter camera. Then, according to the pixel positions of each target LED lamp bead and the rolling shutter parameters of the rolling shutter camera, the scanning timing of each target LED lamp bead scanned by the rolling shutter camera can be obtained. The scanning timing is related to the position of each target LED lamp bead. The target LED lamp bead closer to the bottom starts to be scanned later by the rolling shutter camera. Then, the continuous display time of a single LED lamp bead can be combined to obtain the timing corresponding to the rolling shutter. The display timing of each target LED lamp bead adapted to the progressive scanning characteristics of the machine, and then each target LED lamp bead is controlled to display color according to the display timing, which is equivalent to matching the display of the inner cone image on the screen with the working timing of the progressive scanning exposure of the rolling camera. Therefore, the rolling camera can be used in virtual shooting scenes with higher frame rate image refresh requirements (that is, lower continuous display time of LED lamp beads) without being troubled by the rolling time (that is, rolling time), and greatly improves the display acquisition ratio of the LED lamp beads in the LED screen, that is, improves the utilization rate of the LED screen.

[0096] Based on the above steps S11 to S15 of the embodiment of the present disclosure, Figure 9 A block diagram of another virtual shooting system proposed in an embodiment of the present disclosure is shown as follows: Figure 9As shown, the system includes: a motion capture module for capturing the camera position during the shooting process, obtaining the camera pose information, and sending it to the virtual rendering cluster and the phase calculation module; a screen calibration module for calibrating the LED screen position, and sending it to the virtual rendering cluster and the phase calculation module; a camera calibration module for calibrating other camera information (such as camera internal parameters, rolling shutter time) and sending it to the virtual rendering cluster and the phase calculation module; wherein the screen calibration module and the camera calibration module can be calibrated offline; the virtual rendering cluster is used to render the screen image according to the camera information and screen information (such as the screen position of the LED screen), and the rendered image is sent to the LED rendering module through the playback control module; the playback control module is used to convert the rendered image into a format suitable for the LED screen display, and the LED rendering module is used to drive the LED screen to display the converted format Rendering picture (that is, realizing the picture on the screen); at the same time, the phase calculation module is used to calculate the time phase information of the physical lamp beads on the LED screen being scanned (that is, the target LED lamp beads and the display timing of the target LED lamp beads) according to the camera information and the screen information. The phase information is transmitted through the phase transmission module and received by the phase receiving module and implemented by the phase implementation module. Among them, the phase receiving module, the phase implementation module and the LED rendering module can be located on the same device, so that the phase information can be received and implemented in the LED rendering module, and the target LED lamp beads are dynamically lit in conjunction with the phase information; the rolling shutter camera synchronously shoots 100fps pictures at a high frame rate. If applied to the above-mentioned single-camera green-insertion shooting scene, the 100fps picture captured by the camera can also be split into odd and even parts to obtain background virtual shot video and green screen virtual shot video.

[0097] In practical applications, when using multiple rolling cameras for filming, the start shooting time of each rolling camera can be fine-tuned based on the display timing of the LED beads. Especially in scenarios where multiple background images are displayed alternately, by adjusting the start shooting time of each rolling camera, each rolling camera can be primarily responsible for capturing video of a certain background image, thereby optimizing the shooting experience. In this way, the LED screen displays the specified content at the specified time phase according to the determined display timing, and each rolling camera can simultaneously shoot according to its own determined start shooting time to complete the virtual shooting.

[0098] Traditional virtual shooting technology based on rolling shutter cameras is affected by the rolling shutter characteristics and cannot fully utilize the high refresh rate characteristics of LED screens, making rolling shutter cameras unusable in advanced application scenarios such as multi-camera shooting and multi-scene shooting. However, the display control method for the LED screen proposed in the embodiment of the present disclosure can estimate the exposure phase (i.e., the scanning timing) of the LED screen lamp beads in real time based on the high-precision dynamic compensation camera posture and screen model, and thus plan the display timing of each target LED lamp bead to match the scanning timing of the rolling shutter camera for display, thereby overcoming the influence of the rolling shutter time. Ideally, it can achieve the same high-speed acquisition capability as a global camera, so that the rolling shutter camera can also meet the use of advanced application scenarios such as multi-camera shooting and multi-scene shooting.

[0099] The display control method for LED screens proposed in the disclosed embodiments uses real-time spatial information from the camera and screen to calculate the position of each lamp bead in the camera image. Combining the rolling shutter time and shutter time, the exposure time of each lamp bead can be determined. Furthermore, by combining some positioning system errors and adding some time redundancy to arrange the lighting time of each lamp bead, the LED screen can be guaranteed to meet the requirements of high-frame-rate green-insertion shooting. It can dynamically and precisely arrange the display time of the lamp bead content based on the time when the lamp bead is scanned by the rolling shutter camera, achieving high-speed acquisition of LED screens by rolling shutter cameras, achieving single-machine green-insertion rolling shutter camera shooting, and realizing calibration of the rolling shutter duration of rolling shutter cameras using LED screens.

[0100] Figure 10 A block diagram of a display control device for an LED screen according to an embodiment of the present disclosure is shown. Figure 10 As shown, the device includes:

[0101] An acquisition module 101 is used to acquire the position information of the rolling camera during the virtual shooting process;

[0102] a first determining module 102 for determining, based on the position information of the rolling camera and the screen model of the LED screen, an inner viewing cone image to be displayed on the LED screen and pixel position information of a plurality of target LED lamp beads within the inner viewing cone of the rolling camera on the LED screen, wherein the pixel position information includes pixel positions of the target LED lamp beads projected onto the imaging plane of the rolling camera;

[0103] a second determining module 103 for determining a scanning sequence of each target LED lamp bead based on the start shooting time of the rolling camera, the pixel position information of the plurality of target LED lamp beads, and a rolling parameter of the rolling camera; wherein the rolling parameter represents the rolling duration of the rolling camera, and the scanning sequence includes the start time when each target LED lamp bead starts to be scanned by the rolling camera;

[0104] A third determining module 104 is configured to determine a display timing of the plurality of target LED lamp beads displaying the inner cone image based on a scanning timing of each target LED lamp bead, a shutter duration of the rolling camera, and a continuous display duration of a single LED lamp bead, wherein the display timing represents a start display time and an end display time of each target LED lamp bead;

[0105] The display control module 105 is used to control the multiple target LED lamp beads in the LED screen to display colors according to the display timing of the multiple target LED lamp beads displaying the inner cone image.

[0106] In one possible implementation, the determining of the scanning timing of each target LED lamp bead according to the starting shooting moment of the rolling camera, the pixel position information of the multiple target LED lamp beads, and the rolling parameters of the rolling camera includes: for the i-th row of pixels on the imaging plane of the rolling camera, determining the i-th group of lamp beads corresponding to the i-th row of pixels according to the pixel position information of the multiple target LED lamp beads, the i-th group of lamp beads including a row or multiple target LED lamp beads projected onto the i-th row of pixels, 1≤i≤N, N is The total pixel height of the imaging plane; when i is 1, the starting time for the rolling camera to start scanning the first group of lamp beads corresponding to the first row of pixels is determined according to the starting shooting time of the rolling camera and the rolling time and shutter time of the rolling camera; when i is greater than 1, the starting time for the rolling camera to start scanning the i-th group of lamp beads is determined according to the pixel height of the i-th row of pixels, the rolling parameters, the total pixel height of the imaging plane and the starting time when the rolling camera is to start scanning the first group of lamp beads corresponding to the first row of pixels.

[0107] In one possible implementation, the display timing of the multiple target LED lamp beads displaying the inner cone image is determined based on the scanning timing of each target LED lamp bead, the shutter duration of the rolling camera, and the continuous display duration of a single LED lamp bead, including: taking the difference between the shutter duration of the rolling camera and the continuous display duration of a single LED lamp bead as a redundant duration; for the i-th group of lamp beads corresponding to the i-th row of pixels, determining the start display time of the i-th group of lamp beads based on the start time of the rolling camera to start scanning the i-th group of lamp beads and the redundant duration, wherein the start display time of the i-th group of lamp beads is earlier than the start time when the rolling camera starts scanning; and taking the sum of the start display time of the i-th group of lamp beads and the continuous display duration of the single LED lamp bead as the end display time of the i-th group of lamp beads.

[0108] In a possible implementation, the rolling shutter duration represents the time difference between the start exposure time of the top row of pixels of the rolling shutter camera and the start exposure time of the last row of pixels, wherein the calibration process of the rolling shutter parameters of the rolling shutter camera includes: in the process of controlling the LED screen to alternately display the first image frame and the second image frame and continuously adjusting the display phase of the LED lamp beads in the LED screen, using the rolling shutter camera to shoot the LED screen at a preset acquisition frame rate to obtain a captured video; wherein the picture contents of the first image frame and the second image frame are different; by dividing the video picture of the captured video The method comprises: analyzing the first moment when the first mixed picture just changes to the picture content of the second image frame, and obtaining a second moment when the picture content of the second image frame just does not leak out of the second mixed picture; wherein the first mixed picture includes a top area of ​​the video picture as the picture content of the first image frame and other areas as the picture content of the second image frame, and the second mixed picture includes a bottom area of ​​the video picture as the picture content of the first image frame and other areas as the picture content of the second image frame; and determining the rolling parameters of the rolling camera based on the first moment, the second moment, the single-frame display duration of the second image frame, and the shutter duration of the rolling camera.

[0109] In a possible implementation, the method of determining, based on the posture information of the rolling camera and the screen model of the LED screen, an inner cone image to be displayed on the LED screen and pixel position information of multiple target LED lamp beads in the LED screen and within the inner cone of the rolling camera includes: determining, based on the posture information of the rolling camera, a relative posture of a virtual camera and the screen model in a virtual shooting environment and the inner cone image to be displayed on the LED screen, wherein the posture and internal parameters of the virtual camera and the rolling camera are the same; determining an inner cone area in the screen model and within the inner cone of the virtual camera based on the relative posture, and using multiple LED lamp beads in the inner cone area as target LED lamp beads; and projecting each target LED lamp bead in the screen model onto the imaging plane of the virtual camera based on the spatial position of each target LED lamp bead in the screen model to obtain pixel position information of the multiple target LED lamp beads in the inner cone of the rolling camera.

[0110] In one possible implementation, the inner cone picture to be displayed is any one of at least two pictures to be displayed alternately, wherein the display timing of displaying the inner cone picture according to the multiple target LED lamp beads controls the multiple target LED lamp beads in the LED screen to display colors, including: determining the target display timing of the multiple target LED lamp beads alternately displaying at least two pictures according to the display timing of displaying any one picture by the multiple target LED lamp beads; controlling the individual target LED lamp beads in the LED screen to display colors according to the target display timing of the multiple target LED lamp beads alternately displaying at least two pictures.

[0111] According to the display control device of the embodiment of the present disclosure, the inner cone image and the pixel positions of multiple target LED lamp beads within the inner cone range can be determined by obtaining the posture information of the rolling shutter camera. Then, according to the pixel positions of each target LED lamp bead and the rolling shutter parameters of the rolling shutter camera, the scanning timing of each target LED lamp bead scanned by the rolling shutter camera can be obtained. The scanning timing is related to the position of each target LED lamp bead. The target LED lamp bead closer to the bottom starts to be scanned later by the rolling shutter camera. Then, the continuous display time of a single LED lamp bead can be combined to obtain the timing corresponding to the rolling shutter. The display timing of each target LED lamp bead adapted to the progressive scanning characteristics of the machine, and then each target LED lamp bead is controlled to display color according to the display timing, which is equivalent to matching the display of the inner cone image on the screen with the working timing of the progressive scanning exposure of the rolling camera. Therefore, the rolling camera can be used in virtual shooting scenes with higher frame rate image refresh requirements (that is, lower continuous display time of LED lamp beads) without being troubled by the rolling time (that is, rolling time), and greatly improves the display acquisition ratio of the LED lamp beads in the LED screen, that is, improves the utilization rate of the LED screen.

[0112] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments. For the sake of brevity, it will not be repeated here.

[0113] An embodiment of the present disclosure further provides an electronic device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the above method.

[0114] An embodiment of the present disclosure further provides a non-volatile computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of the above method when executed by a processor.

[0115] An embodiment of the present disclosure further provides a computer program product, including a computer program, or a non-volatile computer-readable storage medium carrying the computer program, wherein the computer program implements the steps of the above method when executed by a processor.

[0116] Figure 11 FIG1 shows a block diagram of an electronic device 1900 according to an embodiment of the present disclosure. For example, the electronic device 1900 can be provided as a server or a terminal device. Figure 11The electronic device 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions executable by the processing component 1922, such as an application. The application stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute the instructions to perform the above-described method.

[0117] The electronic device 1900 may further include a power supply component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input / output interface 1958 (I / O interface). The electronic device 1900 may operate based on an operating system stored in the memory 1932, such as Windows Server 2003. TM , Mac OS X TM , Unix TM ,Linux TM , FreeBSD TM or similar.

[0118] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions that can be executed by the processing component 1922 of the electronic device 1900 to perform the above method.

[0119] A computer-readable storage medium can be a tangible device that can hold and store programs / instructions used by an instruction execution device. A computer-readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination thereof. As used herein, a computer-readable storage medium is not to be construed as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through an electrical wire.

[0120] The computer programs (or computer-readable program instructions) described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions to be stored in the computer-readable storage medium in each computing / processing device.

[0121] The computer program (or computer program instructions) for performing the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The computer readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, by utilizing state information of computer-readable program instructions to personalize and customize an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), the electronic circuit can execute the computer-readable program instructions to implement various aspects of the present disclosure.

[0122] Various aspects of the present disclosure are described herein with reference to flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present disclosure. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0123] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device is generated that implements the functions / actions specified in one or more blocks in the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, where these instructions cause the computer, programmable data processing device, and / or other device to operate in a specific manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing various aspects of the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0124] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0125] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and the part of the module, program segment or instruction contains one or more executable instructions for realizing the prescribed logical function. In some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the prescribed function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0126] While various embodiments of the present disclosure have been described above, the foregoing description is intended to be illustrative, non-exhaustive, and not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is selected to best explain the principles of the embodiments, their practical applications, or technological improvements in the marketplace, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A display control method for an LED screen, characterized in that: The method is applied to capturing a virtual shooting scene of the LED screen using a rolling camera, and includes: During the virtual shooting process, obtaining the position information of the rolling camera; Determining, based on the position information of the rolling camera and the screen model of the LED screen, an inner viewing cone image within the inner viewing cone of the rolling camera to be displayed on the LED screen, and pixel position information of a plurality of target LED lamp beads within the inner viewing cone of the rolling camera in the LED screen, wherein the pixel position information includes pixel positions of the target LED lamp beads projected onto the imaging plane of the rolling camera; Determining a scanning sequence for each target LED lamp bead based on the start shooting time of the rolling camera, the pixel position information of the multiple target LED lamp beads, and the rolling parameters of the rolling camera; wherein the rolling parameters represent the rolling duration of the rolling camera, and the scanning sequence includes the start time when each target LED lamp bead starts to be scanned by the rolling camera; Determining a display timing of the multiple target LED lamp beads displaying the inner cone image based on a scanning timing of each target LED lamp bead, a shutter duration of the rolling camera, and a continuous display duration of a single LED lamp bead, wherein the display timing represents a start display time and an end display time of each target LED lamp bead; and the continuous display duration of a single LED lamp bead is greater than or equal to the shutter duration of the rolling camera; According to the display timing of the inner cone image displayed by the multiple target LED lamp beads, the multiple target LED lamp beads in the LED screen are controlled to display colors.

2. The method according to claim 1, characterized in that The method of determining the scanning timing of each target LED lamp bead according to the start shooting time of the rolling camera, the pixel position information of the plurality of target LED lamp beads, and the rolling parameters of the rolling camera includes: For the i-th row of pixels on the imaging plane of the rolling camera, determine the i-th group of lamp beads corresponding to the i-th row of pixels based on the pixel position information of the multiple target LED lamp beads, where the i-th group of lamp beads includes the multiple target LED lamp beads projected onto the i-th row of pixels, where 1≤i≤N, where N is the total pixel height of the imaging plane; When i is 1, the start time for the rolling camera to start scanning the first group of lamp beads corresponding to the first row of pixels is determined according to the start shooting time of the rolling camera and the rolling time and shutter time of the rolling camera; When i is greater than 1, the starting time when the rolling camera is to start scanning the i-th group of lamp beads is determined based on the pixel height of the i-th row of pixels, the rolling shutter parameters, the total pixel height of the imaging plane, and the starting time when the rolling camera is to start scanning the first group of lamp beads corresponding to the first row of pixels.

3. The method according to claim 1 or 2, characterized in that The step of determining the display timing of the inner cone image displayed by the plurality of target LED lamp beads according to the scanning timing of each target LED lamp bead, the shutter duration of the rolling camera, and the continuous display duration of a single LED lamp bead includes: The difference between the continuous display duration of a single LED lamp bead and the shutter duration of the rolling camera is used as the redundant duration; For the i-th group of lamp beads corresponding to the i-th row of pixels, determine the start display time of the i-th group of lamp beads based on the start time when the rolling camera is about to start scanning the i-th group of lamp beads and the redundant time length, wherein the start display time of the i-th group of lamp beads is earlier than the start time when the rolling camera is about to start scanning; The sum of the start display time of the i-th group of lamp beads and the continuous display time of the single LED lamp bead is used as the end display time of the i-th group of lamp beads.

4. The method according to claim 1, wherein The rolling shutter duration represents the time difference between the start exposure time of the top row of pixels and the start exposure time of the last row of pixels of the rolling shutter camera. The calibration process of the rolling shutter parameters of the rolling shutter camera includes: In the process of controlling the LED screen to alternately display the first image frame and the second image frame and continuously adjusting the display phase of the LED lamp beads in the LED screen, the LED screen is photographed by the rolling camera at a preset acquisition frame rate to obtain a captured video; wherein the image content of the first image frame is different from that of the second image frame; By analyzing the video screen of the captured video, a first moment at which the first mixed screen just changes to the screen content of the second image frame, and a second moment at which the screen content of the second image frame just does not reveal the second mixed screen are obtained; wherein the first mixed screen includes a top area of ​​the video screen that is the screen content of the first image frame and other areas that are the screen content of the second image frame, and the second mixed screen includes a bottom area of ​​the video screen that is the screen content of the first image frame and other areas that are the screen content of the second image frame; The rolling parameters of the rolling camera are determined according to the first moment, the second moment, a single-frame display duration of the second image frame, and a shutter duration of the rolling camera.

5. The method according to claim 1, wherein The step of determining, based on the position information of the rolling camera and the screen model of the LED screen, an inner viewing cone image within the inner viewing cone of the rolling camera to be displayed on the LED screen and pixel position information of a plurality of target LED lamp beads within the inner viewing cone of the rolling camera on the LED screen, includes: Determining, based on the posture information of the rolling camera, the relative posture of a virtual camera and the screen model in a virtual shooting environment and an inner cone image to be displayed on the LED screen, wherein the posture and internal parameters of the virtual camera and the rolling camera are the same; Determine an inner viewing cone area within the inner viewing cone range of the virtual camera in the screen model based on the relative posture, and use multiple LED lamp beads within the inner viewing cone area as target LED lamp beads; Based on the spatial position of each target LED lamp bead in the screen model, each target LED lamp bead in the screen model is projected onto the imaging plane of the virtual camera to obtain pixel position information of multiple target LED lamp beads within the inner viewing cone of the rolling camera.

6. The method according to claim 1, characterized in that The inner cone image to be displayed is any one of at least two images to be displayed alternately, wherein the controlling the multiple target LED lamp beads in the LED screen to display colors according to the display timing of the inner cone image by the multiple target LED lamp beads includes: Determining a target display timing for the multiple target LED lamp beads to alternately display at least two images according to the display timing of the multiple target LED lamp beads displaying any one image; According to the target display timing of the multiple target LED lamp beads alternately displaying at least two pictures, the target LED lamp beads in the LED screen are controlled to display colors.

7. A display control device for an LED screen, characterized in that: The device is applied to use a rolling camera to shoot a virtual shooting scene of the LED screen, and the device includes: An acquisition module, used for acquiring the position information of the rolling camera during the virtual shooting process; a first determining module, configured to determine, based on the position information of the rolling camera and the screen model of the LED screen, an inner viewing cone image to be displayed on the LED screen and within the inner viewing cone of the rolling camera, and pixel position information of a plurality of target LED lamp beads in the LED screen and within the inner viewing cone of the rolling camera, wherein the pixel position information includes pixel positions of the target LED lamp beads projected onto the imaging plane of the rolling camera; a second determining module, configured to determine a scanning sequence of each target LED lamp bead based on a start shooting time of the rolling camera, pixel position information of the plurality of target LED lamp beads, and a rolling parameter of the rolling camera; wherein the rolling parameter represents a rolling duration of the rolling camera, and the scanning sequence includes a start time when each target LED lamp bead starts to be scanned by the rolling camera; a third determining module, configured to determine a display timing of the plurality of target LED lamp beads displaying the inner cone image based on a scanning timing of each target LED lamp bead, a shutter duration of the rolling camera, and a continuous display duration of a single LED lamp bead, wherein the display timing represents a start display moment and an end display moment of each target LED lamp bead; and the continuous display duration of a single LED lamp bead is greater than or equal to the shutter duration of the rolling camera; The display control module is used to control the multiple target LED lamp beads in the LED screen to display colors according to the display timing of the inner cone image displayed by the multiple target LED lamp beads.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 6.

9. A non-volatile computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product comprising a computer program, or a non-volatile computer-readable storage medium carrying a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images