A virtual shooting method and device

Abstract

The present disclosure relates to a virtual shooting method and device, which is applied to virtual shooting by using multiple shooting devices. The method comprises: displaying multiple internal cone pictures on a display screen in a time-sharing manner in a first display period; wherein any shooting device in the multiple shooting devices corresponds to at least one internal cone picture in the multiple internal cone pictures, and different shooting devices correspond to different internal cone pictures; the time-sharing display means that the internal cone picture corresponding to any shooting device is displayed in a first time segment corresponding to the shooting device in the first display period, and the internal cone pictures corresponding to different shooting devices are displayed in different first time segments; and each shooting device in the multiple shooting devices shoots in the corresponding first time segment, so that each shooting device shoots the corresponding internal cone picture. Through the present disclosure, virtual shooting by multiple positions is realized, and the production efficiency and application scenarios of virtual shooting are greatly improved.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of virtual shooting, and in particular to a virtual shooting method and device. BACKGROUND

[0002] Virtual production is a film and television production technology that combines real-time 3D rendering, game engine technology and traditional studio shooting, uses computer graphics and film production technology to create visual content, and uses large LED display technology and live shooting technology to shoot high-definition and realistic film and television content in real time. It provides creators with an immersive environment where they can view and capture the final picture effect in real time. In recent years, in order to improve the production quality of film and television works and reduce the time cost and economic cost of film and television production, virtual production technology has become a popular solution in the industry. However, in virtual production, how to realize multi-camera shooting is a technical blank, which restricts the application scenarios of virtual production and seriously hinders the wider industry application of virtual production. SUMMARY

[0003] Therefore, the present disclosure provides a virtual shooting method, device, electronic equipment, storage medium and computer program product.

[0004] According to an aspect of the present disclosure, a virtual shooting method is provided, which is applied to virtual shooting using multiple shooting devices, and the method comprises:

[0005] In a first display period, multiple inner cone pictures are displayed on a display screen in time division; wherein any shooting device in the multiple shooting devices corresponds to at least one inner cone picture in the multiple inner cone pictures, and different shooting devices correspond to different inner cone pictures; the time division display means that the inner cone picture corresponding to any shooting device is displayed in a first time segment corresponding to the shooting device in the first display period, and the inner cone pictures corresponding to different shooting devices are displayed in different first time segments;

[0006] Each shooting device in the multiple shooting devices shoots in the corresponding first time segment, so that each shooting device shoots the corresponding inner cone picture.

[0007] In a possible implementation, the starting time of each shooting of any shooting device in the multiple shooting devices in the first display period is not earlier than the starting time of the first time segment corresponding to the shooting, and the time length of each shooting is not greater than the time length of the first time segment corresponding to the shooting.

[0008] In a possible implementation, the method further includes: sending, by the synchronization controller, a synchronization signal to each of the display screen and the plurality of photographing devices respectively.

[0009] The display screen displays the first time segment corresponding to the photographing device in response to the synchronization signal, and meanwhile, the photographing device starts photographing in response to the synchronization signal.

[0010] In a possible implementation, the displaying the plurality of internal perspective views on the display screen in time includes:

[0011] The plurality of internal perspective views are displayed on the display screen in sequence, so that the display frame rate of the internal perspective view corresponding to any photographing device in the plurality of photographing devices on the display screen is the same as the photographing frame rate of the photographing device.

[0012] In a possible implementation, the first display period further includes at least one second time segment; and during the second time segment, the photographing device does not perform photographing.

[0013] In a possible implementation, any second time segment in the at least one second time segment displays one internal perspective view in the plurality of internal perspective views.

[0014] The displaying the plurality of internal perspective views on the display screen in time includes:

[0015] According to the order of each first time segment and the order of each second time segment in the first display period, the plurality of internal perspective views are displayed on the display screen in sequence, so that the internal perspective view corresponding to each photographing device is displayed alternately in the first display period.

[0016] In a possible implementation, the time length of the first time segment and the second time segment displaying the same internal perspective view in the first display period is the same.

[0017] In a possible implementation, the plurality of photographing devices are two photographing devices, and the plurality of internal perspective views are two internal perspective views; the first display period includes two first time segments and two second time segments; and the two second time segments are located between the two first time segments, and the two second time segments display the two internal perspective views respectively, so that the internal perspective view corresponding to the two photographing devices is displayed alternately in the first display period.

[0018] In a possible implementation, the method further includes:

[0019] rendering, by a rendering device, based on the parameter data of the plurality of photographing devices and the original to-be-rendered data, to obtain the inner frustum pictures corresponding to each photographing device to be displayed in the first display period, wherein the number of the inner frustum pictures corresponding to any photographing device is related to the photographing frame rate of the photographing device;

[0020] stitching the inner frustum pictures corresponding to each photographing device to be displayed in the first display period to obtain a stitched image;

[0021] splitting, by the display screen, the stitched image to obtain the plurality of inner frustum pictures;

[0022] Alternatively,

[0023] rendering, by a rendering device, based on the parameter data of the plurality of photographing devices and the original to-be-rendered data, to obtain the inner frustum pictures corresponding to each photographing device to be displayed in the first display period, wherein the number of the inner frustum pictures corresponding to any photographing device is related to the photographing frame rate of the photographing device;

[0024] transmitting, in a multi-channel transmission manner, the inner frustum pictures corresponding to each photographing device to be displayed in the first display period to the display screen, so that the display screen obtains the plurality of inner frustum pictures.

[0025] In a possible implementation, the method further includes:

[0026] determining whether there is an overlapping area in the different inner frustum pictures to be displayed in the first display period;

[0027] if there is an overlapping area, determining, for any overlapping area, the gray value of the overlapping area in the corresponding inner frustum pictures;

[0028] calculating a gray compensation value corresponding to the first display period based on the gray value of the overlapping area in the corresponding inner frustum pictures;

[0029] performing gray compensation on the inner frustum pictures corresponding to the overlapping area based on the gray compensation value corresponding to the first display period.

[0030] In a possible implementation, the performing gray compensation on the inner frustum pictures corresponding to the overlapping area based on the gray compensation value corresponding to the first display period includes:

[0031] obtaining a gray compensation value corresponding to a historical display period;

[0032] calculating the mean value of the gray compensation value corresponding to the historical display period and the gray compensation value corresponding to the first display period;

[0033] The mean value is used to compensate the gray scale of each inner view cone picture corresponding to the overlapping area.

[0034] In a possible implementation, the frame rates of any two of the plurality of shooting devices are the same or in a multiple relationship.

[0035] According to another aspect of the present disclosure, a virtual shooting device is provided, which is applied to virtual shooting using a plurality of shooting devices, and the device comprises:

[0036] The display module is configured to display a plurality of inner view cone pictures on the display screen in a time-division manner in a first display period, wherein any shooting device of the plurality of shooting devices corresponds to at least one inner view cone picture of the plurality of inner view cone pictures, and different shooting devices correspond to different inner view cone pictures; the time-division display means that the inner view cone picture corresponding to any shooting device is displayed in a first time segment corresponding to the shooting device in the first display period, and the inner view cone pictures corresponding to different shooting devices are displayed in different first time segments.

[0037] The shooting module is configured to shoot in a corresponding first time segment by each shooting device of the plurality of shooting devices, so that each shooting device shoots a corresponding inner view cone picture.

[0038] According to another aspect of the present disclosure, an electronic device is provided, which comprises a processor, and a memory for storing processor-executable instructions, wherein the processor is configured to implement the above method when executing the instructions stored in the memory.

[0039] According to another aspect of the present disclosure, a non-volatile computer readable storage medium is provided, which stores computer program instructions, wherein the computer program instructions are executed by a processor to implement the above method.

[0040] According to another aspect of the present disclosure, a computer program product is provided, which comprises computer readable code or a non-volatile computer readable storage medium carrying computer readable code, and when the computer readable code is run in a processor of an electronic device, the processor of the electronic device executes the above method.

[0041] In a first display period, a plurality of inner cone pictures are displayed on the display screen in time division; any one of the plurality of shooting devices corresponds to at least one of the plurality of inner cone pictures, and different shooting devices correspond to different inner cone pictures; the time division display means that the inner cone picture corresponding to any one of the shooting devices is displayed in a first time segment corresponding to the shooting device in the first display period, and the inner cone pictures corresponding to different shooting devices are displayed in different first time segments; each of the plurality of shooting devices shoots in the corresponding first time segment, so that each shooting device shoots the corresponding inner cone picture; in this way, by displaying the plurality of inner cone pictures corresponding to the plurality of shooting devices in time division, it is ensured that each shooting device can shoot with the corresponding inner cone picture as the background in the same display period, thereby realizing multi-camera virtual shooting; as an example, the shooting device can be configured with a rolling shutter camera, so that multi-camera virtual shooting can be realized based on the rolling shutter camera, the demand for multi-camera virtual shooting is met, and the production efficiency and application scenarios of virtual shooting are greatly improved.

[0042] Other features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the drawings. BRIEF DESCRIPTION OF DRAWINGS

[0043] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the present disclosure and serve to explain the principles of the present disclosure.

[0044] Figure 1 A structural diagram of a virtual shooting system according to an embodiment of the present disclosure is shown;

[0045] Figure 2 A scene diagram of a double-camera virtual shooting according to an embodiment of the present disclosure is shown;

[0046] Figure 3 A flowchart of a virtual shooting method according to an embodiment of the present disclosure is shown;

[0047] Figure 4 A method flowchart for acquiring a plurality of inner cone pictures according to an embodiment of the present disclosure is shown;

[0048] Figure 5 A distribution diagram of different inner cone pictures on a display screen according to an embodiment of the present disclosure is shown;

[0049] Figure 6 A sorting diagram of a first time segment according to an embodiment of the present disclosure is shown;

[0050] Figure 7FIG. 1 shows a schematic diagram of a first time slice according to an embodiment of the present disclosure;

[0051] Figure 8 FIG. 1 shows a schematic diagram of a first time slice according to an embodiment of the present disclosure;

[0052] Figure 9 FIG. 1 shows a schematic diagram of a first time slice according to an embodiment of the present disclosure;

[0053] Figure 10 FIG. 1 shows a schematic diagram of a first time slice according to an embodiment of the present disclosure;

[0054] Figure 11 FIG. 1 shows a schematic diagram of a first time slice according to an embodiment of the present disclosure;

[0055] Figure 12 FIG. 1 shows a schematic diagram of a first time slice according to an embodiment of the present disclosure;

[0056] Figure 13 FIG. 1 shows a schematic diagram of a first time slice according to an embodiment of the present disclosure;

[0057] Figure 14 FIG. 1 shows a schematic diagram of a first time slice according to an embodiment of the present disclosure;

[0058] Figure 15 FIG. 1 shows a schematic diagram of a first time slice according to an embodiment of the present disclosure;

[0059] Figure 16 FIG. 1 shows a schematic diagram of a first time slice according to an embodiment of the present disclosure;

[0060] Figure 17 FIG. 1 shows a schematic diagram of a first time slice according to an embodiment of the present disclosure; DETAILED DESCRIPTION

[0061] Various exemplary embodiments, features, and aspects of the present disclosure will be described below in detail with reference to the accompanying drawings. The same reference numbers in different drawings denote the same or similar elements. Although various aspects of embodiments are illustrated in the drawings, the drawings are not necessarily drawn to scale unless specifically noted.

[0062] Reference within the specification to "one embodiment" or "an embodiment" or "some embodiments" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present disclosure. The appearance of the phrases "in one embodiment" or "in some embodiments" within the specification, generally means that a particular feature, structure, or characteristic described in connection with that embodiment is included in at least one embodiment of the present disclosure, and that the feature, structure, or characteristic is not necessarily included in all embodiments of the present disclosure. The terms "including," "containing," "having," and variations thereof, are meant to encompass the terms "consisting of" and "consisting essentially of" unless otherwise noted.

[0063] In the present disclosure, "at least one" means one or more, and "multiple" means two or more. The "and / or" describes the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B, which can represent the following cases: A exists alone, A and B exist together, and B exists alone, where A and B can be singular or plural. The character " / " generally represents that the associated objects before and after it are in an "or" relationship. "At least one of the following" or the like means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b, or c can represent a, b, c, a-b, a-c, b-c, or a-b-c, where a, b, and c can be single or multiple.

[0064] In addition, in order to better illustrate the present disclosure, a large number of specific details are given in the specific embodiments below. Those skilled in the art should understand that the present disclosure can also be implemented without some specific details. In some examples, methods, means, elements and circuits that are well known to those skilled in the art are not described in detail, in order to highlight the main idea of the present disclosure.

[0065] Figure 1 A structural diagram of a virtual shooting system according to an embodiment of the present disclosure is shown; as Figure 1As shown, the virtual shooting system can include a broadcast processing device 101 (which can also be referred to as a broadcast controller or a rendering master, etc.), a rendering device 102, a synchronization controller 103, a display screen 104 (three display screens are shown in the figure: display screen 1041, display screen 1042, and display screen 1043), and a shooting device 105; the number of various devices can be flexibly configured according to actual needs, and no limitation is made in this regard; in addition, other devices such as mobile terminals or network devices can also be configured in the virtual shooting system according to needs; and one or more devices in the virtual shooting system can also be integrated into one device according to needs, for example, the rendering device 102, the broadcast processing device 101, and the synchronization controller 103 can be integrated into one device.

[0066] Exemplarily, the broadcast processing device 101 can be connected with the rendering device 102, and can send various control instructions to the rendering device 102 to control the rendering device 102 to render the desired picture. The rendering device 102 can render the picture meeting the requirements in response to the control instruction of the broadcast processing device 101, and can send the rendered picture to the broadcast processing device 101, and the broadcast processing device 101 can perform splicing, decoding, and other processing on the picture from the rendering device 102. As an example, the rendering device 102 can be deployed with a rendering engine to build a virtual environment consistent with the real shooting environment by rendering the picture in real time; as another example, the rendering device 102 can be a distributed rendering cluster.

[0067] Exemplarily, the broadcast processing device 101 can be connected with the display screen 104 to control and manage the display screen 104, for example, the broadcast processing device 101 can be used for data transmission and decoding, such as receiving the picture from the rendering device 102 and decoding it into a format suitable for the display screen 104 to display; can also be used for display control of the display screen 104, such as overall control and scheduling of the display screen 104, including brightness adjustment, color correction, gray scale control, etc., and can also control the display screen 104 to display and refresh the picture; can also be used for partition management of the display screen 104, and the display screen can be divided into multiple independent areas, and each area can display different content. As an example, the broadcast processing device 101 can be configured with a sending card (which can also be referred to as a transmitter) for converting an image into a format suitable for transmission to the display screen 104.

[0068] Exemplarily, the display screen 104 can be an LED screen, a liquid crystal screen, etc., and can be a curved screen or a flat screen, etc. The type, number, size, resolution, etc. of the display screen 104 in the virtual shooting system can be customized according to actual needs, which is not limited. For example, the display screen 104 can be a large display screen composed of many small LED display units. As an example, the display screen 104 can be configured with a receiving card (also referred to as a receiver) to decode and process the received signals so as to be displayed on the display screen 104.

[0069] Exemplarily, the control processing device 101 can be connected with the shooting device 105; the shooting device 105 can transmit the shot picture to the control processing device 101. As an example, the shooting device 105 is configured with a rolling shutter camera; wherein the rolling shutter camera is a camera using a rolling shutter mechanism for image capture, in which the imaging points on the photosensitive sensor are exposed in sequence row by row (or column by column); the rolling shutter camera is widely used in fields such as high-quality video shooting.

[0070] Exemplarily, the synchronization controller 103 can be used to synchronize the control processing device 101 and the shooting device 105, so as to ensure high-precision time synchronization between the control processing device 101 controlling the display screen 104 to display the picture and the shooting device 105 shooting the picture. As an example, the synchronization controller 103 can generate a synchronization signal and send the synchronization signal to the control processing device 101 and the shooting device 105, so that the control processing device 101 and the shooting device 105 are synchronized.

[0071] In the process of virtual shooting, the control processing device 101 in the virtual shooting system can control the rendering device 102 to render the picture, and the control processing device 101 transmits the rendered picture to the display screen 104 for display. At the same time, the actor can perform in front of the display screen 104, and the control processing device 101 controls the shooting device 105 to shoot, so that the actor is the foreground and the picture in the display screen is the background in the shot video frame, thereby realizing virtual shooting.

[0072] In some virtual shooting scenes, the number of shooting devices 105 is multiple, that is, virtual shooting can be performed by multiple positions, and multiple-position shooting can collect more angle pictures in the process of one performance of the actor, thereby improving the shooting efficiency and meeting the needs of different shooting angles. Figure 2 A scene schematic diagram of double-position virtual shooting according to an embodiment of the present disclosure is shown. As shown in Figure 2 The number of shooting devices 105 is 2, that is, camera 1 and camera 2 in the figure, and camera 1 and camera 2 are used to shoot at different angles, respectively.

[0073] Compared with the scene in which the number of the shooting device 105 is one, that is, the virtual shooting is performed by using a single position, the virtual shooting scene of multiple positions is more complex. Embodiments of the present disclosure provide a virtual shooting method, which implements the virtual shooting of multiple positions. As an example, the virtual shooting of multiple positions can be implemented based on a rolling shutter camera, so as to meet the demand of the virtual shooting of multiple positions and greatly improve the production efficiency and application scene of the virtual shooting.

[0074] The virtual shooting method provided by the present disclosure will be described in detail below.

[0075] Figure 3 A flowchart of a virtual shooting method according to an embodiment of the present disclosure is shown. The method can be applied to the virtual shooting performed by using multiple shooting devices. As an example, the method can be executed by the virtual shooting system shown in Figure 1 or one or more component parts in the virtual shooting system, as shown in Figure 3 The method can include the following steps:

[0076] In step 301, in a first display period, multiple internal view cone pictures are displayed on a display screen in time division; wherein any shooting device in the multiple shooting devices corresponds to at least one internal view cone picture in the multiple internal view cone pictures, and different shooting devices correspond to different internal view cone pictures; the display in time division means that the internal view cone picture corresponding to any shooting device is displayed in a first time segment corresponding to the shooting device in the first display period, and the internal view cone pictures corresponding to different shooting devices are displayed in different first time segments.

[0077] As an example, the display screen in this step can be the display screen 104 in the above Figure 1 The multiple internal view cone pictures can be acquired by the play control processing device 101 in the above Figure 1 and provided to the display screen 104 for display. The shooting device can include the shooting device 105 in the above Figure 1

[0078] The first display period means any display period in the process of the virtual shooting performed by using multiple shooting devices. As an example, the length of the first display period can be determined by the least common multiple of the shooting frame rates of the multiple shooting devices, so as to ensure that each shooting device can shoot at least once in each display period, and at least one shooting device shoots only once in each display period. The higher the shooting frame rate of a shooting device is, the more times the shooting device shoots in the first display period; on the contrary, the lower the shooting frame rate of a shooting device is, the fewer times the shooting device shoots in the first display period.

[0079] ​Exemplarily, the shooting frame rates of any two of the plurality of shooting devices are same or in a multiple relationship. Wherein, if the shooting frame rates of any two of the plurality of shooting devices are same, it means that the two shooting devices shoot the same number of times in the first display period; if the shooting frame rates of any two of the plurality of shooting devices are in a multiple relationship, it means that the two shooting devices shoot different number of times in the first display period, but in a multiple relationship.

[0080] As an example, for a virtual shooting scene with two shooting positions, if the shooting frame rates of the two shooting devices are both 25 frames per second, the length of the first display period is 40 ms, and in the 40 ms, the two shooting devices shoot once; if the shooting frame rates of the two shooting devices are 25 frames per second and 50 frames per second respectively, the length of the first display period is 40 ms, and in the 40 ms, one shooting device shoots once and the other shooting device can shoot twice; as another example, for a virtual shooting scene with three shooting positions, if the shooting frame rates of the three shooting devices are 25 frames per second, 50 frames per second and 25 frames per second respectively, the length of the first display period is 40 ms, and in the 40 ms, two shooting devices shoot once and the other shooting device can shoot twice; if the shooting frame rates of the three shooting devices are 25 frames per second, 25 frames per second and 25 frames per second respectively, the length of the first display period is 40 ms, and in the 40 ms, the three shooting devices shoot once; if the shooting frame rates of the three shooting devices are 25 frames per second, 50 frames per second and 100 frames per second respectively, the length of the first display period is 40 ms, and in the 40 ms, one shooting device of the three shooting devices shoots once, one shooting device shoots twice and one shooting device shoots four times.

[0081] In the process of virtual shooting by the plurality of shooting devices, since there are differences in the visual angles of different shooting devices, when any shooting device shoots, the inner frustum picture corresponding to the shooting device needs to be displayed on the display screen, so that the shooting device can shoot the correct picture.

[0082] Wherein, any shooting device of the plurality of shooting devices corresponds to at least one of the plurality of inner frustum pictures in the first display period. Exemplarily, the number of the plurality of inner frustum pictures in the first display period is same as the number of times of shooting by the plurality of shooting devices in the first display period; wherein, for any shooting device, the number of inner frustum pictures corresponding to the shooting device is same as the number of times of shooting by the shooting device in the first display period.

[0083] Wherein, the inner frustum pictures corresponding to different shooting devices of the plurality of shooting devices are different, i.e. the pictures shot by different shooting devices are different; exemplarily, for the same shooting device, the inner frustum picture for each shooting in the first display period can be same or different.

[0084] As an example, for a virtual shooting scene with two shooting positions, if the shooting frame rates of the two shooting devices are both 25 frames / s, the duration of the first display period is 40 ms, and within 40 ms, the two shooting devices are both shot once, and accordingly, two internal view cone pictures need to be displayed within 40 ms; if the shooting frame rates of the two shooting devices are 25 frames / s and 50 frames / s respectively, the duration of the first display period is 40 ms, and within 40 ms, one shooting device is shot once, and the other shooting device can be shot twice, and accordingly, three internal view cone pictures need to be displayed within 40 ms; as another example, for a virtual shooting scene with three shooting positions, if the shooting frame rates of the three shooting devices are 25 frames / s, 50 frames / s and 25 frames / s respectively, the duration of the first display period is 40 ms, and within 40 ms, two shooting devices are both shot once, and the other shooting device can be shot twice, and accordingly, four internal view cone pictures need to be displayed within 40 ms; if the shooting frame rates of the three shooting devices are 25 frames / s, 25 frames / s and 25 frames / s respectively, the duration of the first display period is 40 ms, and within 40 ms, the three shooting devices are all shot once, and accordingly, three internal view cone pictures need to be displayed within 40 ms; if the shooting frame rates of the three shooting devices are 25 frames / s, 50 frames / s and 100 frames / s respectively, the duration of the first display period is 40 ms, and within 40 ms, one shooting device is shot once, one shooting device is shot twice, and one shooting device is shot four times; and accordingly, seven internal view cone pictures need to be displayed within 40 ms.

[0085] The first display period includes a plurality of first time segments, each of the first time segments is used to display one internal view cone picture, and the internal view cone pictures displayed by different first time segments are different. For example, the number of first time segments in the first display period is the same as the number of internal view cone pictures in the plurality of internal view cone pictures in the first display period, that is, the number of first time segments in the first display period is the same as the total number of times that the plurality of shooting devices need to be shot in the first display period. For any shooting device, the first time segment used to display the internal view cone picture corresponding to the shooting device is the first time segment corresponding to the shooting device; if the shooting device needs to be shot once in the first display period, the number of internal view cone pictures corresponding to the shooting device is one, and accordingly, the number of corresponding first time segments is also one; if the shooting device needs to be shot multiple times in the first display period, the number of internal view cone pictures corresponding to the shooting device is multiple, and accordingly, the number of corresponding first time segments is also multiple, and each first time segment corresponds to one internal view cone picture, that is, each shooting corresponds to one time segment. For example, there is no overlap between different first time segments.

[0086] As an example, for a virtual shooting scene with two camera positions, if the shooting frame rates of the two shooting devices are both 25 frames / s, the duration of the first display period is 40 ms, and within 40 ms, the two shooting devices each shoot once. Correspondingly, 40 ms includes two first time segments for displaying two internal cone pictures, respectively. If the shooting frame rates of the two shooting devices are 25 frames / s and 50 frames / s, respectively, the duration of the first display period is 40 ms, and within 40 ms, one shooting device shoots once, and the other shooting device can shoot twice. Correspondingly, 40 ms includes three first time segments for displaying three internal cone pictures, respectively. As another example, for a virtual shooting scene with three camera positions, if the shooting frame rates of the three shooting devices are 25 frames / s, 50 frames / s, and 25 frames / s, respectively, the duration of the first display period is 40 ms, and within 40 ms, two shooting devices each shoot once, and the other shooting device can shoot twice. Correspondingly, 40 ms includes four first time segments for displaying four internal cone pictures, respectively. If the shooting frame rates of the three shooting devices are 25 frames / s, 25 frames / s, and 25 frames / s, respectively, the duration of the first display period is 40 ms, and within 40 ms, the three shooting devices each shoot once. Correspondingly, 40 ms includes three first time segments for displaying three internal cone pictures, respectively. If the shooting frame rates of the three shooting devices are 25 frames / s, 50 frames / s, and 100 frames / s, respectively, the duration of the first display period is 40 ms, and within 40 ms, one of the three shooting devices shoots once, one of the three shooting devices shoots twice, and one of the three shooting devices shoots four times. Correspondingly, 40 ms includes seven first time segments for displaying seven internal cone pictures, respectively.

[0087] The duration of the first time segment determines the duration of the internal cone picture displayed on the display screen, that is, the duration during which the shooting device can capture the internal cone picture. Therefore, the first time segment can also be referred to as a capture time segment. As an example, the duration of each first time segment in the first display period can be determined according to the duration of the first display period and the duration of each shooting of the shooting device corresponding to the first time segment. For any shooting device, the duration of each shooting of the shooting device is a fixed value, which is the duration of exposure of a photosensor in the shooting device. The sum of the durations of the first time segments in the first display period is not more than the duration of the first display period. Meanwhile, for any first time segment, the duration of the first time segment is not less than the duration of each shooting of the shooting device corresponding to the first time segment. In this way, based on the above constraints, by reasonably configuring the durations of the first time segments, the number of shootings of the multiple shooting devices in the first display period and the fact that each shooting can shoot a complete picture as expected can be met.

[0088] As an example, each of the plurality of shooting devices is configured with a rolling shutter camera; wherein the length of each shooting of any shooting device in the first display period is the sum of the shutter time and the rolling time of the shooting device; wherein the shutter time represents the exposure time of each imaging point on the sensor of the rolling shutter camera, and the rolling time represents the time interval between the start of exposure of the first row of imaging points and the start of exposure of the last row of imaging points on the sensor of the rolling shutter camera. Correspondingly, the length of the first time segment corresponding to any shooting device is not less than the sum of the shutter time and the rolling time of the shooting device, so that by reasonably configuring the length of the first time segment, it is ensured that the rolling shutter camera of the shooting device can roll from the first row of imaging points to the last row of imaging points, and the display screen can continuously display the corresponding internal cone picture during the exposure time of all rows of imaging points.

[0089] For example, in the virtual shooting scene of the above Figure 2 two machine positions, the camera 1 and the camera 2 are both rolling shutter cameras, and the shutter time is 5ms and the rolling time is 6ms; the shooting frame rate of the camera 1 and the camera 2 is 25 frames per second, then the length of the first display period can be determined as 40ms, and in the 40ms, the camera 1 and the camera 2 both need to shoot once, correspondingly, 40ms includes two first time segments T1 and T2, which are used to display the internal cone picture corresponding to the camera 1 and the internal cone picture corresponding to the camera 2 respectively, wherein the length of T1 and T2 is not less than the sum of the shutter time and the rolling time, and the sum of the length of T1 and T2 does not exceed the length of the first display period, that is, the length of T1 and T2 is not less than 11ms, and the sum of the length of T1 and T2 is not more than 40ms, then under the two constraints, the specific value of the length of T1 and T2 can be set according to the demand, for example, T1=T2=11ms, or T1=T2=20ms, or T1=14ms, T2=17ms, etc.

[0090] Based on the foregoing description, the duration of each first time segment in the first display period needs to satisfy the following constraints: the sum of the durations of each first time segment does not exceed the duration of the first display period, and for any first time segment, the duration of the first time segment is not less than the duration of each shooting of the corresponding shooting device. Under the condition of satisfying the above constraints, in addition to each first time segment, the first display period can also include at least one second time segment, that is, the first display period can include a plurality of first time segments and at least one second time segment. During the second time segment, the shooting device does not shoot. Therefore, the second time segment can also be referred to as a non-capture time segment. Since the content displayed in the second time segment will not be captured by the shooting device, that is, the content displayed in the second time segment will not affect the picture shot by the virtual shooting. Therefore, the display content of the second time segment can be flexibly configured as needed, for example, a preset image, a random image, one of a plurality of internal cone pictures, and the like.

[0091] Step 302, each shooting device in the plurality of shooting devices respectively shoots in the corresponding first time segment, so that each shooting device shoots the corresponding internal cone picture.

[0092] Exemplarily, the shooting device can be the shooting device 105 shown in the shooting device 105 in the above Figure 1

[0093] This step can be virtual shooting by a plurality of shooting devices; wherein each shooting device shoots with the internal cone picture displayed on the display screen as the background and the entity object as the foreground. Exemplarily, the entity object can include actors, props, scenery, etc. At the same time, any shooting device shoots in the first time segment of the internal cone picture corresponding to the shooting device displayed on the display screen, so that the shooting device can shoot the correct internal cone picture. For any shooting device, if there are a plurality of first time segments corresponding to the shooting device in the first display period, the shooting device shoots in each corresponding first time segment to shoot the internal cone picture displayed in each first time segment, that is, each internal cone picture corresponding to the shooting device.

[0094] In one possible implementation, the starting time of each shooting of any shooting device in the plurality of shooting devices in the first display period is not earlier than the starting time of the first time segment corresponding to the shooting, and the duration of each shooting is not greater than the duration of the first time segment corresponding to the shooting.

[0095] ​Exemplarily, the starting moment of each photographing of any photographing device in the plurality of photographing devices in the first display period is the same as the starting moment of the first time segment corresponding to the photographing. In this way, the photographing device can photograph after the corresponding first time segment starts to display the inner view cone picture or synchronously, so that the photographing device can photograph the corresponding inner view cone picture, and since the length of each photographing does not exceed the length of the first time segment corresponding to the photographing, the photographing device has enough time to photograph a complete inner view cone picture, thereby ensuring that any photographing device can photograph a correct and complete inner view cone picture.

[0096] Exemplarily, the synchronization of the plurality of photographing devices and the display screen can be realized by the synchronization controller. Exemplarily, the synchronization controller can send a synchronization signal to the display screen and any photographing device in the plurality of photographing devices respectively; the display screen displays the inner view cone picture of the first time segment corresponding to the photographing device in response to the synchronization signal, and the photographing device starts photographing in response to the synchronization signal. For example, the synchronization controller can be the synchronization controller 103 in the above Figure 1 The photographing device can be the photographing device 105, and the display screen can be the display screen 104. The synchronization controller 103 can send a synchronization signal to each photographing device 105 and the display screen 104, so as to realize that at the starting moment of any first time segment, the photographing device corresponding to the first time segment starts to photograph this time, that is, the starting moment of each photographing in the first display period is the same as the starting moment of the first time segment corresponding to the photographing; or after the starting moment of any first time segment, the photographing device corresponding to the first time segment starts to photograph this time, that is, the starting moment of each photographing in the first display period is later than the starting moment of the first time segment corresponding to the photographing. For example, the first time segment T1 corresponds to the photographing device P, the synchronization controller 103 can send a synchronization signal 1 to the display screen and the photographing device P, the display screen displays the first time segment 1 after receiving the synchronization signal 1, and the photographing device P starts to photograph after receiving the synchronization signal 1, so as to realize that at the starting moment of the first time segment T1, the photographing device P starts to photograph synchronously.

[0097] In the embodiment of the present disclosure, in the first display period, a plurality of internal view cone pictures are displayed on the display screen in time division; wherein any one of the plurality of shooting devices corresponds to at least one of the plurality of internal view cone pictures, and different shooting devices correspond to different internal view cone pictures; the time division display means that the internal view cone picture corresponding to any one of the shooting devices is displayed in a first time segment corresponding to the shooting device in the first display period, and the internal view cone pictures corresponding to different shooting devices are displayed in different first time segments; each of the plurality of shooting devices shoots in the corresponding first time segment, so that each shooting device shoots the corresponding internal view cone picture; in this way, by displaying the plurality of internal view cone pictures corresponding to the plurality of shooting devices in time division, it is ensured that each shooting device can shoot with the corresponding internal view cone picture as the background in the same display period, thereby realizing multi-camera virtual shooting; as an example, the shooting device can be configured with a rolling shutter camera, so that multi-camera virtual shooting can be realized based on the rolling shutter camera, the demand for multi-camera virtual shooting is met, and the production efficiency and application scenarios of virtual shooting are greatly improved.

[0098] In a possible implementation, the method can further include: obtaining the plurality of internal view cone pictures to be displayed in the first display period. For example, the plurality of internal view cone pictures to be displayed in the first display period can be obtained before the plurality of internal view cone pictures are displayed on the display screen in time division in the first display period in step 301.

[0099] For example, the plurality of internal view cone pictures to be displayed in the first display period can be obtained in the following manner.

[0100] For example, the plurality of internal view cone pictures to be displayed in the first display period can be obtained in the following manner.

[0101] Figure 4 For example, the plurality of internal view cone pictures to be displayed in the first display period can be obtained in the following manner. Figure 4 As shown in the method flowchart for obtaining the plurality of internal view cone pictures according to an embodiment of the present disclosure, the obtaining the plurality of internal view cone pictures to be displayed in the first display period can include:

[0102] In step 401, the rendering device is used to render based on the parameter data of the plurality of shooting devices and the original rendering data to be rendered, so as to obtain the internal view cone picture corresponding to each shooting device in the first display period.

[0103] For example, the rendering device can be the rendering device 102 in the above Figure 1 .

[0104] Exemplarily, before performing this step, the rendering device can acquire the parameter data of each shooting device and the original to-be-rendered data, and then render the inner frustum pictures corresponding to each shooting device. The original to-be-rendered data can be frame data, model geometry data, lighting data, environment data, etc. of a picture of a virtual scene planned to be shot by a user. The parameter data of the shooting device can include intrinsic parameters and extrinsic parameters. The intrinsic parameters of the shooting device can include focal length, optical center offset, distortion parameters, pixel size, and the extrinsic parameters of the shooting device can include position data and attitude data (for example, rotation angle). Since the displacement and attitude of each shooting device can change in real time during the shooting process, the extrinsic parameters of each shooting device can be updated in real time.

[0105] The number of inner frustum pictures corresponding to any shooting device is related to the shooting frame rate of the shooting device. The higher the shooting frame rate of the shooting device is, the more the number of corresponding inner frustum pictures is, and vice versa.

[0106] Exemplarily, for any shooting device, the number of inner frustum pictures corresponding to the shooting device in the first display period is the same as the number of times the shooting device needs to shoot in the first display period, which can be determined by the length of the first display period and the shooting frame rate of the shooting device. The determination of the length of the first display period can refer to the foregoing description. For example, if the shooting frame rate of a shooting device is 25 frames per second and the length of the first display period is 40 ms, the shooting device needs to shoot once in 40 ms, and the number of inner frustum pictures corresponding to the shooting device in the first display period is 1. For another example, if the shooting frame rate of a shooting device is 50 frames per second and the length of the first display period is 40 ms, the shooting device needs to shoot twice in 40 ms, and the number of inner frustum pictures corresponding to the shooting device in the first display period is 2.

[0107] Exemplarily, for any shooting device, the rendering device can acquire the parameter data (such as attitude and rotation angle) of the shooting device in real time. The rendering device can simulate the state of the shooting device according to the parameter data, and render a certain number of inner frustum pictures corresponding to the shooting device based on the original to-be-rendered data. For example, the rendering device can map the original to-be-rendered data according to the parameter data of the shooting device to change the position, focal point, angle of view, etc. of the to-be-rendered data, so that the mapped to-be-rendered data (i.e. inner frustum picture) is adapted to the shooting device.

[0108] Step 402, splicing the inner frustum pictures corresponding to each shooting device to be displayed in the first display period to obtain a frame of spliced image;

[0109] Exemplarily, this step can be performed by the rendering device according to the aboveFigure 1 The middle broadcast control processing device 101 performs, after the rendering device 102 completes rendering of the inner frustum picture corresponding to each shooting device, the rendering device 102 can send the rendered inner frustum picture corresponding to each shooting device to the broadcast control processing device 101; the broadcast control processing device 101 splices the inner frustum picture corresponding to each shooting device to obtain a spliced image, and sends the spliced image to the display screen 104. As an example, the spliced image can be sent to the receiving card configured on the display screen 104 through the sending card configured on the broadcast control processing device 101.

[0110] Exemplarily, the inner frustum pictures corresponding to each shooting device can be spliced based on a preset order. For example, the inner frustum pictures can be spliced according to the number of each shooting device, or according to the order in which the shooting devices start shooting, or according to the order in which the inner frustum pictures corresponding to the shooting devices are displayed in the same display period.

[0111] Considering that when multiple cameras are used for shooting, the shooting angles of different shooting devices may overlap, so the inner frustum pictures corresponding to different shooting devices displayed on the display screen may have overlapping areas; Figure 5 A schematic diagram of the distribution of different inner frustum pictures on a display screen is shown according to an embodiment of the present disclosure; as Figure 5 As shown, the inner frustum picture A corresponding to the camera 1 and the inner frustum picture B corresponding to the camera 2 are displayed on the display screen, and the inner frustum picture A and the inner frustum picture B have an overlapping area. When the picture is switched, the overlapping area will oscillate with high frequency signals, causing a certain high-frequency flicker, which will affect the vision of the on-site staff, so in the virtual shooting scene of multiple cameras, it is necessary to reduce the flicker of the human eye when watching the screen caused by the overlapping area.

[0112] In a possible implementation, before the splicing of the inner frustum pictures corresponding to each shooting device to be displayed in the first display period, the method further includes: determining whether the different inner frustum pictures to be displayed in the first display period have overlapping areas; if there are overlapping areas, determining the gray value of the overlapping area in the corresponding inner frustum pictures for any overlapping area; calculating the gray compensation value corresponding to the first display period based on the gray value of the overlapping area in the corresponding inner frustum pictures; and performing gray compensation on the inner frustum pictures corresponding to the overlapping area based on the gray compensation value corresponding to the first display period.

[0113] Exemplarily, the determining the gray scale value of the overlapping region in each corresponding inner view frustum image comprises: determining an average gray scale value of the overlapping region in each corresponding inner view frustum image. It can be understood that the number of the corresponding inner view frustum images of any overlapping region is at least two. In addition to the average gray scale value, the gray scale value of the overlapping region in each corresponding inner view frustum image can also be expressed in other ways, for example, the median gray scale value of all gray scale values of the overlapping region in each corresponding inner view frustum image can be determined instead of the average gray scale value, and the like. Those skilled in the art can make a selection as needed.

[0114] Exemplarily, the calculating the gray scale compensation value corresponding to the first display period based on the gray scale value of the overlapping region in each corresponding inner view frustum image comprises: calculating a difference value of the average gray scale values of the overlapping region in each corresponding inner view frustum image, and taking the product of a preset proportion parameter and the difference value as the corresponding gray scale compensation value in the first display period. Wherein, if the number of the corresponding inner view frustum images of the overlapping region exceeds two, a plurality of difference values of the average gray scale values of the overlapping region in different corresponding inner view frustum images can be obtained by pairwise calculation, and the final difference value can be obtained by averaging the plurality of difference values. Wherein, the difference value can be represented by any numerical value reflecting the difference size, such as a difference value or a ratio value. The preset proportion parameter can be set arbitrarily as needed.

[0115] Exemplarily, the performing gray scale compensation on each inner view frustum image corresponding to the overlapping region based on the gray scale compensation value corresponding to the first display period comprises: obtaining a gray scale compensation value corresponding to a historical display period; calculating the average of the gray scale compensation value corresponding to the historical display period and the gray scale compensation value corresponding to the first display period; and performing gray scale compensation on each inner view frustum image corresponding to the overlapping region based on the average. Wherein, the historical display period can include one or more display periods before the first display period. In this way, by averaging the gray scale compensation values of the historical display periods, the gray scale fluctuation caused by a higher compensation value is reduced, so that the compensation of the picture in the time dimension is more gentle, that is, the difference compensation changes slowly over time, rather than suddenly or dramatically.

[0116] For example, the above Figure 5Taking two inner cone pictures with overlapping area as an example, the overlapping area C = A & B can be calculated in real time according to the inner cone picture A and the inner cone picture B, and then the average gray value L_CA of the inner cone picture A in the overlapping area and the average gray value L_CB of the inner cone picture B in the overlapping area can be counted, and then the difference value Dif = L_CA - L_CB of the average gray value L_CA of the inner cone picture A in the overlapping area and the average gray value L_CB of the inner cone picture B in the overlapping area can be calculated; and then the gray compensation value Delta = -alpha * Dif is calculated, wherein alpha is a proportional parameter, for example, 0.5 can be taken, and the gray compensation value is the gray compensation value corresponding to the current display period, and then the average value Delta_t = tsmooth(Delta, Delta1, Delta2…) is calculated in combination with the gray compensation value corresponding to the historical display period, wherein Delta1, Delta2… represent the gray compensation values corresponding to the historical different display periods, and tsmooth represents time low-pass processing, that is, the average value of these gray compensation values is taken; the inner cone picture A is compensated based on the average value Delta_t A_pro = A’ + Delta_t, wherein A_pro is the gray value of the inner cone picture A after gray compensation, and A’ is the original gray value of the inner cone picture A; the inner cone picture B is compensated based on the average value Delta_t B_pro = B’ - Delta_t, wherein B_pro is the gray value of the inner cone picture B after gray compensation, and B’ is the original gray value of the inner cone picture B. In this way, the gray difference of different inner cone pictures corresponding to the overlapping area is compensated into the inner cone picture A and the inner cone picture B, and the difference compensation is ensured to be slowly changed in time.

[0117] Further, if the inner view frustum picture A, the inner view frustum picture B and the inner view frustum picture C exist in the same overlapping area, the gray scale compensated gray scale values can be obtained in the above manner between each other, and after three times of processing, the average of the gray scale compensated gray scale values of the same inner view frustum picture is taken as the final gray scale value of the inner view frustum picture. For example, first, based on the average gray scale value L_CA of the inner view frustum picture A in the overlapping area and the average gray scale value L_CB of the inner view frustum picture B in the overlapping area, A_pro and B_pro are obtained in the above manner; second, based on the average gray scale value L_CA of the inner view frustum picture A in the overlapping area and the average gray scale value L_CC of the inner view frustum picture C in the overlapping area, A_pro and C_pro are obtained in the above manner; third, based on the average gray scale value L_CC of the inner view frustum picture C in the overlapping area and the average gray scale value L_CB of the inner view frustum picture B in the overlapping area, C_pro and B_pro are obtained in the above manner; further, the average of the A_pro obtained in the first time and the A_pro obtained in the second time is taken as the final gray scale compensated gray scale value of the inner view frustum picture A, the average of the B_pro obtained in the first time and the B_pro obtained in the third time is taken as the final gray scale compensated gray scale value of the inner view frustum picture B, and the average of the C_pro obtained in the second time and the C_pro obtained in the third time is taken as the final gray scale compensated gray scale value of the inner view frustum picture C. When there are more than three overlapping inner view frustum pictures, the above manner can be used for gray scale compensation.

[0118] The above gray scale compensation process can be performed at any link before the multiple inner view frustum pictures are displayed on the display screen in time division, so that the inner view frustum pictures displayed on the display screen are gray scale compensated.

[0119] In step 403, the display screen splits the spliced image to obtain the multiple inner view frustum pictures.

[0120] For example, the step can be performed by the display screen 104 in the above Figure 1 For example, the display screen 104 configured with a receiving card splits the spliced image to obtain the multiple inner view frustum pictures, i.e., the inner view frustum pictures corresponding to each shooting device in the first display period, after receiving the spliced image.

[0121] For example, the spliced image can be split based on the splicing order when the inner view frustum pictures corresponding to each shooting device to be displayed in the first display period are spliced.

[0122] In the embodiments of the present disclosure, the rendering device is used to render based on the parameter data of the plurality of shooting devices and the original to-be-rendered data to obtain the inner frustum pictures corresponding to each shooting device to be displayed in the first display period; the inner frustum pictures corresponding to each shooting device to be displayed in the first display period are spliced to obtain a spliced image; the spliced image is split by the display screen to obtain the plurality of inner frustum pictures; and thus the inner frustum pictures required for real-time rendering and the transmission of the inner frustum pictures are realized, so that the display screen can be used for time-sharing display. As an example, after the inner frustum pictures required for rendering are obtained, further flicker reduction processing is performed, so as to reduce the flicker of the human eye when watching the screen in the overlapping area, effectively suppress the flicker perception of the naked eye of the on-site personnel, and improve the shooting comfort of the on-site personnel.

[0123] Method two,

[0124] In a possible implementation, the obtaining of the plurality of inner frustum pictures to be displayed in the first display period further includes: using a rendering device to render based on the parameter data of the plurality of shooting devices and the original to-be-rendered data to obtain the inner frustum pictures corresponding to each shooting device to be displayed in the first display period; wherein the number of the inner frustum pictures corresponding to any shooting device is related to the shooting frame rate of the shooting device; and the inner frustum pictures corresponding to each shooting device to be displayed in the first display period are transmitted to the display screen in a multi-channel transmission manner, so that the display screen obtains the plurality of inner frustum pictures.

[0125] The specific description of rendering based on the parameter data of the plurality of shooting devices and the original to-be-rendered data by using the rendering device can be referred to the related description in step 401.

[0126] For example, the control processing device and the display screen have a plurality of data transmission channels. After the rendering device generates the inner frustum pictures corresponding to each shooting device to be displayed in the first display period, the control processing device sends the inner frustum pictures to the display screen in a multi-channel transmission manner through the plurality of data transmission channels. For example, the control processing device can be configured with a sending card, and the display screen can be configured with a receiving card. The sending card and the receiving card have a plurality of data transmission channels. The sending card sends the inner frustum pictures corresponding to each shooting device to be displayed in the first display period rendered by the rendering device to the receiving card through the plurality of data transmission channels, and the receiving card receives the inner frustum pictures through the plurality of data transmission channels, so as to realize the transmission of the inner frustum pictures corresponding to each shooting device to be displayed in the first display period to the display screen in a multi-channel transmission manner.

[0127] Exemplarily, before the multiple inner view frustums corresponding to the multiple photographing devices in the first display period are transmitted to the display screen in a multi-channel transmission manner, it can be determined whether there is an overlapping area in the different inner view frustums to be displayed in the first display period. If there is an overlapping area, for any overlapping area, the gray value of the overlapping area in the corresponding inner view frustums is determined. Based on the gray value of the overlapping area in the corresponding inner view frustums, the gray compensation value corresponding to the first display period is calculated. The gray compensation is performed on the inner view frustums corresponding to the overlapping area based on the gray compensation value corresponding to the first display period. The possible gray compensation manner can refer to the related description in the foregoing.

[0128] The possible implementation manner of displaying the multiple inner view frustums on the display screen in the step 301 is exemplarily described below.

[0129] Scenario one, the first display period includes the first time segment.

[0130] In this scenario, the first display period only includes the first time segment. Therefore, in the first display period, only the inner view frustums corresponding to the multiple photographing devices can be displayed. The determination manner of the number and length of the first time segments in the first display period can refer to the related description in the foregoing.

[0131] In a possible implementation manner, the displaying the multiple inner view frustums on the display screen includes: sequentially displaying the multiple inner view frustums on the display screen, so that the display frame rate of the inner view frustum corresponding to any photographing device in the multiple photographing devices on the display screen is the same as the photographing frame rate of the photographing device.

[0132] Exemplarily, the order of the first time segments in the first display period can be determined according to the photographing frame rate of each photographing device in the multiple photographing devices. The multiple inner view frustums are sequentially displayed on the display screen according to the order of the first time segments in the first display period, so that the display frame rate of the inner view frustum corresponding to any photographing device in the multiple photographing devices on the display screen is the same as the photographing frame rate of the photographing device.

[0133] Exemplarily, the shooting frame rates of the plurality of shooting devices are same, and in the first display period, each shooting device needs to shoot once, corresponding to one first time segment. Then, the order of the first time segments in the first display period can be determined according to the order of shooting start of the shooting devices, or a preset order, or a randomly generated order. Then, based on the order of the first time segments, the corresponding internal view cone picture is displayed on the display screen in the first time segment corresponding to each shooting device in turn, and correspondingly, each shooting device shoots once in the corresponding first time segment, thereby shooting the corresponding internal view cone picture. In this way, for any shooting device, one internal view cone picture corresponding to the shooting device is displayed in the same display period, and the shooting device shoots once in the same display period, thereby realizing that the display frame rate of the internal view cone picture corresponding to any shooting device in the plurality of shooting devices on the display screen is same as the shooting frame rate of the shooting device, meeting the virtual shooting requirement of multiple positions.

[0134] Exemplarily, in the virtual shooting process, the number, duration and order of the first time segments in each display period are same, that is, the data, duration and order of the first time segments can be determined only in the initial display period.

[0135] For example, Figure 6 Fig. 1 shows a schematic diagram of the order of a first time segment according to an embodiment of the present disclosure; as Figure 6 shown, it is a virtual shooting scene of two positions, and the shooting frame rates of the two shooting devices are same. T1 and T2 respectively represent the first time segments corresponding to the shooting devices; in the same display period, T1 and T2 can be arranged in the order of T1 first and T2 second (i.e. Figure 6 as shown in Fig. 1), or can also be arranged in the order of T2 first and T1 second. In this way, in the same display period, one internal view cone picture corresponding to T1 and T2 is displayed respectively, and T1 and T2 both shoot once, and the shooting frame rates of T1 and T2 are same as the display frame rates of the internal view cone pictures corresponding to T1 and T2 on the display screen.

[0136] Figure 7 Fig. 2 shows a schematic diagram of the order of a first time segment according to an embodiment of the present disclosure; as Figure 7 shown, it is a virtual shooting scene of three positions, and the shooting frame rates of the three shooting devices are same. T1, T2 and T3 respectively represent the first time segments corresponding to the shooting devices, and in the same display period, T1, T2 and T3 can be arranged in the order of T1, T2 and T3 (i.e. Figure 7In the case shown in FIG. 1, the order of T1, T2 and T3 can be T1, T2, T3, or can be T2, T1, T3, or can be T3, T1, T2, and so on. It can be understood that the order of T1, T2 and T3 can be flexibly set, as long as there is only one T1, T2 or T3 in each display period, and the order is the same in different display periods. In the same display period, one inner cone picture corresponding to T1, T2 or T3 is displayed, and T1, T2 or T3 is photographed once. The frame rate of T1, T2 or T3 is the same as the display frame rate of the inner cone picture corresponding to T1, T2 or T3 on the display screen.

[0137] In the second scenario, the first display period includes a first time segment and a second time segment. The number of the first time segment and the second time segment can be one or more.

[0138] In this scenario, the first display period includes a first time segment and a second time segment, so that the inner cone picture corresponding to each photographing device can be displayed in the first display period. After determining the number, duration and order of the first time segment, the number, duration, order and display content of the second time segment can be further flexibly set. The determination of the duration and number of the first time segment in the first display period can refer to the related description in the foregoing.

[0139] Exemplarily, in the case that the frame rates of at least two photographing devices in the plurality of photographing devices are different, for a plurality of first time segments corresponding to the same photographing device in the first display period, the order of other first time segments and each second time segment in the first display period can be configured according to requirements, as long as these first time segments are arranged at equal intervals in the first display period.

[0140] Considering that the content displayed in the second time segment in the virtual photographing process can be seen by the on-site staff, the content displayed in the second time segment can be reasonably designed to reduce the flicker caused by the on-site staff viewing the screen with the naked eye as much as possible. Exemplarily, any second time segment displays one of the plurality of inner cone pictures. In this way, the rendered inner cone picture is directly reused as the display content of the second time segment, which is more convenient. At the same time, since the difference between different inner cone pictures in the same display period is usually not too large, the flicker of the picture is reduced.

[0141] Exemplarily, the number and length of the second time segments in the first display period can be configured according to requirements under the condition of meeting the constraints; for example, the constraints can include that the total length of each second time segment in the first display period and the total length of each first time segment are equal to the length of the first display period. As an example, the lengths of the first time segment and the second time segment for displaying the same internal view cone picture in the first display period are close. As another example, the lengths of the first time segment and the second time segment for displaying the same internal view cone picture in the first display period are the same, thereby reducing flicker.

[0142] For example, Figure 8 A schematic diagram of the ordering of a first time segment according to an embodiment of the present disclosure is shown. As shown in Figure 8 The virtual shooting scene shown is of double camera positions, and the shooting frame rates of the two shooting devices are the same. T1 and T2 respectively represent the first time segments corresponding to the shooting devices; T' represents the second time segment. As shown in Figure 8 In the same display period, one T1, one T2, and one T' are included, and are arranged in the order of T1, T', and T2 in sequence, so that the shooting frame rate of any shooting device is the same as the display frame rate of the corresponding internal view cone picture.

[0143] Figure 9 A schematic diagram of the ordering of a first time segment according to an embodiment of the present disclosure is shown. As shown in Figure 9 The virtual shooting scene shown is of double camera positions, and the shooting frame rate of one shooting device is twice that of the other shooting device. T1 and T2 respectively represent the first time segments corresponding to the shooting devices; T' represents the second time segment. As shown in Figure 8 In the same display period, two T1, one T2, and one T' are included, and are arranged in the order of T1, T2, T1, and T' in sequence, where the length of T' is the same as that of T2, so that the shooting frame rate of any shooting device is the same as the display frame rate of the corresponding internal view cone picture.

[0144] Figure 10 A schematic diagram of the ordering of a first time segment according to an embodiment of the present disclosure is shown. As shown in Figure 10 The virtual shooting scene shown is of three camera positions, and the shooting frame rates of two shooting devices are the same, which are twice the shooting frame rate of the other shooting device. T1, T2, and T3 respectively represent the first time segments corresponding to the shooting devices, and T' represents the second time segment, as shown in Figure 9 In the same display period, two T1, two T2, one T3, and one T' are included, and are arranged in the order of T1, T2, T3, T1, T2, and T' in sequence, where the length of T' is the same as that of T3, so that the shooting frame rate of any shooting device is the same as the display frame rate of the corresponding internal view cone picture.

[0145] Figure 11 Fig. 1 shows a schematic diagram of the ordering of the first time segments according to an embodiment of the present disclosure; as shown, it is a virtual shooting scene of three camera positions, and the shooting frame rates of the three shooting devices are the same. T1, T2 and T3 respectively represent the first time segments corresponding to the three shooting devices, and T' represents the second time segment. As shown, in the same display period, one T1, one T2, one T3 and two T' are included, and they are arranged in the order of T1, T', T2, T' and T3 in sequence, so that the shooting frame rate of any shooting device is the same as the display frame rate of the corresponding internal view cone picture. Figure 11 Figure 11 Fig. 2 shows a schematic diagram of the ordering of the first time segments according to an embodiment of the present disclosure; as shown, it is a virtual shooting scene of two camera positions, and the shooting frame rates of the two shooting devices are the same. T1 and T2 respectively represent the first time segments corresponding to the two shooting devices, and T' represents the second time segment. As shown, in the same display period, one T1, one T2 and two T' are included, and they are arranged in the order of T1, T', T2 and T' in sequence, so that the shooting frame rate of any shooting device is the same as the display frame rate of the corresponding internal view cone picture, and at the same time, the display refresh rate is improved, further reducing the influence of flicker.

[0146] Figure 12 Figure 12 Fig. 3 shows a schematic diagram of the ordering of the first time segments according to an embodiment of the present disclosure; as shown, it is a virtual shooting scene of three camera positions, and the shooting frame rates of the three shooting devices are the same. T1, T2 and T3 respectively represent the first time segments corresponding to the three shooting devices, and T' represents the second time segment. As shown, in the same display period, one T1, one T2, one T3 and three T' are included, and they are arranged in the order of T1, T', T2, T', T3 and T' in sequence, so that the shooting frame rate of any shooting device is the same as the display frame rate of the corresponding internal view cone picture, and at the same time, the display refresh rate is improved, further reducing the influence of flicker. Figure 12

[0147] Figure 13 Fig. 4 shows a schematic diagram of the ordering of the first time segments according to an embodiment of the present disclosure; as shown, it is a virtual shooting scene of three camera positions, and the shooting frame rates of the three shooting devices are the same. T1, T2 and T3 respectively represent the first time segments corresponding to the three shooting devices, and T' represents the second time segment. As shown, in the same display period, one T1, one T2, one T3 and three T' are included, and they are arranged in the order of T1, T', T2, T', T3 and T' in sequence, so that the shooting frame rate of any shooting device is the same as the display frame rate of the corresponding internal view cone picture, and at the same time, the display refresh rate is improved, further reducing the influence of flicker. Figure 13 Figure 13 In a possible implementation, any second time segment in the at least one second time segment displays one internal view cone picture in the plurality of internal view cone pictures; and the displaying the plurality of internal view cone pictures on the display screen in time-sharing manner comprises: according to the ordering of the first time segments and the ordering of the second time segments in the first display period, displaying the plurality of internal view cone pictures on the display screen in sequence, so that the internal view cone pictures corresponding to the shooting devices are displayed alternately in the first display period.

[0148]

[0149] ​​​​​Exemplarily, the order of each second time segment in the first display period and the inner view frustum picture to be displayed in each second time segment can be determined according to the order of each first time segment in the first display period and the inner view frustum picture to be displayed in each first time segment; and the plurality of inner view frustum pictures are displayed on the display screen in sequence according to the order of each first time segment in the first display period and the order of each second time segment, so that the inner view frustum pictures corresponding to the plurality of shooting devices are alternately displayed in the first display period.

[0150] Exemplarily, the time length of the first time segment and the second time segment of the same inner view frustum picture in the first display period is close or the same, so that the influence of flicker is further reduced on the basis of alternating display.

[0151] As an example, the plurality of shooting devices are two shooting devices, and the plurality of inner view frustum pictures are two inner view frustum pictures; the first display period includes two first time segments and two second time segments; the two second time segments are located between the two first time segments, and the two second time segments display the two inner view frustum pictures respectively, so that the inner view frustum pictures corresponding to the two shooting devices are alternately displayed in the first display period.

[0152] For example, Figure 14 A schematic diagram showing the change of display content on the display screen over time according to an embodiment of the present disclosure is shown as follows: Figure 14 As shown, the horizontal axis represents time, and the vertical axis includes two levels, high and low. The high level represents the display of the inner view frustum picture A, and the low level represents the display of the inner view frustum picture B, which correspond to the two shooting devices respectively. Taking the shooting frame rate of the two shooting devices as 25 frames per second as an example, that is, one frame of image is shot every 40 ms, the time length of the first display period can be determined as 40 ms, and in the 40 ms, the inner view frustum picture A and the inner view frustum picture B are displayed respectively. Two first time segments are needed in the first display period, and the time length of each first time segment is not less than the time length of each shooting of the shooting device. Taking the shutter time length of the shooting device as 5 ms and the rolling shutter time length as 6 ms as an example, the first time segment needs at least 11 ms; assuming Figure 14T1=T4=11 ms is the first time period, T2, T3 is the second time period, then T2+T3=18 ms, T2=T3=9 ms can be taken, wherein T1 and T3 display the inner visual cone picture A (i.e., T1 displays A(1, t1) and T3 displays A(2, t1) in the figure), T2 and T4 display the inner visual cone picture B (i.e., T2 displays B(1, t1) and T4 displays B(2, t1) in the figure), in this way, T1, T2, T3, T4 alternately display the inner visual cone pictures corresponding to the two shooting devices, meeting the requirement that the display screen displays the inner visual cone pictures and the shooting devices synchronously shoot, and at the same time, the flicker of naked eyes can be better reduced, and the viewing comfort of the on-site personnel such as actors and shooting teams is improved.

[0153] In the embodiments of the present disclosure, by configuring the order of each first time segment and / or the order of each second time segment in the first display period, the multi-camera virtual shooting requirement is met, and at the same time, the flicker is controlled by reasonable order of each first time segment and / or second time segment, the flicker perception of naked eyes is suppressed, and the shooting comfort of on-site personnel is improved.

[0154] For example, the virtual shooting system shown in the above Figure 1 is used to perform the virtual shooting of the double-camera position shown in the above Figure 2 , the virtual shooting method of the double-camera position shown in the above Figure 15 is used to perform the virtual shooting of the double-camera position shown in the above Figure 15As shown, the rendering device 102 acquires the parameter data (internal parameters and real-time updated external parameters) of the camera 1 and the camera 2 and the original data to be rendered, renders the inner view cone picture A and the inner view cone picture B according to the parameter data of the camera 1 and the camera 2, wherein the inner view cone picture A corresponds to the camera 1 and the inner view cone picture B corresponds to the camera 2. The rendering device 102 transmits the rendered inner view cone picture A and the inner view cone picture B to the control processing device 101. The control processing device 101 receives the inner view cone picture A and the inner view cone picture B, splices the inner view cone picture A and the inner view cone picture B to obtain a frame image AB, and can perform flicker reduction according to the overlap of the inner view cone picture A and the inner view cone picture B. The sending card of the control processing device 101 sends the spliced image AB to the receiving card configured by the display screen 104 according to the shooting phase requirement (for example, the order of starting shooting of the camera 1 and the camera 2). The receiving card configured by the display screen 104 receives the spliced image AB, first splits the image AB to separate the inner view cone picture A and the inner view cone picture B, and then performs time-sharing on-screen display according to the order of the first time segment and / or the second time segment. The display screen 104 displays the inner view cone picture A and the inner view cone picture B in time-sharing manner. The camera 1 captures an image with the inner view cone picture A as the background, and the camera 2 captures an image with the inner view cone picture B as the background. During this period, the synchronization controller 103 locks the shooting frame rate of the sending card and the camera, and adjusts the shutter phase (that is, the time when shooting starts) of the camera 1 and the camera 2 respectively, so that the camera 1 and the camera 3 can capture the inner view cone picture A and the inner view cone picture B respectively without picture aliasing. In this way, the actual needs of virtual shooting are combined to meet the dual-camera shooting requirements, and the flicker perception of the naked eye is suppressed through reasonable on-screen content sorting control and flicker reduction, thereby improving the shooting comfort.

[0155] Based on the same inventive concept of the above method embodiments, the embodiments of the present disclosure also provide a virtual shooting device which can be used to execute the technical solutions described in the above method embodiments.

[0156] Figure 16 FIG. 1 shows a structure schematic diagram of a virtual shooting device according to an embodiment of the present disclosure, which is applied to virtual shooting by using multiple shooting devices, such as Figure 16As shown, the apparatus can comprise: a display module 1601, configured to display a plurality of internal view cone pictures on the display screen in a time-sharing manner in a first display period; wherein any one of the plurality of shooting devices corresponds to at least one of the plurality of internal view cone pictures, and different shooting devices correspond to different internal view cone pictures; the time-sharing display means that the internal view cone picture corresponding to any one of the shooting devices is displayed in the first time segment corresponding to the shooting device in the first display period, and the internal view cone pictures corresponding to different shooting devices are displayed in different first time segments; a shooting module 1602, configured to shoot in the corresponding first time segment by each of the plurality of shooting devices, so that each shooting device shoots the corresponding internal view cone picture.

[0157] In the embodiments of the present disclosure, a plurality of internal view cone pictures are displayed on the display screen in a time-sharing manner in a first display period; wherein any one of the plurality of shooting devices corresponds to at least one of the plurality of internal view cone pictures, and different shooting devices correspond to different internal view cone pictures; the time-sharing display means that the internal view cone picture corresponding to any one of the shooting devices is displayed in the first time segment corresponding to the shooting device in the first display period, and the internal view cone pictures corresponding to different shooting devices are displayed in different first time segments; each of the plurality of shooting devices shoots in the corresponding first time segment, so that each shooting device shoots the corresponding internal view cone picture; in this way, by displaying a plurality of internal view cone pictures corresponding to a plurality of shooting devices in a time-sharing manner, it is ensured that each shooting device can shoot with the corresponding internal view cone picture as the background in the same display period, thereby realizing multi-camera virtual shooting; as an example, the shooting device can be configured with a rolling shutter camera, so that multi-camera virtual shooting can be realized based on the rolling shutter camera, meeting the demand of multi-camera virtual shooting, and greatly improving the production efficiency and application scenarios of virtual shooting.

[0158] In a possible implementation, the starting moment of each shooting of any one of the plurality of shooting devices in the first display period is not earlier than the starting moment of the first time segment corresponding to the shooting, and the time length of each shooting is not greater than the time length of the first time segment corresponding to the shooting.

[0159] In a possible implementation, the apparatus further comprises a synchronization module, configured to: send a synchronization signal to the display screen and any one of the plurality of shooting devices through a synchronization controller; the display module 1601 is further configured to: the display screen displays the internal view cone picture of the first time segment corresponding to the shooting device in response to the synchronization signal, and at the same time, the shooting device starts shooting in response to the synchronization signal.

[0160] In a possible implementation, the display module 1601 is further configured to: display the plurality of internal view cone pictures on the display screen in sequence, so that the display frame rate of the internal view cone picture corresponding to any one of the plurality of shooting devices on the display screen is the same as the shooting frame rate of the shooting device.

[0161] In a possible implementation, the first display period further includes at least one second time segment; and during the second time segment, the shooting device does not perform shooting.

[0162] In a possible implementation, any one of the at least one second time segment displays one internal view cone picture of the plurality of internal view cone pictures; and the display module 1601 is configured to: display the plurality of internal view cone pictures on the display screen in sequence according to the order of each first time segment and the order of each second time segment in the first display period, so that the internal view cone pictures corresponding to each shooting device are alternately displayed in the first display period.

[0163] In a possible implementation, the time length of the first time segment and the second time segment in which the same internal view cone picture is displayed in the first display period is the same.

[0164] In a possible implementation, the plurality of shooting devices are two shooting devices, and the plurality of internal view cone pictures are two internal view cone pictures; the first display period includes two first time segments and two second time segments; and the two second time segments are located between the two first time segments, and the two second time segments display the two internal view cone pictures respectively, so that the internal view cone pictures corresponding to the two shooting devices are alternately displayed in the first display period.

[0165] In a possible implementation, the apparatus further includes an acquisition module configured to: render, by a rendering device, based on the parameter data of the plurality of photographing devices and the original to-be-rendered data, the inner frustum pictures corresponding to each photographing device to be displayed in the first display period, wherein the number of inner frustum pictures corresponding to any photographing device is related to the photographing frame rate of the photographing device; splice the inner frustum pictures corresponding to each photographing device to be displayed in the first display period to obtain a spliced image; split the spliced image by the display screen to obtain the plurality of inner frustum pictures; or render, by a rendering device, based on the parameter data of the plurality of photographing devices and the original to-be-rendered data, the inner frustum pictures corresponding to each photographing device to be displayed in the first display period, wherein the number of inner frustum pictures corresponding to any photographing device is related to the photographing frame rate of the photographing device; and transmit, in a multi-channel transmission manner, the inner frustum pictures corresponding to each photographing device to be displayed in the first display period to the display screen, so that the display screen obtains the plurality of inner frustum pictures.

[0166] In a possible implementation, the acquisition module is further configured to: determine whether there is an overlapping area in different inner frustum pictures to be displayed in the first display period; if there is an overlapping area, determine, for any overlapping area, the gray value of the overlapping area in the corresponding inner frustum pictures; calculate a gray compensation value corresponding to the first display period based on the gray values of the overlapping area in the corresponding inner frustum pictures; and perform gray compensation on the inner frustum pictures corresponding to the overlapping area based on the gray compensation value corresponding to the first display period.

[0167] In a possible implementation, the acquisition module is configured to: acquire a gray compensation value corresponding to a historical display period; calculate a mean value of the gray compensation value corresponding to the historical display period and the gray compensation value corresponding to the first display period; and perform gray compensation on the inner frustum pictures corresponding to the overlapping area based on the mean value.

[0168] In a possible implementation, the photographing frame rates of any two photographing devices in the plurality of photographing devices are the same or in a multiple relationship.

[0169] In some embodiments, the apparatus provided by the embodiments of the present disclosure has functions or includes modules that can be used to execute the methods described in the above method embodiments, and the specific implementation can refer to the description of the above method embodiments. For brevity, it will not be described here.

[0170] The embodiments of the present disclosure also provide a computer-readable storage medium having stored thereon computer program instructions, which, when executed by a processor, implement the above method. The computer-readable storage medium can be a volatile or non-volatile computer-readable storage medium.

[0171] The embodiment of the present disclosure further provides an electronic device, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to implement the above method when executing the instructions stored in the memory.

[0172] The embodiment of the present disclosure further provides a computer program product, comprising computer readable code or a non-volatile computer readable storage medium carrying computer readable code, when the computer readable code is run in the processor of the electronic device, the processor in the electronic device executes the above method.

[0173] Figure 17 A block diagram of an electronic device 1900 according to an embodiment of the present disclosure is shown. For example, the device 1900 can be provided as a server or a terminal device. Referring to Figure 17 , the device 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions executable by the processing component 1922, such as an application program. The application program stored in the memory 1932 can include one or more than one module each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute the instructions to perform the above method.

[0174] The device 1900 can also include a power supply component 1926 configured to perform power management of the device 1900, a wired or wireless network interface 1950 configured to connect the device 1900 to a network, and an input / output interface 1958 (I / O interface). The device 1900 can operate based on an operating system stored in the memory 1932, such as Windows Server TM , MacOS X TM , Unix TM , Linux TM , FreeBSD TM or the like.

[0175] In an exemplary embodiment, a non-volatile computer readable storage medium, such as the memory 1932 including computer program instructions executable by the processing component 1922 of the device 1900 to complete the above method, is also provided.

[0176] The present disclosure can be a system, a method, and / or a computer program product. The computer program product can include a computer readable storage medium having computer readable program instructions embodied therewith, which are executable by a processor to implement various aspects of the present disclosure.

[0177] Computer readable storage media can be tangible storage media which can retain and store instructions for use by an instruction execution device. Computer readable storage media can be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer readable storage media include the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or raised structures in a groove having instructions recorded thereon, and any suitable combination of the foregoing. A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber-optic cable), or electrical signals transmitted through a wire.

[0178] Computer readable program instructions described herein can be downloaded to respective computing / processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and / or a wireless network. The network can comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing / processing device.

[0179] Computer readable program instructions for carrying out operations of the present disclosure can be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer readable program instructions can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate array (FPGA), or programmable logic array (PLA) can execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present disclosure.

[0180] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0181] These computer readable program instructions can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. These computer readable program instructions can also be stored in a computer readable storage medium that can include random access memory (RAM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), flash memory or other data storage device. When the computer readable program instructions are loaded into the computer and other programmable data processing apparatus, a series of operational steps are implemented that provide processes such that the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0182] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer, other programmable data processing apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0183] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer, other programmable data processing apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0184] Embodiments of the present disclosure have been described above, and the description is intended to be illustrative, and not restrictive, of the disclosed embodiments. Many modifications and variations of the described embodiments are possible, and all such modifications and variations are intended to be within the scope of the described embodiments. The description used herein is intended to best explain the principles of the various embodiments, the practical application, and the best mode of using the present disclosure, and to enable others skilled in the art to understand the disclosure, various embodiments, and / or various implementations detailed herein. Any terminology used herein should not be considered limiting of the disclosure, various embodiments, and / or various implementations detailed herein.

Claims

1. A virtual photographing method characterized by comprising: The method is applied to virtual shooting by using multiple shooting devices, and the method comprises the following steps: In a first display period, multiple internal view cone pictures are displayed on a display screen in time division; wherein, any shooting device in the multiple shooting devices corresponds to at least one internal view cone picture in the multiple internal view cone pictures, and different shooting devices correspond to different internal view cone pictures; the internal view cone picture corresponding to any shooting device is displayed in a first time segment corresponding to the shooting device in the first display period, and the internal view cone pictures corresponding to different shooting devices are displayed in different first time segments; the first display period further comprises at least one second time segment; wherein, during the second time segment, none of the multiple shooting devices performs shooting; one internal view cone picture in the multiple internal view cone pictures is displayed in any second time segment in the at least one second time segment; Each shooting device in the multiple shooting devices performs shooting in the corresponding first time segment, so that each shooting device shoots the corresponding internal view cone picture. The method further comprises the following steps: It is judged whether there is an overlapping area in different internal view cone pictures to be displayed in the first display period; If there is an overlapping area, for any overlapping area, the gray value of the overlapping area in the corresponding internal view cone picture is determined; Based on the gray value of the overlapping area in the corresponding internal view cone picture, a gray compensation value corresponding to the first display period is calculated; The internal view cone pictures corresponding to the overlapping area are compensated in gray based on the gray compensation value corresponding to the first display period.

2. The method of claim 1, wherein, The starting moment of each shooting of any shooting device in the multiple shooting devices in the first display period is not earlier than the starting moment of the first time segment corresponding to the shooting, and the time length of each shooting is not longer than the time length of the first time segment corresponding to the shooting.

3. The method of claim 1, wherein, The method further comprises the following steps: a synchronization controller sends a synchronization signal to the display screen and any shooting device in the multiple shooting devices respectively; The display screen displays the internal view cone picture of the first time segment corresponding to the shooting device in response to the synchronization signal, and at the same time, the shooting device starts shooting in response to the synchronization signal.

4. The method of claim 1, wherein, The multiple internal view cone pictures are displayed on the display screen in time division, which comprises the following steps: The multiple internal view cone pictures are displayed on the display screen in sequence, so that the display frame rate of the internal view cone picture corresponding to any shooting device in the multiple shooting devices on the display screen is the same as the shooting frame rate of the shooting device.

5. The method of claim 1, wherein, The multiple internal view cone pictures are displayed on the display screen in time division, which comprises the following steps: According to the sequence of each first time segment and the sequence of each second time segment in the first display period, the multiple internal view cone pictures are displayed on the display screen in sequence, so that the internal view cone picture corresponding to each shooting device is displayed alternately in the first display period.

6. The method of claim 5, wherein, The time length of the first time segment and the second time segment of the same internal view cone picture in the first display period is the same.

7. The method of claim 6, wherein, The multiple shooting devices are two shooting devices, and the multiple inner view cone pictures are two inner view cone pictures; the first display period includes two first time segments and two second time segments; the two second time segments are located between the two first time segments, and the two second time segments display the two inner view cone pictures respectively, so that the inner view cone pictures corresponding to the two shooting devices are alternately displayed in the first display period.

8. The method of claim 1, wherein, The method further comprises: rendering, by a rendering device, based on the parameter data of the multiple shooting devices and original rendering data to be rendered, to obtain the inner view cone pictures corresponding to each shooting device to be displayed in the first display period; wherein the number of the inner view cone pictures corresponding to any shooting device is related to the shooting frame rate of the shooting device; stitching the inner view cone pictures corresponding to each shooting device to be displayed in the first display period to obtain a stitched image; splitting the stitched image by the display screen to obtain the multiple inner view cone pictures; or rendering, by a rendering device, based on the parameter data of the multiple shooting devices and original rendering data to be rendered, to obtain the inner view cone pictures corresponding to each shooting device to be displayed in the first display period; wherein the number of the inner view cone pictures corresponding to any shooting device is related to the shooting frame rate of the shooting device; transmitting the inner view cone pictures corresponding to each shooting device to be displayed in the first display period to the display screen in a multi-channel transmission manner, so that the display screen obtains the multiple inner view cone pictures.

9. The method of claim 1, wherein, The gray scale compensation of each inner view cone picture corresponding to the overlapping area based on the gray scale compensation value corresponding to the first display period comprises: obtaining a gray scale compensation value corresponding to a historical display period; calculating the mean value of the gray scale compensation value corresponding to the historical display period and the gray scale compensation value corresponding to the first display period; gray scale compensating each inner view cone picture corresponding to the overlapping area based on the mean value.

10. The method of claim 1, wherein, The shooting frame rates of any two shooting devices in the multiple shooting devices are the same or in a multiple relationship.

11. A virtual camera, characterized by comprising: The device is applied to virtual shooting by using multiple shooting devices, and the device comprises: a display module configured to display multiple inner view cone pictures on a display screen in a time-sharing manner in a first display period; wherein any shooting device in the multiple shooting devices corresponds to at least one inner view cone picture in the multiple inner view cone pictures, and the inner view cone pictures corresponding to different shooting devices are different; the time-sharing display means that the inner view cone picture corresponding to any shooting device is displayed in a first time segment corresponding to the shooting device in the first display period, and the inner view cone pictures corresponding to different shooting devices are displayed in different first time segments; the first display period further includes at least one second time segment; wherein none of the multiple shooting devices performs shooting during the second time segment; and any second time segment in the at least one second time segment displays one inner view cone picture in the multiple inner view cone pictures. The display module is further configured to: determine whether different internal view cone pictures to be displayed in the first display period have an overlapping area; if the overlapping area exists, determine, for any overlapping area, a gray value of the overlapping area in each corresponding internal view cone picture; calculate a gray compensation value corresponding to the first display period based on the gray values of the overlapping area in each corresponding internal view cone picture; and perform gray compensation on each internal view cone picture corresponding to the overlapping area based on the gray compensation value corresponding to the first display period. The photographing module is configured to perform photographing by each photographing device in the plurality of photographing devices in a corresponding first time segment, so that each photographing device photographs a corresponding internal view cone picture.

12. An electronic device, comprising: The apparatus comprises: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to implement the method of any one of claims 1 to 10 when executing the instructions stored in the memory.

13. A non-transitory computer readable storage medium having stored thereon computer program instructions, wherein, The computer program instructions, when executed by the processor, implement the method of any one of claims 1 to 10.

Images