Picture frame image processing method, control device and storage medium

Abstract

The embodiment of the invention provides a picture frame image processing method, control equipment and a storage medium, relates to the technical field of image processing, is applied to the control equipment, and comprises the following steps: controlling a phase synchronization signal generator to send the same phase synchronization signal to at least two cameras and a video server; performing time sequence synchronization on the display time sequence of the plurality of picture frame images and the phase signals of the corresponding cameras to obtain a time sequence synchronization result; and sending the plurality of picture frame images to a display screen according to a time sequence synchronization result, so that the display time sequence of each picture frame image in the overlapping region is consistent with the shutter exposure time sequence of the camera of the corresponding inner view cone in the display process. By adopting the technical scheme, the display process of the spliced picture frame image is processed, and the view finding range of multi-camera shooting does not need to be considered, so that the limitation of the view finding range of multi-camera shooting is relieved, and the virtual manufacturing efficiency is improved.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of image processing technology, and in particular to a frame image processing method, a control device, and a storage medium. Background Art

[0002] In curtain wall-based virtual film and television production, the camera's on-screen viewing area is called the "inner cone," while the area outside the camera's field of view is called the "outer cone." The "inner cone" image is rendered in real time based on the camera's viewpoint to ensure accurate perspective. In this scenario, if two or more cameras simultaneously capture the same area of ​​the curtain wall, their "inner cones" will overlap and block each other. This means that only one camera can capture the correct "inner cone" image, while the others, due to their blocked "inner cones," will be unable to capture the correct image.

[0003] Therefore, in traditional virtual shooting, when using multiple cameras to shoot simultaneously, the photographer must carefully control the shooting range of each camera to prevent overlapping occlusions between the "inner cones", which limits the shooting freedom.

[0004] Therefore, there is an urgent need for a frame image processing method that can use any number of cameras to frame the scene within the refresh rate range allowed by the screen, thereby removing the framing range restrictions of multi-camera shooting and improving the efficiency of virtual production. Summary of the Invention

[0005] In order to solve the above technical problems or at least partially solve the above technical problems, the embodiments of the present disclosure provide a picture frame image processing method, a control device and a storage medium.

[0006] A first aspect of an embodiment of the present disclosure provides a method for processing a picture frame, which is applied to a control device. The method includes:

[0007] A phase synchronization signal generator is controlled to send a same phase synchronization signal to at least two cameras and a video server; wherein the same phase synchronization signal is used to control the timing synchronization of the at least two cameras and the video server; the at least two cameras are used to perform a shutter exposure action after receiving the same phase synchronization signal; the inner viewing cones of the at least two cameras have an overlapping area; the video server obtains a stitched picture frame image from a rendering server after receiving the same phase synchronization signal; the stitched picture frame image is obtained by the rendering server rendering and stitching multiple picture frame images; each picture frame image is a composite picture of the inner viewing cone and outer viewing cone of the corresponding camera;

[0008] Performing timing synchronization on the display timing of the plurality of frame images and the phase signal of the corresponding camera to obtain a timing synchronization result;

[0009] The multiple frame images are sent to the display screen according to the result of the timing synchronization, so that during the display process, the display timing of each frame image in the overlapping area is consistent with the shutter exposure timing of the camera in the corresponding inner viewing cone.

[0010] A second aspect of the present disclosure provides a method for processing a frame image, which is applied to an image processing system, wherein the image processing system includes a video server, a rendering server, and at least two cameras; the method includes:

[0011] Generate multiple frame images in real time by the rendering server, render and stitch the multiple frame images to obtain a stitched frame image;

[0012] After acquiring the stitched frame image, the video server shifts each of the stitched frame images according to a displacement offset corresponding to the frame image to obtain a plurality of frame images; wherein the displacement offset corresponding to the frame image is a preset displacement of the frame image at a stitching sequence number of the stitched frame image minus 1 times; wherein the preset displacement is a frame image resolution when the rendering server renders the frame images;

[0013] The video server sequentially sends the plurality of frame images to a display screen in a preset order; wherein the display screen receives the frame images and displays the frame images;

[0014] When each camera receives the same phase synchronization signal, the initial phase offset of the camera is adjusted to the target phase offset of the camera according to the corresponding relationship between the exposure timing of each camera and the display timing of the frame image; wherein each camera has a different framing range;

[0015] The exposure time of the camera is controlled according to the target phase offset of the camera, and the frame image displayed on the display screen is photographed so that the exposure timing of the camera is consistent with the display timing of the corresponding frame image within the camera's viewing range.

[0016] A third aspect of the present disclosure provides a control device, the control device comprising:

[0017] A control module is configured to control a phase synchronization signal generator to send a same phase synchronization signal to at least two cameras and a video server; wherein the same phase synchronization signal is used to control the timing synchronization of the at least two cameras and the video server; the at least two cameras are configured to perform a shutter exposure action after receiving the same phase synchronization signal; the inner viewing cones of the at least two cameras have an overlapping area; the video server obtains a stitched picture frame image from a rendering server after receiving the same phase synchronization signal; the stitched picture frame image is obtained by the rendering server rendering and stitching multiple picture frame images; each picture frame image is a composite picture of the inner viewing cone and outer viewing cone of the corresponding camera;

[0018] A synchronization module, configured to synchronize the display timing of the plurality of frame images with the phase signal of the corresponding camera to obtain a timing synchronization result;

[0019] A sending module is used to send the multiple picture frame images to the display screen according to the result of the timing synchronization, so that during the display process, the display timing of each of the picture frame images in the overlapping area is consistent with the shutter exposure timing of the camera of the corresponding inner cone.

[0020] A fourth aspect of the embodiments of the present disclosure provides an image processing system, the image processing system comprising a video server, a rendering server, and at least two cameras; the image processing system comprises:

[0021] The rendering server is used to generate multiple picture frame images in real time, and render and stitch the multiple picture frame images to obtain a stitched picture frame image;

[0022] The video server is configured to, after acquiring the stitched frame image, shift each of the frame images in the stitched frame image according to a displacement offset corresponding to the frame image, thereby obtaining a plurality of frame images; wherein the displacement offset corresponding to the frame image is a preset displacement of the frame image by a stitching sequence number of the stitched frame image minus 1 times; wherein the preset displacement is a frame image resolution when the rendering server renders the frame images;

[0023] The video server is used to send the plurality of picture frame images to the display screen in sequence according to a preset order; wherein the display screen receives the picture frame images and displays the picture frame images;

[0024] Each of the cameras is configured to adjust the initial phase offset of the camera to the target phase offset of the camera according to the corresponding relationship between the exposure timing of each camera and the display timing of the frame image when receiving the same phase synchronization signal; wherein each of the cameras has a different framing range;

[0025] The exposure time of the camera is controlled according to the target phase offset of the camera, and the frame image displayed on the display screen is photographed so that the exposure timing of the camera is consistent with the display timing of the corresponding frame image within the camera's viewing range.

[0026] A fifth aspect of the embodiments of the present disclosure provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the method of the first aspect or the second aspect can be implemented.

[0027] An embodiment of the present disclosure provides a method for processing a picture frame image, which is applied to a control device, the method comprising: controlling a phase synchronization signal generator to send a same phase synchronization signal to at least two cameras and a video server; wherein the same phase synchronization signal is used to control the timing synchronization of the at least two cameras and the video server; the at least two cameras are used to perform a shutter exposure action after receiving the same phase synchronization signal; there is an overlapping area between the inner viewing cones of the at least two cameras; after receiving the same phase synchronization signal, the video server obtains a stitched picture frame image from a rendering server; the stitched picture frame image is obtained by the rendering server rendering and stitching multiple picture frame images; each picture frame image is a composite picture of the inner viewing cone and the outer viewing cone of the corresponding camera; the display timing of the multiple picture frame images is synchronized with the phase signal of the corresponding camera to obtain a timing synchronization result; and the multiple picture frame images are sent to a display screen according to the timing synchronization result, so that during the display process, the display timing of each picture frame image in the overlapping area is consistent with the shutter exposure timing of the camera of the corresponding inner viewing cone. By adopting this technical solution, since multiple frame images taken by multiple cameras are stitched together and the display process of the stitched frame images is processed, there is no need to consider the framing range of multi-camera shooting, thereby removing the limitation of the framing range of multi-camera shooting and improving the efficiency of virtual production. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0029] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0030] Figure 1 This is a schematic diagram of overlapping occlusion of inner viewing cones when shooting with multiple cameras, provided by an embodiment of the present disclosure;

[0031] Figure 2 This is a structural diagram of a picture frame image processing scenario provided by an embodiment of the present disclosure;

[0032] Figure 3 This is a flow chart of a method for processing a frame image provided by an embodiment of the present disclosure;

[0033] Figure 4 This is a flow chart of a method for processing a frame image provided by an embodiment of the present disclosure;

[0034] Figure 5 This is a flow chart of a method for processing a frame image provided by an embodiment of the present disclosure;

[0035] Figure 6 This is a schematic diagram of a timeline of multi-camera shooting provided by an embodiment of the present disclosure;

[0036] Figure 7 is a structural diagram of a control device provided by an embodiment of the present disclosure;

[0037] Figure 8 It is a structural diagram of an image processing system provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0038] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features therein can be combined with each other in the absence of conflict.

[0039] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0040] In a virtual film and television production based on a curtain wall, if multiple cameras are shooting the same area of ​​the screen at the same time, the inner cones of these cameras will overlap and block each other. In this case, only one camera can capture the correct inner cone image, while the other cameras cannot capture the correct image because their inner cones are blocked. For a clearer explanation, please refer to Figure 1 A schematic diagram showing overlapping occlusion of inner viewing cones when shooting with multiple cameras is shown.

[0041] The method in this embodiment is implemented based on a frame image processing system. Figure 2 The structural diagram of a picture frame image processing scenario is shown in FIG. Figure 2 It can be seen that the picture frame image processing scenario is realized by the image processing system, the phase synchronization signal generator, the control device and the display screen.

[0042] Now, the following will be described with reference to specific embodiments. Figure 3 FIG. 1 is a flow chart of a method for processing a frame image provided by an embodiment of the present disclosure, which can be executed by a control device. Figure 3 As shown, the method provided in this embodiment includes the following steps:

[0043] S301, control the phase synchronization signal generator to send the same phase synchronization signal to at least two cameras and a video server; wherein the same phase synchronization signal is used to control the timing synchronization of the at least two cameras and the video server; the at least two cameras are used to perform a shutter exposure action after receiving the same phase synchronization signal; there is an overlapping area between the inner viewing cones of the at least two cameras; the video server obtains a stitched picture frame image from the rendering server after receiving the same phase synchronization signal; the stitched picture frame image is obtained after the rendering server renders and stitches multiple picture frame images; each picture frame image is a composite picture of the inner viewing cone and the outer viewing cone of the corresponding camera.

[0044] In one example, a phase synchronization signal generator simultaneously sends the same phase synchronization signal to a video server and at least two cameras to control the timing synchronization between the at least two cameras and the video server. Specifically, after receiving the same phase synchronization signal, the video server obtains a stitched frame image from a rendering server. The at least two cameras are: camera A, camera B, and camera C. After receiving the phase synchronization signal, camera A performs its shutter exposure action; after receiving the phase synchronization signal, camera B performs its shutter exposure action; and after receiving the phase synchronization signal, camera C performs its shutter exposure action.

[0045] In one example, there is an overlapping area between the inner viewing cones of at least two cameras.

[0046] In one example, each frame image is a composite image of the inner and outer viewing cones of the corresponding camera. For example, frame image A is a composite image of the inner and outer viewing cones of camera A, and frame image B is a composite image of the inner and outer viewing cones of camera B. A stitched frame image is obtained by rendering and stitching multiple frame images on a rendering server. For example, a stitched frame image can be obtained by stitching frame image A and frame image B.

[0047] Furthermore, the rendering server's rendering and stitching of multiple frame images runs in parallel with the video server's acquisition of stitched frame images from the rendering server upon receiving a phase synchronization signal. In other words, there's no coupling between the rendering server and the video server. This setup offers the advantage of requiring sub-microsecond timing accuracy for the phase signal controlling the camera shutter, while the rendering server is limited by rendering delays of up to 10-30 milliseconds, far from meeting the response speed and accuracy requirements for frame switching.

[0048] S302 : Perform timing synchronization on the display timing of the plurality of frame images and the phase signal of the corresponding camera to obtain a timing synchronization result.

[0049] In one example, the plurality of frame images may be frame image A and frame image B, wherein frame image A is a composite image of the inner and outer viewing cones of camera A, and frame image A corresponds to camera A. A display timing of frame image A is time-synchronized with a phase signal of camera A. Frame image B is a composite image of the inner and outer viewing cones of camera B, and frame image B corresponds to camera B. A display timing of frame image B is time-synchronized with a phase signal of camera B, and a result of the timing synchronization is that the display timing of frame image A is time-synchronized with the phase signal of camera A, and the display timing of frame image B is time-synchronized with the phase signal of camera B.

[0050] S303 , sending the plurality of frame images to the display screen according to the result of the timing synchronization, so that during the display process, the display timing of each frame image in the overlapping area is consistent with the shutter exposure timing of the camera of the corresponding inner viewing cone.

[0051] In one example, picture frame image A and picture frame image B are sent to the display screen in sequence so that during the display process, the display timing of picture frame image A is consistent with the shutter exposure timing of camera A, and the display timing of picture frame image B is consistent with the shutter exposure timing of camera B.

[0052] The disclosed embodiment provides a frame image processing method for use in a control device, the method comprising: controlling a phase synchronization signal generator to send the same phase synchronization signal to at least two cameras and a video server; performing timing synchronization on the display timing of multiple frame images with the phase signals of the corresponding cameras to obtain a timing synchronization result; and sending the multiple frame images to a display screen according to the timing synchronization result, so that during the display process, the display timing of each frame image in the overlapping area is consistent with the shutter exposure timing of the camera of the corresponding inner cone. By adopting this technical solution, since multiple frame images captured by multiple cameras are spliced ​​together and the display process of the spliced ​​frame images is processed, there is no need to consider the framing range of multi-camera shooting, thereby removing the limitation of the framing range of multi-camera shooting and improving the efficiency of virtual production.

[0053] Figure 4 The following is a flow chart of a method for processing a frame image provided by an embodiment of the present disclosure. The present disclosure embodiment is optimized based on the above embodiment, and the present disclosure embodiment can be combined with various optional solutions in one or more of the above embodiments and applied to a control device.

[0054] like Figure 4 As shown, the picture frame image processing method may include the following steps:

[0055] S401, control the phase synchronization signal generator to send the same phase synchronization signal to at least two cameras and a video server; wherein the same phase synchronization signal is used to control the timing synchronization of the at least two cameras and the video server; the at least two cameras are used to perform a shutter exposure action after receiving the same phase synchronization signal; there is an overlapping area between the inner viewing cones of the at least two cameras; the video server obtains a stitched picture frame image from the rendering server after receiving the same phase synchronization signal; the stitched picture frame image is obtained after the rendering server renders and stitches multiple picture frame images; each picture frame image is a composite picture of the inner viewing cone and the outer viewing cone of the corresponding camera.

[0056] In one example, at least two cameras are configured to perform shutter exposure after receiving the same phase synchronization signal, including:

[0057] Each camera is configured to adjust an initial phase offset of the camera to a target phase offset of the camera according to a corresponding relationship between the exposure timing of each camera and the display timing of the frame image when receiving the same phase synchronization signal; wherein each camera has a different framing range;

[0058] Each camera is used to control the camera's exposure time according to the camera's target phase offset, and shoot the frame image displayed on the display screen, so that the camera captures the corresponding frame image within the camera's field of view; wherein the camera's exposure time is not greater than the display refresh interval of the display screen.

[0059] In one example, for clarity, camera A is used as an example. The exposure timing of camera A corresponds to the display timing of frame image A. The initial phase offset of camera A is adjusted to the target phase offset of camera A according to the corresponding relationship.

[0060] After obtaining the target phase offset of camera A, the exposure time of camera A is controlled according to the target phase offset of camera A, and the frame image A displayed on the display screen is photographed so that camera A only captures frame image A when photographing. To ensure that camera A only captures frame image A when photographing, the exposure time of camera A needs to be set to no longer than the display refresh interval of the display screen.

[0061] S402 : Perform timing synchronization on the display timing of the plurality of frame images and the phase signal of the corresponding camera to obtain a timing synchronization result.

[0062] In an example, this step may refer to the content of step S302.

[0063] S403 , sending the plurality of frame images to the display screen according to the result of the timing synchronization, so that during the display process, the display timing of each frame image in the overlapping area is consistent with the shutter exposure timing of the camera of the corresponding inner viewing cone.

[0064] In an example, this step may refer to the content of step S303.

[0065] S404: The display screen is used to receive the picture frame image and display the picture frame image; wherein the duration for the display screen to display the picture frame image is the display refresh interval duration.

[0066] In one example, after receiving picture frame image A, the display screen displays picture frame image A; after receiving picture frame image B, the display screen displays picture frame image B; and after receiving picture frame image C, the display screen displays picture frame image C. Specifically, the display duration of picture frame image A is the display refresh interval duration of the display screen. The display duration of picture frame image B is the display refresh interval duration of the display screen. The display duration of picture frame image C is the display refresh interval duration of the display screen.

[0067] The disclosed embodiment provides a method for processing a frame image, which is applied to a control device. The method includes: each camera is used to adjust the camera's initial phase offset to the camera's target phase offset according to the corresponding relationship between each camera's exposure timing and the display timing of the frame image when receiving the same phase synchronization signal; each camera is used to control the camera's exposure time according to the camera's target phase offset, and shoot the frame image displayed on the display screen, so that the camera captures the corresponding frame image within the camera's field of view. Using this technical solution, each camera opens the shutter for exposure in sequence according to the phase signal, and each camera can just capture its corresponding inner cone within the exposure time. This achieves the separate display and shooting of multiple overlapping inner cones in the time domain.

[0068] Figure 5 This is a flow chart of a method for processing a frame image provided by an embodiment of the present disclosure. The method can be executed by an image processing system, which includes a video server, a rendering server, and at least two cameras. Figure 5 As shown, the method provided in this embodiment includes the following steps:

[0069] S501 : Generate multiple frame images in real time through a rendering server, render and stitch the multiple frame images, and obtain stitched frame images.

[0070] In one example, each frame image is obtained by synthesizing the inner and outer view cones of a camera. For the sake of convenience, let's assume that a camera is identified as camera A. The spatial position, orientation, and viewing range of camera A are first received. The outer view cone and inner view cone A are then rendered. The rendered inner view cone A and the rendered outer view cone are then synthesized to generate frame image A. For another example, let's assume that a camera is identified as camera B. The spatial position, orientation, and viewing range of camera B are first received. The outer view cone and inner view cone B are then rendered. The rendered inner view cone B and the rendered outer view cone are then synthesized to generate frame image B. That is, different cameras have different inner view cones, but the outer view cones are the same.

[0071] In this embodiment, if there are three picture frame images, the three picture frame images need to be spliced ​​to obtain a spliced ​​picture frame image.

[0072] S502. After obtaining the stitched picture frame image, the video server shifts the position of each picture frame image in the stitched picture frame image according to the displacement offset corresponding to the picture frame image to obtain multiple picture frame images; wherein the displacement offset corresponding to the picture frame image is a preset displacement of the picture frame image at a stitching sequence number of the stitched picture frame image minus 1 times; wherein the preset displacement amount is the picture frame image resolution when the rendering server renders the picture frame image.

[0073] In one example, the preset displacement is the picture frame image resolution when the rendering server renders the picture frame image. The picture frame image resolution is pre-set. The stitching sequence number of the picture frame image in the stitching picture frame image refers to the sequence number of the picture frame image when stitching. For example, the stitching picture frame image is stitched according to picture frame image A, picture frame image B and picture frame image C, then the stitching sequence number of picture frame image A in the stitching picture frame image is 1, the stitching sequence number of picture frame image B in the stitching picture frame image is 2, and the stitching sequence number of picture frame image C in the stitching picture frame image is 3. Then the displacement offset corresponding to picture frame image A is 0 times the preset displacement, the displacement offset corresponding to picture frame image B is 1 times the preset displacement, and the displacement offset corresponding to picture frame image C is 2 times the preset displacement.

[0074] In one example, after obtaining the stitched frame image, the video server shifts the position of each frame image in the stitched frame image according to the displacement offset corresponding to the frame image to obtain multiple frame images, including:

[0075] The initial displacement of the i-th frame image in the spliced ​​frame image is shifted by i-1 times the preset displacement to obtain the target displacement of the i-th frame image; wherein i is a positive integer;

[0076] The i-th frame image is displayed according to the target displacement to obtain the i-th frame image.

[0077] In this embodiment, i is the stitching sequence number of the picture frame image in the stitching picture frame image.

[0078] For example, if i=1, the initial displacement of the first frame image in the stitched frame image is shifted by 0 times the preset displacement, that is, the initial displacement of the first frame image is not shifted, and the target displacement of the first frame image is obtained, and then the first frame image is displayed at this position. If i=2, the initial displacement of the second frame image in the stitched frame image is shifted by 1 times the preset displacement, and the target displacement of the second frame image is obtained, and then the second frame image is displayed at this position.

[0079] In this embodiment, the direction of movement is related to the direction in which the frame images of the stitched picture are stitched together, wherein the direction in which the frame images of the stitched picture are stitched together can be horizontal, vertical, or any other direction.

[0080] S503: Sending a plurality of frame images to a display screen in sequence according to a preset order via a video server; wherein the display screen receives the frame images and displays the frame images.

[0081] In one example, the preset sequence is that the stitching sequence number of the frame images when stitching the frame images refers to the sequence number of the frame images when stitching. Then, the video server first sends frame image A to the display screen, then sends frame image B to the display screen, and finally sends frame image C to the display screen. After receiving frame image A, the display screen displays frame image A; after receiving frame image B, it displays frame image B; and after receiving frame image C, it displays frame image C.

[0082] S504. When each camera receives the same phase synchronization signal, the initial phase offset of each camera is adjusted to the target phase offset of the camera according to the corresponding relationship between the exposure timing of each camera and the display timing of the picture frame image; wherein the framing range of each camera is different.

[0083] In one example, when each camera receives the same phase synchronization signal, the initial phase offset of each camera is adjusted to the target phase offset of the camera according to the corresponding relationship between the exposure timing of each camera and the display timing of the picture frame image, including:

[0084] The initial phase offset of the i-th camera is adjusted according to i-1 times the phase offset reference value to obtain the target phase offset of the i-th camera; wherein the phase offset reference value is equal to the display refresh interval of the display screen.

[0085] In one example, the initial phase offset of the first camera is adjusted by 0 times the phase offset reference value to obtain the target phase offset of the first camera. The initial phase offset of the second camera is adjusted by 1 times the phase offset reference value to obtain the target phase offset of the second camera. The initial phase offset of the third camera is adjusted by 2 times the phase offset reference value to obtain the target phase offset of the third camera.

[0086] In this embodiment, the phase offset reference value is equal to the display refresh interval of the display screen.

[0087] S505 , controlling the exposure time of the camera according to the target phase offset of the camera, and shooting the frame images displayed on the display screen, so that the exposure timing of the camera is consistent with the display timing of the corresponding frame images within the camera's framing range.

[0088] In one example, controlling the exposure time of a camera according to a target phase offset of the camera to capture a frame image displayed on a display screen includes:

[0089] The exposure time of the i-th camera is controlled according to the target phase offset of the i-th camera, and the i-th frame image displayed on the display screen is captured; wherein the duration of the i-th frame image displayed on the display screen is the display refresh interval duration of the display screen.

[0090] In one example, since the determination process of the target phase offset of the camera is related to the display refresh interval of the display screen, the exposure timing of the camera can be made consistent with the display timing of the corresponding frame image within the camera's viewing range.

[0091] Specifically, the exposure time of the first camera is controlled according to the target phase offset of the first camera, and the first frame image displayed on the display screen is photographed, so that the first camera can only photograph the first frame image displayed on the display screen.

[0092] For details, see Figure 6 A schematic diagram of a timeline of multi-camera shooting is shown.

[0093] The disclosed embodiments provide a frame image processing method applicable to multiple cameras, the method comprising: generating multiple frame images in real time via a rendering server, rendering and stitching the multiple frame images to obtain a stitched frame image, shifting each frame image in the stitched frame image according to a displacement offset corresponding to the frame image after obtaining the stitched frame image via a video server to obtain multiple frame images, and sequentially sending the multiple frame images to a display screen via the video server in a preset order, adjusting the initial phase offset of each camera to a target phase offset of the camera according to the corresponding relationship between the exposure timing of each camera and the display timing of the frame images when each camera receives the same phase synchronization signal; controlling the exposure time of the camera according to the target phase offset of the camera, and capturing the frame images displayed on the display screen so that the exposure timing of the camera is consistent with the display timing of the corresponding frame images within the camera's field of view. The present technical solution can achieve the purpose of displaying multiple spatially overlapping inner cones separately in the temporal domain.

[0094] Figure 7This is a schematic diagram of the structure of a control device provided by an embodiment of the present disclosure. Figure 7 As shown, the control device 70 includes:

[0095] Control module 701 is used to control a phase synchronization signal generator to send the same phase synchronization signal to at least two cameras and a video server; wherein the same phase synchronization signal is used to control the timing synchronization of the at least two cameras and the video server; the at least two cameras are used to perform a shutter exposure action after receiving the same phase synchronization signal; there is an overlapping area between the inner viewing cones of the at least two cameras; after receiving the same phase synchronization signal, the video server obtains a stitched picture frame image from the rendering server; the stitched picture frame image is obtained by rendering and stitching multiple picture frame images by the rendering server; each picture frame image is a composite picture of the inner viewing cone and the outer viewing cone of the corresponding camera.

[0096] The synchronization module 702 is used to synchronize the display timing of multiple frame images with the phase signal of the corresponding camera to obtain a timing synchronization result.

[0097] The sending module 703 is used to send multiple frame images to the display screen according to the result of timing synchronization, so that during the display process, the display timing of each frame image in the overlapping area is consistent with the shutter exposure timing of the camera of the corresponding inner cone.

[0098] In one example, at least two cameras are configured to perform shutter exposure after receiving the same phase synchronization signal, including:

[0099] Each camera is configured to adjust an initial phase offset of the camera to a target phase offset of the camera according to a corresponding relationship between the exposure timing of each camera and the display timing of the frame image when receiving the same phase synchronization signal; wherein each camera has a different framing range;

[0100] Each camera is used to control the camera's exposure time according to the camera's target phase offset, and shoot the frame image displayed on the display screen, so that the camera captures the corresponding frame image within the camera's field of view; wherein the camera's exposure time is not greater than the display refresh interval of the display screen.

[0101] In one example, after sending module 703, the control device 70 includes:

[0102] The display screen is used to receive the picture frame image and display the picture frame image; wherein, the duration for the display screen to display the picture frame image is the display refresh interval duration.

[0103] The control device provided in this embodiment can execute the method of any of the above embodiments, and its execution method and beneficial effects are similar, which will not be repeated here.

[0104] Figure 8 FIG. 1 is a schematic diagram of the structure of an image processing system provided by an embodiment of the present disclosure. Figure 8 As shown, the image processing system 80 includes a video server 801, a rendering server 802 and at least two cameras 803; the image processing system 80 includes:

[0105] The rendering server 802 is used to generate multiple frame images in real time, and perform rendering and stitching processing on the multiple frame images to obtain stitched frame images.

[0106] The video server 801 is used to, after obtaining the stitched picture frame image, shift the position of each picture frame image in the stitched picture frame image according to the displacement offset corresponding to the picture frame image, so as to obtain multiple picture frame images; wherein the displacement offset corresponding to the picture frame image is a preset displacement of the picture frame image at the stitching sequence number of the stitched picture frame image - 1 times; wherein the preset displacement is the picture frame image resolution when the rendering server renders the picture frame image.

[0107] The video server 801 is used to send a plurality of frame images to a display screen in sequence according to a preset order; wherein the display screen receives the frame images and displays the frame images.

[0108] Each camera 803 is used to adjust the initial phase offset of the camera to the target phase offset of the camera according to the corresponding relationship between the exposure timing of each camera and the display timing of the picture frame image when receiving the same phase synchronization signal; wherein the viewing range of each camera is different.

[0109] The exposure time of the camera is controlled according to the target phase offset of the camera, and the frame image displayed on the display screen is photographed so that the exposure timing of the camera is consistent with the display timing of the corresponding frame image within the camera's framing range.

[0110] In one example, the video server 801 is used to:

[0111] The initial displacement of the i-th frame image in the spliced ​​frame image is shifted by i-1 times the preset displacement to obtain the target displacement of the i-th frame image; wherein i is a positive integer;

[0112] The i-th frame image is displayed according to the target displacement to obtain the i-th frame image.

[0113] In one example, each camera 803 is used to:

[0114] The initial phase offset of the i-th camera is adjusted according to i-1 times the phase offset reference value to obtain the target phase offset of the i-th camera; wherein the phase offset reference value is equal to the display refresh interval of the display screen.

[0115] In one example, each camera 803 is used to:

[0116] The exposure time of the i-th camera is controlled according to the target phase offset of the i-th camera, and the i-th frame image displayed on the display screen is captured; wherein the duration of the i-th frame image displayed on the display screen is the display refresh interval duration of the display screen.

[0117] The image processing system provided in this embodiment can execute the method of any of the above embodiments. Its execution method and beneficial effects are similar and will not be described in detail here.

[0118] It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or component. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0119] In some embodiments, the client and server can communicate using any currently known or future developed network protocol, such as HTTP (HyperText Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0120] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0121] The computer-readable medium carries one or more programs, and when the one or more programs are executed by the electronic device, the electronic device executes the method.

[0122] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including, but not limited to, object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0123] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0124] The units involved in the embodiments described in this disclosure may be implemented in software or hardware, wherein the name of a unit does not necessarily limit the unit itself.

[0125] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0126] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0127] The embodiments of the present disclosure also provide a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the method of any of the above embodiments can be implemented. The execution method and beneficial effects are similar and will not be repeated here.

[0128] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device that includes the element.

[0129] The above are merely specific embodiments of the present disclosure, intended to enable those skilled in the art to understand and implement the present disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure is not limited to these embodiments, but is to be construed in the broadest manner consistent with the principles and novel features disclosed herein.

Claims

1. A method for processing a frame image, characterized in that: Applied to a control device, the method includes: A phase synchronization signal generator is controlled to send a same phase synchronization signal to at least two cameras and a video server; wherein the same phase synchronization signal is used to control the timing synchronization of the at least two cameras and the video server; the at least two cameras are used to perform a shutter exposure action after receiving the same phase synchronization signal; the inner viewing cones of the at least two cameras have an overlapping area; the video server obtains a stitched picture frame image from a rendering server after receiving the same phase synchronization signal; the stitched picture frame image is obtained by the rendering server rendering and stitching multiple picture frame images; each picture frame image is a composite picture of the inner viewing cone and outer viewing cone of the corresponding camera; Performing timing synchronization on the display timing of the plurality of frame images and the phase signal of the corresponding camera to obtain a timing synchronization result; The multiple frame images are sent to the display screen according to the result of the timing synchronization, so that during the display process, the display timing of each frame image in the overlapping area is consistent with the shutter exposure timing of the camera in the corresponding inner viewing cone.

2. The method according to claim 1, characterized in that The at least two cameras are configured to perform a shutter exposure action after receiving the same phase synchronization signal, including: Each of the cameras is configured to adjust the initial phase offset of the camera to the target phase offset of the camera according to the corresponding relationship between the exposure timing of each camera and the display timing of the picture frame image when receiving the same phase synchronization signal; wherein each of the cameras has a different framing range; Each of the cameras is used to control the exposure time of the camera according to the target phase offset of the camera, and shoot the frame image displayed on the display screen, so that the camera captures the frame image corresponding to the camera's field of view; wherein the exposure time of the camera is not greater than the display refresh interval of the display screen.

3. The method according to claim 1, characterized in that After sending the plurality of picture frame images to a display screen according to the result of the timing synchronization, the method includes: The display screen is used to receive the picture frame image and display the picture frame image; wherein, the duration for the display screen to display the picture frame image is the display refresh interval duration.

4. A method for processing a frame image, characterized in that: Applied to an image processing system, the image processing system includes a video server, a rendering server and at least two cameras; the method includes: Generate multiple frame images in real time by the rendering server, render and stitch the multiple frame images to obtain a stitched frame image; After acquiring the stitched frame image, the video server shifts each of the stitched frame images according to a displacement offset corresponding to the frame image to obtain a plurality of frame images; wherein the displacement offset corresponding to the frame image is a preset displacement of the frame image at a stitching sequence number of the stitched frame image minus 1 times; wherein the preset displacement is a frame image resolution when the rendering server renders the frame images; The video server sequentially sends the plurality of frame images to a display screen in a preset order; wherein the display screen receives the frame images and displays the frame images; When each camera receives the same phase synchronization signal, the initial phase offset of the camera is adjusted to the target phase offset of the camera according to the corresponding relationship between the exposure timing of each camera and the display timing of the frame image; wherein each camera has a different framing range; The exposure time of the camera is controlled according to the target phase offset of the camera, and the frame image displayed on the display screen is photographed so that the exposure timing of the camera is consistent with the display timing of the corresponding frame image within the camera's viewing range.

5. The method according to claim 4, characterized in that After the video server obtains the stitched picture frame image, each picture frame image in the stitched picture frame image is shifted according to a displacement offset corresponding to the picture frame image to obtain a plurality of picture frame images, including: Performing a position shift on the initial displacement of the i-th frame image in the stitched frame image according to i-1 times the preset displacement to obtain a target displacement of the i-th frame image; wherein i is a positive integer; The i-th picture frame image is displayed according to the target displacement to obtain the i-th picture frame image.

6. The method according to claim 4, characterized in that The adjusting, when each camera receives the same phase synchronization signal, the initial phase offset of the camera to the target phase offset of the camera according to the corresponding relationship between the exposure timing of each camera and the display timing of the picture frame image, includes: The initial phase offset of the i-th camera is adjusted according to i-1 times the phase offset reference value to obtain the target phase offset of the i-th camera; wherein the phase offset reference value is equal to the display refresh interval of the display screen.

7. The method according to claim 6, characterized in that The step of controlling the exposure time of the camera according to the target phase offset of the camera and photographing the frame image displayed on the display screen includes: The exposure time of the i-th camera is controlled according to the target phase offset of the i-th camera, and the i-th frame image displayed on the display screen is photographed; wherein the duration of the i-th frame image displayed on the display screen is the display refresh interval duration of the display screen.

8. A control device, characterized in that: The control device includes: A control module is configured to control a phase synchronization signal generator to send a same phase synchronization signal to at least two cameras and a video server; wherein the same phase synchronization signal is used to control the timing synchronization of the at least two cameras and the video server; the at least two cameras are configured to perform a shutter exposure action after receiving the same phase synchronization signal; the inner viewing cones of the at least two cameras have an overlapping area; the video server obtains a stitched picture frame image from a rendering server after receiving the same phase synchronization signal; the stitched picture frame image is obtained by the rendering server rendering and stitching multiple picture frame images; each picture frame image is a composite picture of the inner viewing cone and outer viewing cone of the corresponding camera; A synchronization module, configured to synchronize the display timing of the plurality of frame images with the phase signal of the corresponding camera to obtain a timing synchronization result; A sending module is used to send the multiple picture frame images to the display screen according to the result of the timing synchronization, so that during the display process, the display timing of each of the picture frame images in the overlapping area is consistent with the shutter exposure timing of the camera of the corresponding inner cone.

9. An image processing system, characterized in that: The image processing system includes a video server, a rendering server and at least two cameras; the image processing system includes: The rendering server is used to generate multiple picture frame images in real time, and render and stitch the multiple picture frame images to obtain a stitched picture frame image; The video server is configured to, after acquiring the stitched frame image, shift each of the frame images in the stitched frame image according to a displacement offset corresponding to the frame image, thereby obtaining a plurality of frame images; wherein the displacement offset corresponding to the frame image is a preset displacement of the frame image by a stitching sequence number of the stitched frame image minus 1 times; wherein the preset displacement is a frame image resolution when the rendering server renders the frame images; The video server is used to send the plurality of picture frame images to the display screen in sequence according to a preset order; wherein the display screen receives the picture frame images and displays the picture frame images; Each of the cameras is configured to adjust the initial phase offset of the camera to the target phase offset of the camera according to the corresponding relationship between the exposure timing of each camera and the display timing of the frame image when receiving the same phase synchronization signal; wherein each of the cameras has a different framing range; The exposure time of the camera is controlled according to the target phase offset of the camera, and the frame image displayed on the display screen is photographed so that the exposure timing of the camera is consistent with the display timing of the corresponding frame image within the camera's viewing range.

10. A computer-readable storage medium, characterized in that The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 3 is implemented or the method according to any one of claims 4 to 7 is executed.

Images