Spatial calibration method and device

Abstract

The invention relates to a spatial calibration method and device, and the method comprises the steps: generating an identification graph according to the basic information of a screen used for virtual shooting in response to an identification graph generation instruction from electronic equipment, and controlling the screen to display the identification graph; pictures of the identification graph shot by the cameras are collected to obtain collected images, tracking information for tracking the poses of the cameras is obtained, and the number of the cameras is one or more; and if the number of the calibration images determined in the acquired images corresponding to any one of the one or more cameras reaches a preset number, performing spatial calibration according to the calibration image of the camera, the pose information corresponding to each calibration image and the screen model of the screen, and generating a spatial calibration result of the camera. According to the invention, parallel execution of spatial calibration of the camera to be used in virtual shooting can be realized, the speed and efficiency of spatial calibration are improved, and user operation is facilitated.

Description

Technical Field

[0001] The present disclosure relates to the technical field of virtual shooting, and in particular to a space calibration method and device. Background Art

[0002] Virtual shooting is a shooting method that uses computer-generated images and real-time rendering technology to create. During the virtual shooting process, at least one camera will be used to shoot at the same time. In order to ensure the accuracy of the shooting picture, staff are required to perform spatial calibration on all cameras on site before the formal shooting. However, in the related art, spatial calibration takes a long time, is inefficient, and is inconvenient. Summary of the invention

[0003] In view of this, the present disclosure proposes a spatial calibration method and device.

[0004] According to one aspect of the present disclosure, a spatial calibration method is provided, the method comprising:

[0005] In response to an identification graphic generation instruction from the electronic device, generating an identification graphic according to basic information of a screen used for virtual shooting, and controlling the screen to display the identification graphic;

[0006] Capturing a picture of the identification graphic shot by a camera to obtain a captured image, and obtaining tracking information for tracking the position and posture of the camera, wherein the number of the camera is one or more;

[0007] If the number of calibration images determined in the captured images corresponding to any one of the one or more cameras reaches a preset number, spatial calibration is performed according to the calibration images of the camera, the pose information corresponding to each of the calibration images, and the screen model of the screen to generate a spatial calibration result of the camera;

[0008] The pose information corresponding to each calibration image is determined based on the tracking information, and the spatial calibration result is used to indicate the relative position relationship between the corresponding camera and the screen.

[0009] In a possible implementation, the method further includes:

[0010] According to the image captured by the camera and the tracking information, an instruction is given to adjust the position and / or posture of the camera for capturing the identification graphic.

[0011] In a possible implementation, instructing the camera to adjust the position and / or posture of the camera for photographing the identification graphic according to the image captured by the camera and the tracking information includes:

[0012] Determine, according to the images captured by the cameras and the tracking information, the field of view coverage and screen coverage of each camera and the movable range of each camera in the tracking field corresponding to the current screen;

[0013] According to the field of view coverage, screen coverage and movable range of each camera, an instruction is given to adjust the position and / or posture of each camera for shooting the identification graphic.

[0014] In a possible implementation, the method further includes:

[0015] Performing content recognition on the captured image, if the identification pattern included in the captured image meets the requirements of the calibration image, determining the captured image as the calibration image, and counting the number of calibration images of each camera;

[0016] The position and posture information corresponding to the calibration image is determined according to the tracking information of the camera corresponding to the calibration image.

[0017] In a possible implementation, if the identification pattern included in the acquired image meets the requirement of the calibration image, determining the acquired image as the calibration image includes:

[0018] If the identification graphics included in the captured image meet the requirements of the calibration image, and the captured image is only captured in the picture taken when the camera is in a stationary state, then the captured image is determined as the calibration image, wherein the tracking information of the camera corresponding to the calibration image includes stable tracking information obtained when the camera is in a stationary state.

[0019] In a possible implementation, if the identification pattern included in the acquired image meets the requirement of the calibration image, determining the acquired image as the calibration image includes:

[0020] If the identification graphics included in the captured image meet the requirements of the calibration image, and the captured image is captured in the pictures taken when the camera is in a stationary or non-stationary state, then the captured image is determined as the calibration image, wherein there is a tracking delay between the tracking information of the camera corresponding to the calibration image and the time information of the calibration image.

[0021] In a possible implementation, the method further includes:

[0022] Determining a tracking delay of the tracking device based on time information of a first captured image and time information of a first tracking information after the camera switches its state from being in a stationary state to being in a moving state;

[0023] Among them, determining the posture information corresponding to the calibration image according to the tracking information of the camera corresponding to the calibration image includes: determining the posture information corresponding to the calibration image from the tracking information of the corresponding camera according to the time information of the calibration image and the tracking delay.

[0024] In a possible implementation, capturing a picture of the identification graphic captured by a camera to obtain a captured image includes:

[0025] In response to a capture instruction from the electronic device and / or a detected capture trigger operation, the camera captures the image of the identification graphic to obtain a captured image.

[0026] According to another aspect of the present disclosure, a spatial calibration device is provided, the device comprising:

[0027] A graphics generation module, for responding to an identification graphics generation instruction from an electronic device, generating an identification graphics according to basic information of a screen used for virtual shooting, and controlling the screen to display the identification graphics;

[0028] A data receiving module, used to acquire a captured image from a picture of the identification graphic taken by a camera, and obtain tracking information for tracking the position and posture of the camera, wherein the number of the camera is one or more;

[0029] A calibration module, configured to perform spatial calibration according to the calibration images of the camera, the position and posture information corresponding to each of the calibration images, and the screen model of the screen, and generate a spatial calibration result of the camera if the number of calibration images determined in the captured images corresponding to any camera of the one or more cameras reaches a preset number;

[0030] The pose information corresponding to each calibration image is determined based on the tracking information, and the spatial calibration result is used to indicate the relative position relationship between the corresponding camera and the screen.

[0031] In a possible implementation manner, the device further includes:

[0032] The posture adjustment module is used to instruct the camera to adjust the position and / or posture of the identification graphic according to the picture taken by the camera and the tracking information.

[0033] In a possible implementation, instructing the camera to adjust the position and / or posture of the camera for photographing the identification graphic according to the image captured by the camera and the tracking information includes:

[0034] Determine, according to the images captured by the cameras and the tracking information, the field of view coverage and screen coverage of each camera and the movable range of each camera in the tracking field corresponding to the current screen;

[0035] According to the field of view coverage, screen coverage and movable range of each camera, an instruction is given to adjust the position and / or posture of each camera for shooting the identification graphic.

[0036] In a possible implementation manner, the device further includes:

[0037] An image determination module, configured to perform content recognition on the captured image, determine the captured image as a calibration image if the identification pattern included in the captured image meets the requirements of the calibration image, and count the number of calibration images of each camera;

[0038] The posture determination module is used to determine the posture information corresponding to the calibration image according to the tracking information of the camera corresponding to the calibration image.

[0039] In a possible implementation, if the identification pattern included in the acquired image meets the requirement of the calibration image, determining the acquired image as the calibration image includes:

[0040] If the identification graphics included in the captured image meet the requirements of the calibration image, and the captured image is only captured in the picture taken when the camera is in a stationary state, then the captured image is determined as the calibration image, wherein the tracking information of the camera corresponding to the calibration image includes stable tracking information obtained when the camera is in a stationary state.

[0041] In a possible implementation, if the identification pattern included in the acquired image meets the requirement of the calibration image, determining the acquired image as the calibration image includes:

[0042] If the identification graphics included in the captured image meet the requirements of the calibration image, and the captured image is captured in the pictures taken when the camera is in a stationary or non-stationary state, then the captured image is determined as the calibration image, wherein there is a tracking delay between the tracking information of the camera corresponding to the calibration image and the time information of the calibration image.

[0043] In a possible implementation, the tracking delay of the tracking device is determined based on time information of a first captured image and time information of a first tracking information after the camera switches its state, wherein the state switches from a stationary state to a moved state of the camera;

[0044] Among them, determining the posture information corresponding to the calibration image according to the tracking information of the camera corresponding to the calibration image includes: determining the posture information corresponding to the calibration image from the tracking information of the corresponding camera according to the time information of the calibration image and the tracking delay.

[0045] In a possible implementation, capturing a picture of the identification graphic captured by a camera to obtain a captured image includes:

[0046] In response to a capture instruction from the electronic device and / or a detected capture trigger operation, the camera captures the image of the identification graphic to obtain a captured image.

[0047] According to another aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to implement the above method when executing the instructions stored in the memory.

[0048] According to another aspect of the present disclosure, a non-volatile computer-readable storage medium is provided, on which computer program instructions are stored, wherein the computer program instructions implement the above method when executed by a processor.

[0049] According to another aspect of the present disclosure, a computer program product is provided, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above method.

[0050] The spatial calibration method and device provided by the embodiment of the present disclosure, in response to the identification graphic generation instruction from the electronic device, generates the identification graphic according to the basic information of the screen used for virtual shooting, and controls the screen to display the identification graphic; the picture of the identification graphic shot by the camera is captured to obtain the captured image, and the tracking information for tracking the position and posture of the camera is obtained, and the number of the cameras is one or more; if the number of calibration images determined in the captured image corresponding to any camera of the one or more cameras reaches a preset number, the spatial calibration is performed according to the calibration image of the camera, the position and posture information corresponding to each calibration image and the screen model of the screen, and the spatial calibration result of the camera is generated. The parallel execution of the spatial calibration of the camera to be used in the virtual shooting can be realized, that is, the captured images corresponding to one or more cameras are collected in parallel, and the tracking information is obtained in parallel. When the number of calibration images in the captured image of any camera reaches a preset number, the spatial calibration is performed on the camera until the spatial calibration of all cameras is completed, thereby improving the speed and efficiency of the spatial calibration and facilitating user operation.

[0051] Further features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the disclosure and, together with the description, serve to explain the principles of the disclosure.

[0053] Figure 1 A schematic diagram of the structure of a virtual shooting system according to an embodiment of the present disclosure is shown.

[0054] Figure 2 A schematic diagram showing an application of a spatial calibration method according to an embodiment of the present disclosure is shown.

[0055] Figure 3 A flowchart of a spatial calibration method according to an embodiment of the present disclosure is shown.

[0056] Figure 4 is a block diagram of a device 800 for spatial calibration according to an exemplary embodiment. DETAILED DESCRIPTION

[0057] Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0058] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0059] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present disclosure.

[0060] Spatial calibration is a process used in many fields to ensure the spatial accuracy and consistency of measurement, imaging or positioning systems. Its main purpose is to eliminate or reduce spatial deviations caused by system errors, environmental changes or the characteristics of the equipment itself, thereby improving the accuracy of measurement and imaging. In virtual shooting, spatial calibration is also a crucial step. The positional relationship between the real camera and the real screen is determined, and then the positional relationship between the virtual camera and the virtual screen is calibrated based on the positional relationship between the real camera and the real screen. It ensures the precise alignment between the real camera and the virtual scene, thereby achieving a high-quality virtual-real fusion effect. In the related art, the implementation process of spatial calibration is: the screen displaying the virtual scene is controlled to display the logo graphic, the camera operator moves the camera and notifies the data collection personnel to collect the camera's image and posture information after the camera is stationary. After multiple acquisitions, the spatial calibration of the camera is performed based on the multiple acquired images and posture information, and the above process is repeated until all cameras have completed the spatial calibration. However, it is obvious that the spatial calibration of only one camera at a time is inefficient and slow.

[0061] In order to solve the above technical problems, the embodiment of the present disclosure provides a spatial calibration method and device, which responds to the identification graphic generation instruction from the electronic device, generates the identification graphic according to the basic information of the screen used for virtual shooting, and controls the screen to display the identification graphic; collects the picture of the identification graphic captured by the camera to obtain the collection image, and obtains the tracking information for tracking the position and posture of the camera, and the number of the cameras is one or more; if the number of calibration images determined in the collection image corresponding to any camera of the one or more cameras reaches a preset number, the spatial calibration is performed according to the calibration image of the camera, the position and posture information corresponding to each calibration image and the screen model of the screen, and the spatial calibration result of the camera is generated. The spatial calibration of the camera to be used in the virtual shooting can be performed in parallel, that is, the collection images corresponding to one or more cameras are collected in parallel, and the tracking information is obtained in parallel. When the number of calibration images in the collection image of any camera reaches a preset number, the spatial calibration is performed on the camera until the spatial calibration of all cameras is completed, thereby improving the speed and efficiency of the spatial calibration and facilitating user operation.

[0062] To facilitate the description of the implementation of the spatial calibration method and device provided in the embodiment of the present disclosure, the virtual shooting system is first schematically described below. Figure 1 As shown, the virtual shooting system may include a camera, a tracking device, a switch, a screen, a rendering cluster, an LED processor, and a synchronization signal generator.

[0063] There can be one or more cameras, each of which can shoot and obtain the shooting picture at the shooting site, and can transmit the picture content to the control machine in the rendering cluster in real time through the SDI signal. The tracking device is bound to the camera, and the tracking device can track the position and posture of the camera in real time to form tracking information, and broadcast the tracking information to the local area network on site through the network. Among them, the tracking information represents the absolute position and absolute posture of the camera, such as the position and posture in the world coordinate system.

[0064] The switch can build a local area network at the shooting site to achieve communication between devices in the local area network. The switch can receive tracking information broadcast by the tracking device and send the tracking information to the rendering cluster. In order to ensure the real-time data transmission, the switch needs to have a high bandwidth, for example, the switch can be a 10G switch.

[0065] The screen may be an LED screen, which may include one or more, Figure 1 The example is a plurality of LED screens. Each LED screen includes at least one LED box, one side of the LED box is an LED panel, and the plurality of LED panels are spliced ​​into the display area of ​​the screen. The rendering cluster can render the virtual scene and display the rendered image of the virtual scene on the screen. The rendering cluster includes a control machine and at least one rendering server ( Figure 1 The example is multiple rendering servers). During the virtual shooting process, after receiving the picture sent back by the camera, the control machine determines the picture to be rendered based on the picture, and then assigns the picture to be rendered to multiple rendering servers for rendering. The rendering server executes the rendering task assigned by the control machine to obtain the rendering result, and sends the rendering result to the LED processor. Among them, the rendering server can send the rendered image to be displayed (that is, the rendering result) to the LED processor through the DP signal. A rendering server can correspond to one or more LED boxes, and the rendering server can render the image displayed by the corresponding one or more LED boxes. The LED processor can be a hardware device for controlling the screen, and the LED processor can include at least one ( Figure 1 An example is a plurality of LED processors), one LED processor may correspond to one or more LED boxes, and the LED processor may control the one or more LED boxes corresponding thereto.

[0066] The sync signal generator can generate sync signal pulses and send the sync signal to the camera, the tracking device, the rendering cluster, and the LED processor, so that the camera, the tracking device, the rendering cluster, and the LED processor can be synchronized. In some examples, the sync signal generator can transmit the sync signal in the form of an SDI signal.

[0067] The above is only an example of a virtual shooting system. The virtual shooting system may also include more or fewer devices as needed. The above devices may be physically integrated into the same device or distributed on different devices.

[0068] like Figure 2 , Figure 3 As shown, the spatial calibration method provided by the embodiment of the present disclosure includes steps S101 to S105. The method can be applied to any device with processing capabilities, such as Figure 1 The control machine in the virtual shooting system shown in the figure can be Figure 1 The virtual shooting system shown in the figure provides a local area network on site and can be controlled by the user Figure 2 The electronic device shown communicates with each other. The spatial calibration method is performed before the virtual shooting officially starts to ensure the quality of the virtual shooting.

[0069] In step S101, in response to an identification graphic generation instruction from an electronic device, an identification graphic is generated according to basic information of a screen used for virtual shooting, and the screen is controlled to display the identification graphic.

[0070] In this embodiment, the electronic device can generate an identification graphic generation instruction according to the user's operation settings, and send the identification graphic generation instruction to the control machine through the field local area network to start the entire space calibration process.

[0071] In some embodiments, a calibration control for starting spatial calibration may be displayed in the electronic device so that the user can trigger the calibration control, and the electronic device may generate an identification graphic generation instruction in response to the user's triggering operation on the calibration control. The calibration control may be displayed in the spatial calibration interface, and the spatial calibration interface may also display the user with the pictures currently captured by each camera and the tracking information corresponding to the pictures, the camera number and other camera-related information, so that the user can determine whether to start spatial calibration based on the pictures captured by each camera. For example, if there are three cameras on site, if the captured pictures displayed in the spatial calibration interface include the pictures currently captured by these three cameras, it can be determined that the cameras are in place and can start working, and spatial calibration can be started. The user can trigger the calibration control, and the electronic device issues an identification graphic generation instruction.

[0072] In some embodiments, an application corresponding to space calibration may be pre-installed in the electronic device. When it is detected that the application is triggered, the control electronic device is connected to the control machine through the on-site local area network, and the real-time images captured by each camera and the corresponding related information are obtained from the control machine, and then the space calibration interface is displayed to the user.

[0073] In this embodiment, the identification graphic is a text or symbol that is easy to identify. The identification graphic should have a clear boundary and a large color difference with the background, so that after the camera captures an image including the identification graphic, the identification graphic can be accurately identified from the captured image to determine the position of the feature points on the identification graphic on the captured image. In some embodiments, the identification graphic can be an Aruco code (also known as an Aruco mark, an Aruco label, an Aruco QR code) picture, a checkerboard picture, etc., and the present disclosure does not limit this. After generating the identification graphic, the control machine sends the corresponding rendering task to the corresponding rendering server in the rendering cluster, so that the rendering server renders the image required for the screen display of the identification graphic, and then sends the image to the LED processor, so that the LED processor finally controls the screen to display the image, so that the identification graphic can be displayed on the screen. One or more parameters such as the type of the identification graphic, the number of feature points, the arrangement method, and the density can be pre-set in the control machine, or calculated by the control machine based on screen parameters such as screen size. Optionally, at least some parameters can also be set by the user through an electronic device, and the present disclosure does not limit this.

[0074] In step S102, a captured image is obtained by capturing a picture of the identification graphic taken by a camera, and tracking information for tracking the position and posture of the camera is obtained. The number of the camera may be one or more.

[0075] In this embodiment, after the logo graphic is displayed on the screen, the camera operator can control the camera to shoot the screen. One or more cameras can shoot at the same time, and transmit the pictures taken by each camera back to the control machine of the rendering cluster. The control machine will further collect the pictures taken by the camera according to a preset frequency, or in response to the collection instructions from the electronic device or in response to the detected collection trigger operation to obtain a collection image. Collection can be understood as extracting at least part of the frames in the captured picture. During the picture shooting process, the tracking device will also send the determined tracking information to the control machine with the help of the on-site local area network.

[0076] In some embodiments, in the space calibration interface displayed by the electronic device to the user, the acquisition control for each camera can also be displayed to the user. When the user is watching the pictures taken by each camera, if the current picture meets the requirements of the calibration image, the acquisition control of the camera can be triggered, and then the electronic device will respond to the triggering of the acquisition control, generate an acquisition instruction and send it to the control machine. Then the control machine can respond to the acquisition instruction to perform image acquisition of the shooting picture, obtain the acquisition image for the acquisition instruction and further determine the tracking information corresponding to the acquisition image, and finally determine the calibration image corresponding to the acquisition instruction from the acquisition image corresponding to the acquisition instruction. In this way, the user can control the electronic device to perform acquisition control of the shooting picture. Each time the user issues an acquisition instruction, an image acquisition for the shooting picture is performed to obtain a calibration image and corresponding tracking information, which can improve the efficiency, speed and accuracy of the calibration image acquisition. Similarly, the control machine can also display the space calibration interface for the user like the electronic device, so that the user can trigger the acquisition control displayed by the control machine to issue an acquisition trigger operation. Among them, in the case where no acquisition instruction and acquisition trigger operation are received, image acquisition and tracking information acquisition can be performed on the shooting screen at a preset frequency to obtain multiple acquired images and their corresponding tracking information, and then the control machine determines the calibration image from these acquired images.

[0077] In a possible implementation, the method may further include: according to the picture taken by the camera and the tracking information, instructing the camera to adjust the position and / or posture of the identification graphic. Wherein, after the user watching the electronic device, the electronic device and / or the control machine determine that the position and / or posture of the camera needs to be adjusted according to the picture taken by the camera and the tracking information, the adjustment required for the camera will be further sent to the corresponding staff. Wherein, the adjustment information can be sent to the terminal such as the mobile phone of the operator of the camera, so that the operator can directly understand what adjustments need to be made to the camera controlled by him through the adjustment information through the handheld terminal. It is also possible to control the device (such as a speaker, etc.) of the virtual shooting scene to play the adjustment prompt, which can be the text that the operator of the camera can see and / or the voice that can be heard, so that the operator can adjust the camera in time. Alternatively, the adjustment information can also be sent directly to the control device that controls the position and posture of the camera, so that the control device can directly adjust the position and / or posture of the camera based on the adjustment information. The implementation method of adjusting the posture of the camera can be set according to actual needs, and the present disclosure does not limit this. Among them, the electronic device and the control machine can directly generate adjustment information and send and / or control the device to send an adjustment prompt, and the user watching the electronic device can convey instructions to the camera operator to adjust the position and / or posture of the camera by operating the electronic device, the control machine, the on-site intercom, etc. For example, if it is determined that the picture currently captured by a certain camera does not have corresponding tracking information, it can be determined that the current position of the camera may no longer be within the movable range and the tracking device can no longer track the position and posture of the camera. The camera needs to be adjusted to the movable range, and the position and posture of the camera can be adjusted. For another example, if a certain camera does not capture the screen, or the range of the captured screen is too small to obtain a calibration picture, the operator or control device can also be notified to adjust the position and / or posture of the camera.

[0078] In some embodiments, instructing the adjustment of the position and / or posture of the camera for shooting the identification graphic based on the picture captured by the camera and the tracking information may include: determining the field of view coverage and screen coverage of each camera and the movable range of each camera in the tracking field corresponding to the current screen based on the picture captured by the camera and the tracking information; instructing the adjustment of the position and / or posture of each camera for shooting the identification graphic based on the field of view coverage, screen coverage and movable range of each camera.

[0079] The camera's field of view coverage may refer to the ratio of the screen area contained in the camera's field of view. The camera's screen coverage may refer to the ratio of the screen area captured by the camera to the total screen area. The movable range of the camera in the tracking field corresponding to the current screen may refer to the range in which the camera can be tracked by the tracking device in the virtual shooting scene.

[0080] In this embodiment, the tracking device can only track the position of the camera within the movable range. Therefore, if the tracking device does not return tracking information, it can be determined that the camera where the tracking device is located has moved out of the movable range. Based on the position information in the last tracking information returned by the tracking device before the tracking information is returned, a boundary point of the movable range of the camera can be determined, and then the movable range of the camera can be determined based on the determined multiple boundary points. In order to achieve spatial calibration, it is necessary to ensure that the camera is always within the movable range. Therefore, if it is determined that the camera can return to the shooting screen normally but the tracking information is not returned, it is determined that the camera position needs to be adjusted. The camera can be moved back to the movable range based on the most recent tracking information of the camera and the movable range of the camera.

[0081] In this embodiment, if it is determined that the camera's field of view coverage is not within the field of view coverage range required for spatial calibration. For example, if the field of view coverage is too small, the camera can be controlled to move in a direction that increases the distance between the camera and the screen, and the camera's posture can be adjusted (such as changing the angle between the camera and the screen) to expand the area of ​​the screen area that can be covered within the field of view. If it is determined that the camera's screen coverage is not within the screen coverage range required for spatial calibration. For example, if the screen coverage is too small, the camera can be controlled to move in a direction that reduces the distance between the camera and the screen, and the camera's posture can be adjusted (such as changing the angle between the camera and the screen) to make the screen occupy a larger proportion in the picture.

[0082] In step S103, the content of the captured image is identified. If the identification graphic included in the captured image meets the requirements of the calibration image, the captured image is determined as the calibration image; and the position information corresponding to the calibration image is determined according to the tracking information of the camera corresponding to the calibration image. Among them, the identification graphic included in the captured image meets the requirements of the calibration image, which may include: the captured image includes an identification graphic with a clarity higher than the clarity threshold, and the identification graphic with a clarity higher than the clarity threshold may mean that the size of the identification graphic in the captured image meets the requirements of spatial calibration, and the clarity of the identification graphic is also higher than the clarity threshold (the clarity threshold can be set according to the requirements of spatial calibration). For example, if the identification graphic is an Aruco code picture, "the size of the identification graphic in the captured image meets the requirements of spatial calibration" may mean that the number of Aruco codes in the captured image meets the requirements of spatial calibration. Among them, the implementation methods of determining the calibration image and its tracking information include the following implementation methods 1 and 2, which are described below respectively.

[0083] Implementation method 1: If the identification pattern included in the acquisition image meets the requirements of the calibration image, and the acquisition image is only acquired in the picture taken when the camera is in a stationary state, the acquisition image is determined as the calibration image, wherein the tracking information of the camera corresponding to the calibration image includes the stable tracking information obtained when the camera is in a stationary state. In this case, when the control machine acquires the acquisition image from the picture taken by the camera, it is necessary to determine whether the camera is in a stationary state when the acquisition image is taken. In the case of being in a stationary state, if the identification pattern included in the acquisition image meets the requirements of the calibration image, it can be used as the calibration image. In this case, since the camera is stationary when taking the acquisition image, the tracking information remains unchanged, and the tracking information with the same or similar time information as the time information of the acquisition image and in a stable state (for example, unchanged for a period of time) can be selected as the tracking information corresponding to the calibration image. Among them, the time information corresponding to the calibration image can be the timestamp of the camera taking the calibration image, and the time information of the tracking information can be the timestamp of the tracking device collecting the tracking information, and the timestamp can be the timestamp on the synchronization signal.

[0084] Among them, the control machine can detect multiple frames of captured images in the collected image to determine whether the collected image has jitter. If there is no jitter, it can be determined that the collected image is taken by the camera in a static state, and at least some frames in the multiple frames of captured images are used as calibration images. If there is jitter, it can be determined that the collected image is taken by the camera in a moving state. The jitter detection method can be implemented by combining feature point matching, frame difference detection, regional motion analysis and other methods with existing technologies, and the present disclosure does not limit this. Or the control machine can also first determine whether the picture taken by the camera has jitter in a similar way, and collect the collected image from a group of adjacent pictures without jitter.

[0085] Implementation method two: If the identification pattern included in the acquired image meets the requirements of the calibration image, then the acquired image is determined as the calibration image, including: if the identification pattern included in the acquired image meets the requirements of the calibration image, and the acquired image is acquired in the picture taken when the camera is in a stationary or non-stationary state, then the acquired image is determined as the calibration image, wherein there is a tracking delay between the tracking information of the camera corresponding to the calibration image and the time information of the calibration image.

[0086] Among them, although the tracking device is bound to the camera, the frame rate of the camera capturing the picture is different from the frame rate of the tracking device obtaining the tracking information, so the time information corresponding to the captured picture returned by the camera at the same time and the time information corresponding to the tracking information of the captured picture may not be synchronized, and there is a delay. In addition, since the synchronization signal has been sent to each device of the virtual shooting system, the captured image and the tracking information can be updated according to the frequency of the synchronization signal, so that the delay between the tracking information and the captured image can be guaranteed to be fixed. After the tracking delay is calculated in advance, the position information corresponding to each calibration image can be determined from the tracking information based on the calibration image and the time information of each tracking information. In the case of introducing the tracking delay, the controller can acquire the captured image from the picture captured by the camera in a stationary or non-stationary state, that is, there is no need to consider whether the camera is stationary. Therefore, the method may also include: before determining the tracking information of the calibration image, based on the time information of the first captured picture of the camera after the state switching and the time information of the first tracking information, determining the tracking delay of the tracking device, the state switching is the state where the camera changes from a stationary state to a moved state. This is because when the camera is suddenly moved, the state of the camera switches, which will cause the tracking information and the shooting picture to jump. The tracking delay of the tracking device can be determined based on the time information corresponding to the first shooting picture after the jump and the time information corresponding to the first tracking information after the jump.

[0087] Among them, the control machine can detect each frame of the shooting picture and use whether the shooting picture is jittering as the basis for whether the camera has switched states. If there is jittering, the time information corresponding to the first shooting picture after the jitter can be determined. In addition, the control machine can compare the tracking information, because the posture of the camera is not fixed when it is stationary. If the camera state is switched, the posture information in the tracking information will also jump. Therefore, whether the posture information in the tracking information jumps can be used as the basis for whether the camera has changed. The posture information in the tracking information can be compared to determine the time information of the first tracking information whose posture information has changed. Obviously, the first shooting picture taken after the camera state is switched must correspond to the first tracking information, and the difference in the time information between the two can be used as the tracking delay Δt. For each shooting picture after the first shooting picture, assuming that the timestamp of the shooting picture is t, the tracking information with the timestamp t+Δt is taken as the tracking information of the shooting picture. The tracking delay can be calculated only once and used all the time in the future.

[0088] In this way, through the second implementation method, at least one camera operator is required to stand in front of the screen to operate each camera to achieve spatial calibration. Even if the camera moves, spatial calibration can continue by calculating the tracking delay. There is no need to wait for the camera to be still before collecting data. This is simpler, faster, more efficient and more stable.

[0089] In step S104, the number of calibration images of each camera is counted. Then, the number of calibration images of each camera is judged. If the number of calibration images of any camera reaches the preset number, step S105 is executed. If the number of calibration images of any camera does not reach the preset number, step S103 is continued to select a new calibration image from the new collected images until the number of calibration images of the camera reaches the preset number. The preset number can be set according to actual needs. For example, the preset number can be 8, and the present disclosure does not limit this.

[0090] In step S105, spatial calibration is performed based on the calibration images of the cameras, the pose information corresponding to each of the calibration images, and the screen model of the screen to generate a spatial calibration result for the corresponding camera. As described above, the pose information corresponding to each of the calibration images is determined based on the tracking information. The spatial calibration result is used to indicate the relative position relationship between the corresponding camera and the screen. Spatial calibration can be completed based on related technologies. In this way, the calibration images of each camera are acquired synchronously, and then the cameras whose number of calibration images reaches a preset number are sequentially spatially calibrated, thereby realizing parallel spatial calibration in the case of multiple cameras.

[0091] In a possible implementation, the method may further include: generating the screen model according to the basic information of the screen and the calibration image. In this way, compared with building a screen model based on hand-measured data, it is more efficient, more accurate, faster, and can save manpower and material resources.

[0092] Among them, after setting up the screen for virtual shooting, the position and shape of the screen need to remain constant, and then space calibration can be started to determine the relative position relationship between the camera and the screen used for virtual shooting. Since the screen can include one or more LED boxes, if the screen includes two or more LED boxes, the LED boxes can be spliced ​​in a row and column arrangement to obtain screens of various shapes. Therefore, the basic information of the screen may include the number of rows and columns of the LED boxes, the size information of a single LED box, the resolution information of a single LED box, the resolution information of the entire screen, etc. The basic information of the screen can be input by the user through an electronic device, or obtained by other means, and the present disclosure does not limit this.

[0093] Among them, since the screen is usually a continuous screen, that is, Figure 1 The screen shown is a continuous display area composed of multiple LED screens, so the relative positions between the LED screens can also be represented by the angles between the LED boxes located in the boundary area, so it is necessary to determine the angles between the adjacent LED boxes in the screen. Then, the overall structure of the screen can be determined based on the basic information of the screen, and the relative positions between different LED screens and the angles between adjacent LED boxes in different LED screens can be determined based on the multiple calibration images collected, and finally a screen model of the screen is generated. The screen model may refer to a three-dimensional model of the screen. In the case where the screen includes multiple screens, the screen model may also include multiple screen models. For example, the screen includes a floor screen, a ceiling screen, and a curved vertical screen, then the screen model corresponding to the screen includes three models corresponding to the floor screen, the ceiling screen, and the curved vertical screen, respectively.

[0094] The present disclosure also provides a spatial calibration device, which includes:

[0095] A graphics generation module, for responding to an identification graphics generation instruction from an electronic device, generating an identification graphics according to basic information of a screen used for virtual shooting, and controlling the screen to display the identification graphics;

[0096] A data receiving module, used to acquire a captured image from a picture of the identification graphic taken by a camera, and obtain tracking information for tracking the position and posture of the camera, wherein the number of the camera is one or more;

[0097] A calibration module, configured to perform spatial calibration according to the calibration images of the camera, the position and posture information corresponding to each of the calibration images, and the screen model of the screen, and generate a spatial calibration result of the camera if the number of calibration images determined in the captured images corresponding to any camera of the one or more cameras reaches a preset number;

[0098] The pose information corresponding to each calibration image is determined based on the tracking information, and the spatial calibration result is used to indicate the relative position relationship between the corresponding camera and the screen.

[0099] In a possible implementation manner, the device further includes:

[0100] The posture adjustment module is used to instruct the camera to adjust the position and / or posture of the identification graphic according to the picture taken by the camera and the tracking information.

[0101] In a possible implementation, instructing the camera to adjust the position and / or posture of the camera for photographing the identification graphic according to the image captured by the camera and the tracking information includes:

[0102] Determine, according to the images captured by the cameras and the tracking information, the field of view coverage and screen coverage of each camera and the movable range of each camera in the tracking field corresponding to the current screen;

[0103] According to the field of view coverage, screen coverage and movable range of each camera, an instruction is given to adjust the position and / or posture of each camera for shooting the identification graphic.

[0104] In a possible implementation manner, the device further includes:

[0105] An image determination module, configured to perform content recognition on the captured image, determine the captured image as a calibration image if the identification pattern included in the captured image meets the requirements of the calibration image, and count the number of calibration images of each camera;

[0106] The posture determination module is used to determine the posture information corresponding to the calibration image according to the tracking information of the camera corresponding to the calibration image.

[0107] In a possible implementation, if the identification pattern included in the acquired image meets the requirement of the calibration image, determining the acquired image as the calibration image includes:

[0108] If the identification graphics included in the captured image meet the requirements of the calibration image, and the captured image is only captured in the picture taken when the camera is in a stationary state, then the captured image is determined as the calibration image, wherein the tracking information of the camera corresponding to the calibration image includes stable tracking information obtained when the camera is in a stationary state.

[0109] In a possible implementation, if the identification pattern included in the acquired image meets the requirement of the calibration image, determining the acquired image as the calibration image includes:

[0110] If the identification graphics included in the captured image meet the requirements of the calibration image, and the captured image is captured in the pictures taken when the camera is in a stationary or non-stationary state, then the captured image is determined as the calibration image, wherein there is a tracking delay between the tracking information of the camera corresponding to the calibration image and the time information of the calibration image.

[0111] In a possible implementation manner, the device further includes:

[0112] A tracking delay determination module, used to determine the tracking delay of the tracking device based on the time information of the first captured image and the time information of the first tracking information after the state switching of the camera, wherein the state switching is the state where the camera changes from a stationary state to a moved state;

[0113] Among them, determining the posture information corresponding to the calibration image according to the tracking information of the camera corresponding to the calibration image includes: determining the posture information corresponding to the calibration image from the tracking information of the corresponding camera according to the time information of the calibration image and the tracking delay.

[0114] In a possible implementation, capturing a picture of the identification graphic captured by a camera to obtain a captured image includes:

[0115] In response to a capture instruction from the electronic device and / or a detected capture trigger operation, the camera captures the image of the identification graphic to obtain a captured image.

[0116] It should be noted that although the above embodiments are used as examples to introduce the spatial calibration method and device, those skilled in the art will appreciate that the present disclosure should not be limited thereto. In fact, the user can flexibly set each step and module according to personal preferences and / or actual application scenarios, as long as it complies with the technical solution of the present disclosure.

[0117] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0118] The embodiment of the present disclosure also provides a computer-readable storage medium on which computer program instructions are stored, and the computer program instructions implement the above method when executed by a processor. The computer-readable storage medium can be a volatile or non-volatile computer-readable storage medium.

[0119] An embodiment of the present disclosure further proposes an electronic device, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to implement the above method when executing the instructions stored in the memory.

[0120] The embodiments of the present disclosure also provide a computer program product, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above method.

[0121] Figure 4 8 is a block diagram of an apparatus 800 for spatial calibration according to an exemplary embodiment. For example, the apparatus 800 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0122] Reference Figure 4 , the device 800 may include one or more of the following components: a processing component 802 , a memory 804 , a power component 806 , a multimedia component 808 , an audio component 810 , an input / output interface 812 (I / O interface), a sensor component 814 , and a communication component 816 .

[0123] The processing component 802 generally controls the overall operation of the device 800, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0124] The memory 804 is configured to store various types of data to support operations on the device 800. Examples of such data include instructions for any application or method operating on the device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0125] The power supply component 806 provides power to the various components of the device 800. The power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 800.

[0126] The multimedia component 808 includes a screen that provides an output interface between the device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0127] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC), and when the device 800 is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 804 or sent via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

[0128] The input / output interface 812 provides an interface between the processing component 802 and the peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0129] The sensor assembly 814 includes one or more sensors for providing various aspects of the status assessment of the device 800. For example, the sensor assembly 814 can detect the open / closed state of the device 800, the relative positioning of components, such as the display and keypad of the device 800, and the sensor assembly 814 can also detect the position change of the device 800 or a component of the device 800, the presence or absence of user contact with the device 800, the orientation or acceleration / deceleration of the device 800, and the temperature change of the device 800. The sensor assembly 814 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 814 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0130] The communication component 816 is configured to facilitate wired or wireless communication between the device 800 and other devices. The device 800 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0131] In an exemplary embodiment, the apparatus 800 may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above methods.

[0132] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 804 including computer program instructions, which can be executed by the processor 820 of the device 800 to perform the above method.

[0133] The present disclosure may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0134] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be interpreted as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.

[0135] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0136] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0137] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0138] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0139] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0140] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.

[0141] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A spatial calibration method, characterized in that: The method comprises: In response to an identification graphic generation instruction from the electronic device, generating an identification graphic according to basic information of a screen used for virtual shooting, and controlling the screen to display the identification graphic; Capturing a picture of the identification graphic shot by a camera to obtain a captured image, and obtaining tracking information for tracking the position and posture of the camera, wherein the number of the camera is one or more; If the number of calibration images determined in the captured images corresponding to any one of the one or more cameras reaches a preset number, spatial calibration is performed according to the calibration images of the camera, the pose information corresponding to each of the calibration images, and the screen model of the screen to generate a spatial calibration result of the camera; The pose information corresponding to each calibration image is determined based on the tracking information, and the spatial calibration result is used to indicate the relative position relationship between the corresponding camera and the screen.

2. The method according to claim 1, characterized in that The method further comprises: According to the image captured by the camera and the tracking information, an instruction is given to adjust the position and / or posture of the camera for capturing the identification graphic.

3. The method according to claim 1, characterized in that According to the picture captured by the camera and the tracking information, instructing the camera to adjust the position and / or posture of the camera for capturing the identification graphic, including: Determine, according to the images captured by the cameras and the tracking information, the field of view coverage and screen coverage of each camera and the movable range of each camera in the tracking field corresponding to the current screen; According to the field of view coverage, screen coverage and movable range of each camera, an instruction is given to adjust the position and / or posture of each camera for shooting the identification graphic.

4. The method according to claim 1, characterized in that: The method further comprises: Performing content recognition on the captured image, if the identification pattern included in the captured image meets the requirements of the calibration image, determining the captured image as the calibration image, and counting the number of calibration images of each camera; The position and posture information corresponding to the calibration image is determined according to the tracking information of the camera corresponding to the calibration image.

5. The method according to claim 4, characterized in that If the identification pattern included in the collected image meets the requirements of the calibration image, determining the collected image as the calibration image includes: If the identification graphics included in the captured image meet the requirements of the calibration image, and the captured image is only captured in the picture taken when the camera is in a stationary state, then the captured image is determined as the calibration image, wherein the tracking information of the camera corresponding to the calibration image includes stable tracking information obtained when the camera is in a stationary state.

6. The method according to claim 4, characterized in that If the identification pattern included in the collected image meets the requirements of the calibration image, determining the collected image as the calibration image includes: If the identification graphics included in the captured image meet the requirements of the calibration image, and the captured image is captured in the pictures taken when the camera is in a stationary or non-stationary state, then the captured image is determined as the calibration image, wherein there is a tracking delay between the tracking information of the camera corresponding to the calibration image and the time information of the calibration image.

7. The method according to claim 6, characterized in that The method further comprises: Determining a tracking delay based on time information of a first captured image and time information of a first tracking information after a state switching of the camera, wherein the state switching is when the camera changes from a stationary state to a moved state; Among them, determining the posture information corresponding to the calibration image according to the tracking information of the camera corresponding to the calibration image includes: determining the posture information corresponding to the calibration image from the tracking information of the corresponding camera according to the time information of the calibration image and the tracking delay.

8. The method according to claim 1, characterized in that The process of capturing a picture of the logo pattern taken by a camera to obtain a captured image includes: In response to a capture instruction from the electronic device and / or a detected capture trigger operation, the camera captures the image of the identification graphic to obtain a captured image.

9. A spatial calibration device, characterized in that: The device comprises: A graphics generation module, for generating a logo graphic according to basic information of a screen used for virtual shooting in response to a logo graphic generation instruction from an electronic device, and controlling the screen to display the logo graphic; A data receiving module, used to acquire a captured image from a picture of the identification graphic taken by a camera, and obtain tracking information for tracking the position and posture of the camera, wherein the number of the camera is one or more; A calibration module, configured to perform spatial calibration according to the calibration images of the camera, the position and posture information corresponding to each of the calibration images, and the screen model of the screen, and generate a spatial calibration result of the camera if the number of calibration images determined in the captured images corresponding to any camera of the one or more cameras reaches a preset number; The pose information corresponding to each calibration image is determined based on the tracking information, and the spatial calibration result is used to indicate the relative position relationship between the corresponding camera and the screen.

10. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to implement the method described in any one of claims 1 to 8 when executing the instructions stored in the memory.

11. A non-volatile computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 8 is implemented.

12. A computer program product comprising computer readable code, or a non-volatile computer readable storage medium carrying computer readable code, characterized in that: When the computer readable code runs in a processor of an electronic device, the processor in the electronic device executes the method according to any one of claims 1 to 8.

Images