Calibration method for virtual camera, related apparatus, device, and storage medium

Abstract

The present application relates to the technical field of computer vision. Disclosed are a calibration method for a virtual camera, a related apparatus, a device, and a storage medium. The method in the present application comprises: when a target object model corresponding to an entity photographing device has been acquired, displaying, by means of a head-mounted display device, the target object model linked with an operation controller; when a contact plane of the operation controller is in a fitted state with a contact plane of the entity photographing device, displaying, by means of the head-mounted display device, that a camera of the target object model is in an overlapped state with a camera of the entity photographing device; and in response to a spatial calibration operation on a virtual camera, calibrating the virtual camera on the basis of spatial information of the operation controller, wherein the spatial information comprises target position information and a target rotation angle. In the present application, there is no need to perform extra calculation when a virtual camera is calibrated, and a relatively high accuracy is achieved, thereby improving the calibration efficiency of the virtual camera and enhancing the user experience.

Description

A virtual camera calibration method, related device, equipment and storage medium

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on September 4, 2023, application number 202311137301.4, and application name “A virtual camera calibration method, related devices, equipment and storage medium”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the fields of extended reality technology and computer vision technology, and in particular to virtual camera calibration technology. Background Art

[0003] Mixed reality capture (MRC) is a recording technology that blends the user's extended reality (XR) experience with their real-world movements. During MRC, the physical camera and virtual camera must be calibrated to ensure their positions and orientations are as consistent as possible.

[0004] In related technologies, the user first fixes the position of the physical camera and positions it, thereby also using that position as the location of the virtual camera. The user then puts on the headset, picks up the controller, and faces the physical camera to capture image 1. Simultaneously, the virtual camera captures image 2. The user moves image 2 so that the controller and headset overlap in images 1 and 2. Finally, a trigonometric function is used to calculate the deflection angle, which is used to calibrate the virtual camera's heading.

[0005] However, during the virtual camera orientation calibration process, due to inaccurate orientation, users may need to repeatedly remove and re-install their headsets to adjust the orientation. This results in inefficient virtual camera orientation calibration and compromises the user experience. Currently, no effective solution has been proposed to address this issue.

[0006] Summary of the Invention

[0007] The embodiments of the present application provide a virtual camera calibration method, related apparatus, device, and storage medium, which can achieve high accuracy when calibrating a virtual camera without performing additional calculations, thereby improving the calibration efficiency of the virtual camera and enhancing the user experience.

[0008] In view of this, the present application provides, on one hand, a virtual camera calibration method, which is executed by a computer device and includes:

[0009] When a target object model corresponding to the physical camera device is obtained, the target object model linked to the operation controller is displayed through the head-mounted display device, wherein a contact plane of the target object model is aligned with a contact plane of the operation controller, and a camera position of the target object model corresponds to a camera position of the physical camera device;

[0010] When the contact plane of the operation controller is in contact with the contact plane of the physical shooting device, the camera displaying the target object model through the head display device and the camera of the physical shooting device are in an overlapping state;

[0011] In response to a spatial calibration operation for the virtual camera, the virtual camera is calibrated according to spatial information of the operation controller, wherein the spatial information includes target position information and target orientation angle.

[0012] Another aspect of the present application provides a virtual camera calibration method, which is performed by a computer device and includes:

[0013] When the contact plane of the operation controller is in contact with the contact plane of the physical camera device, the target position information and the current orientation angle of the operation controller are obtained;

[0014] Determining a target orientation angle of the manipulation controller according to the current orientation angle and a plane perpendicular vector of the manipulation controller, wherein the plane perpendicular vector represents a vector perpendicular to a contact plane of the manipulation controller;

[0015] The virtual camera is calibrated according to the target position information and target orientation angle of the operation controller.

[0016] Another aspect of the present application provides a virtual camera calibration device, which is deployed on a computer device and includes:

[0017] A display module is configured to, upon acquiring a target object model corresponding to the physical camera device, display the target object model linked to the operation controller via a head-mounted display device, wherein a contact plane of the target object model is aligned with a contact plane of the operation controller, and a camera position of the target object model corresponds to a camera position of the physical camera device;

[0018] The display module is further configured to cause a camera of the head-mounted display device displaying a target object model to overlap with a camera of the physical shooting device when the contact plane of the operation controller and the contact plane of the physical shooting device are in contact with each other;

[0019] The calibration module is configured to calibrate the virtual camera in response to a spatial calibration operation on the virtual camera and according to spatial information of the operation controller, wherein the spatial information includes target position information and target orientation angle.

[0020] Another aspect of the present application provides a virtual camera calibration device, which is deployed on a computer device and includes:

[0021] an acquisition module, configured to acquire target position information and a current orientation angle of the operation controller when the contact plane of the operation controller is in contact with the contact plane of the physical photographing device;

[0022] a determination module, configured to determine a target orientation angle of the operation controller based on a current orientation angle and a plane perpendicular vector of the operation controller, wherein the plane perpendicular vector represents a vector perpendicular to a contact plane of the operation controller;

[0023] The calibration module is used to calibrate the virtual camera according to the target position information and target orientation angle of the operation controller.

[0024] On the other hand, the present application provides an extended reality device, including a display screen, a processor, a memory, and a communication module;

[0025] A display screen, used to provide a display function;

[0026] A communication module, used to establish a communication connection with a physical shooting device;

[0027] Memory for storing computer programs;

[0028] A processor is used to implement the above-mentioned methods when executing a computer program.

[0029] On the other hand, the present application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the above-mentioned methods when executing the computer program.

[0030] Another aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, which implements the above-mentioned methods when the computer program is executed by a processor.

[0031] Another aspect of the present application provides a computer program product, including a computer program, which implements the above-mentioned methods when executed by a processor.

[0032] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0033] In an embodiment of the present application, a method for calibrating a virtual camera is provided. First, a target object model linked to an operation controller is displayed through a head-mounted display device, and the camera position of the target object model corresponds to the camera position of the physical shooting device. When the contact plane of the operation controller and the contact plane of the physical shooting device are in a fit state, it means that the plane vertical vector constructed based on the contact plane of the operation controller is the same as the orientation angle of the physical shooting device. The camera of the target object model displayed through the head-mounted display device and the camera of the physical shooting device are in an overlapping state. The camera position of the target object model overlaps with the camera position of the physical shooting device, which means that the position of the virtual camera is the camera position of the target object model. Based on this, when spatial calibration is required, the virtual camera can be calibrated according to the spatial information of the operation controller in response to the spatial calibration operation for the virtual camera. As a result, no additional calculation is required when calibrating the virtual camera, and a higher accuracy can be achieved, thereby improving the calibration efficiency of the virtual camera and enhancing the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] FIG1 is a schematic diagram of an architecture for implementing virtual reality hybrid recording in an embodiment of the present application;

[0035] FIG2 is a schematic diagram of an implementation environment of a virtual camera calibration method according to an embodiment of the present application;

[0036] FIG3 is a flow chart of a virtual camera calibration method according to an embodiment of the present application;

[0037] FIG4 is a schematic diagram of the linkage between the handle device and the target object model in an embodiment of the present application;

[0038] FIG5 is a schematic diagram of the connection between the handle device and the physical shooting device according to an embodiment of the present application;

[0039] FIG6 is a schematic diagram showing how to mate a handle device with a physical camera device according to an embodiment of the present application;

[0040] FIG7 is a schematic diagram showing a first demonstration animation in an embodiment of the present application;

[0041] FIG8 is a schematic diagram of determining a plane vertical vector based on a handle device in an embodiment of the present application;

[0042] FIG9 is a schematic diagram showing a prompt for using a handle device to touch a camera of a physical shooting device according to an embodiment of the present application;

[0043] FIG10 is a schematic diagram showing a method for attaching a handle device to a physical shooting device according to an embodiment of the present application;

[0044] FIG11 is a schematic diagram showing a second demonstration animation in an embodiment of the present application;

[0045] FIG12 is a schematic diagram of detecting the fitting state of a device based on a pressure sensor in an embodiment of the present application;

[0046] FIG13 is a schematic diagram of enabling the perspective function in an embodiment of the present application;

[0047] FIG14 is a schematic diagram showing a calibration prompt message according to an embodiment of the present application;

[0048] FIG15 is a comparative schematic diagram of auxiliary bonding of a camera device in an embodiment of the present application;

[0049] FIG16 is a schematic diagram of implementing virtual reality hybrid recording in an embodiment of the present application;

[0050] FIG17 is another schematic flow chart of the virtual camera calibration method according to an embodiment of the present application;

[0051] FIG18 is a schematic diagram of an overall process of calibrating a virtual camera in an embodiment of the present application;

[0052] FIG19 is a schematic diagram of a virtual camera calibration device according to an embodiment of the present application;

[0053] FIG20 is another schematic diagram of a virtual camera calibration device according to an embodiment of the present application;

[0054] Figure 21 is a structural diagram of the terminal in an embodiment of the present application. DETAILED DESCRIPTION

[0055] The embodiments of the present application provide a virtual camera calibration method, related apparatus, device, and storage medium, which can achieve high accuracy when calibrating a virtual camera without performing additional calculations, thereby improving the calibration efficiency of the virtual camera and enhancing the user experience.

[0056] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the numbers used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can, for example, be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "corresponding to" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0057] MRC can provide a more engaging and powerful viewing experience for people who are not wearing a headset. These features not only add new ways to view XR applications, but also significantly increase user stickiness and potential sales, helping to bring in more revenue. MRC technology is used to render third-person perspective images and integrate them with real-life images. In applications that support MRC, the scene is generally logically divided into foreground and background parts. After turning on the MRC function, the headset device draws the foreground content and background content of the current game state of the "first-person player" from a "third-person perspective". The foreground and background image frames are encoded and transmitted to the terminal (for example, a mobile phone or computer). The terminal performs portrait cutout on the current camera stream and then synthesizes the final video in the order of background, cutout portrait, and foreground.

[0058] In the MRC process, a key step is to calibrate the physical camera and the virtual camera, requiring the physical camera and the virtual camera to have the same position and orientation angle as much as possible. In this application, the physical camera can be understood as the camera of the physical camera.

[0059] Based on this, the present application provides a method for calibrating a virtual camera, taking the operation controller as a handle device, and the handle device as a ring handle device as an example, there is a ring on the ring handle device, and the ring forms a plane, and the vertical vector of the plane can be marked as a heading angle 1. When a mobile phone is used as a real camera (i.e., a physical shooting device), the heading angle 2 of the mobile phone camera (i.e., the physical camera) and the vertical vector of the mobile phone plane are generally the same. When the user holds the ring handle device against the plane of the mobile phone, the heading angle 1 and the heading angle 2 coincide. At this time, recording the angle of the ring handle device in the virtual scene can calibrate the heading angles of the virtual camera and the mobile phone camera, and can achieve very high accuracy.

[0060] Regarding the virtual camera calibration method provided in this application, MRC technology can help users record themselves playing in virtual scenes, making it possible to share virtual reality (VR) images that are otherwise unshareable. In the XR field, this technology is widely used in VR live streaming, game advertising, and sharing services. The following will introduce this technology in conjunction with game live streaming services.

[0061] First, the user needs to prepare a set of XR equipment (including a head-mounted display and a handle device), a mobile phone, and a tripod with a mobile phone holder. Based on this, the user starts the XR device and demarcates a safe play area that is spacious enough, and selects a suitable location at the edge of the safe play area to fix the tripod. Then, clamp the phone firmly and keep the phone camera screen able to cover the shooting area. Next, start the MRC recording application (application, APP) on the XR device side, and then choose to establish a connection with the XR device in the assistant APP on the mobile phone. After the connection is successful, the physical camera and the virtual camera can be calibrated. The calibration process can adopt the virtual camera calibration method provided in this application. After the calibration is completed, the mobile phone displays the MRC recording interface, and the user can click the record button to start recording. Then, the user returns to the safe play area and starts the game. After the game is over, the video can be saved on the mobile phone.

[0062] It should be noted that the above application scenarios are only examples, and the virtual camera calibration method provided in this embodiment can also be applied to other scenarios, which are not limited here.

[0063] For ease of understanding, the MRC architecture will be introduced below in conjunction with Figure 1. Please refer to Figure 1. Figure 1 is a schematic diagram of an architecture for implementing virtual reality mixed recording in an embodiment of the present application. As shown in the figure, the head display device and the terminal respectively include a calibration module and a recording module. The head display device and the terminal establish a communication connection by accessing the same mobile hotspot (wireless fidelity, Wi-Fi). After the device is successfully connected, the process of aligning the virtual camera with the physical camera is triggered by the terminal. After the calibration is completed, the head display device transmits the video stream in the virtual scene to the terminal through wireless projection technology. The terminal performs portrait cutout on the video stream and fuses it to obtain the final video.

[0064] Understandably, MRC utilizes a deep learning cutout model to extract characters from their surroundings and superimpose them into a virtual space. Therefore, it requires computer vision (CV) technology. CV is the science of enabling machines to "see." Specifically, it refers to machine vision techniques such as using cameras and computers to replace the human eye in identifying and measuring objects. Further image processing is performed to create images more suitable for human observation or transmission to instrumentation. As a scientific discipline, CV studies related theories and technologies, attempting to build artificial intelligence systems capable of extracting information from images or multidimensional data. CV technologies typically include image processing, image recognition, image semantic understanding, image retrieval, optical character recognition (OCR), video processing, video semantic understanding, video content / action recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping, and common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0065] The method provided in this application can be applied to the implementation environment shown in Figure 2, which includes an XR device and a terminal 130, wherein the XR device includes an avatar device 110 and a handle device 120. The XR device and the terminal 130 can communicate through a communication network 140. The communication network 140 uses standard communication technology and / or protocols, typically the Internet, but can also be any network, including but not limited to Bluetooth, local area network (LAN), metropolitan area network (MAN), wide area network (WAN), mobile, dedicated network or any combination of virtual private networks). In some embodiments, customized or dedicated data communication technology can be used to replace or supplement the above-mentioned data communication technology.

[0066] The head-mounted display device 110 involved in this application may also be referred to as a head-mounted display (HMD). By sending optical signals to the eyes through the head-mounted display device 110, different effects such as VR, augmented reality (AR), and mixed reality (MR) can be achieved.

[0067] The handle device 120 involved in this application is an operation controller. Typically, the head display device 110 is accompanied by a handle device 120, and the user can use the handle device 120 to perform operations, such as buttons, touchpads, and triggers, to complete actions in the virtual space.

[0068] The terminals 130 involved in this application include, but are not limited to, mobile phones, tablet computers, laptop computers, desktop computers, intelligent voice interaction devices, smart home appliances, vehicle-mounted terminals, aircraft, etc. The client is deployed on the terminal 130 and can be run on the terminal 130 in the form of a browser or an independent APP.

[0069] In conjunction with the above implementation environment, after entering calibration mode, in step A1, the user can view the target object model linked to the handle device 120 through the head-mounted display device 110. In step A2, the user aligns the contact plane of the handle device 120 with the contact plane of the terminal 130. At this point, in step A3, the user can view through the head-mounted display device 110 that the camera of the target object model overlaps with the camera of the terminal 130. Based on this, in step A4, the user triggers a spatial calibration operation for the virtual camera, thereby achieving calibration of the virtual camera.

[0070] After entering the recording mode, in step A5, the head-mounted display device 110 sends a virtual scene video stream to the terminal 130 via the communication network 140, wherein the virtual scene video stream includes multiple frames of virtual foreground images and multiple frames of virtual background images. At the same time, in step A6, the terminal 130 is recording a real scene video stream, wherein the real scene video stream includes multiple frames of real scene images. The terminal 130 calls the deep learning cutout model to perform portrait cutout on each frame of the real scene image to obtain each frame of cutout image. In step A7, the terminal 130 fuses the cutout image, virtual foreground image, and virtual background image corresponding to the same timestamp to obtain a frame of synthesized recorded image. The synthesized video stream is obtained based on the synthesized recorded images of each frame.

[0071] It should be noted that the virtual camera calibration method provided in the embodiment of the present application can be implemented by a computer device, which can be an XR device, or an XR device and a terminal, or an XR device, a terminal and a server. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud computing services. The terminal can be a smartphone, a tablet computer, a laptop computer, etc., but is not limited to this. The terminal and the server can be directly or indirectly connected via wired or wireless communication, and this application does not limit this.

[0072] In combination with the above introduction, the virtual camera calibration method in this application will be introduced below. Please refer to Figure 3. The virtual camera calibration method in the embodiment of this application can be completed by the XR device independently, or by the XR device and the terminal in cooperation, or by the XR device, the terminal and the server in cooperation. The method of this application includes:

[0073] 210. When a target object model corresponding to the physical camera device is obtained, the target object model linked to the operation controller is displayed via the head-mounted display device, wherein a contact plane of the target object model is aligned with a contact plane of the operation controller, and a camera position of the target object model corresponds to a camera position of the physical camera device;

[0074] In one or more embodiments, a user can select a target object model through a terminal, and use the target object model to simulate the appearance of a physical shooting device, that is, the target object model is a model that simulates the appearance of a physical shooting device. Based on this, the user can display the target object model that is linked to the operation controller through a head-mounted display device. Among them, the head-mounted display device involved in this application can specifically be an XR device, which can achieve different effects such as VR, AR, and MR. It should be noted that the physical shooting device involved in this application can be a hardware device that actually exists and is used to shoot images or videos, and is used to capture images of real scenes. The physical shooting device can be a terminal (for example, a mobile phone, a tablet computer, etc.) or an independent camera device, which is not limited here.

[0075] The operation controller can be an input device for interacting with a virtual scene. The embodiment of the present application does not limit the type of the operation controller. For example, it can be a handle device, a glove controller, etc., wherein the handle device can be a handle device of various shapes, for example, it can be a ring handle device.

[0076] For ease of understanding, taking the operation controller as a handle device as an example, please refer to Figure 4. Figure 4 is a schematic diagram of the linkage between the handle device and the target object model in an embodiment of the present application. As shown in the figure, B1 is used to indicate the handle device, wherein the annular surface of the handle device belongs to the contact plane of the handle device. B2 is used to indicate the target object model. B3 is used to indicate the camera of the target object model. Since the target object model can simulate the appearance of the physical shooting device, the camera position of the target object model has a corresponding relationship with the camera position of the physical shooting device, that is, the camera position of the target object model is the same as or close to the camera position of the physical shooting device (for example, the deviation range is less than or equal to 3 mm).

[0077] Since the contact plane of the target object model remains in contact with the contact plane of the handle device, when the handle device moves, the target object model will also move in a corresponding manner. Taking the Quest platform using the Unity engine as an example, before the virtual camera is calibrated, a target object model is fitted on the contact plane of the handle device, that is, a rectangular model is fitted on the contact plane (for example, a torus) of the handle model node as the target object model. At the initial moment, the target object model uses the handle model node as the parent node. Therefore, when the position of the target object model relative to the handle device remains unchanged, it will always be fitted on the contact plane of the handle device.

[0078] Understandably, developing applications on the Quest platform requires its software development kit (SDK). The SDK is a software library that contains some platform interfaces and resources. For example, the SDK can be used to obtain information such as whether a controller button is pressed and the official controller model.

[0079] 220. When the contact plane of the operating controller is in contact with the contact plane of the physical photographing device, the camera displaying the target object model through the head display device and the camera of the physical photographing device are in an overlapping state;

[0080] In one or more embodiments, the target object model can guide the user to use the operation controller to fit the physical shooting device, which has a better effect in the interaction effect. Based on the see-through technology, the user can see the augmented reality scene through the head-mounted display device. Based on this, the user can use the operation controller to fit the physical shooting device. When the operation controller and the physical shooting device are accurately fitted, the user can see through the head-mounted display device that the camera of the target object model and the camera of the physical shooting device are overlapping. It should be noted that the "overlapping state" involved above can be complete overlap (i.e., 100% overlap) or an overlap greater than a certain threshold (for example, the overlap rate reaches more than 95%).

[0081] It can be understood that see-through refers to a technology that can capture real scenes through the forward-facing camera in the head-mounted display device and display them on the screen in real time, allowing users to see the real scenes even when wearing the head-mounted display device.

[0082] For ease of understanding, taking the operation controller as a handle device as an example, please refer to Figure 5. Figure 5 is a schematic diagram of the fit between the handle device and the physical shooting device in an embodiment of the present application. As shown in the figure, B1 is used to indicate the handle device, wherein the annular surface of the handle device belongs to the contact plane of the handle device. B4 is used to indicate the physical shooting device. B5 is used to indicate the camera of the physical shooting device. When the handle device and the physical shooting device are accurately fitted, the camera of the physical shooting device indicated by B5 overlaps with the camera of the target object model indicated by B3 in Figure 4.

[0083] 230. In response to a spatial calibration operation for the virtual camera, calibrate the virtual camera according to spatial information of the operation controller, wherein the spatial information includes target position information and target orientation angle.

[0084] In one or more embodiments, if the contact plane of the operation controller is in contact with the contact plane of the physical shooting device, and the camera of the target object model is in an overlapping state with the camera of the physical shooting device, the user can trigger a specific key event through the operation controller, thereby triggering a spatial calibration operation for the virtual camera. Then, the parent node of the target object model is updated to the scene root node. Based on this, in the virtual scene, the target object model no longer follows the operation controller, and its position relative to the scene remains unchanged. Therefore, at this time, the position information of the camera of the target object model is the position information of the virtual camera, and the orientation angle of the operation controller is the orientation angle of the virtual camera.

[0085] The camera position of the target object model is pre-set. The transformation information (transform) of the operation controller in the world coordinate system can be directly obtained through the existing methods in the XR development platform SDK. Among them, the transformation information includes target position information (position) and current orientation angle (rotation). Based on this, according to the positional relationship between the camera position of the target object model and the handle model, as well as the target position information of the operation controller in the world coordinate system, the position information of the camera of the target object model in the world coordinate system can be derived. In addition, based on the current orientation angle of the operation controller and the plane vertical vector of the operation controller, the target orientation angle of the operation controller can be calculated. Among them, the plane vertical vector represents a vector perpendicular to the contact plane of the operation controller.

[0086] It is understood that, based on the spatial information of the operating controller, on the one hand, the position information of the camera of the target object model in the world coordinate system can be derived, and thus, the position information of the camera of the target object model in the world coordinate system can be used as the position information of the virtual camera in the world coordinate system. On the other hand, the target heading angle of the operating controller can be calculated and used as the heading angle of the virtual camera in the world coordinate system.

[0087] It should be noted that the term "in response to" in this application is used to indicate the conditions or states upon which the execution of an operation depends. When certain conditions or states are met, one or more operations may be executed. These operations may be in real time or with a certain delay.

[0088] In an embodiment of the present application, a virtual camera calibration method is provided. Through the above-described method, the virtual camera can be calibrated based on the position information and orientation angle of the operating controller. This eliminates the need for additional calculations when calibrating the virtual camera and achieves a high degree of accuracy, thereby improving the efficiency of virtual camera calibration.

[0089] It is understood that physical camera devices may have multiple models, and the appearance of different physical camera devices may vary. Therefore, corresponding object models can be established for different physical camera device models. Therefore, in order to improve the accuracy of the target object model, the target object model can be obtained based on the model information of the physical camera device.

[0090] It should be noted that object models of different models of physical shooting devices can be stored in different locations, such as locally or on a server. Based on different storage methods, the method of obtaining the target object model according to the model information of the physical shooting device may be different. Based on one or more embodiments corresponding to the above Figure 3, another optional embodiment provided by the embodiment of the present application, when the target object model corresponding to the physical shooting device is obtained, before the target object model linked to the operation controller is displayed through the head display device, it may also include:

[0091] In response to a model call request, the target object model corresponding to the entity shooting device is obtained from a locally stored model resource collection, wherein the model call request carries the model information of the entity shooting device, the model resource collection includes at least one object model, each object model corresponds to a model information, and the target object model corresponds to the model information carried by the model call request.

[0092] In one or more embodiments, a method for determining the target object model is introduced. It can be seen from the aforementioned embodiments that the physical shooting device or terminal can read the model information of the physical shooting device and send a model call request carrying the model information to the XR device. If the physical shooting device and the terminal (for example, a mobile phone, a tablet computer, etc.) are the same device, the physical shooting device will send its own model information to the XR device. If the physical shooting device (for example, an external camera, etc.) and the terminal (for example, a mobile phone, a tablet computer, etc.) are not the same device, after the physical shooting device establishes a communication connection with the terminal, the terminal can read the model information of the physical shooting device and send the model information of the physical shooting device to the XR device.

[0093] After receiving a model call request, the XR device first parses the request to obtain the model information of the physical camera. Then, based on the physical camera model information, it queries the locally stored model resource collection to determine whether an object model corresponding to the physical camera model exists. If an object model corresponding to the physical camera model exists, it is used as the target object model.

[0094] Secondly, in an embodiment of the present application, a method for determining a target object model is provided. Through the above method, the XR device reads the object model corresponding to the model information locally, thereby obtaining a target object model that matches the appearance of the physical camera device, thereby obtaining more accurate location information.

[0095] In another optional embodiment provided in accordance with one or more embodiments corresponding to FIG. 3 above, after obtaining a target object model corresponding to a physical camera, the following steps may be further included before displaying the target object model linked to an operation controller via a head-mounted display device:

[0096] In response to the model call request, sending a model download request to the server, wherein the model call request carries the model information of the physical shooting device, and the model download request carries the model information of the physical shooting device;

[0097] A target object model corresponding to the physical shooting device sent by the server is received, wherein the target object model corresponds to the model information carried in the model download request.

[0098] In one or more embodiments, another method for determining the target object model is introduced. As can be seen from the above embodiments, the physical camera device or terminal can read the model information of the physical camera device and send a model call request carrying the model information to the XR device.

[0099] After receiving the model call request, the XR device first parses the model call request to obtain the model information of the physical camera device. Then, the model information of the physical camera device is encapsulated into a model download request and the model download request is sent to the server. Based on this, after receiving the model download request, the server parses the model download request to obtain the model information of the physical camera device. Then, based on the model information of the physical camera device, the server queries the model resource collection to see whether there is an object model corresponding to the model information of the physical camera device. If there is an object model corresponding to the model information of the physical camera device, the object model is used as the target object model. Finally, the server feeds back the target object model to the XR device.

[0100] Secondly, in the embodiment of the present application, another method of determining the target object model is provided. Through the above method, the XR device downloads the object model that matches the model information from the server. On the one hand, it is possible to obtain a target object model that matches the appearance of the physical shooting device, thereby obtaining more accurate location information. On the other hand, storing various object models on the server side can save local storage resources of the XR device, which is conducive to optimizing the performance of the XR device.

[0101] In another optional embodiment provided in accordance with one or more embodiments corresponding to FIG. 3 above, after acquiring a target object model corresponding to a physical camera, displaying the target object model linked to an operation controller via a head-mounted display device may further include:

[0102] Displaying a first prompt message through the head-mounted display device, wherein the first prompt message is used to prompt the operator to align the contact plane of the operation controller with the contact plane of the physical shooting device, and to overlap the target object model with the physical shooting device;

[0103] or,

[0104] Playing a first voice message through the head-mounted display device, wherein the first voice message is used to prompt the operator to align the contact plane of the operation controller with the contact plane of the physical camera device, and to overlap the target object model with the physical camera device;

[0105] or,

[0106] The first prompt message is displayed through the head display device, and the first voice message is played.

[0107] In one or more embodiments, a method for prompting users to perform relevant operations when using a target object model is described. As can be seen from the aforementioned embodiments, based on see-through technology, users can view an augmented reality scene through a head-mounted display. Therefore, the augmented reality scene can be used to guide users in performing relevant operations. Based on this, after the user sees the target object model linked to the operation controller, a prompt can be provided to guide the user to align the contact surface of the operation controller with the contact surface of the physical camera device.

[0108] The following will be combined with Figure 6, taking the operation controller being a handle device as an example to introduce different prompting methods. Figure 6 is a schematic diagram of prompting the use of the handle device to fit the physical shooting device in an embodiment of the present application.

[0109] Method 1: Interface prompt;

[0110] During implementation, please refer to Figure 6 (A), where C1 indicates the first prompt message. After enabling see-through, the user can see not only the target object model linked to the handle device through the head-mounted display, but also the physical camera device. Based on this, the first prompt message indicated by C1 can guide the user to align the contact plane of the handle device with the contact plane of the physical camera device. When the alignment is accurate, the camera of the target object model and the camera of the physical camera device are in an overlapping state.

[0111] Method 2: Voice prompt;

[0112] During implementation, please refer to Figure (B) in Figure 6 , where C2 is used to indicate the first voice message. After enabling see-through, the user can see the target object model linked to the handle device and the physical camera through the head-mounted display. Based on this, the first voice message indicated by C2 can guide the user to align the contact plane of the handle device with the contact plane of the physical camera. When the alignment is accurate, the camera of the target object model and the camera of the physical camera are in an overlapping state.

[0113] Method 3: Combination of interface prompts and voice prompts;

[0114] During implementation, please refer to Figure (C) in Figure 6, where C3 is used to indicate the first voice message and C4 is used to indicate the first prompt message. After turning on see-through, the user can see the target object model and the physical shooting device linked to the handle device through the head-mounted display device. Based on this, the first voice message indicated by C3 and the first prompt message indicated by C4 can guide the user to align the contact plane of the handle device with the contact plane of the physical shooting device. When the alignment is accurate, the camera of the target object model and the camera of the physical shooting device are in an overlapping state.

[0115] Secondly, an embodiment of the present application provides a method for prompting the user to perform relevant operations when using a target object model. Through this method, the head-mounted display device can provide the user with relevant prompts for aligning the operation controller with the physical camera device. This can more intuitively guide the user to perform the correct operation, reduce the learning cost, and thus improve the effectiveness of virtual camera calibration.

[0116] Based on one or more embodiments corresponding to FIG. 3 above, another optional embodiment provided in the embodiment of the present application may further include:

[0117] A first demonstration animation is displayed through a head-mounted display device, wherein the first demonstration animation is used to guide a process of calibrating a virtual camera using an operation controller.

[0118] In one or more embodiments, a method for playing a demonstration animation using a target object model is described. As can be seen from the aforementioned embodiments, before calibrating a virtual camera, a user can view a first demonstration animation through a head-mounted display (HMD) or a terminal. This first demonstration animation allows the user to more intuitively understand how to calibrate the virtual camera. The following describes the first demonstration animation with diagrams.

[0119] Taking the operation controller as a handle device as an example, for ease of understanding, please refer to Figure 7. Figure 7 is a schematic diagram showing the first demonstration animation in an embodiment of the present application. As shown in the figure, the first demonstration animation prompts the user to first fix the position and orientation of the physical shooting device through the information indicated by D1, and put on the head display device and hold the handle device to enter the virtual scene. The prompt content indicated by D1 can be presented in text form or in voice form. Next, the first demonstration animation prompts the user to align the contact plane of the handle device with the contact plane of the physical shooting device through the information indicated by D2. Finally, the first demonstration animation prompts the user to press the button B of the handle device through the information indicated by D3 when the contact plane of the handle device is aligned with the contact plane of the physical shooting device to achieve automatic calibration of the virtual camera.

[0120] It should be noted that the “button B” of the handle device is only a schematic illustration. In actual applications, the user can customize other buttons of the handle device as “button B”, which is not limited here.

[0121] Secondly, an embodiment of the present application provides a method for playing a demonstration animation using a target object model. Through this method, the demonstration animation guides the user through the process of calibrating the virtual camera using the target object model. The demonstration animation enhances user understanding through a visual experience and helps users understand the calibration process in a simple and understandable manner, thereby improving the communication of the message.

[0122] Based on one or more embodiments corresponding to FIG. 3 above, in another optional embodiment provided by the present application, in response to a spatial calibration operation for a virtual camera, calibrating the virtual camera according to spatial information of an operation controller may specifically include:

[0123] In response to a spatial calibration operation for the virtual camera, obtaining target position information and a current orientation angle of the operation controller;

[0124] Determine the position information of the virtual camera according to the target position information and the camera position of the target object model;

[0125] Determining a target orientation angle of the manipulation controller according to the current orientation angle and a plane perpendicular vector of the manipulation controller, wherein the plane perpendicular vector represents a vector perpendicular to a contact plane of the manipulation controller;

[0126] The virtual camera is calibrated using its position information and target orientation angle.

[0127] In one or more embodiments, a method for calibrating a virtual camera using a target object model is described. As can be seen from the aforementioned embodiments, when the contact plane of the operation controller aligns with the contact plane of the physical camera, and the camera of the target object model overlaps with the camera of the physical camera, the user can trigger a spatial calibration operation for the virtual camera. Based on this, calling the "object.getTransform" interface based on the Unity engine can obtain the transformation information of the operation controller in the world coordinate system, that is, obtain the target position information and current orientation angle of the operation controller in the world coordinate system.

[0128] It should be noted that the "position information" involved in this application can be expressed as coordinates (x, y, z), and the "orientation angle" involved in this application can be expressed as Euler angles (xAngle, yAngle, zAngle). The following describes how to determine the position information of a virtual camera and the orientation angle of a virtual camera.

[0129] 1. The location information of the virtual camera;

[0130] On the one hand, since the camera position of the target object model is pre-set, when the camera of the target object model overlaps with the camera of the physical camera, the camera position of the target object model can be directly used as the camera position of the physical camera. On the other hand, since the positional relationship between the target object model and the handle model is also pre-set, the positional offset between the camera position of the target object model and the operation controller (i.e., the offset in the three directions of the X-axis, Y-axis, and Z-axis) is also known.

[0131] In summary, when the camera of the target object model overlaps with the camera of the physical shooting device, the target position information of the operation controller in the world coordinate system is offset calculated based on the position offset between the camera position of the target object model and the operation controller, and the position information of the camera of the target object model in the world coordinate system is obtained. Assume that the position offset is expressed as (Δx, Δy, Δz), and the target position information is expressed as (x t ,y t ,z t ), then the position information of the camera of the target object model in the world coordinate system can be expressed as (x t +Δx,y t +Δy,z t +Δz).

[0132] It can be understood that the position information of the camera of the target object model in the world coordinate system is also the position information of the camera of the physical shooting device in the world coordinate system. Therefore, the position information of the camera of the physical shooting device in the world coordinate system can also be expressed as (x t +Δx,y t +Δy,z t +Δz). When calibrating the virtual camera, it is necessary to unify the virtual camera's position information in the world coordinate system with the position information of the physical camera's camera in the world coordinate system. Therefore, the virtual camera's position information can be directly used as the virtual camera's position information in the world coordinate system to complete the virtual camera's position calibration.

[0133] 2. The orientation angle of the virtual camera;

[0134] Taking the operation controller as an example of a handle device, for ease of understanding, please refer to Figure 8, which is a schematic diagram of determining a plane vertical vector based on a handle device in an embodiment of the present application. As shown in the figure, taking a ring handle device as an example, there is a ring on the ring handle device, and the ring constitutes the contact plane of the handle device. Among them, the plane vertical vector is a vector perpendicular to the contact plane of the handle device, and the plane vertical vector and the initial orientation angle of the handle device are at a fixed angle. Therefore, the plane vertical vector can be obtained by calculating the cosine value of the rotation angle component of the plane vertical vector and the rotation angle component of the handle device.

[0135] The following example illustrates the process of calculating the target orientation angle of a handle device.

[0136] Assume that the initial orientation angle of the controller device is represented by "rotation_init". The initial orientation angle can be set to (0,0,0). The initial orientation angle of the controller device is the orientation angle of the controller device captured when the XR device is started.

[0137] Assume that the plane perpendicular vector is represented by "n". Here, the plane perpendicular vector can be set to "(x1, y1, z1)".

[0138] Assume that the current orientation angle of the controller device is represented by "rotation_ct". The current orientation angle can be set to (angle_x, angle_y, angle_z). The current orientation angle of the controller device is the orientation angle of the controller device captured when the user triggers the spatial calibration operation.

[0139] Assume that the target orientation angle of the controller device in the world coordinate system is represented as "n_rotated".

[0140] First, the plane perpendicular vector is converted to a unit vector (ie, a normalized vector) in the following manner: n_normalized = n / ||n||; Formula (1)

[0141] Where n_normalized represents a unit vector and n represents a plane-normal vector.

[0142] Then, calculate the rotation matrix of the handle device as follows:

[0143] Among them, R x Indicates the rotation matrix corresponding to the angle_x angle of the handle device around the X axis. y Indicates the rotation matrix corresponding to the angle_y angle of the handle device around the Y axis. zIndicates the rotation matrix corresponding to the controller device rotating by angle_z around the Z axis. (angle_x, angle_y, angle_z) represents the current heading angle.

[0144] Therefore, the final target rotation matrix can be obtained by multiplying each rotation matrix in sequence as follows: R = R z *R y *R x ; Formula (5)

[0145] Where R represents the target rotation matrix.

[0146] Finally, the target orientation angle of the handle device is calculated as follows: n_rotated = R*n_normalized; Formula (6)

[0147] Where n_rotated represents the target orientation angle of the controller device in the world coordinate system, R represents the target rotation matrix, and n_normalized represents the unit vector.

[0148] It's understandable that the target heading angle of the controller device in the world coordinate system is also the heading angle of the physical camera device's camera in the world coordinate system. When calibrating the virtual camera's heading angle, it's necessary to unify the virtual camera's heading angle in the world coordinate system with the physical camera device's camera's heading angle in the world coordinate system. Therefore, simply using the controller device's target heading angle in the world coordinate system as the virtual camera's heading angle in the world coordinate system can complete the virtual camera's heading angle calibration.

[0149] Secondly, in an embodiment of the present application, a method for calibrating a virtual camera using a target object model is provided. Through this method, on the one hand, the position information of the virtual camera can be directly determined using the known target position information and the camera position of the target object model. On the other hand, the orientation angle of the virtual camera can be calculated using the known current orientation angle and the plane perpendicular vector. As can be seen, the steps for calibrating the virtual camera are relatively simple, no additional calculations are required, and a highly accurate calibration effect can be achieved.

[0150] Based on one or more embodiments corresponding to FIG. 3 above, another optional embodiment provided by the embodiment of the present application may further include:

[0151] If the target object model corresponding to the physical shooting device is not obtained, then when the operation controller is in contact with the camera of the physical shooting device, in response to the position recording operation for the operation controller, obtaining target position information of the operation controller;

[0152] When the contact plane of the operation controller is aligned with the contact plane of the physical camera, in response to an angle recording operation on the operation controller, obtaining a target orientation angle of the operation controller;

[0153] The virtual camera is calibrated according to the target position information and the target orientation angle.

[0154] In one or more embodiments, a method for calibrating a virtual camera without a target object model is described. As can be seen from the aforementioned embodiments, if a target object model that matches the physical camera cannot be found, it is necessary to additionally record the target position information of the operating controller in the world coordinate system. In this case, the target position information is the position information of the camera of the physical camera in the world coordinate system.

[0155] During implementation, the user can use the operation controller to touch the camera of the physical camera device. When the contact is made, the position recording operation of the operation controller is triggered to obtain the target position information of the operation controller. In addition, the user needs to align the contact plane of the operation controller with the contact plane of the physical camera device. When the contact is made, the angle recording operation of the operation controller is triggered to obtain the current orientation angle of the operation controller.

[0156] It is understood that, on the one hand, the target position information of the operation controller can be directly used as the position information of the virtual camera in the world coordinate system. On the other hand, the target heading angle of the operation controller can be calculated based on the current heading angle and the plane vertical vector, and thus, the target heading angle can be used as the heading angle of the virtual camera in the world coordinate system.

[0157] Secondly, an embodiment of the present application provides a method for calibrating a virtual camera without a target object model. This method allows the user to use an operating controller to touch the camera of the physical camera if a target object model matching the physical camera cannot be obtained. This increases the flexibility and operability of the solution.

[0158] In another optional embodiment provided in accordance with one or more embodiments corresponding to FIG. 3 above, when an operation controller is in contact with a camera of a physical photographing device, in response to a position recording operation on the operation controller, before obtaining target position information of the operation controller, the following steps may be further included:

[0159] Displaying a second prompt message through the head-mounted display device, wherein the second prompt message is used to prompt the operator to contact the camera of the physical shooting device;

[0160] or,

[0161] Playing a second voice message through the head-mounted display device, wherein the second voice message is used to prompt the operator to touch the operation controller to the camera of the physical shooting device;

[0162] or,

[0163] The second prompt message is displayed through the head display device, and the second voice message is played.

[0164] In one or more embodiments, a method for prompting a user to perform a location recording operation is described. As can be seen from the aforementioned embodiments, based on see-through technology, a user can see an augmented reality scene through a head-mounted display device. Therefore, the augmented reality scene can be used to guide the user in performing relevant operations. Based on this, before the user triggers the location recording operation, a prompt can be provided to guide the user in placing an operating controller in contact with the camera of a physical camera device.

[0165] The following will be combined with Figure 9, taking the operation controller being a handle device as an example to introduce different prompting methods. Figure 9 is a relevant schematic diagram of prompting the use of a handle device to touch the camera of a physical shooting device in an embodiment of the present application.

[0166] Method 1: Interface prompt;

[0167] During implementation, please refer to Figure 9 (A), where E1 indicates the second prompt message. After enabling see-through, the user can not only see the relevant prompts through the head-mounted display device, but also see the physical camera device. Based on this, the second prompt message indicated by E1 can guide the user to touch the handle device with the camera of the physical camera device. If the contact is accurate, the user can trigger the location recording operation for the handle device.

[0168] Method 2: Voice prompt;

[0169] In implementation, please refer to Figure 9 (B), where E2 indicates the second voice message. After enabling see-through, the user can not only see relevant prompts through the head-mounted display device, but also see the physical camera device. Based on this, the second voice message indicated by E2 can guide the user to touch the handle device with the camera of the physical camera device. If the contact is accurate, the user can trigger the location recording operation for the handle device.

[0170] Method 3: Combination of interface prompts and voice prompts;

[0171] In implementation, refer to Figure 9 (C), where E3 indicates the second voice message and E4 indicates the second prompt message. After enabling see-through, the user can not only see the relevant prompts through the head-mounted display, but also the physical camera. Based on this, the second voice message indicated by E3 and the second prompt message indicated by E4 can guide the user to place the controller device in contact with the camera of the physical camera. If the contact is accurate, the user can trigger a location recording operation for the controller device.

[0172] Furthermore, an embodiment of the present application provides a method for prompting the user to perform a position recording operation. Through this method, the head-mounted display device can provide the user with relevant prompts for using the operation controller to record the camera position of the physical camera device. This can more intuitively guide the user to perform the correct operation, reduce the learning cost, and thus improve the effectiveness of virtual camera calibration.

[0173] In another optional embodiment provided in accordance with one or more embodiments corresponding to FIG. 3 above, when the contact plane of the operation controller aligns with the contact plane of the physical camera, in response to the angle recording operation on the operation controller, before obtaining the target orientation angle of the operation controller, the following steps may be further included:

[0174] Displaying a third prompt message through the head-mounted display device, wherein the third prompt message is used to prompt the operator to align the contact plane of the operation controller with the contact plane of the physical camera device;

[0175] or,

[0176] Playing a third voice message through the head-mounted display device, wherein the third voice message is used to prompt the operator to align the contact plane of the operation controller with the contact plane of the physical camera device;

[0177] or,

[0178] The third prompt message is displayed through the head display device, and the third voice message is played.

[0179] In one or more embodiments, a method for prompting a user to perform an angle recording operation is described. As can be seen from the aforementioned embodiments, based on see-through technology, a user can see an augmented reality scene through a head-mounted display. Therefore, the augmented reality scene can be used to guide the user in performing relevant operations. Based on this, before the user triggers the angle recording operation, a prompt can be provided to guide the user in aligning the operating controller with the physical camera device.

[0180] The following will be combined with Figure 10, taking the operation controller being a handle device as an example to introduce different prompting methods. Figure 10 is a relevant schematic diagram of prompting the use of a handle device to fit the physical shooting device in an embodiment of the present application.

[0181] Method 1: Interface prompt;

[0182] During implementation, please refer to Figure 10 (A), where F1 indicates the third prompt message. After enabling see-through, the user can not only see the relevant prompts through the head-mounted display, but also see the physical camera. Based on this, the third prompt message indicated by F1 can guide the user to align the contact plane of the handle device with the contact plane of the physical camera. If the alignment is accurate, the user can trigger the angle recording operation for the handle device.

[0183] Method 2: Voice prompt;

[0184] In implementation, please refer to Figure 10 (B), where F2 is used to indicate a third voice message. After enabling see-through, the user not only sees relevant prompts through the head-mounted display, but also sees the physical camera. Based on this, the third voice message indicated by F2 can guide the user to align the contact plane of the controller device with the contact plane of the physical camera device. When the alignment is accurate, the user can trigger an angle recording operation for the controller device.

[0185] Method 3: Combination of interface prompts and voice prompts;

[0186] During implementation, please refer to Figure 10 (C), where F3 indicates the third voice message and F4 indicates the third prompt message. After enabling see-through, the user can not only see the relevant prompts through the head-mounted display device, but also see the physical camera device. Based on this, the third voice message indicated by F3 and the fourth prompt message indicated by F4 can guide the user to align the contact plane of the handle device with the contact plane of the physical camera device. If the alignment is accurate, the user can trigger the angle recording operation for the handle device.

[0187] Furthermore, an embodiment of the present application provides a method for prompting the user to perform an angle recording operation. Through this method, the head-mounted display device can provide the user with relevant prompts for using the operation controller to align with the physical camera device. This can more intuitively guide the user to perform the correct operation, reduce learning costs, and further improve the effectiveness of virtual camera calibration.

[0188] In another optional embodiment provided in accordance with one or more embodiments corresponding to FIG. 3 above, if the target object model corresponding to the physical photographing device is not obtained, then when the operation controller is in contact with the camera of the physical photographing device, in response to a position recording operation for the operation controller, obtaining target position information of the operation controller may specifically include:

[0189] If the target object model corresponding to the physical shooting device is not obtained, a second demonstration animation is displayed through the head display device, wherein the second demonstration animation is used to guide the process of calibrating the virtual camera using the operation controller;

[0190] In response to a playback end operation for the second demonstration animation, when the operation controller is in contact with a camera of the physical shooting device, in response to a position recording operation for the operation controller, target position information of the operation controller is acquired.

[0191] In one or more embodiments, a method for playing a demonstration animation without using a target object model is described. As can be seen from the aforementioned embodiments, before performing virtual camera calibration, the user can view a related second demonstration animation through a head-mounted display device or terminal. This second demonstration animation allows the user to more intuitively understand how to calibrate the virtual camera. The following describes the relevant content of this second demonstration animation with reference to diagrams.

[0192] Taking the operation controller as a handle device as an example, for ease of understanding, please refer to Figure 11. Figure 11 is a schematic diagram showing the second demonstration animation in an embodiment of the present application. As shown in the figure, the second demonstration animation prompts the user to first fix the position and orientation of the physical shooting device through the information indicated by G1, and then put on the head display device and hold the handle device to enter the virtual scene. The prompt content indicated by G1 can be presented in text form or in voice form. Next, the second demonstration animation prompts the user to use the handle device to contact the camera position of the physical shooting device through the information indicated by G2, and press button A to trigger the position recording operation. Finally, the second demonstration animation prompts the user to press button B of the handle device to trigger the angle recording operation when the contact plane of the handle device is aligned with the contact plane of the physical shooting device through the information indicated by G3. This achieves automatic calibration of the virtual camera.

[0193] It should be noted that the "Button A" and "Button B" of the handle device are only for reference. In actual application, the user can customize other buttons of the handle device as "Button A" or "Button B", which is not limited here.

[0194] Furthermore, an embodiment of the present application provides a method for playing a demonstration animation without using a target object model. Through this method, the demonstration animation guides the user through the virtual camera calibration process. The demonstration animation enhances user understanding through a visual experience and helps users understand the calibration process in a simple and understandable manner, thereby improving the message conveyed.

[0195] Based on one or more embodiments corresponding to FIG. 3 above, in another optional embodiment provided by the present application, in response to an angle recording operation on an operation controller, obtaining a target orientation angle of the operation controller may specifically include:

[0196] In response to an angle recording operation on the operation controller, obtaining a current orientation angle of the operation controller;

[0197] The target orientation angle of the operation controller is determined according to the current orientation angle and the plane perpendicular vector of the operation controller, wherein the plane perpendicular vector represents a vector perpendicular to the contact plane of the operation controller.

[0198] In one or more embodiments, a method for obtaining a target orientation angle based on an angle recording operation is described. As can be seen from the aforementioned embodiments, when the contact plane of an operating controller aligns with the contact plane of a physical camera, the user can trigger an angle recording operation for the virtual camera. Based on this, calling the "object.getTransform" interface within the Unity engine can obtain the transformation information of the operating controller in the world coordinate system, namely, the target position information and current orientation angle of the operating controller in the world coordinate system.

[0199] It should be noted that the method for calculating the target orientation angle of the operation controller based on the current orientation angle and the plane vertical vector of the operation controller can refer to formulas (1) to (6) of the above embodiments and is not limited here.

[0200] Furthermore, in an embodiment of the present application, a method for obtaining the target heading angle based on an angle recording operation is provided. Through this method, the heading angle of the virtual camera can be calculated based on the current heading angle and the plane perpendicular vector. This eliminates the need for additional calculations and achieves a highly accurate calibration effect.

[0201] Based on one or more embodiments corresponding to FIG. 3 above, in another optional embodiment provided by the embodiment of the present application, N pressure sensors are deployed on the contact plane of the operation controller, where N is an integer greater than 1;

[0202] When the contact plane of the operation controller is aligned with the contact plane of the physical shooting device, before the camera displaying the target object model through the head display device and the camera of the physical shooting device are overlapped, the method may further include:

[0203] Acquire at least one set of pressure parameter sets through N pressure sensors, wherein each set of pressure parameter sets includes at least one pressure value, and the at least one pressure value is derived from at least one pressure sensor deployed continuously;

[0204] According to each set of pressure parameters, a contact state between a contact plane of the operation controller and a contact plane of the physical shooting device is determined.

[0205] In one or more embodiments, a method for detecting the fit of a device using pressure sensors is described. As can be seen from the aforementioned embodiments, N pressure sensors can be deployed on the contact surface of the operating controller, each of which is used to collect a corresponding pressure value. Based on the pressure values ​​collected by each pressure sensor, pressure balance can be detected to determine the fit between the operating controller and the physical camera device.

[0206] The following is an introduction using the example of an operation controller that is a ring handle device. Among them, there is a ring on the ring handle device, and there is a circle of infrared light strips on the ring. The XR device can locate the position and angle of the handle in space through an infrared camera. For ease of understanding, please refer to Figure 12, which is a schematic diagram of a device fitting state based on a pressure sensor detection device in an embodiment of the present application, wherein H1 to H8 respectively represent pressure sensors deployed on the contact plane of the ring handle device. In this case, N is equal to 8. Three different situations are shown in Figure 12, which will be analyzed separately below.

[0207] Case 1;

[0208] As shown in Figure 12 (A), two sets of pressure parameter sets are currently obtained. Therefore, it can be determined that the ring handle device and the physical shooting device are in contact. Among them, the first set of pressure parameter sets includes the pressure values ​​detected by pressure sensor H2, the pressure values ​​detected by pressure sensor H3, and the pressure values ​​detected by pressure sensor H4, and the second set of pressure parameter sets includes the pressure values ​​detected by pressure sensor H6, the pressure values ​​detected by pressure sensor H7, and the pressure values ​​detected by pressure sensor H8. Since three points can determine a plane, if there are at least three pressure sensors that can detect pressure values, and the difference between these pressure values ​​is small, it means that the contact plane of the ring handle device and the contact plane of the physical shooting device are in contact.

[0209] For example, taking a set of pressure parameters as an example, assuming that the difference between the maximum and minimum pressure values ​​in the set of pressure parameters is less than or equal to the pressure difference threshold, it means that the difference between these pressure values ​​is small. In actual applications, other methods can also be used to determine the size of the pressure difference, which is not limited here.

[0210] Case 2:

[0211] As shown in Figure 12 (B), a set of pressure parameter sets is currently obtained, so it can be determined that the ring handle device and the physical shooting device are in contact. Among them, the set of pressure parameter sets includes the pressure value detected by the pressure sensor H1. Although the ring handle device and the physical shooting device are in contact, since the number of pressure sensors that can detect pressure values ​​is less than 3, it is impossible to ensure that the contact plane of the ring handle device and the contact plane of the physical shooting device are in contact. Based on this, the user can be prompted to make corresponding adjustments.

[0212] Case three;

[0213] As shown in Figure 12 (C), a set of pressure parameters is currently obtained, so it can be determined that the ring handle device and the physical camera device are in contact. The set of pressure parameters includes the pressure value detected by pressure sensor H2, the pressure value detected by pressure sensor H3, and the pressure value detected by pressure sensor H4.

[0214] Based on this set of pressure parameters, we can see that while at least three pressure sensors are able to detect pressure, the pressure values ​​vary significantly. This indicates a pressure imbalance between the ring handle device and the physical camera, suggesting the ring handle device may be tilted. Based on this, the user can be prompted to make appropriate adjustments.

[0215] Secondly, in an embodiment of the present application, a method for detecting the fit of the device based on a pressure sensor is provided. Through the above method, the pressure sensor deployed on the contact surface of the ring handle device can be used to detect the fit force. Thus, by analyzing the balance of pressure, the fit between the ring handle device and the physical shooting device can be determined, thereby prompting the user to perform accurate operations, thereby improving the accuracy of virtual camera calibration.

[0216] Based on one or more embodiments corresponding to FIG. 3 above, another optional embodiment provided by the embodiment of the present application may further include:

[0217] In response to a function activation operation on the head-mounted display device, activating a see-through function of the head-mounted display device;

[0218] Displaying a target object model linked to an operation controller through a head-mounted display device may include:

[0219] Based on the perspective function of the head-mounted display device, the target object model linked with the operation controller is displayed through the head-mounted display device;

[0220] The camera that displays the target object model through the head-mounted display device overlaps with the camera of the physical shooting device, which may include:

[0221] Based on the perspective function of the head-mounted display device, the camera that displays the target object model through the head-mounted display device overlaps with the camera of the physical shooting device.

[0222] In one or more embodiments, a method for enabling a see-through feature is described. As can be seen from the aforementioned embodiments, after a user puts on a head-mounted display (HMD), if they need to see objects in a real scene (e.g., a physical camera), they also need to enable the see-through feature.

[0223] For ease of understanding, taking the operation controller as a handle device as an example, please refer to Figure 13, which is a schematic diagram of enabling the see-through function in an embodiment of the present application. As shown in Figure 13 (A), the user can see the prompt indicated by I1 in the virtual scene, for example, "Please double-tap the head-mounted display device to switch to the see-through function." Based on this, when the user double-taps the head-mounted display device, the function activation operation for the head-mounted display device is triggered, thereby enabling the see-through function of the head-mounted display device. It is understandable that in actual applications, the function activation operation can also be triggered by other means, such as voice input or gesture input.

[0224] As shown in Figure 13 (B), after turning on the see-through function, the user can see the physical camera indicated by I2 through the head display device. As a result, the user can use the handle device to align with the physical camera device.

[0225] As shown in Figure 13 (C), after turning on the see-through function, if the user accurately aligns the contact plane of the handle device with the contact plane of the physical shooting device, the user can see through the head display device that the camera of the target object model and the camera of the physical shooting device are overlapping.

[0226] Secondly, an embodiment of the present application provides a method for activating the see-through function. Through the above method, based on the see-through function, on the one hand, users can interact with the external real world without removing the head-mounted display device, thereby enhancing the sustainability of the XR device experience. On the other hand, integrating AR functions into the see-through function can enable users to experience more AR functions and realize the interaction between the real world and the virtual world.

[0227] Based on one or more embodiments corresponding to FIG. 3 above, another optional embodiment provided by the embodiment of the present application may further include:

[0228] Display a calibration prompt message, wherein the calibration prompt message is used to prompt whether to calibrate the virtual camera;

[0229] If a confirmation operation is performed in response to the calibration prompt message, the step of displaying the target object model linked to the operation controller through the head-mounted display device when the target object model corresponding to the physical shooting device is acquired is performed;

[0230] If a cancel operation is performed on the calibration prompt message, the virtual camera is calibrated according to the historical spatial information of the operation controller, wherein the historical spatial information includes the spatial information of the operation controller acquired last time.

[0231] In one or more embodiments, a method for triggering the virtual camera calibration process is introduced. As can be seen from the aforementioned embodiments, if the physical camera used for MRC is fixed in the same position, the user can skip the virtual camera calibration process when restarting the XR device. Based on this, the user can choose whether to skip the virtual camera calibration process through the terminal or XR device.

[0232] For ease of understanding, please refer to Figure 14, which is a schematic diagram of a calibration prompt message displayed in an embodiment of the present application. As shown in the figure, J1 is used to indicate a calibration prompt message, which is used to prompt the user whether the virtual camera needs to be calibrated. If the user clicks the confirmation control indicated by J2, a confirmation operation for the calibration prompt message is triggered, thereby continuing the calibration process of the virtual camera. If the user clicks the cancel control indicated by J3, a cancel operation for the calibration prompt message is triggered, thereby calibrating the virtual camera according to the historical spatial information of the operation controller. Among them, the historical spatial information includes the spatial information of the operation controller obtained last time.

[0233] Secondly, in an embodiment of the present application, a method for triggering the virtual camera calibration process is provided. In this way, the user can skip the step of calibrating the virtual camera when the position of the physical camera device remains unchanged. This can improve the flexibility of the solution and save operation time.

[0234] On the basis of one or more embodiments corresponding to FIG3 above, in another optional embodiment provided by the embodiment of the present application, the physical shooting device is a camera device, and the camera device includes a camera and a camera base;

[0235] Wherein, the camera is fixed to the camera base;

[0236] The camera base is provided with a detachable panel, which has a hollow area for passing the camera;

[0237] It may also include:

[0238] When the contact plane of the operation controller is in contact with the detachable panel, it is determined that the contact plane of the operation controller is in contact with the contact plane of the physical shooting device.

[0239] In one or more embodiments, a method for assisting with lamination using a detachable panel is provided. As can be seen from the aforementioned embodiments, if the terminal (e.g., a mobile phone, tablet, etc.) has low pixel count, the user can also choose to use the camera as a physical capture device. That is, the camera establishes a communication connection with the terminal and transmits the captured video stream to the terminal for subsequent processing.

[0240] For easier understanding, please refer to Figure 15, which is a comparative diagram of assisting the bonding of a camera device in an embodiment of the present application. As shown in Figure 15 (A), K1 indicates the camera, and K2 indicates the camera base. The camera is fixed to the camera base. Because the contact surface of the camera device may be uneven, a detachable panel can be used to assist in bonding the operating controller to the camera device.

[0241] As shown in Figure 15 (B), K3 is used to indicate a detachable panel. The detachable panel indicated by K3 has a hollow area, through which the camera indicated by K1 can pass, thereby preventing the detachable panel from blocking the camera's field of view. The detachable panel indicated by K3 can be fixed to the camera base indicated by K2. Based on this, the user can align the contact surface of the operation controller with the detachable panel, thereby achieving alignment between the contact surface of the operation controller and the contact surface of the physical camera device.

[0242] Secondly, in the embodiment of the present application, a method of using a detachable panel to assist in bonding is provided. Through the above method, considering that the contact surface of the camera device may be uneven, the detachable panel can provide a flat contact surface for the camera device, thereby facilitating the user to bond the operating controller to the camera device.

[0243] Based on one or more embodiments corresponding to FIG3 above, in another optional embodiment provided by the embodiment of the present application, the calibrated virtual camera includes a foreground virtual camera and a background virtual camera;

[0244] It may also include:

[0245] Rendering the foreground space of the virtual scene through a foreground virtual camera to obtain a virtual foreground image;

[0246] Rendering the background space of the virtual scene by using a background virtual camera to obtain a virtual background image, wherein the virtual background image and the virtual background image correspond to the same timestamp;

[0247] A virtual foreground image and a virtual background image are sent to a physical shooting device so that the physical shooting device generates and displays a synthetic recorded image for a timestamp based on the virtual foreground image, the virtual background image and the cutout image, wherein the cutout image is generated by cutout processing of a real scene image, and the real scene image is obtained by shooting with a camera of the physical shooting device.

[0248] In one or more embodiments, a method for implementing MRC is described. As can be seen from the aforementioned embodiments, after completing virtual camera calibration, the user can enable the MRC function to record video. To achieve video synchronization, the virtual foreground image, virtual background image, and cutout image corresponding to the same timestamp are synthesized to produce a single composite recorded image. Finally, the composite recorded images corresponding to each timestamp are sequentially output to produce a composite video stream.

[0249] It should be noted that the following will use a timestamp as an example to introduce the process of generating a synthetic recorded image. At the same time, it is assumed that the physical shooting device is a mobile phone.

[0250] For ease of understanding, please refer to Figure 16, which is a schematic diagram of implementing virtual reality mixed recording in an embodiment of the present application. As shown in the figure, L1 is used to indicate a virtual camera, wherein the virtual camera includes a foreground virtual camera and a background virtual camera, and the foreground virtual camera and the background virtual camera have the same position information and orientation angle. The foreground virtual camera is used to render the foreground, and the background virtual camera is used to render the background. It can be understood that the foreground refers to the scene space from the virtual camera to the user, and the background refers to the scene space from the user to infinity. L2 is used to indicate the physical shooting device.

[0251] Based on this, taking a specific timestamp as an example, the foreground space of the virtual scene is rendered using the foreground virtual camera to obtain the virtual foreground image corresponding to that timestamp. Similarly, the background space of the virtual scene is rendered using the background virtual camera to obtain the virtual background image corresponding to that timestamp. Thus, the virtual foreground and background images are transmitted to the physical camera using wireless projection technology.

[0252] At the same time, the physical camera is also recording footage containing the user, thereby obtaining an image of the real scene corresponding to the timestamp. The physical camera then uses a deep learning cutout model to cut out the person's portrait from the real scene image, generating a cutout image. Finally, the physical camera overlays the virtual background image, followed by the cutout image, and finally the virtual foreground image, on a single frame for fusion, resulting in a composite recorded image.

[0253] Secondly, in the embodiments of the present application, a method for implementing MRC is provided. By using the above method, when calibrating the virtual camera, a highly accurate calibration result can be obtained, thereby obtaining a video stream with a better synthesis effect after MRC. This is conducive to improving the user experience.

[0254] In combination with the above introduction, the virtual camera calibration method in this application will be introduced below. Please refer to Figure 17. The virtual camera calibration method in the embodiment of this application can be completed by the XR device independently, or by the XR device and the terminal in cooperation, or by the XR device, the terminal and the server in cooperation. The method of this application includes:

[0255] 310. When the contact plane of the operation controller is in contact with the contact plane of the physical camera device, obtain target position information and current orientation angle of the operation controller;

[0256] In one or more embodiments, a user can align an operation controller with a physical camera. Specifically, when the contact plane of the operation controller aligns with the contact plane of the physical camera, existing methods in the XR development platform SDK can be used to directly obtain the transformation information of the operation controller in the world coordinate system. This transformation information includes target position information and current heading angle.

[0257] 320. Determine a target orientation angle of the operation controller based on the current orientation angle and a plane perpendicular vector of the operation controller, wherein the plane perpendicular vector represents a vector perpendicular to the contact plane of the operation controller;

[0258] In one or more embodiments, a target orientation angle of the operation controller can be calculated based on the current orientation angle and the plane perpendicular vector of the operation controller. The plane perpendicular vector is a vector perpendicular to the contact plane of the operation controller, and the plane perpendicular vector forms a fixed angle with the initial orientation angle of the operation controller. The target orientation angle of the operation controller in the world coordinate system is the orientation angle of the camera of the physical camera in the world coordinate system.

[0259] It should be noted that the method for calculating the target orientation angle of the operation controller based on the current orientation angle and the plane vertical vector of the operation controller can refer to formulas (1) to (6) of the above embodiments and is not limited here.

[0260] 330. Calibrate the virtual camera according to the target position information and the target orientation angle of the operation controller.

[0261] In one or more embodiments, if a target object model is used, then when the camera of the target object model overlaps with the camera of the physical camera, the target position information of the operation controller in the world coordinate system is offset calculated based on the position offset between the camera position of the target object model and the operation controller, thereby obtaining the position information of the camera of the target object model in the world coordinate system. The position information of the camera of the target object model in the world coordinate system is the position information of the camera of the physical camera in the world coordinate system.

[0262] If the target object model is not used, when the operation controller touches the camera of the physical shooting device, the position information of the camera of the physical shooting device in the world coordinate system can be directly obtained.

[0263] It is understood that when calibrating the virtual camera, it is necessary to unify the virtual camera's position information in the world coordinate system with the position information of the physical camera's camera in the world coordinate system. Therefore, the virtual camera's position calibration can be completed by using the physical camera's camera's position information in the world coordinate system as the virtual camera's position information in the world coordinate system.

[0264] It's understandable that when calibrating the virtual camera's position, it's necessary to align the virtual camera's orientation angle in the world coordinate system with the physical camera's orientation angle in the world coordinate system. Therefore, calibrating the virtual camera's orientation angle can be accomplished by using the controller's target orientation angle in the world coordinate system as the virtual camera's orientation angle in the world coordinate system.

[0265] In an embodiment of the present application, a virtual camera calibration method is provided. Through the above-described method, the virtual camera can be calibrated based on the position information and orientation angle of the operating controller. This eliminates the need for additional calculations when calibrating the virtual camera and achieves a high degree of accuracy, thereby improving the efficiency of virtual camera calibration.

[0266] In another optional embodiment provided in accordance with one or more embodiments corresponding to FIG. 17 above, when the contact plane of the operation controller and the contact plane of the physical camera are in contact with each other, obtaining target position information and current orientation angle of the operation controller may include:

[0267] When a target object model corresponding to the physical camera device is acquired, when the contact plane of the operation controller and the contact plane of the physical camera device are in contact, a camera displaying the target object model through the head-mounted display device and a camera of the physical camera device are in an overlapping state, wherein the contact plane of the target object model and the contact plane of the operation controller are in contact, and a position of the camera of the target object model corresponds to a position of the camera of the physical camera device;

[0268] In response to a space calibration operation for the virtual camera, target position information and a current orientation angle of the operation controller are acquired.

[0269] In one or more embodiments, a method for calibrating a virtual camera using a target object model is described. As can be seen from the aforementioned embodiments, when the contact plane of the operation controller aligns with the contact plane of the physical camera, and the camera of the target object model overlaps with the camera of the physical camera, the user can trigger a spatial calibration operation for the virtual camera. Based on this, calling the "object.getTransform" interface based on the Unity engine can obtain the transformation information of the operation controller in the world coordinate system, that is, obtain the target position information and current orientation angle of the operation controller in the world coordinate system.

[0270] Secondly, in an embodiment of the present application, a method for calibrating a virtual camera using a target object model is provided. Through the above method, on the one hand, the position information of the virtual camera can be directly determined using the known target position information and the camera position of the target object model. There is no need for the user to touch the camera of the physical shooting device using the operation controller, thereby improving the calibration efficiency. On the other hand, the orientation angle of the virtual camera can be calculated using the known current orientation angle and the plane vertical vector. It can be seen that the steps for calibrating the virtual camera are relatively simple, no additional calculations are required, and a high-precision calibration effect can be obtained.

[0271] In another optional embodiment provided in accordance with one or more embodiments corresponding to FIG. 17 above, when the contact plane of the operation controller and the contact plane of the physical camera are in contact with each other, obtaining target position information and current orientation angle of the operation controller may include:

[0272] If the target object model corresponding to the physical shooting device is not obtained, then when the operation controller is in contact with the camera of the physical shooting device, in response to the position recording operation for the operation controller, obtaining target position information of the operation controller;

[0273] When the contact plane of the operation controller is aligned with the contact plane of the physical shooting device, the current orientation angle of the operation controller is acquired in response to the angle recording operation on the operation controller.

[0274] In one or more embodiments, a method for calibrating a virtual camera without using a target object model is described. As can be seen from the aforementioned embodiments, a user can use an operation controller to contact the camera of a physical camera device, and upon contact, trigger a position recording operation for the operation controller to obtain target position information of the operation controller. The user also needs to align the contact plane of the operation controller with the contact plane of the physical camera device, and upon alignment, trigger an angle recording operation for the operation controller to obtain the current orientation angle of the operation controller.

[0275] Secondly, an embodiment of the present application provides a method for calibrating a virtual camera without using a target object model. This method allows, if a target object model that matches the physical camera cannot be obtained, to locate the physical camera's camera position by touching the controller. This improves the flexibility and operability of the solution.

[0276] In conjunction with the above embodiments, the following will take a specific scenario as an example to introduce the calibration process of the virtual camera. In this scenario, the operating controller is a handle device. For ease of understanding, please refer to Figure 18, which is a schematic diagram of the overall process of calibrating the virtual camera in the embodiment of the present application. As shown in the figure, specifically:

[0277] In step S1, taking the physical shooting device as a mobile phone as an example, the user is guided to fix the mobile phone through the mobile phone.

[0278] In step S2, the user is further guided by the mobile phone to wear the head display device and hold the handle device.

[0279] In step S3, the user can see a virtual scene through the head display device. In the virtual scene, a rectangular object is generated on the contact plane of the handle device. The rectangular object is the target object model. The target object model fits the contact plane of the handle device.

[0280] In step S4 , the user turns on the see-through function.

[0281] In step S5, the user is guided by the mobile phone or head display device to use the handle device to fit the plane of the mobile phone, and press a specific button on the handle device after confirming the fit.

[0282] In step S6, the handle device continuously detects specific key events. After receiving the specific key event, the parent node corresponding to the target object model is set as the scene root node.

[0283] In step S7 , based on this, a virtual camera is generated based on the position information and orientation angle of the target object model, wherein the virtual camera includes a virtual foreground camera and a virtual background camera.

[0284] The virtual camera calibration device in the present application is described in detail below. Please refer to FIG19 , which is a schematic diagram of an embodiment of the virtual camera calibration device in the present application. The virtual camera calibration device 40 includes:

[0285] Display module 410 is configured to, upon acquiring a target object model corresponding to the physical camera device, display the target object model linked to the operation controller via a head-mounted display device, wherein a contact plane of the target object model is aligned with a contact plane of the operation controller, and a camera position of the target object model corresponds to a camera position of the physical camera device;

[0286] The display module 410 is further configured to display the target object model through the head-mounted display device in an overlapping state with the camera of the physical shooting device when the contact plane of the operation controller and the contact plane of the physical shooting device are in contact with each other;

[0287] The calibration module 420 is configured to calibrate the virtual camera according to the spatial information of the operation controller in response to the spatial calibration operation for the virtual camera, wherein the spatial information includes target position information and target orientation angle.

[0288] Optionally, based on the embodiment corresponding to FIG. 19 , the embodiment of the present application provides another embodiment of the virtual camera calibration device 40 , wherein the virtual camera calibration device 40 further includes an acquisition module 430 ;

[0289] The acquisition module 430 is used to obtain the target object model corresponding to the physical shooting device from the locally stored model resource collection in response to a model call request before displaying the target object model linked to the operation controller through the head display device when the target object model corresponding to the physical shooting device is obtained, wherein the model call request carries the model information of the physical shooting device, the model resource collection includes at least one object model, each object model corresponds to a model information, and the target object model corresponds to the model information carried by the model call request.

[0290] Optionally, based on the embodiment corresponding to FIG. 19 , the embodiment of the present application provides another embodiment of the virtual camera calibration device 40 , wherein the virtual camera calibration device further includes a processing module 440 ;

[0291] Processing module 440 is configured to, upon acquiring a target object model corresponding to the physical camera device, send a model download request to the server in response to a model call request before displaying the target object model linked to the operation controller via the head-mounted display device, wherein the model call request carries model information of the physical camera device, and the model download request carries model information of the physical camera device;

[0292] The processing module 440 is further configured to receive a target object model corresponding to the physical shooting device sent by the server, wherein the target object model corresponds to the model information carried in the model download request.

[0293] Optionally, based on the embodiment corresponding to FIG. 19 , the present application provides another embodiment of the virtual camera calibration device 40 :

[0294] The display module 410 is further configured to, after acquiring the target object model corresponding to the physical camera device, display the target object model linked to the operation controller via the head-mounted display device, and then display a first prompt message via the head-mounted display device, wherein the first prompt message is configured to prompt the user to align the contact plane of the operation controller with the contact plane of the physical camera device, and to overlap the target object model with the physical camera device.

[0295] or,

[0296] The display module 410 is further configured to play a first voice message through the head-mounted display device, wherein the first voice message is configured to prompt the user to align the contact plane of the operating controller with the contact plane of the physical camera device, and to overlap the target object model with the physical camera device.

[0297] or,

[0298] The display module 410 is further configured to display the first prompt message through the head display device and play the first voice message.

[0299] Optionally, based on the embodiment corresponding to FIG. 19 , the present application provides another embodiment of a virtual camera calibration device 40 :

[0300] The display module 410 is further configured to display a first demonstration animation through a head-mounted display device, wherein the first demonstration animation is configured to guide a process of calibrating the virtual camera using an operation controller.

[0301] Optionally, based on the embodiment corresponding to FIG. 19 , the present application provides another embodiment of a virtual camera calibration device 40 :

[0302] The calibration module 420 is specifically configured to obtain target position information and current orientation angle of the operation controller in response to a spatial calibration operation on the virtual camera;

[0303] Determine the position information of the virtual camera according to the target position information and the camera position of the target object model;

[0304] Determining a target orientation angle of the manipulation controller according to the current orientation angle and a plane perpendicular vector of the manipulation controller, wherein the plane perpendicular vector represents a vector perpendicular to a contact plane of the manipulation controller;

[0305] The virtual camera is calibrated using its position information and target orientation angle.

[0306] Optionally, based on the embodiment corresponding to FIG. 19 , the present application provides another embodiment of a virtual camera calibration device 40 :

[0307] The acquisition module 430 is further configured to, if the target object model corresponding to the physical photographing device is not acquired, acquire target position information of the operation controller in response to a position recording operation on the operation controller when the operation controller is in contact with the camera of the physical photographing device;

[0308] The acquisition module 430 is further configured to acquire a target orientation angle of the operation controller in response to an angle recording operation on the operation controller when the contact plane of the operation controller aligns with the contact plane of the physical camera device;

[0309] The calibration module 420 is further configured to calibrate the virtual camera according to the target position information and the target orientation angle.

[0310] Optionally, based on the embodiment corresponding to FIG. 19 , the present application provides another embodiment of a virtual camera calibration device 40 :

[0311] The display module 410 is further configured to, in response to a position recording operation on the operation controller and before obtaining target position information of the operation controller, display a second prompt message through the head-mounted display device when the operation controller is in contact with the camera of the physical camera device, wherein the second prompt message is used to prompt the operator to contact the camera of the physical camera device;

[0312] or,

[0313] The display module 410 is further configured to play a second voice message through the head-mounted display device, wherein the second voice message is configured to prompt the operator to contact the camera of the physical shooting device;

[0314] or,

[0315] The display module 410 is further configured to display the second prompt message through the head display device and play the second voice message.

[0316] Optionally, based on the embodiment corresponding to FIG. 19 , the present application provides another embodiment of a virtual camera calibration device 40 :

[0317] The display module 410 is further configured to, in a case where the contact plane of the operation controller is aligned with the contact plane of the physical camera device, display a third prompt message via the head-mounted display device in response to an angle recording operation on the operation controller and before obtaining a target orientation angle of the operation controller, wherein the third prompt message is configured to prompt the user to align the contact plane of the operation controller with the contact plane of the physical camera device;

[0318] or,

[0319] The display module 410 is further configured to play a third voice message through the head-mounted display device, wherein the third voice message is used to prompt the operator to align the contact surface of the operating controller with the contact surface of the physical camera device;

[0320] or,

[0321] The display module 410 is further configured to display a third prompt message through the head display device and play a third voice message.

[0322] Optionally, based on the embodiment corresponding to FIG. 19 , the present application provides another embodiment of a virtual camera calibration device 40 :

[0323] The acquisition module 430 is specifically configured to display a second demonstration animation through the head-mounted display device if the target object model corresponding to the physical shooting device is not acquired, wherein the second demonstration animation is used to guide the process of calibrating the virtual camera using the operation controller;

[0324] In response to a playback end operation for the second demonstration animation, when the operation controller is in contact with a camera of the physical shooting device, in response to a position recording operation for the operation controller, target position information of the operation controller is acquired.

[0325] Optionally, based on the embodiment corresponding to FIG. 19 , the present application provides another embodiment of a virtual camera calibration device 40 :

[0326] The acquisition module 430 is specifically configured to acquire a current orientation angle of the operation controller in response to an angle recording operation on the operation controller;

[0327] The target orientation angle of the operation controller is determined according to the current orientation angle and the plane perpendicular vector of the operation controller, wherein the plane perpendicular vector represents a vector perpendicular to the contact plane of the operation controller.

[0328] Optionally, based on the embodiment corresponding to FIG. 19 , the present application provides another embodiment of a virtual camera calibration device 40 , wherein N pressure sensors are deployed on the contact plane of the operation controller, where N is an integer greater than 1;

[0329] The acquisition module 430 is further configured to acquire, through the N pressure sensors, at least one set of pressure parameter sets when the contact plane of the operation controller is in contact with the contact plane of the physical capture device and before a camera displaying a target object model on the head-mounted display device overlaps with a camera of the physical capture device, wherein each set of pressure parameter sets includes at least one pressure value, and the at least one pressure value is derived from at least one pressure sensor deployed continuously;

[0330] The processing module 440 is further configured to determine the contact state between the contact plane of the operation controller and the contact plane of the physical shooting device according to each set of pressure parameters.

[0331] Optionally, based on the embodiment corresponding to FIG. 19 , the present application provides another embodiment of the virtual camera calibration device 40 :

[0332] The processing module 440 is further configured to enable a see-through function of the head-mounted display device in response to a function activation operation on the head-mounted display device;

[0333] The display module 410 is specifically configured to display the target object model linked to the operation controller through the head-mounted display device based on the perspective function of the head-mounted display device.

[0334] Optionally, based on the embodiment corresponding to FIG. 19 , the present application provides another embodiment of the virtual camera calibration device 40 :

[0335] The display module 410 is specifically configured to display the target object model through the camera of the head-mounted display device in an overlapping state with the camera of the physical shooting device based on the perspective function of the head-mounted display device.

[0336] Optionally, based on the embodiment corresponding to FIG. 19 , the present application provides another embodiment of the virtual camera calibration device 40 :

[0337] The display module 410 is further configured to display a calibration prompt message, wherein the calibration prompt message is used to prompt whether to calibrate the virtual camera;

[0338] The processing module 440 is further configured to, in response to a confirmation operation on the calibration prompt message, execute a step of displaying the target object model linked to the operation controller via the head-mounted display device when a target object model corresponding to the physical camera device is acquired;

[0339] The calibration module 420 is further configured to calibrate the virtual camera according to the historical spatial information of the operation controller in response to a cancel operation on the calibration prompt message, wherein the historical spatial information includes the spatial information of the operation controller acquired last time.

[0340] Optionally, based on the embodiment corresponding to FIG. 19 , the embodiment of the present application provides another embodiment of a virtual camera calibration device 40 , wherein the physical shooting device is a camera device, and the camera device includes a camera and a camera base;

[0341] Wherein, the camera is fixed to the camera base;

[0342] The camera base is provided with a detachable panel, which has a hollow area for passing the camera;

[0343] The processing module 440 is further configured to determine whether the contact plane of the operation controller is in contact with the contact plane of the physical shooting device when the contact plane of the operation controller is in contact with the detachable panel.

[0344] Optionally, based on the embodiment corresponding to FIG. 19 , the present application provides another embodiment of a virtual camera calibration device 40 , wherein the calibrated virtual cameras include a foreground virtual camera and a background virtual camera;

[0345] The processing module 440 is further configured to render the foreground space of the virtual scene through the foreground virtual camera to obtain a virtual foreground image;

[0346] The processing module 440 is further configured to render the background space of the virtual scene through the background virtual camera to obtain a virtual background image, wherein the virtual background image and the virtual background image correspond to the same timestamp;

[0347] The processing module 440 is also used to send a virtual foreground image and a virtual background image to the physical shooting device, so that the physical shooting device generates and displays a synthetic recorded image for the timestamp based on the virtual foreground image, the virtual background image and the cutout image, wherein the cutout image is generated after cutout processing of the real scene image, and the real scene image is obtained by shooting through the camera of the physical shooting device.

[0348] The virtual camera calibration device in the present application is described in detail below. Please refer to FIG. 20 , which is a schematic diagram of another embodiment of the virtual camera calibration device in the embodiment of the present application. The virtual camera calibration device 50 includes:

[0349] An acquisition module 510 is configured to acquire target position information and a current orientation angle of the operation controller when the contact plane of the operation controller is in contact with the contact plane of the physical photographing device;

[0350] a determination module 520 for determining a target orientation angle of the manipulation controller based on the current orientation angle and a plane perpendicular vector of the manipulation controller, wherein the plane perpendicular vector represents a vector perpendicular to a contact plane of the manipulation controller;

[0351] The calibration module 530 is used to calibrate the virtual camera according to the target position information and the target orientation angle of the operation controller.

[0352] Optionally, based on the embodiment corresponding to FIG. 20 , the present application provides another embodiment of the virtual camera calibration device 50 :

[0353] The acquisition module 510 is specifically configured to, upon acquiring a target object model corresponding to a physical camera device, cause a camera displaying the target object model through a head-mounted display device to overlap with a camera display of the physical camera device when a contact plane of the operation controller and a contact plane of the physical camera device are aligned, wherein the contact plane of the target object model and the contact plane of the operation controller are aligned, and a position of the camera display of the target object model corresponds to a position of a camera display of the physical camera device;

[0354] In response to a space calibration operation for the virtual camera, target position information and a current orientation angle of the operation controller are acquired.

[0355] Optionally, based on the embodiment corresponding to FIG. 20 , the present application provides another embodiment of the virtual camera calibration device 50 :

[0356] an acquisition module 510 , specifically configured to acquire target position information of the operation controller in response to a position recording operation on the operation controller when the operation controller is in contact with the camera of the physical shooting device if the target object model corresponding to the physical shooting device is not acquired;

[0357] When the contact plane of the operation controller is aligned with the contact plane of the physical shooting device, the current orientation angle of the operation controller is acquired in response to the angle recording operation on the operation controller.

[0358] The present application also provides a terminal, as shown in FIG21. For ease of explanation, only the parts related to the present application are shown. For specific technical details not disclosed, please refer to the method part of the present application. In the present application, the terminal is an XR device as an example for explanation:

[0359] FIG21 is a block diagram showing a partial structure of an XR device related to a terminal provided in an embodiment of the present application. Referring to FIG21 , the XR device includes components such as a radio frequency (RF) circuit 610, a memory 620, an input unit 630, a display unit 640, a sensor 650, an audio circuit 660, a mobile hotspot (wireless fidelity, WiFi) module 670, a processor 680, and a power supply 690. The input unit 630 may include a touch panel 631 and other input devices 632, the display unit 640 may include a display panel 641, an audio circuit 660, a speaker 661, and a microphone 662 that can provide an audio interface between the user and the XR device. Those skilled in the art will understand that the XR device structure shown in FIG21 does not constitute a limitation on the XR device, and may include more or fewer components than shown, or combine certain components, or arrange components differently.

[0360] Although not shown, the XR device may also include a camera, a Bluetooth module, etc., which will not be described here.

[0361] The steps executed by the terminal in the above embodiment may be based on the terminal structure shown in FIG21 .

[0362] A computer-readable storage medium is also provided in an embodiment of the present application, on which a computer program is stored. When the computer program is executed by a processor, the steps of the methods described in the above embodiments are implemented.

[0363] A computer program product is also provided in an embodiment of the present application, including a computer program. When the computer program is executed by a processor, the steps of the methods described in the above embodiments are implemented.

[0364] It is understandable that in the specific implementation of this application, related data such as user information and operation information are involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0365] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0366] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0367] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0368] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0369] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a server or terminal device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store computer programs, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0370] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A virtual camera calibration method, the method being executed by a computer device, comprising: When the target object model corresponding to the physical shooting device is obtained, the target object model linked with the operation controller is displayed through the head display device, wherein the contact plane of the target object model is in contact with the contact plane of the operation controller, and the camera position of the target object model corresponds to the camera position of the physical shooting device; When the contact plane of the operation controller is in contact with the contact plane of the physical photographing device, the camera of the target object model displayed by the head display device is in an overlapping state with the camera of the physical photographing device; In response to a spatial calibration operation for a virtual camera, the virtual camera is calibrated according to spatial information of the operation controller, wherein the spatial information includes target position information and a target orientation angle.

2. According to the calibration method of claim 1, when the target object model corresponding to the physical shooting device is obtained, before the target object model linked to the operation controller is displayed through the head display device, the method also includes: In response to a model call request, the target object model corresponding to the entity shooting device is obtained from a locally stored model resource collection, wherein the model call request carries model information of the entity shooting device, the model resource collection includes at least one object model, each object model corresponds to a model information, and the target object model corresponds to the model information carried by the model call request.

3. According to the calibration method of claim 1, when the target object model corresponding to the physical shooting device is obtained, before the target object model linked to the operation controller is displayed through the head display device, the method also includes: In response to the model calling request, sending a model downloading request to the server, wherein the model calling request carries the model information of the entity photographing device, and the model downloading request carries the model information of the entity photographing device; The target object model corresponding to the physical shooting device sent by the server is received, wherein the target object model corresponds to the model information carried in the model download request.

4. The calibration method according to any one of claims 1 to 3, wherein after acquiring the target object model corresponding to the physical shooting device, displaying the target object model linked to the operation controller through the head display device, the method further comprises: Displaying a first prompt message through the head display device, wherein the first prompt message is used to prompt that the contact plane of the operation controller is aligned with the contact plane of the physical shooting device, and the target object model is overlapped with the physical shooting device; or, Playing a first voice message through the head display device, wherein the first voice message is used to prompt the user to fit the contact plane of the operation controller to the contact plane of the physical shooting device, and to overlap the target object model with the physical shooting device; or, The first prompt message is displayed through the head display device, and the first voice message is played.

5. The calibration method according to any one of claims 1 to 4, further comprising: A first demonstration animation is displayed through the head display device, wherein the first demonstration animation is used to guide a process of calibrating the virtual camera using the operation controller.

6. The calibration method according to any one of claims 1 to 5, wherein in response to a spatial calibration operation for a virtual camera, calibrating the virtual camera according to spatial information of the operation controller comprises: In response to a space calibration operation for the virtual camera, obtaining target position information and a current orientation angle of the operation controller; Determining the position information of the virtual camera according to the target position information and the camera position of the target object model; Determining a target orientation angle of the operation controller according to the current orientation angle and a plane vertical vector of the operation controller, wherein the plane vertical vector represents a vector perpendicular to a contact plane of the operation controller; The virtual camera is calibrated using the position information of the virtual camera and the target orientation angle.

7. The calibration method according to any one of claims 1 to 6, further comprising: If the target object model corresponding to the physical photographing device is not obtained, in response to a position recording operation for the operation controller, obtaining target position information of the operation controller when the operation controller is in contact with the camera of the physical photographing device; When the contact plane of the operation controller is aligned with the contact plane of the physical photographing device, in response to an angle recording operation on the operation controller, obtaining a target orientation angle of the operation controller; The virtual camera is calibrated according to the target position information and the target orientation angle.

8. The calibration method according to claim 7, wherein when the operation controller is in contact with the camera of the physical photographing device, before obtaining the target position information of the operation controller in response to the position recording operation of the operation controller, the method further comprises: Displaying a second prompt message through the head display device, wherein the second prompt message is used to prompt the operation controller to contact the camera of the physical shooting device; or, Playing a second voice message through the head display device, wherein the second voice message is used to prompt the operation controller to contact the camera of the physical shooting device; or, The second prompt message is displayed through the head display device, and the second voice message is played.

9. The calibration method according to claim 7 or 8, wherein when the contact plane of the operation controller is aligned with the contact plane of the physical photographing device, in response to an angle recording operation on the operation controller, before obtaining the target orientation angle of the operation controller, the method further comprises: Displaying a third prompt message through the head display device, wherein the third prompt message is used to prompt the contact plane of the operation controller to be aligned with the contact plane of the physical shooting device; or, Playing a third voice message through the head display device, wherein the third voice message is used to prompt the contact plane of the operation controller to be aligned with the contact plane of the physical shooting device; or, The third prompt message is displayed through the head display device, and the third voice message is played.

10. The calibration method according to any one of claims 7 to 9, wherein if the target object model corresponding to the physical photographing device is not obtained, when the operation controller is in contact with a camera of the physical photographing device, in response to a position recording operation for the operation controller, obtaining target position information of the operation controller comprises: If the target object model corresponding to the physical shooting device is not obtained, displaying a second demonstration animation through the head display device, wherein the second demonstration animation is used to guide the process of calibrating the virtual camera using the operation controller; In response to the playback end operation for the second demonstration animation, when the operation controller is in contact with the camera of the physical shooting device, in response to the position recording operation for the operation controller, the target position information of the operation controller is obtained.

11. The calibration method according to any one of claims 7 to 10, wherein in response to the angle recording operation for the operation controller, obtaining the target orientation angle of the operation controller comprises: In response to an angle recording operation on the operation controller, obtaining a current orientation angle of the operation controller; The target orientation angle of the operation controller is determined according to the current orientation angle and a plane perpendicular vector of the operation controller, wherein the plane perpendicular vector represents a vector perpendicular to a contact plane of the operation controller.

12. The calibration method according to any one of claims 1 to 11, wherein N pressure sensors are disposed on the contact plane of the operation controller, wherein: Said N is an integer greater than 1; When the contact plane of the operation controller is in contact with the contact plane of the physical photographing device, before the camera of the target object model displayed by the head display device is in an overlapping state with the camera of the physical photographing device, the method further includes: Acquire at least one set of pressure parameter sets through the N pressure sensors, wherein each set of pressure parameter sets includes at least one pressure value, and the at least one pressure value is derived from at least one pressure sensor deployed continuously; According to each set of pressure parameter sets, a contact state between a contact plane of the operation controller and a contact plane of the physical photographing device is determined.

13. The calibration method according to any one of claims 1 to 12, further comprising: In response to a function activation operation on the head display device, activating a perspective function of the head display device; The target object model displayed by the head display device in linkage with the operation controller includes: Based on the perspective function of the head display device, the target object model linked with the operation controller is displayed through the head display device; The camera for displaying the target object model through the head display device and the camera of the physical shooting device are in an overlapping state, including: Based on the perspective function of the head display device, the camera of the head display device that displays the target object model is in an overlapping state with the camera of the physical shooting device.

14. The calibration method according to any one of claims 1 to 13, further comprising: Displaying a calibration prompt message, wherein the calibration prompt message is used to prompt whether to calibrate the virtual camera; If a confirmation operation is performed on the calibration prompt message in response, the step of displaying the target object model linked to the operation controller through the head display device when the target object model corresponding to the physical shooting device is acquired is performed; If a cancel operation is performed on the calibration prompt message in response, the virtual camera is calibrated according to the historical spatial information of the operation controller, wherein the historical spatial information includes the spatial information of the operation controller acquired last time.

15. The calibration method according to any one of claims 1 to 14, wherein the physical shooting device is a camera device, and the camera device comprises a camera and a camera base; in, The camera is fixed to the camera base; The camera base is provided with a detachable panel, the detachable panel is provided with a hollow area, and the hollow area is used for passing the camera; The method further comprises: When the contact plane of the operation controller is in contact with the detachable panel, it is determined that the contact plane of the operation controller is in contact with the contact plane of the physical shooting device.

16. The calibration method according to any one of claims 1 to 15, wherein the calibrated virtual camera comprises a foreground virtual camera and a background virtual camera; The method further comprises: Rendering the foreground space of the virtual scene by the foreground virtual camera to obtain a virtual foreground image; Rendering the background space of the virtual scene by the background virtual camera to obtain a virtual background image, wherein the virtual background image and the virtual background image correspond to the same timestamp; The virtual foreground image and the virtual background image are sent to the physical shooting device, so that the physical shooting device generates and displays a synthetic recorded image for the timestamp based on the virtual foreground image, the virtual background image and the cutout image, wherein the cutout image is generated by cutout processing of a real scene image, and the real scene image is obtained by shooting with a camera of the physical shooting device.

17. A virtual camera calibration method, the method being executed by a computer device, comprising: When the contact plane of the operation controller is in contact with the contact plane of the physical shooting device, the target position information and the current orientation angle of the operation controller are obtained; Determining a target orientation angle of the operation controller according to the current orientation angle and a plane vertical vector of the operation controller, wherein the plane vertical vector represents a vector perpendicular to a contact plane of the operation controller; The virtual camera is calibrated according to the target position information and the target orientation angle of the operation controller.

18. The calibration method according to claim 17, wherein when the contact plane of the operation controller is in contact with the contact plane of the physical photographing device, obtaining the target position information and the current orientation angle of the operation controller comprises: When the target object model corresponding to the physical photographing device is obtained, when the contact plane of the operation controller is in contact with the contact plane of the physical photographing device, the camera of the target object model displayed by the head display device is in an overlapping state with the camera of the physical photographing device, wherein the contact plane of the target object model is in contact with the contact plane of the physical photographing device. The touch plane is in contact with the contact plane of the operation controller, and the camera position of the target object model corresponds to the camera position of the physical shooting device; In response to a space calibration operation for a virtual camera, the target position information and the current orientation angle of the operation controller are acquired.

19. The calibration method according to claim 17, wherein when the contact plane of the operation controller is in contact with the contact plane of the physical photographing device, obtaining the target position information and the current orientation angle of the operation controller comprises: If the target object model corresponding to the physical photographing device is not obtained, in response to a position recording operation for the operation controller, obtaining target position information of the operation controller when the operation controller is in contact with the camera of the physical photographing device; When the contact plane of the operation controller is aligned with the contact plane of the physical shooting device, the current orientation angle of the operation controller is acquired in response to an angle recording operation on the operation controller.

20. A virtual camera calibration device, the device being deployed on a computer device, comprising: A display module, configured to display the target object model linked to the operation controller through a head display device when a target object model corresponding to the physical shooting device is acquired, wherein a contact plane of the target object model is in contact with a contact plane of the operation controller, and a camera position of the target object model corresponds to a camera position of the physical shooting device; The display module is further configured to display, through the head display device, a camera of the target object model and a camera of the physical shooting device in an overlapping state when the contact plane of the operation controller and the contact plane of the physical shooting device are in a contact state; The calibration module is used to calibrate the virtual camera in response to a spatial calibration operation for the virtual camera according to spatial information of the operation controller, wherein the spatial information includes target position information and target orientation angle.

21. A virtual camera calibration device, the device being deployed on a computer device, comprising: An acquisition module, used for acquiring target position information and current orientation angle of the operation controller when the contact plane of the operation controller is in contact with the contact plane of the physical shooting device; a determination module, configured to determine a target orientation angle of the operation controller according to the current orientation angle and a plane vertical vector of the operation controller, wherein the plane vertical vector represents a vector perpendicular to a contact plane of the operation controller; A calibration module is used to calibrate the virtual camera according to the target position information and the target orientation angle of the operation controller.

22. An extended reality device, comprising: Display screen, processor, memory and communication module; The display screen is used to provide a display function; The communication module is used to establish a communication connection with the physical shooting device; The memory is used to store computer programs; The processor is used to execute the computer program to implement the steps of the calibration method according to any one of claims 1 to 16, or to implement the steps of the calibration method according to any one of claims 17 to 19.

23. A computer device comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the calibration method according to any one of claims 1 to 16 are implemented, or the steps of the calibration method according to any one of claims 17 to 19 are implemented.

24. A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the computer program implements the steps of the calibration method according to any one of claims 1 to 16, or implements the steps of the calibration method according to any one of claims 17 to 19.

25. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the steps of the calibration method according to any one of claims 1 to 16, or the computer program implements the steps of the calibration method according to any one of claims 17 to 19.

Images