Pose calibration method, device, equipment, storage medium and computer program product

Abstract

The application discloses a pose calibration method, device, equipment, storage medium and computer program product. The method comprises the following steps: acquiring N shooting data; constructing M shooting data groups based on the N shooting data; traversing the M shooting data groups, determining the mth relative pose of a positioning device and a camera device and the mth pose of an electronic screen in the positioning device space based on H shooting data in the mth shooting data group currently traversed and position information of a reference image in the electronic screen space; and selecting a target relative pose of the positioning device and the camera device and a target pose of the electronic screen in the positioning device space from the M relative poses of the positioning device and the camera device and the M poses of the electronic screen in the positioning device space determined based on the M shooting data groups, so that the relative pose relationship among the positioning device, the camera device and the electronic screen can be calibrated.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence, and in particular to a pose calibration method and device, equipment, a storage medium and a computer program product. BACKGROUND

[0002] Virtual production refers to various digital workflows and methods using computer-aided production and film visualization production; under the promotion of real-time rendering technology, it can include pre-production virtual rehearsal, real-time motion capture of virtual characters, green screen virtual production, and electronic screen virtual production, etc. In the virtual production scene, it is usually necessary to calibrate the relative pose relationship between the positioning device, the camera device and the electronic screen, so that the pose of the virtual camera device relative to the virtual screen can be determined based on the real-time pose of the positioning device in the positioning device space, and further, the target image to be displayed in the electronic screen at the current shooting angle can be rendered in real time according to the determined pose of the virtual camera device relative to the virtual screen, so that the camera device can shoot the target image rendered in real time. Therefore, how to calibrate the relative pose relationship between the positioning device, the camera device and the electronic screen is a research hotspot at present. SUMMARY

[0003] The embodiments of the present application provide a pose calibration method, device, equipment, storage medium and computer program product, which can calibrate the relative pose relationship between the positioning device, the camera device and the electronic screen.

[0004] In one aspect, the embodiments of the present application provide a pose calibration method, comprising:

[0005] Obtaining N pieces of shooting data; any shooting data comprises a shooting image and a pose of a positioning device in a positioning device space when the shooting image is obtained; wherein the shooting image is obtained by a camera device bound to the positioning device from any shooting angle of a reference image in an electronic screen, and N is a positive integer;

[0006] Constructing M pieces of shooting data groups based on the N pieces of shooting data; the number of shooting data in any shooting data group is H, H∈[Z, N], Z is a quantity threshold, and M is a positive integer;

[0007] Iterating the M pieces of shooting data groups, and for the H pieces of shooting data in the mth shooting data group in the current iteration, determining the mth relative pose of the positioning device and the camera device and the mth pose of the electronic screen in the positioning device space based on the H pieces of shooting data and position information of the reference image in the electronic screen space; m∈[1, M];

[0008] select a target relative pose of the positioning device and the camera device and a target pose of the electronic screen in the positioning device space from the M relative poses of the positioning device and the camera device and the M poses of the electronic screen in the positioning device space determined based on the M sets of photographing data; the target relative pose and the target pose are used to determine a pose of a virtual camera device relative to a virtual screen.

[0009] In an aspect, an embodiment of the present application provides a pose calibration device, comprising:

[0010] a obtaining unit configured to obtain N photographing data; any photographing data comprises a photographing image and a pose of a positioning device in a positioning device space when the photographing image is obtained; wherein the photographing image is obtained by a camera device bound to the positioning device from any photographing angle on a reference image in an electronic screen, and N is a positive integer;

[0011] a processing unit configured to construct M sets of photographing data based on the N photographing data; a number of photographing data in any set of photographing data is H, H∈[Z, N], Z is a number threshold, and M is a positive integer;

[0012] the processing unit is further configured to traverse the M sets of photographing data, and determine, based on H photographing data in a current mth set of photographing data and position information of the reference image in an electronic screen space, an mth relative pose of the positioning device and the camera device and an mth pose of the electronic screen in the positioning device space; m∈[1, M];

[0013] the processing unit is further configured to select a target relative pose of the positioning device and the camera device and a target pose of the electronic screen in the positioning device space from the M relative poses of the positioning device and the camera device and the M poses of the electronic screen in the positioning device space determined based on the M sets of photographing data; the target relative pose and the target pose are used to determine a pose of a virtual camera device relative to a virtual screen.

[0014] In an aspect, an embodiment of the present application provides an electronic device, characterized in that the electronic device comprises an input interface and an output interface, and further comprises:

[0015] a processor adapted to implement one or more instructions; and

[0016] a computer storage medium storing one or more instructions, the one or more instructions being adapted to be loaded by the processor and execute the above-mentioned pose calibration method.

[0017] In one aspect, an embodiment of the present application provides a computer storage medium, characterized in that the computer storage medium stores computer program instructions, and the computer program instructions are executed by a processor to implement the posture calibration method.

[0018] In one aspect, an embodiment of the present application provides a computer program product, which comprises a computer program stored in a computer storage medium; a processor of an electronic device reads the computer program from the computer storage medium, and the processor executes the computer program to enable the electronic device to implement the posture calibration method.

[0019] In an embodiment of the present application, after N pieces of shooting data including shooting images and postures of corresponding positioning devices in a positioning device space are acquired, M pieces of shooting data groups can be constructed based on the N pieces of shooting data; then, by traversing the M pieces of shooting data groups, for H pieces of shooting data in the mth piece of shooting data group in the current traversal, based on the H pieces of shooting data and position information of the reference image in the electronic screen space, the mth relative posture of the positioning device and the camera device and the mth posture of the electronic screen in the positioning device space can be determined; then, from the M relative postures of the positioning device and the camera device and the M postures of the electronic screen in the positioning device space determined based on the M pieces of shooting data groups, a target relative posture of the positioning device and the camera device and a target posture of the electronic screen in the positioning device space can be selected; the target relative posture of the positioning device and the camera device and the target posture of the electronic screen in the positioning device space can be determined based on the N pieces of shooting data including the shooting images and the postures of the corresponding positioning devices in the positioning device space, that is, the relative posture relationship among the positioning device, the camera device and the electronic screen can be calibrated based on multiple pieces of shooting data including the shooting images and the postures of the corresponding positioning devices in the positioning device space; and the expansion of the shooting data group required for determining the target relative posture and the target posture is realized by combining multiple pieces of shooting data. BRIEF DESCRIPTION OF DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort.

[0021] Figure 1 is a structural schematic diagram of a virtual production system provided by an embodiment of the present application;

[0022] Figure 2is a flowchart of a pose calibration method provided by an embodiment of the present application;

[0023] Figure 3a is a schematic diagram of a corner point in a reference image provided by an embodiment of the present application;

[0024] Figure 3b is another schematic diagram of a corner point in a reference image provided by an embodiment of the present application;

[0025] Figure 4 is a schematic diagram of determining three-dimensional position information of each corner point in a reference image under an electronic screen space provided by an embodiment of the present application;

[0026] Figure 5 is a flowchart of another pose calibration method provided by an embodiment of the present application;

[0027] Figure 6 is a flowchart of another pose calibration method provided by an embodiment of the present application;

[0028] Figure 7 is a schematic diagram of a configuration information entry interface provided by an embodiment of the present application;

[0029] Figure 8 is a schematic diagram of a pose calibration interface provided by an embodiment of the present application;

[0030] Figure 9 is a schematic diagram of updating a target relative pose and a target pose provided by an embodiment of the present application;

[0031] Figure 10 is a flowchart of updating a target relative pose and a target pose provided by an embodiment of the present application;

[0032] Figure 11 is a structural schematic diagram of a pose calibration apparatus provided by an embodiment of the present application;

[0033] Figure 12 is a structural schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0034] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0035] Artificial Intelligence (AI) is the theory, method, technology and application system that use digital computer or machine controlled by digital computer to simulate, extend and expand human intelligence, perceive environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology of computer science, which attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that machines have the functions of perception, reasoning and decision-making.

[0036] Artificial intelligence technology is a comprehensive discipline, involving a wide range of fields, both hardware and software level technology. Artificial intelligence basic technology generally includes technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics, etc. Artificial intelligence software technology mainly includes computer vision technology (CV), speech processing technology, natural language processing technology, and machine learning (ML) / deep learning (DL) and several other major directions.

[0037] Among them, computer vision is a science that studies how to make machines "see". Further, it refers to using cameras and computers to replace human eyes to identify, determine and measure targets, and further process graphics, so that computer processing becomes images more suitable for human eye observation or transmission to instrument detection. As a scientific discipline, computer vision researches related theories and technologies, and attempts to establish artificial intelligence systems that can obtain information from images or multidimensional data. Computer vision technology usually includes image processing, image recognition, image semantic understanding, image retrieval, optical character recognition (OCR), video processing, video semantic understanding, video content / behavior recognition, three-dimensional (3-dimensional, 3D) technology, three-dimensional object reconstruction, virtual reality (VR), augmented reality (AR), mixed reality (MR), extended reality (Extended Reality, XR), simultaneous localization and mapping, etc.

[0038] Based on the above-mentioned extended reality technology in the computer vision technology, an attitude calibration scheme is provided, which can construct M groups of shooting data based on N groups of shooting data after obtaining the N groups of shooting data; any shooting data includes a shooting image and a pose of the positioning device in the positioning device space when the shooting image is shot, and the number of shooting data in any shooting data group is H, wherein the shooting image is obtained by a camera device bound to the positioning device from any shooting angle, the reference image in the electronic screen, N is a positive integer, M is a positive integer, H [Z, N], and Z is a quantity threshold. Traversing the M groups of shooting data, for the H groups of shooting data in the mth group of shooting data in the current traversal, based on the H groups of shooting data and the position information of the reference image in the electronic screen space, the mth relative attitude between the positioning device and the camera device and the mth attitude of the electronic screen in the positioning device space are determined, m [1, M]; and then the target relative attitude between the positioning device and the camera device and the target attitude of the electronic screen in the positioning device space can be selected from the M relative attitudes between the positioning device and the camera device and the M attitudes of the electronic screen in the positioning device space determined based on the M groups of shooting data; wherein the target relative attitude and the target attitude are used to determine the attitude of the virtual camera device relative to the virtual screen.

[0039] In a specific implementation, the attitude calibration scheme can be executed by an electronic device, which can be a terminal device or a server; here, the terminal device can include but is not limited to a computer, a smart phone, a tablet computer, a notebook computer, a smart home appliance, a vehicle-mounted terminal, a smart wearable device, etc.; here, the server can be a standalone physical server, or a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content distribution networks (CDN), and basic cloud computing services such as big data and artificial intelligence platforms. Optionally, the attitude calibration scheme can also be executed by other electronic devices with computing power, alone or in cooperation, and the embodiments of the present application do not make any limitation.

[0040] The attitude calibration scheme can be applied to a virtual production scene, based on which, an embodiment of the present application provides a virtual production system, referring to Figure 1 , a structural schematic diagram of a virtual production system provided by an embodiment of the present application. Figure 1The illustrated virtual production system can include an electronic screen 101, a positioning device 102, a camera device 103, an electronic device 104, a virtual screen 105, and a virtual camera device 106. Among them, the electronic screen 101 can be an LED screen that can display multimedia data such as images, videos, and videos; the positioning device 102 can be a commonly used positioning device (i.e., a commonly used Tracker) in the virtual production scene; the camera device 103 can be an imaging device such as a video camera or a camera; the camera device 103 is bound to the positioning device 102, and the camera device 103 and the positioning device 102 satisfy the rigid body relationship. Among them, the virtual screen 105 can be a virtual model obtained by modeling the electronic screen 101; the virtual camera device 106 can be a virtual model corresponding to the camera device 103; in other words, when the electronic screen 101 and the camera device 103 exist in the virtual production scene, the virtual screen 105 and the virtual camera device 106 also exist; further, the virtual screen 105 and the virtual camera device 106 can be obtained based on any electronic device, and further optionally, the virtual screen 105 and the virtual camera device 106 can be obtained based on the Unreal Engine deployed on the electronic device, and the electronic device in this embodiment is taken as an example to be described.

[0041] In a specific implementation, the electronic screen 101 displays a reference image; the camera device 103 bound to the positioning device 102 captures the reference image in the electronic screen 101 from N shooting angles to obtain N captured images, and sends the N captured images to the electronic device 104; when the camera device 103 captures the reference image in the electronic screen 101 from any shooting angle of the N shooting angles, the positioning device 102 sends the pose of the positioning device in the positioning device space under the corresponding shooting angle to the electronic device 104. After obtaining the N captured data, the electronic device 104 can construct M captured data groups based on the N captured data; wherein any captured data in the N captured data includes a captured image, and the pose of the positioning device in the positioning device space when the corresponding captured image is captured; further, the electronic device 104 can traverse the M captured data groups, for the H captured data in the mth captured data group in the current traversal, based on the H captured data and the position information of the reference image in the electronic screen space, determine the mth relative pose of the positioning device and the camera device, and the mth pose of the electronic screen in the positioning device space; and then the target relative pose of the positioning device and the camera device, and the target pose of the electronic screen in the positioning device space can be selected from the M relative poses of the positioning device and the camera device, and the M poses of the electronic screen in the positioning device space determined based on the M captured data groups.

[0042] In one application scenario, there is an object A placed in front of the electronic screen 101; the positioning device 102 sends the posture of the positioning device in the positioning device space under the current shooting angle (referred to as the current posture of the positioning device in the positioning device space) to the electronic device 104. After the electronic device 104 receives the current posture of the positioning device in the positioning device space, the electronic device 104 can determine the current posture of the virtual camera relative to the virtual screen based on the target relative posture of the positioning device and the camera, the target posture of the electronic screen in the positioning device space, and the current posture of the positioning device in the positioning device space; and then the electronic device 104 can render the target image that needs to be displayed in the electronic screen 101 under the current shooting angle based on the current posture of the virtual camera relative to the virtual screen, and send the target image to the electronic screen 101; after receiving the target image, the electronic screen 101 can display the target image, so that the camera 103 can shoot the target image and the object A under the current shooting angle; further, the change of the target image displayed in the electronic screen 101 when the positioning device 102 moves can be realized, and the change is adapted to the movement of the positioning device 102. Since the camera 103 is bound to the positioning device 102, the change is adapted to the movement of the camera 103, and the adaptive target image can be accurately output for the change of the shooting angle, so that the reality of the shot picture can be improved.

[0043] It should be particularly noted that in the specific embodiments of the present application, data related to the object is involved, and when the embodiments of the present application are applied to specific products or technologies, the permission or consent of the object needs to be obtained, and the collection, use and processing of the related data need to comply with the local laws, regulations and standards.

[0044] Based on the above posture calibration scheme, the embodiments of the present application provide a posture calibration method. Referring to Figure 2 , a flowchart of a posture calibration method provided by the embodiments of the present application is shown. Figure 2 The posture calibration method shown can be executed by an electronic device. Figure 2 The posture calibration method shown can include the following steps:

[0045] S201, acquiring N pieces of shooting data.

[0046] Any shooting data includes a shooting image and the posture of the positioning device in the positioning device space when the shooting image is shot; the shooting image can be obtained by a camera bound to the positioning device from any shooting angle; N is a positive integer, and the number of shooting data can be set according to specific requirements, for example, the number of shooting data can be set to 8, 9, etc.

[0047] In an embodiment, the reference image can be any image used for reference in determining the relative pose of the positioning device and the camera, and the target pose of the electronic screen under the positioning device, for example, the reference image can be a checkerboard image, a marker image (i.e., an Aruco image), etc., and the embodiment of the present application is described by taking the reference image as a checkerboard image as an example; further optionally, when the reference image is a checkerboard image, each checkerboard in the reference image needs to be displayed in the electronic screen when the reference image is displayed in the electronic screen; further optionally, the reference image can be generated by the electronic device and sent to the electronic screen, and the size of the reference image, the display position of the reference image in the electronic screen, etc. can be set according to specific requirements.

[0048] S202, constructing M groups of shooting data based on the N groups of shooting data.

[0049] Wherein, the number of shooting data in any group of shooting data is H, H∈[Z, N], Z is a quantity threshold, and M is a positive integer; the quantity threshold can be set according to specific requirements, for example, the quantity threshold can be set as 4, 5, etc.

[0050] S203, traversing the M groups of shooting data, and determining the mth relative pose of the positioning device and the camera, and the mth pose of the electronic screen under the positioning device based on the H groups of shooting data and the position information of the reference image in the electronic screen space for the H groups of shooting data in the mth group of shooting data currently traversed.

[0051] Wherein, m∈[1, M].

[0052] In an embodiment, the position information of the reference image in the electronic screen space can include three-dimensional position information of each corner point in the reference image in the electronic screen space. The corner point in the reference image is a point of interest in the reference image, which can be set according to specific requirements; for example, if the reference image is a checkerboard image, referring to Figure 3a , a schematic diagram of a corner point in a reference image provided by the embodiment of the present application, at this time, the corner point in the reference image is the vertex of each checkerboard, referring to Figure 3bAs another schematic diagram of the embodiment of the present application for determining the three-dimensional position information of the corner points in the reference image, the corner points in the reference image are the intersection points of any four chessboard grids. The three-dimensional position information of each corner point in the reference image in the electronic screen space refers to the coordinate information of each corner point in the reference image in the three-dimensional coordinate system constructed based on the electronic screen when the reference image is displayed in the electronic screen. Optionally, when the electronic screen is a rectangular screen, the left upper corner of the rectangular screen can be used as the origin to construct the three-dimensional coordinate system. When the electronic screen is a special-shaped screen (for example, a circular screen), the three-dimensional coordinate system can be constructed based on the three-dimensional vertex information of the virtual screen corresponding to the electronic screen. The embodiment of the present application is described taking the rectangular screen as an example.

[0053] Since the reference image can be generated by the electronic device and sent to the electronic screen, based on this, the electronic device can also obtain the electronic screen configuration information and the reference image configuration information before obtaining the N pieces of shooting data, generate the reference image according to the reference image configuration information, and send the reference image to the electronic screen for display. The electronic screen configuration information can be used to determine the coordinate system information of the three-dimensional coordinate system constructed based on the electronic screen, and the reference image configuration information can be used to determine the three-dimensional position information of each corner point in the reference image in the electronic screen space. In a feasible implementation manner, if the reference image is a chessboard image, the electronic screen configuration information can include the size of the electronic screen, and the reference image configuration information can include the size of the reference image, the display position of the reference image in the electronic screen, and the size of the chessboard grid. For example, referring to Figure 4 As a schematic diagram provided by the embodiment of the present application for determining the three-dimensional position information of each corner point in the reference image in the electronic screen space, if the size of the electronic screen is 5 meters long, 6 meters wide, and 0 meters high, the size of the reference image is 2 meters long, 3 meters wide, and 0 meters high, the size of the chessboard grid is 0.5 meters long, 0.5 meters wide, and 0 meters high, and the display position of the reference image in the electronic screen indicates that the three-dimensional position information of the left upper corner of the reference image in the electronic screen space is (0.5, 0.5, 0), the three-dimensional position information of each corner point in the reference image in the electronic screen space can be determined according to the reference image configuration information, for example, the three-dimensional position information of the corner point in the reference image in the electronic screen space as indicated by the mark 401 is (1, 1, 0). In another feasible implementation manner, if the reference image is an Aruco image, the electronic device can obtain the corresponding MarkId according to the part of the Aruco image in the shooting image during the process of determining the three-dimensional position information of each corner point in the reference image in the electronic screen space, and then the corresponding three-dimensional position information can be obtained by mapping the MarkId.

[0054] S204, selecting a target relative pose of the positioning device and the camera device and a target pose of the electronic screen in the positioning device space from the M relative poses of the positioning device and the camera device and the M poses of the electronic screen in the positioning device space determined based on the M sets of shooting data.

[0055] The target relative pose and the target pose are used to determine a pose of a virtual camera device relative to a virtual screen, and the target relative pose and the target pose are determined based on a same set of shooting data in the M sets of shooting data.

[0056] In the embodiments of the present application, after N sets of shooting data including shooting images and poses of corresponding positioning devices in a positioning device space are obtained, M sets of shooting data can be constructed based on the N sets of shooting data. Then, for H sets of shooting data in a mth set of shooting data in the M sets of shooting data, a mth relative pose of the positioning device and the camera device and a mth pose of the electronic screen in the positioning device space can be determined based on the H sets of shooting data and position information of the reference image in the electronic screen space. Then, a target relative pose of the positioning device and the camera device and a target pose of the electronic screen in the positioning device space can be selected from the M relative poses of the positioning device and the camera device and the M poses of the electronic screen in the positioning device space determined based on the M sets of shooting data. The target relative pose of the positioning device and the camera device and the target pose of the electronic screen in the positioning device space can be determined based on the N sets of shooting data including shooting images and poses of corresponding positioning devices in the positioning device space. That is, the relative pose relationship among the positioning device, the camera device and the electronic screen can be calibrated based on multiple sets of shooting data including shooting images and poses of corresponding positioning devices in the positioning device space. Moreover, the combination of multiple sets of shooting data can expand the sets of shooting data required to determine the target relative pose and the target pose.

[0057] Based on the above-mentioned relative pose calibration method, another relative pose calibration method is provided in the embodiments of the present application. Referring to Figure 5 The flowchart of another relative pose calibration method provided in the embodiments of the present application is shown. Figure 5 The relative pose calibration method shown can be executed by an electronic device. Figure 5 The relative pose calibration method shown can include the following steps:

[0058] S501, obtaining N sets of shooting data.

[0059] Any one of the shooting data comprises a shooting image, and a pose of the positioning device in a positioning device space when the shooting image is obtained; the shooting image is obtained by a camera device bound to the positioning device from any shooting angle of a reference image in the electronic screen, N is a positive integer; the related process of step S501 is similar to the related process of step S201, and details are not repeated here.

[0060] S502, constructing M shooting data groups based on N shooting data.

[0061] Any one of the shooting data comprises a shooting image, and a pose of the positioning device in a positioning device space when the shooting image is obtained; the shooting image is obtained by a camera device bound to the positioning device from any shooting angle of a reference image in the electronic screen, N is a positive integer; the related process of step S501 is similar to the related process of step S201, and details are not repeated here.

[0062] S503, traversing the M shooting data groups, and determining, for H shooting data in the mth shooting data group in the current traversal, a mth relative pose of the positioning device and the camera device and a mth pose of the electronic screen in the positioning device space based on the H shooting data and position information of the reference image in the electronic screen space.

[0063] Wherein, m∈[1,M]。

[0064] In one embodiment, the electronic device determines, for H shooting data in the mth shooting data group in the current traversal, a mth relative pose of the positioning device and the camera device and a mth pose of the electronic screen in the positioning device space based on the H shooting data and position information of the reference image in the electronic screen space, which can include: determining, based on a shooting image in the hth shooting data of the H shooting data and the position information of the reference image in the electronic screen space, a hth reference pose of the electronic screen relative to the camera device, to obtain H reference poses of the electronic screen relative to the camera device; determining, based on the poses of the positioning device in the positioning device space in the H shooting data and the H reference poses, the mth relative pose of the positioning device and the camera device; determining, based on the poses of the positioning device in the positioning device space in the H shooting data, the H reference poses, and the mth relative pose of the positioning device and the camera device, the mth pose of the electronic screen in the positioning device space; wherein, h∈[1,H]。

[0065] In a specific implementation, the photographed image in the hth photographed data can be referred to as the hth photographed image; the electronic device determines the hth reference pose of the electronic screen relative to the camera device based on the photographed image in the hth photographed data in the H photographed data and the position information of the reference image under the electronic screen space, which can include: obtaining the two-dimensional position information of each corner point in the hth photographed image in the hth photographed image; determining the hth reference pose of the electronic screen relative to the camera device according to the position mapping relationship between the three-dimensional position information of each corner point in the reference image under the electronic screen space and the two-dimensional position information of each corner point in the hth photographed image in the hth photographed image. Since the hth photographed image is obtained by the camera device bound to the positioning device from a certain shooting angle, each corner point in the hth photographed image corresponds to each corner point in the reference image one-to-one, and the position of each corner point in the hth photographed image in the hth photographed image can be regarded as the projection position of each corner point in the reference image displayed on the electronic screen. Based on this, when the electronic device determines the hth reference pose of the electronic screen relative to the camera device according to the three-dimensional position information of each corner point in the reference image under the electronic screen space and the position mapping relationship between the two-dimensional position information of each corner point in the hth photographed image in the hth photographed image, the PNP (perspective-n-point) algorithm in the pose estimation algorithm can be used for solving; further, when the PNP algorithm is used to solve the hth reference pose of the electronic screen relative to the camera device, the intrinsic parameter of the camera device also needs to be used; optionally, the intrinsic parameter of the camera device can be determined by Zhang Zhengyou calibration method. When the electronic device obtains the two-dimensional position information of each corner point in the hth photographed image in the hth photographed image, the UV information of the corresponding corner point found by using the OpenCV (Open Computer Vision) can be used.

[0066] Further, when the electronic device determines the mth relative pose of the positioning device and the camera device based on the pose of the positioning device in the positioning device space in the H photographed data and the H reference poses of the electronic screen relative to the camera device, the hand-eye calibration algorithm can be used.

[0067] Further, the pose of the positioning device in the hth photographing data in the positioning device space can be referred to as the hth pose of the positioning device in the positioning device space; the electronic device determines the mth pose of the electronic screen in the positioning device space based on the poses of the positioning device in the positioning device space in the H photographing data, the H reference poses, and the mth relative pose of the positioning device and the camera device, which can include: determining the hth pose of the camera device in the positioning device space based on the hth pose of the positioning device in the positioning device space and the mth relative pose of the positioning device and the camera device, to obtain H poses of the camera device in the positioning device space; and determining the mth pose of the electronic screen in the positioning device space based on the H poses of the camera device in the positioning device space and the H reference poses of the electronic screen relative to the camera device. Since the pose of the electronic screen in the positioning device space satisfies the loop: the pose of the electronic screen in the positioning device space * the pose of the electronic screen relative to the camera device = the pose of the camera device in the positioning device space, and since the pose of the electronic screen relative to the camera device includes H, i.e., the H reference poses of the electronic screen relative to the camera device, and the pose of the camera device in the positioning device space includes H, i.e., the H poses of the camera device in the positioning device space, based on this, the mth pose of the electronic screen in the positioning device space determined based on the H poses of the camera device in the positioning device space and the H reference poses of the electronic screen relative to the camera device can be the optimal solution of the pose of the electronic screen in the positioning device space obtained by using a decomposition algorithm (for example, a singular value decomposition algorithm, i.e., an SVD (Singular Value Decomposition) algorithm), i.e., the pose of the electronic screen in the positioning device space under the optimal solution is taken as the mth pose of the electronic screen in the positioning device space.

[0068] S504, for the mth photographing data set, determining a calibration error corresponding to the mth photographing data set based on the mth relative pose of the positioning device and the camera device, the mth pose of the electronic screen in the positioning device space, and the poses of the positioning device in the positioning device space in the N photographing data.

[0069] The calibration error corresponding to the mth photographing data set is used to measure the accuracy of the mth relative pose of the positioning device and the camera device and the mth pose of the electronic screen in the positioning device space.

[0070] In one embodiment, the electronic device determines the calibration error corresponding to the mth set of photographed data based on the mth relative pose of the positioning device and the camera device, the mth pose of the electronic screen in the positioning device space, and the poses of the positioning device in the positioning device space in the N sets of photographed data, which can include: determining the nth contrast pose of the electronic screen relative to the camera device based on the mth relative pose of the positioning device and the camera device, the mth pose of the electronic screen in the positioning device space, and the pose of the positioning device in the positioning device space in the nth set of photographed data in the N sets of photographed data, to obtain N contrast poses of the electronic screen relative to the camera device; determining the calibration error corresponding to the mth set of photographed data according to the differences between the N contrast poses of the electronic screen relative to the camera device and the N reference poses of the electronic screen relative to the camera device; wherein the nth reference pose of the N reference poses of the electronic screen relative to the camera device is determined based on the photographed image in the nth set of photographed data and the position information of the reference image in the electronic screen space, n ∈ [1, N].

[0071] In a specific implementation, the photographed image in the nth set of photographed data can be referred to as the nth photographed image; and the electronic device determines the nth reference pose of the N reference poses of the electronic screen relative to the camera device based on the photographed image in the nth set of photographed data and the position information of the reference image in the electronic screen space, which can include: obtaining the two-dimensional position information of each corner point in the nth photographed image; and determining the nth reference pose of the electronic screen relative to the camera device according to the position mapping relationship between the three-dimensional position information of each corner point in the reference image in the electronic screen space and the two-dimensional position information of each corner point in the nth photographed image; the related process is similar to the related process of determining the hth reference pose of the electronic screen relative to the camera device based on the photographed image in the hth set of photographed data in the H sets of photographed data and the position information of the reference image in the electronic screen space, and will not be described here.

[0072] Further, the pose of the positioning device in the n-th photographing data in the positioning device space can be referred to as the n-th pose of the positioning device in the positioning device space, and the electronic device determines the n-th contrast pose of the electronic screen relative to the camera based on the m-th relative pose of the positioning device relative to the camera, the m-th pose of the electronic screen in the positioning device space, and the pose of the positioning device in the n-th photographing data in the positioning device space, which can include: determining the n-th pose of the camera in the positioning device space based on the n-th pose of the positioning device in the positioning device space and the m-th relative pose of the positioning device relative to the camera; and determining the n-th contrast pose of the electronic screen relative to the camera based on the n-th pose of the camera in the positioning device space and the m-th pose of the electronic screen in the positioning device space.

[0073] Further, the electronic device determines the calibration error corresponding to the m-th photographing data set according to the differences between the N contrast poses of the electronic screen relative to the camera and the N reference poses of the electronic screen relative to the camera, which can include: determining the n-th reference error corresponding to the m-th photographing data set according to the difference between the n-th contrast pose of the electronic screen relative to the camera and the n-th reference pose of the electronic screen relative to the camera in the N contrast poses of the electronic screen relative to the camera and the N reference poses of the electronic screen relative to the camera, to obtain the N reference errors corresponding to the m-th photographing data set; and performing error analysis processing on the N reference errors corresponding to the m-th photographing data set to obtain the calibration error corresponding to the m-th photographing data set. Optionally, the calibration error corresponding to the m-th photographing data set obtained by the electronic device performing error analysis processing on the N reference errors corresponding to the m-th photographing data set includes but is not limited to the following: an average error obtained by performing average processing on the N reference errors corresponding to the m-th photographing data set, and a sum error obtained by performing sum processing on the N reference errors corresponding to the m-th photographing data set.

[0074] In a specific implementation, the electronic device determines the n th reference error corresponding to the m th set of shooting data according to the difference between the n th comparison attitude of the electronic screen relative to the camera and the n th reference attitude of the electronic screen relative to the camera, which can include: determining the n th shooting image corresponding to the n th reference attitude from the shooting images in the N sets of shooting data according to the n th reference attitude of the electronic screen relative to the camera; obtaining the two-dimensional position information of each corner point in the n th shooting image in the n th shooting image; determining the mapped two-dimensional position information of each corner point in the reference image in the n th shooting image according to the three-dimensional position information of each corner point in the reference image in the electronic screen space and the n th comparison attitude of the electronic screen relative to the camera; and determining the n th reference error corresponding to the m th set of shooting data based on the difference between the mapped two-dimensional position information of each corner point in the reference image in the n th shooting image and the two-dimensional position information of each corner point in the n th shooting image in the n th shooting image.

[0075] In a specific implementation, the electronic device determines the mapped two-dimensional position information of each corner point in the reference image in the n th shooting image according to the three-dimensional position information of each corner point in the reference image in the electronic screen space and the n th comparison attitude of the electronic screen relative to the camera, which can be implemented by using a model projection function. In the process of determining the n th reference error corresponding to the m th set of shooting data based on the difference between the mapped two-dimensional position information of each corner point in the reference image in the n th shooting image and the two-dimensional position information of each corner point in the n th shooting image in the n th shooting image, the distance of the corresponding corner point in the n th shooting image can be determined based on the mapped two-dimensional position information of each corner point in the reference image in the n th shooting image and the two-dimensional position information of the corresponding corner point in the n th shooting image in the n th shooting image. Further, the distance of any corresponding corner point in the n th shooting image can be determined as the n th reference error corresponding to the m th set of shooting data, or the sum of distances determined based on the distances of each corresponding corner point in the n th shooting image can be determined as the n th reference error corresponding to the m th set of shooting data, or the average distance determined based on the distances of each corresponding corner point in the n th shooting image can be determined as the n th reference error corresponding to the m th set of shooting data, and so on. The distance of the corresponding corner point in the n th shooting image can be a physical distance, a pixel distance, and so on. The embodiments of the present application are described by taking the pixel distance as an example.

[0076] S505, from the calibration errors corresponding to the M sets of shooting data, determine the minimum calibration error.

[0077] S506, determine the relative pose of the positioning device and the camera device indicated by the minimum calibration error as a target relative pose, and determine the pose of the electronic screen in the positioning device space indicated by the minimum calibration error as a target pose.

[0078] The target relative pose and the target pose are used to determine the pose of the virtual camera device relative to the virtual screen.

[0079] In the embodiments of the present application, after obtaining N pieces of shooting data including a shooting image and a pose of a corresponding positioning device in a positioning device space, M pieces of shooting data groups can be constructed based on the N pieces of shooting data. Then, by traversing the M pieces of shooting data groups, for H pieces of shooting data in the mth piece of shooting data group in the current traversal, the mth relative pose of the positioning device and the camera device and the mth pose of the electronic screen in the positioning device space can be determined based on the H pieces of shooting data and the position information of the reference image in the electronic screen space. Then, for the mth piece of shooting data group, the mth calibration error corresponding to the mth piece of shooting data group can be determined based on the mth relative pose of the positioning device and the camera device, the mth pose of the electronic screen in the positioning device space, and the poses of the positioning device in the positioning device space in the N pieces of shooting data. The minimum calibration error is determined from the calibration errors corresponding to the M pieces of shooting data groups. Then, the relative pose of the positioning device and the camera device indicated by the minimum calibration error is determined as a target relative pose, and the pose of the electronic screen in the positioning device space indicated by the minimum calibration error is determined as a target pose. That is, in the process of selecting the target relative pose of the positioning device and the camera device and the target pose of the electronic screen in the positioning device space from the M relative poses of the positioning device and the camera device and the M poses of the electronic screen in the positioning device space determined based on the M pieces of shooting data groups, the relative pose of the positioning device and the camera device and the pose of the electronic screen in the positioning device space corresponding to the target shooting data group indicated by the minimum calibration error can be selected, which can improve the accuracy of the determined target relative pose and target pose, i.e., the calibration accuracy of the relative pose relationship among the positioning device, the camera device, and the electronic screen.

[0080] Based on the related embodiments of the above attitude calibration method, the embodiments of the present application provide another attitude calibration method. Referring to Figure 6 The flowchart of another attitude calibration method provided by the embodiments of the present application is shown. Figure 6 The attitude calibration method shown can be executed by an electronic device. Figure 6 The attitude calibration method shown can include the following steps:

[0081] S601, obtaining N pieces of shooting data.

[0082] Each of the captured data includes a captured image and the orientation of the positioning device in the positioning device space when the captured image is captured; wherein the captured image is obtained by a camera device attached to the positioning device capturing a reference image on the electronic screen from any shooting angle, and N is a positive integer.

[0083] In one embodiment, before acquiring N sets of captured data, the electronic device can also acquire electronic screen configuration information and reference image configuration information, generate a reference image based on the reference image configuration information, and send the reference image to the electronic screen for display. The electronic screen configuration information can be used to determine the coordinate system information of the three-dimensional coordinate system constructed based on the electronic screen, and the reference image configuration information can be used to determine the three-dimensional position information of each corner point in the reference image in the electronic screen space.

[0084] In one feasible implementation, if the reference image is a chessboard diagram, the electronic screen configuration information may include the size of the electronic screen; the reference image configuration information may include the size of the reference image, the display position of the reference image on the electronic screen, and the size of the chessboard squares; Figure 4 Taking the reference image displayed on the electronic screen as an example, the dimensions of the electronic screen are 5 meters long, 6 meters wide, and 0 meters high. The dimensions of the reference image are 2 meters long, 3 meters wide, and 0 meters high. The dimensions of the checkerboard grid are 0.5 meters long, 0.5 meters wide, and 0 meters high. Referring to the display position of the reference image on the electronic screen, and considering that the three-dimensional position information of the upper left corner of the reference image in the electronic screen space is (0.5, 0.5, 0), the three-dimensional position information of each corner point in the reference image in the electronic screen space can be determined based on the above reference image configuration information. For example, as shown by mark 401, the three-dimensional position information of the corner point in the reference image in the electronic screen space is (1, 1, 0).

[0085] In another feasible implementation, if the reference image is a chessboard diagram, the electronic screen configuration information may include the size of the electronic screen, the number of electronic screen blocks, the size of the electronic screen after block division, and the pixel size of the electronic screen after block division. The reference image configuration information may include the size of the reference image (specifically, it can be represented by the number of electronic screen blocks included in the reference image) and the display position of the reference image on the electronic screen (specifically, it can be represented by the index of the electronic screen block corresponding to the chessboard square indicated by the upper left corner of the reference image when displayed on the electronic screen). Figure 4The display is shown as an example of the reference image in the electronic screen. At this time, the size of the electronic screen included in the electronic screen configuration information is 5 meters long, 6 meters wide, and 0 meters high. The number of electronic screen blocks is 10 blocks long and 12 blocks wide. The size of the blocked electronic screen is 0.5 meters long and 0.5 meters wide. The pixel size of the blocked electronic screen indicates a resolution of 216*216. The size of the reference image included in the reference image configuration information can be 4 blocks long and 6 blocks wide when the number of blocked electronic screens included in the reference image is used to represent the size of the reference image. The display position of the reference image in the electronic screen can be represented by the index of the blocked electronic screen corresponding to the chessboard indicated by the upper left corner of the reference image when displayed in the electronic screen. It can be (2, 2).

[0086] Further, the electronic screen configuration information and the reference image configuration information can be object entered. Specifically, it can be entered through the configuration information entry interface displayed in the electronic device. Referring to Figure 7 A configuration information entry interface provided by an embodiment of the present application; the configuration information entry interface can include an electronic screen configuration information entry area as shown in 701. The memory of the electronic device can store various sizes, pixel sizes, and other information of the electronic screen entered in advance. By selecting the electronic screen to be configured in the electronic screen selection area as shown in 702, the electronic device can obtain the size and pixel size of the corresponding electronic screen from the memory. At this time, the screen name of the selected electronic screen can be displayed in the electronic screen configuration information entry area as shown in 701, specifically as "LED screen name". Further, the number of electronic screen blocks in the electronic screen configuration information entry area as shown in 701 is specifically displayed as "LED screen block". The size of the blocked electronic screen in the electronic screen configuration information entry area as shown in 701 is specifically displayed as "block physical size". The pixel size of the blocked electronic screen in the electronic screen configuration information entry area as shown in 701 is specifically displayed as "block pixel size". Optionally, after entering the number of electronic screen blocks, the electronic device can generate the size of the blocked electronic screen and the pixel size of the blocked electronic screen according to the number of electronic screen blocks, the size of the electronic screen, and the pixel size, and display them in the corresponding entry area.

[0087] The configuration information input interface can include a reference image configuration information input area as indicated by 703. When the size of the reference image is represented by the number of the divided electronic screens in the reference image, in the reference image configuration information input area as indicated by 703, it is specifically displayed as "generate block length and width". When the display position of the reference image in the electronic screen is represented by the index of the divided electronic screen corresponding to the chessboard indicated by the upper left corner of the reference image when displayed in the electronic screen, in the reference image configuration information input area as indicated by 703, it is specifically displayed as "upper left corner index". Further, the type of the reference image can be selected in the area as indicated by 703, for example, a chessboard image (chessboard) or an Aruco image. When the Aruco image is selected, the start marker identification (i.e., start MarkId) needs to be set.

[0088] Further optionally, the configuration information input interface can further include a configuration operation area as indicated by 704. The configuration operation area can include a new component, a modified component, a deleted component, a vertex binding component (specifically displayed as "bind vertex"), and a reference image generation component (specifically displayed as "generate chessboard / Aruco image"). Through triggering of the new component, the modified component, and the deleted component, the addition, modification, and deletion of information such as the size and pixel size of the electronic screen can be respectively realized. Through triggering of the vertex binding component, the binding of the related information of the reference image and the three-dimensional coordinate system constructed based on the electronic screen can be realized. Through triggering of the reference image generation component, the reference image can be generated and sent to the electronic screen for display, and further the three-dimensional position information of the corner points in the reference image under the electronic screen space can be obtained.

[0089] Further optionally, the configuration information input interface can further include a chessboard information configuration area as indicated by 705. The chessboard information configuration area can realize the input of the number of horizontal corner points, the number of vertical corner points, the size of each horizontal grid, and the size of each vertical grid. The related information input in the chessboard information configuration area is used to generate and print the chessboard image when the chessboard image needs to be output and printed.

[0090] S602, constructing M groups of shooting data based on N groups of shooting data.

[0091] Wherein, the number of shooting data in any group of shooting data is H, H∈[Z, N], Z is a quantity threshold, and M is a positive integer.

[0092] S603, traversing the M groups of shooting data, for the H shooting data in the mth group of shooting data currently traversed, based on the H shooting data and the position information of the reference image in the electronic screen space, determining the mth relative pose of the positioning device and the camera device and the mth pose of the electronic screen in the positioning device space.

[0093] wherein m∈[1, M].

[0094] S604, for the mth group of shooting data, based on the mth relative pose of the positioning device and the camera device, the mth pose of the electronic screen in the positioning device space, and the pose of the positioning device in the positioning device space in the N groups of shooting data, determining the calibration error corresponding to the mth group of shooting data.

[0095] wherein the calibration error corresponding to the mth group of shooting data is used to measure the accuracy of the mth relative pose of the positioning device and the camera device and the mth pose of the electronic screen in the positioning device space.

[0096] S605, determining the minimum calibration error from the calibration errors corresponding to the M groups of shooting data.

[0097] S606, determining the relative pose of the positioning device and the camera device indicated by the minimum calibration error as the target relative pose, and determining the pose of the electronic screen in the positioning device space indicated by the minimum calibration error as the target pose.

[0098] wherein the target relative pose and the target pose are used to determine the pose of the virtual camera device relative to the virtual screen; the related processes of steps S601 to S606 are similar to the related processes of steps S501 to S506 described above, and will not be repeated here.

[0099] In one embodiment, the electronic device can further output the minimum calibration error; output N reference errors corresponding to the target photographing data set indicated by the minimum calibration error; obtain the mapping two-dimensional position information of each corner point in the reference image in the nth photographing image as the target two-dimensional position information of each corner point in the nth photographing image in the reference image, which is obtained in the process of determining the nth reference error corresponding to the target photographing data set; and map and display the target two-dimensional position information of each corner point in the nth photographing image in the reference image to the nth photographing image, n∈[1, N]; so that the subject can intuitively understand the calibration error corresponding to the target relative pose of the positioning device and the camera device and the target pose of the electronic screen under the positioning device, and intuitively understand the target relative pose and the target pose, and the reference error determined based on each comparison pose and the corresponding reference pose of the electronic screen relative to the camera device, that is, the subject can intuitively understand which photographing data in the N photographing data brings larger reference error. Further optionally, the electronic device can further output the indication information of the other photographing data in the N photographing data except for each photographing data in the target photographing data set indicated by the minimum calibration error, to prompt the subject to delete the other photographing data.

[0100] Referring to Figure 8A schematic diagram of a pose calibration interface provided by an embodiment of the present application; the pose calibration interface can include a shooting data entry area as indicated by 801, after receiving the shooting data, the electronic device can store the shooting data in a folder, based on which, the shooting data can be obtained from the corresponding folder by entering the corresponding pose folder and pose file name in the shooting data entry area; further, the pose calibration interface can also include a reference error display area as indicated by 802, for displaying the N reference errors corresponding to the target shooting data group indicated by the minimum calibration error; at this time, the first reference error is the reference error 5.07 indicated by index 0, the second reference error is the reference error 8.87 indicated by index 1, and so on; by triggering the calculation component in the reference error display area, the electronic device can output the minimum calibration error, which can be displayed in the calibration error display area as indicated by 803, wherein the minimum calibration error is 9.13484, and the target shooting data group includes the shooting data indicated by indexes 0, 2, 4, 5, and 6, so indexes 1 and 3 can be output to prompt the object to delete the shooting data indicated by indexes 1 and 3; the pose calibration interface can also include an error comparison display area as indicated by 804, which can display the mapping of the target two-dimensional position information of each corner point in the reference image in the nth shooting image to the image after the nth shooting image. Further optionally, the pose calibration interface can also include a camera parameter display area as indicated by 805, for displaying the intrinsic parameters of the camera, specifically including: focal length Fx and Fy, the position of the camera optical center in the image Center, and the distortion coefficient K1; further optionally, the pose calibration interface can also include a first pose display area as indicated by 806, for displaying the pose of the positioning device in the positioning device space; further optionally, the pose calibration interface can also include a second pose display area as indicated by 807, for displaying the target relative pose of the positioning device and the camera. Further optionally, the pose calibration interface can also include a camera physical parameter display area as indicated by 808, for configuring the size of the camera internal sensor, which, together with the intrinsic parameters of the camera, can determine the difference between the calculated focal length and the real focal length. It is known that the related configuration is not necessary.

[0101] S607, obtaining K test data.

[0102] Any test data includes a test image and the pose of the positioning device in the positioning device space when the test image is shot, wherein the test image is obtained by shooting the reference image in the electronic screen from any shooting angle by the camera bound to the positioning device, K is a positive integer.

[0103] S608, determining the kth test reference pose of the electronic screen relative to the camera device based on the test image in the kth test data and the position information of the reference image in the electronic screen space, to obtain K test reference poses of the electronic screen relative to the camera device.

[0104] Wherein, k∈[1, K]; the related process of step S608 is similar to the related process of determining the nth reference pose of the N reference poses of the electronic screen relative to the camera device based on the photographed image in the nth photographed data and the position information of the reference image in the electronic screen space, which will not be repeated here.

[0105] S609, determining the kth test comparison pose of the electronic screen relative to the camera device based on the target relative pose of the positioning device and the camera device, the target pose of the electronic screen in the positioning device space, and the pose of the positioning device in the positioning device space in the kth test data, to obtain K test comparison poses of the electronic screen relative to the camera device.

[0106] Wherein, the related process of step S609 is similar to the related process of determining the nth comparison pose of the electronic screen relative to the camera device based on the mth relative pose of the positioning device and the camera device, the mth pose of the electronic screen in the positioning device space, and the pose of the positioning device in the positioning device space in the nth photographed data, which will not be repeated here.

[0107] S610, determining the kth test reference error according to the difference between the kth test comparison pose of the electronic screen relative to the camera device and the kth test reference pose of the electronic screen relative to the camera device, to obtain K test reference errors.

[0108] Wherein, the related process of step S610 is similar to the related process of determining the nth reference error corresponding to the mth photographed data set according to the difference between the nth comparison pose of the N comparison poses of the electronic screen relative to the camera device and the nth reference pose of the N reference poses of the electronic screen relative to the camera device, which will not be repeated here.

[0109] S611, selecting L test data from the K test data as photographed data according to the K test reference errors, and adding to the N photographed data, to update the target relative pose and the target pose based on the updated photographed data.

[0110] The test reference error corresponding to the L test data is greater than other test reference errors in the K test reference errors except for the test reference error corresponding to the L test data. The L test data is selected from the K test data as the shooting data according to the K test reference errors and is added to the N shooting data, so as to select the L test data with greater test reference error from the K test data, that is, select the L test data with poor shooting angle effect, and add the L test data to the N shooting data, so that the target relative attitude and the target attitude can be iteratively updated based on the updated shooting data, and the accuracy of the target relative attitude and the target attitude is further improved. In a feasible implementation manner, the L test data is selected from the K test data as the shooting data according to the K test reference errors, and the test data with a test reference error greater than a reference error threshold can be selected from the K test data. The reference error threshold can be set according to specific requirements, for example, set to 10 pixels. In another feasible implementation manner, the L test data is selected from the K test data as the shooting data according to the K test reference errors, and the test data with a test reference error arranged in the top L can be selected from the K test data. At this time, L can be a value set according to specific requirements.

[0111] In another feasible implementation manner, the electronic device can also delete, from the N shooting data, other shooting data except for each shooting data in the target shooting data group indicated by the target relative attitude, to obtain deleted shooting data. At this time, the electronic device can select the L test data from the K test data as the shooting data according to the K test reference errors and add the L test data to the N shooting data, which can include selecting the L test data from the K test data as the shooting data according to the K test reference errors and adding the L test data to the deleted shooting data. That is, the electronic device can delete, from the N shooting data, other shooting data except for each shooting data in the target shooting data group indicated by the minimum calibration error, to obtain deleted shooting data, and then select the L test data from the K test data as the shooting data according to the K test reference errors and add the L test data to the deleted shooting data, so as to iteratively update the target relative attitude and the target attitude based on the updated shooting data. Since the other shooting data in the N shooting data except for each shooting data in the target shooting data group indicated by the minimum calibration error is not the shooting data used to determine the target relative attitude and the target attitude, the other shooting data is deleted, and the L test data is further selected from the K test data as the shooting data according to the K test reference errors and added to the deleted shooting data, so as to accelerate the rate of iteratively updating the target relative attitude and the target attitude based on the updated shooting data.

[0112] For example, referring to Figure 9The schematic diagram of updating the target relative pose and the target pose provided by the embodiment of the present application is as follows. After triggering the start component in the pose updating interface as indicated by the label 901, the electronic device generates the corresponding test reference error according to the continuously input test data, and maps and displays the target two-dimensional position information of each corner point in the reference image corresponding to the test reference error in the test image in the area as indicated by the label 902. In the case where the test reference error is greater than the reference error threshold, the test data is recorded. After triggering the end component in the pose updating interface, the electronic device ends the input of the test data, and after the serialization option as indicated by the label 903 is selected, adds the recorded test data as the photographed data to the deleted photographed data, and performs serialization processing to obtain the updated photographed data. Then, after the calculation component is triggered, the target relative pose and the target pose are updated based on the updated photographed data. At this time, the number of the updated photographed data is 8, the minimum calibration error determined based on the updated photographed data, and the 8 reference errors and the like corresponding to the target photographed data group indicated by the minimum calibration error are displayed in the pose calibration page as indicated by the label 904. The minimum calibration error is 5.562366, the target photographed data group includes the photographed data indicated by the indexes 0, 1, 2 and 7, so the indexes 3, 4, 5 and 6 can be output to prompt the object to delete the photographed data indicated by the indexes 3, 4, 5 and 6. The first reference error is the reference error 6.50 indicated by the index 0, the second reference error is the reference error 1.78 indicated by the index 1, and the like. Optionally, the pose updating interface as indicated by the label 901 can further include a test data recording component (specifically displayed as “record frame”) so that the corresponding test data can be recorded by triggering the test data recording component.

[0113] Referring to Figure 10A flowchart for updating a target relative pose and a target pose is provided for an embodiment of the present application. An electronic device can obtain electronic screen configuration information and reference image configuration information, generate a reference image according to the reference image configuration information, and send the reference image to the electronic screen for display. The electronic screen configuration information can be used to determine coordinate system information of a three-dimensional coordinate system constructed based on the electronic screen, and the reference image configuration information can be used to determine three-dimensional position information of each corner point in the reference image in the electronic screen space. Further, the electronic device can obtain N pieces of shooting data, any piece of shooting data including a shooting image and a pose of a positioning device in a positioning device space when the shooting image is obtained, and construct M pieces of shooting data groups based on the N pieces of shooting data. For H pieces of shooting data in the mth piece of shooting data group in the current iteration, based on the shooting image in the hth piece of shooting data and the position information of the reference image in the electronic screen space, the hth reference pose of the electronic screen relative to the camera device is determined to obtain H reference poses of the electronic screen relative to the camera device. Based on the pose of the positioning device in the positioning device space in the H pieces of shooting data and the H reference poses, the mth relative pose of the positioning device and the camera device is determined. Based on the pose of the positioning device in the positioning device space in the H pieces of shooting data, the H reference poses, and the mth relative pose of the positioning device and the camera device, the mth pose of the electronic screen in the positioning device space is determined. For the mth piece of shooting data group, based on the mth relative pose of the positioning device and the camera device, the mth pose of the electronic screen in the positioning device space, and the pose of the positioning device in the positioning device space in the N pieces of shooting data, the calibration error corresponding to the mth piece of shooting data group is determined. From the calibration errors corresponding to the M pieces of shooting data groups, the minimum calibration error is determined. The relative pose of the positioning device and the camera device indicated by the minimum calibration error is determined as the target relative pose, and the pose of the electronic screen in the positioning device space indicated by the minimum calibration error is determined as the target pose.

[0114] Further, the N photographing data can be updated based on the K test data, the target relative pose and the target pose, so as to iteratively update the target relative pose and the target pose based on the updated photographing data. Specifically, the electronic device can determine a kth test reference pose of the electronic screen relative to the camera based on a test image in the kth test data in the K test data and the position information of the reference image under the electronic screen space, so as to obtain K test reference poses of the electronic screen relative to the camera; determine a kth test comparison pose of the electronic screen relative to the camera based on the target relative pose of the positioning device and the camera, the target pose of the electronic screen under the positioning device space and the pose of the positioning device under the positioning device space in the kth test data in the K test data, so as to obtain K test comparison poses of the electronic screen relative to the camera; determine a kth test reference error according to the difference between the kth test comparison pose of the electronic screen relative to the camera and the kth test reference pose of the electronic screen relative to the camera, so as to obtain K test reference errors; and select L test data from the K test data as photographing data according to the K test reference errors, and add the L test data to the N photographing data, so as to iteratively update the target relative pose and the target pose based on the updated photographing data.

[0115] In the embodiments of this application, after obtaining the target relative pose of the positioning device and the camera device based on the N pieces of shooting data, and the target pose of the electronic screen in the space of the positioning device, K pieces of test data can be obtained. Based on the test image in the kth test data in the K pieces of test data and the position information of the reference image in the space of the electronic screen, the kth test reference pose of the electronic screen relative to the camera device is determined to obtain K test reference poses of the electronic screen relative to the camera device. Based on the target relative pose of the positioning device and the camera device, the target pose of the electronic screen in the space of the positioning device, and the pose of the positioning device in the space of the positioning device in the kth test data in the K pieces of test data, the kth test comparison pose of the electronic screen relative to the camera device is determined to obtain K test comparison poses of the electronic screen relative to the camera device. According to the difference between the kth test comparison pose of the electronic screen relative to the camera device and the kth test reference pose of the electronic screen relative to the camera device, the kth test reference error is determined to obtain K test reference errors. Then, according to the K test reference errors, L pieces of test data can be selected from the K pieces of test data as shooting data and added to the N pieces of shooting data to update the target relative pose and the target pose based on the updated shooting data. The L pieces of test data with larger test reference errors can be selected from the K pieces of test data, that is, the L pieces of test data with poor shooting angle effect are selected and added to the N pieces of shooting data, so that the target relative pose and the target pose can be updated based on the updated shooting data, and the accuracy of the target relative pose and the target pose is further improved, that is, the calibration accuracy of the relative pose relationship among the positioning device, the camera device and the electronic screen is further improved. In addition, the other shooting data except the shooting data in the target shooting data group indicated by the target relative pose can be deleted from the N pieces of shooting data to obtain the deleted shooting data, and the selected L pieces of test data are added to the deleted shooting data as shooting data, which can speed up the rate of updating the target relative pose and the target pose based on the updated shooting data.

[0116] Based on the related embodiments of the above attitude calibration method, the embodiments of this application provide an attitude calibration device. Referring to Figure 11 The attitude calibration device provided in the embodiments of this application can include an acquisition unit 1101 and a processing unit 1102. Figure 11 The attitude calibration device shown can run the following units:

[0117] The acquisition unit 1101 is configured to acquire N pieces of shooting data; any piece of shooting data comprises a shooting image and a pose of a positioning device under a positioning device space when the shooting image is obtained; the shooting image is obtained by a camera device bound to the positioning device from any shooting angle of a reference image in an electronic screen; N is a positive integer;

[0118] The processing unit 1102 is configured to construct M groups of shooting data based on the N pieces of shooting data; the number of pieces of shooting data in any group of shooting data is H, H is included in [Z, N], Z is a quantity threshold, and M is a positive integer;

[0119] The processing unit 1102 is further configured to traverse the M groups of shooting data, determine an mth relative pose of the positioning device and the camera device and an mth pose of the electronic screen under the positioning device space based on H pieces of shooting data in a currently traversed mth group of shooting data, the H pieces of shooting data, and position information of the reference image under an electronic screen space; m is included in [1, M];

[0120] The processing unit 1102 is further configured to select a target relative pose of the positioning device and the camera device and a target pose of the electronic screen under the positioning device space from the M relative poses of the positioning device and the camera device and the M poses of the electronic screen under the positioning device space determined based on the M groups of shooting data; the target relative pose and the target pose are used to determine a pose of a virtual camera device relative to a virtual screen.

[0121] In one embodiment, when the processing unit 1102 selects the target relative pose of the positioning device and the camera device and the target pose of the electronic screen under the positioning device space from the M relative poses of the positioning device and the camera device and the M poses of the electronic screen under the positioning device space determined based on the M groups of shooting data, the processing unit 1102 specifically performs the following operations:

[0122] For the mth group of shooting data, the processing unit 1102 determines a calibration error corresponding to the mth group of shooting data based on the mth relative pose of the positioning device and the camera device, the mth pose of the electronic screen under the positioning device space, and the poses of the positioning device under the positioning device space in the N pieces of shooting data; the calibration error corresponding to the mth group of shooting data is used to measure the accuracy of the mth relative pose of the positioning device and the camera device and the mth pose of the electronic screen under the positioning device space;

[0123] The processing unit 1102 determines a minimum calibration error from the calibration errors corresponding to the M groups of shooting data;

[0124] determining the relative pose of the positioning device and the camera device indicated by the minimum calibration error as the target relative pose, and determining the pose of the electronic screen in the positioning device space indicated by the minimum calibration error as the target pose.

[0125] In one embodiment, when the processing unit 1102 determines the calibration error corresponding to the mth photographing data set based on the mth relative pose of the positioning device and the camera device, the mth pose of the electronic screen in the positioning device space, and the pose of the positioning device in the positioning device space in the N photographing data, the processing unit 1102 specifically performs the following operations:

[0126] determining the nth contrast pose of the electronic screen relative to the camera device based on the mth relative pose of the positioning device and the camera device, the mth pose of the electronic screen in the positioning device space, and the pose of the positioning device in the positioning device space in the nth photographing data in the N photographing data; n ∈ [1, N];

[0127] determining the calibration error corresponding to the mth photographing data set according to the difference between the N contrast poses of the electronic screen relative to the camera device and the N reference poses of the electronic screen relative to the camera device; wherein the nth reference pose of the N reference poses of the electronic screen relative to the camera device is determined based on the photographed image in the nth photographing data and the position information of the reference image in the electronic screen space.

[0128] In one embodiment, the position information of the reference image in the electronic screen space includes: three-dimensional position information of each corner point in the reference image in the electronic screen space; the photographed image in the nth photographing data is referred to as the nth photographed image.

[0129] When the processing unit 1102 determines the nth reference pose of the N reference poses of the electronic screen relative to the camera device based on the photographed image in the nth photographing data and the position information of the reference image in the electronic screen space, the processing unit 1102 specifically performs the following operations:

[0130] obtaining two-dimensional position information of each corner point in the nth photographed image in the nth photographed image;

[0131] According to the position mapping relationship between the three-dimensional position information of each corner point in the reference image under the electronic screen space and the two-dimensional position information of each corner point in the nth photographed image in the nth photographed image, a first reference pose of the electronic screen relative to the camera device is determined.

[0132] In one embodiment, when the processing unit 1102 determines the calibration error corresponding to the mth photographed data set according to the differences between the N comparative poses of the electronic screen relative to the camera device and the N reference poses of the electronic screen relative to the camera device, the processing unit 1102 specifically performs the following operations:

[0133] According to the difference between the nth comparative pose of the N comparative poses of the electronic screen relative to the camera device and the nth reference pose of the N reference poses of the electronic screen relative to the camera device, an nth reference error corresponding to the mth photographed data set is determined to obtain N reference errors corresponding to the mth photographed data set.

[0134] The N reference errors corresponding to the mth photographed data set are subjected to error analysis processing to obtain the calibration error corresponding to the mth photographed data set.

[0135] In one embodiment, the position information of the reference image under the electronic screen space includes three-dimensional position information of each corner point in the reference image under the electronic screen space.

[0136] When the processing unit 1102 determines the nth reference error corresponding to the mth photographed data set according to the difference between the nth comparative pose of the N comparative poses of the electronic screen relative to the camera device and the nth reference pose of the N reference poses of the electronic screen relative to the camera device, the processing unit 1102 specifically performs the following operations:

[0137] According to the first reference pose of the electronic screen relative to the camera device, an nth photographed image corresponding to the first reference pose is determined from the N photographed data.

[0138] The two-dimensional position information of each corner point in the nth photographed image is obtained.

[0139] According to the three-dimensional position information of each corner point in the reference image under the electronic screen space and the nth comparative pose of the electronic screen relative to the camera device, the mapped two-dimensional position information of each corner point in the reference image in the nth photographed image is determined.

[0140] determine an n-th reference error corresponding to the m-th shooting data group based on the mapped two-dimensional position information of each corner point in the reference image in the n-th shooting image and a difference between the two-dimensional position information of each corner point in the n-th shooting image in the n-th shooting image.

[0141] In one embodiment, the position information of the reference image in the electronic screen space includes three-dimensional position information of each corner point in the reference image in the electronic screen space.

[0142] The processing unit 1102 is further configured to:

[0143] output the minimum calibration error;

[0144] output N reference errors corresponding to a target shooting data group indicated by the minimum calibration error;

[0145] obtain the mapped two-dimensional position information of each corner point in the reference image in the n-th shooting image as target two-dimensional position information of each corner point in the reference image in the n-th shooting image in a process of determining the n-th reference error corresponding to the target shooting data group;

[0146] map the target two-dimensional position information of each corner point in the reference image in the n-th shooting image to the n-th shooting image for display.

[0147] In one embodiment, the obtaining unit 1101 is further configured to obtain K test data; any test data includes a test image and a pose of the positioning device in the positioning device space when the test image is shot, and K is a positive integer;

[0148] The processing unit 1102 is further configured to determine a k-th test reference pose of the electronic screen relative to the camera device based on the test image in the k-th test data in the K test data and the position information of the reference image in the electronic screen space, to obtain K test reference poses of the electronic screen relative to the camera device; k ∈ [1, K];

[0149] The processing unit 1102 is further configured to determine a k-th test comparison pose of the electronic screen relative to the camera device based on the target relative pose of the positioning device and the camera device, the target pose of the electronic screen in the positioning device space, and the pose of the positioning device in the positioning device space in the k-th test data in the K test data, to obtain K test comparison poses of the electronic screen relative to the camera device;

[0150] The processing unit 1102 is further configured to determine a kth test reference error according to a difference between a kth test relative pose of the electronic screen relative to the camera device and a kth test reference pose of the electronic screen relative to the camera device, to obtain K test reference errors.

[0151] The processing unit 1102 is further configured to select L test data from the K test data as shooting data according to the K test reference errors, and add the L test data to the N shooting data, to iteratively update the target relative pose and the target pose based on updated shooting data, wherein the L test data correspond to test reference errors greater than other test reference errors in the K test reference errors except for the test reference errors corresponding to the L test data.

[0152] In an embodiment, the processing unit 1102 is further configured to delete, from the N shooting data, shooting data other than each shooting data in a target shooting data group indicated by the target relative pose, to obtain deleted shooting data.

[0153] When the processing unit 1102 selects L test data from the K test data as shooting data according to the K test reference errors and adds the L test data to the N shooting data, the processing unit 1102 performs the following operations:

[0154] Selects L test data from the K test data as shooting data according to the K test reference errors and adds the L test data to the deleted shooting data.

[0155] In an embodiment, when the processing unit 1102 determines a mth relative pose of the positioning device relative to the camera device and a mth pose of the electronic screen in the positioning device space based on the H shooting data and position information of the reference image in the electronic screen space, the processing unit 1102 performs the following operations:

[0156] Determines a hth reference pose of the electronic screen relative to the camera device based on a shooting image in a hth shooting data in the H shooting data and the position information of the reference image in the electronic screen space, to obtain H reference poses of the electronic screen relative to the camera device, h∈[1, H];

[0157] Determines a mth relative pose of the positioning device relative to the camera device based on a pose of the positioning device in the positioning device space in the H shooting data and the H reference poses;

[0158] Based on the orientation of the positioning device in the positioning device space from the H captured data, the H reference orientations, and the m-th relative orientation between the positioning device and the camera device, the m-th orientation of the electronic screen in the positioning device space is determined.

[0159] In one embodiment, the attitude of the positioning device in the positioning device space in the h-th captured data among the H captured data is called the h-th attitude of the positioning device in the positioning device space.

[0160] When the processing unit 1102 determines the m-th pose of the electronic screen in the positioning device space based on the pose of the positioning device in the positioning device space from the H captured data, the H reference poses, and the m-th relative pose between the positioning device and the camera device, it specifically performs the following operations:

[0161] Based on the h-th pose of the positioning device in the positioning device space and the m-th relative pose between the positioning device and the camera device, the h-th pose of the camera device in the positioning device space is determined, so as to obtain H poses of the camera device in the positioning device space.

[0162] Based on the H poses of the camera device in the positioning device space and the H reference poses of the electronic screen relative to the camera device, the m-th pose of the electronic screen in the positioning device space is determined.

[0163] According to one embodiment of this application, Figure 2 , Figure 5 as well as Figure 6 The attitude calibration method shown can involve various steps that can be achieved by... Figure 11 This is performed by the individual units in the attitude calibration device shown. For example, Figure 2 The step S201 shown can be performed by Figure 11 The acquisition unit 1101 in the attitude calibration device shown performs this action; Figure 2 Steps S202 to S204 shown can be derived from Figure 11 The attitude calibration device shown is executed by the processing unit 1102. For example, Figure 5 The step S501 shown can be performed by Figure 11 The acquisition unit 1101 in the attitude calibration device shown performs this action; Figure 5 Steps S502 to S506 shown can be derived from... Figure 11 The attitude calibration device shown is executed by the processing unit 1102. For example, Figure 6 Steps S601 and S607 shown can be derived from... Figure 11 The acquisition unit 1101 in the attitude calibration device shown performs this action;Figure 6 The steps S602 to S606, S608 to S611 shown can be executed by the processing unit 1102 in the attitude calibration device shown. Figure 11 The processing unit 1102 in the attitude calibration device shown can execute the steps S602 to S606, S608 to S611 shown.

[0164] According to another embodiment of the present application, Figure 11 The units in the attitude calibration device shown can be respectively or all combined into one or several other units to constitute, or some of the units can be further split into a plurality of units with smaller functions to constitute, which can achieve the same operation without affecting the implementation of the technical effects of the embodiments of the present application. The above units are divided based on logical functions, and in actual applications, the functions of one unit can also be implemented by multiple units, or the functions of multiple units can be implemented by one unit. In other embodiments of the present application, the attitude calibration device divided based on logical functions can also include other units, and in actual applications, these functions can also be implemented by other units, and can be implemented by multiple units in cooperation.

[0165] According to another embodiment of the present application, the attitude calibration device as shown in Figure 2 , Figure 5 and Figure 6 the attitude calibration device as shown in Figure 11 and the attitude calibration method of the embodiments of the present application can be constructed by running the computer program (including program codes) related to each step of the corresponding method as shown in

[0166] In the embodiments of the present application, after N pieces of shooting data including the shooting image and the pose of the corresponding positioning device in the positioning device space are acquired, M pieces of shooting data groups can be constructed based on the N pieces of shooting data. Then, by traversing the M pieces of shooting data groups, for the H pieces of shooting data in the mth piece of shooting data group in the current traversal, the mth relative pose of the positioning device and the camera device and the mth pose of the electronic screen in the positioning device space can be determined based on the H pieces of shooting data and the position information of the reference image in the electronic screen space. Then, the target relative pose of the positioning device and the camera device and the target pose of the electronic screen in the positioning device space can be selected from the M relative poses of the positioning device and the camera device and the M poses of the electronic screen in the positioning device space determined based on the M pieces of shooting data groups. The target relative pose of the positioning device and the camera device and the target pose of the electronic screen in the positioning device space can be determined based on the N pieces of shooting data including the shooting image and the pose of the corresponding positioning device in the positioning device space, that is, the relative pose relationship among the positioning device, the camera device and the electronic screen can be calibrated based on multiple pieces of shooting data including the shooting image and the pose of the corresponding positioning device in the positioning device space, and the shooting data group required for determining the target relative pose and the target pose is expanded by combining multiple pieces of shooting data.

[0167] Based on the related embodiments of the above-mentioned pose calibration method and the pose calibration device embodiments, the present application further provides an electronic device. Referring to Figure 12 , a structural schematic diagram of an electronic device provided by the embodiments of the present application. Figure 12 The electronic device shown can at least include a processor 1201, an input interface 1202, an output interface 1203 and a computer storage medium 1204. Among them, the processor 1201, the input interface 1202, the output interface 1203 and the computer storage medium 1204 can be connected through a bus or other means.

[0168] The computer storage medium 1204 can be stored in the memory of the electronic device, and the computer storage medium 1204 is used to store a computer program, the computer program includes program instructions, and the processor 1201 is used to execute the program instructions stored by the computer storage medium 1204. The processor 1201 (or CPU (Central Processing Unit, Central Processor)) is the computing core and control core of the electronic device, which is suitable for implementing one or more instructions, and is particularly suitable for loading and executing one or more instructions to implement the above-mentioned pose calibration method process or corresponding function.

[0169] The embodiment of the present application further provides a computer storage medium (Memory), which is a memory device in an electronic device and is used for storing programs and data. It can be understood that the computer storage medium herein can include a built-in storage medium in the terminal, and of course can also include an expansion storage medium supported by the terminal. The computer storage medium provides a storage space, and the storage space stores an operating system of the terminal. Furthermore, the storage space also stores one or more instructions which are suitable for being loaded and executed by the processor 1201, and the instructions can be one or more computer programs (including program codes). It should be noted that the computer storage medium herein can be a high-speed random access memory (RAM) memory, or a non-volatile memory such as at least one disk memory; and optionally, the computer storage medium can be at least one computer storage medium located away from the processor.

[0170] In one embodiment, the one or more instructions stored in the computer storage medium can be loaded and executed by the processor 1201 to implement the corresponding steps of the method in the above-mentioned posture calibration method embodiment. Figure 2 、 Figure 5 and Figure 6 In a specific implementation, the processor 1201 can be used to:

[0171] obtain N pieces of shooting data; any piece of shooting data includes a shooting image and a posture of the positioning device in a positioning device space when the shooting image is obtained; wherein the shooting image is obtained by a camera device bound to the positioning device from any shooting angle, and the shooting image is a reference image in an electronic screen; and N is a positive integer;

[0172] construct M pieces of shooting data groups based on the N pieces of shooting data; the number of shooting data in any piece of shooting data group is H, H ∈ [Z, N], Z is a quantity threshold, and M is a positive integer;

[0173] traverse the M pieces of shooting data groups, and based on H pieces of shooting data in a current traversed mth piece of shooting data group and position information of the reference image in an electronic screen space, determine an mth relative posture of the positioning device and the camera device and an mth posture of the electronic screen in the positioning device space; m ∈ [1, M];

[0174] selecting, from the M relative poses of the positioning device and the camera device determined based on the M sets of photographed data and the M poses of the electronic screen in the positioning device space, a target relative pose of the positioning device and the camera device and a target pose of the electronic screen in the positioning device space; the target relative pose and the target pose are used to determine a pose of a virtual camera device relative to a virtual screen.

[0175] In one embodiment, when the processor 1201 selects, from the M relative poses of the positioning device and the camera device determined based on the M sets of photographed data and the M poses of the electronic screen in the positioning device space, a target relative pose of the positioning device and the camera device and a target pose of the electronic screen in the positioning device space, the processor 1201 specifically performs the following operations:

[0176] For the mth set of photographed data, based on the mth relative pose of the positioning device and the camera device, the mth pose of the electronic screen in the positioning device space, and the poses of the positioning device in the positioning device space in the N sets of photographed data, a calibration error corresponding to the mth set of photographed data is determined; the calibration error corresponding to the mth set of photographed data is used to measure the accuracy of the mth relative pose of the positioning device and the camera device and the mth pose of the electronic screen in the positioning device space;

[0177] From the calibration errors corresponding to the M sets of photographed data, a minimum calibration error is determined;

[0178] The relative pose of the positioning device and the camera device indicated by the minimum calibration error is determined as the target relative pose, and the pose of the electronic screen in the positioning device space indicated by the minimum calibration error is determined as the target pose.

[0179] In one embodiment, when the processor 1201 determines the calibration error corresponding to the mth set of photographed data based on the mth relative pose of the positioning device and the camera device, the mth pose of the electronic screen in the positioning device space, and the poses of the positioning device in the positioning device space in the N sets of photographed data, the processor 1201 specifically performs the following operations:

[0180] Based on the mth relative pose of the positioning device and the camera device, the mth pose of the electronic screen in the positioning device space, and the pose of the positioning device in the positioning device space in the nth set of photographed data in the N sets of photographed data, an nth contrast pose of the electronic screen relative to the camera device is determined to obtain N contrast poses of the electronic screen relative to the camera device; n∈[1, N];

[0181] determine the calibration error corresponding to the mth set of photographed data according to the difference between the N comparative poses of the electronic screen relative to the camera device and the N reference poses of the electronic screen relative to the camera device; wherein the nth reference pose of the N reference poses of the electronic screen relative to the camera device is determined based on the photographed image in the nth photographed data and the position information of the reference image in the electronic screen space.

[0182] In an embodiment, the position information of the reference image in the electronic screen space includes: three-dimensional position information of each corner point in the reference image in the electronic screen space; the photographed image in the nth photographed data is referred to as the nth photographed image.

[0183] When the processor 1201 determines the nth reference pose of the N reference poses of the electronic screen relative to the camera device based on the photographed image in the nth photographed data and the position information of the reference image in the electronic screen space, the following operations are specifically performed:

[0184] Obtain two-dimensional position information of each corner point in the nth photographed image in the nth photographed image.

[0185] Determine the nth reference pose of the electronic screen relative to the camera device according to the position mapping relationship between the three-dimensional position information of each corner point in the reference image in the electronic screen space and the two-dimensional position information of each corner point in the nth photographed image in the nth photographed image.

[0186] In an embodiment, when the processor 1201 determines the calibration error corresponding to the mth set of photographed data according to the difference between the N comparative poses of the electronic screen relative to the camera device and the N reference poses of the electronic screen relative to the camera device, the following operations are specifically performed:

[0187] Determine the nth reference error corresponding to the mth set of photographed data according to the difference between the nth comparative pose of the N comparative poses of the electronic screen relative to the camera device and the nth reference pose of the N reference poses of the electronic screen relative to the camera device, to obtain the N reference errors corresponding to the mth set of photographed data.

[0188] Perform error analysis processing on the N reference errors corresponding to the mth set of photographed data to obtain the calibration error corresponding to the mth set of photographed data.

[0189] In an embodiment, the position information of the reference image in the electronic screen space comprises: three-dimensional position information of each corner point in the reference image in the electronic screen space.

[0190] The processor 1201 determines an n th reference error corresponding to the m th shooting data set according to a difference between an n th comparison posture of the electronic screen relative to the camera and an n th reference posture of the electronic screen relative to the camera.

[0191] According to the n th reference posture of the electronic screen relative to the camera, an n th shooting image corresponding to the n th reference posture is determined from the shooting images in the N shooting data.

[0192] Obtain two-dimensional position information of each corner point in the n th shooting image in the n th shooting image.

[0193] According to the three-dimensional position information of each corner point in the reference image in the electronic screen space and the n th comparison posture of the electronic screen relative to the camera, the mapped two-dimensional position information of each corner point in the reference image in the n th shooting image is determined.

[0194] Based on the difference between the mapped two-dimensional position information of each corner point in the reference image in the n th shooting image and the two-dimensional position information of each corner point in the n th shooting image in the n th shooting image, the n th reference error corresponding to the m th shooting data set is determined.

[0195] In an embodiment, the position information of the reference image in the electronic screen space comprises: three-dimensional position information of each corner point in the reference image in the electronic screen space.

[0196] The processor 1201 is further configured to:

[0197] Output the minimum calibration error.

[0198] Output the N reference errors corresponding to the target shooting data set indicated by the minimum calibration error.

[0199] Obtain the mapped two-dimensional position information of each corner point in the reference image in the n th shooting image as the target two-dimensional position information of each corner point in the reference image in the n th shooting image in the process of determining the n th reference error corresponding to the target shooting data set.

[0200] Map the target two-dimensional position information of each corner point in the reference image in the nth captured image to the nth captured image for display.

[0201] In one embodiment, the processor 1201 is further configured to:

[0202] Obtain K test data, any test data including a test image and a pose of the positioning device in a positioning device space when the test image is captured, K being a positive integer;

[0203] Determine a kth test reference pose of the electronic screen relative to the camera device based on the test image in the kth test data in the K test data and the position information of the reference image in the electronic screen space, to obtain K test reference poses of the electronic screen relative to the camera device, k∈[1, K];

[0204] Determine a kth test comparison pose of the electronic screen relative to the camera device based on the target relative pose of the positioning device and the camera device, the target pose of the electronic screen in the positioning device space, and the pose of the positioning device in the positioning device space in the kth test data in the K test data, to obtain K test comparison poses of the electronic screen relative to the camera device;

[0205] Determine a kth test reference error based on the difference between the kth test comparison pose of the electronic screen relative to the camera device and the kth test reference pose of the electronic screen relative to the camera device, to obtain K test reference errors;

[0206] Select L test data from the K test data as captured data to be added to the N captured data according to the K test reference errors, to update the target relative pose and the target pose based on the updated captured data; wherein the test reference errors corresponding to the L test data are greater than the test reference errors other than the test reference errors corresponding to the L test data among the K test reference errors.

[0207] In one embodiment, the processor 1201 is further configured to delete, from the N captured data, captured data other than the captured data in the target captured data group indicated by the target relative pose, to obtain deleted captured data;

[0208] When the processor 1201 selects L test data from the K test data as captured data to be added to the N captured data according to the K test reference errors, the processor 1201 specifically performs the following operations:

[0209] According to the K test reference errors, L test data are selected from the K test data as the photographed data and added to the deleted photographed data.

[0210] In one embodiment, the processor 1201 determines the mth relative pose of the positioning device relative to the camera device and the mth pose of the electronic screen in the positioning device space based on the H photographed data and the position information of the reference image in the electronic screen space, and specifically performs the following operations:

[0211] Based on the photographed image in the hth photographed data of the H photographed data and the position information of the reference image in the electronic screen space, the hth reference pose of the electronic screen relative to the camera device is determined to obtain the H reference poses of the electronic screen relative to the camera device, h∈[1, H];

[0212] Based on the poses of the positioning device in the positioning device space in the H photographed data and the H reference poses, the mth relative pose of the positioning device relative to the camera device is determined.

[0213] Based on the poses of the positioning device in the positioning device space in the H photographed data, the H reference poses, and the mth relative pose of the positioning device relative to the camera device, the mth pose of the electronic screen in the positioning device space is determined.

[0214] In one embodiment, the pose of the positioning device in the positioning device space in the hth photographed data of the H photographed data is referred to as the hth pose of the positioning device in the positioning device space;

[0215] When the processor 1201 determines the mth pose of the electronic screen in the positioning device space based on the poses of the positioning device in the positioning device space in the H photographed data, the H reference poses, and the mth relative pose of the positioning device relative to the camera device, it specifically performs the following operations:

[0216] Based on the hth pose of the positioning device in the positioning device space and the mth relative pose of the positioning device relative to the camera device, the hth pose of the camera device in the positioning device space is determined to obtain the H poses of the camera device in the positioning device space.

[0217] Based on the H poses of the camera device in the positioning device space and the H reference poses of the electronic screen relative to the camera device, the mth pose of the electronic screen in the positioning device space is determined.

[0218] The embodiment of the present application provides a computer program product, the computer program product comprises a computer program, the computer program is stored in a computer storage medium; the processor of an electronic device reads the computer program from the computer storage medium, the processor executes the computer program, and the electronic device executes the method embodiment shown in the above Figure 2 、 Figure 5 and Figure 6 . Wherein, the computer readable storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM) and the like.

[0219] The above is merely a specific implementation of the present application, but the protection scope of the present application is not limited to this, any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method of pose calibration, the method comprising: The method comprises the following steps: acquiring N pieces of shooting data; any piece of shooting data comprises a shooting image and a pose of a positioning device in a positioning device space when the shooting image is shot; the shooting image is obtained by a camera device bound to the positioning device from any shooting angle of a reference image in an electronic screen, and N is a positive integer; constructing M pieces of shooting data groups based on the N pieces of shooting data; the number of pieces of shooting data in any piece of shooting data group is H, H ∈ [Z, N], Z is a quantity threshold, and M is a positive integer; iterating the M pieces of shooting data groups, and determining an mth relative pose of the positioning device and the camera device and an mth pose of the electronic screen in the positioning device space based on H pieces of shooting data in an mth piece of shooting data group currently iterated, the H pieces of shooting data, and position information of the reference image in an electronic screen space; m ∈ [1, M]; selecting a target relative pose of the positioning device and the camera device and a target pose of the electronic screen in the positioning device space from the M relative poses of the positioning device and the camera device and the M poses of the electronic screen in the positioning device space determined based on the M pieces of shooting data groups; the target relative pose and the target pose are used to determine a pose of a virtual camera device relative to a virtual screen; wherein the selecting the target relative pose of the positioning device and the camera device and the target pose of the electronic screen in the positioning device space comprises: determining a calibration error corresponding to the mth piece of shooting data group based on the mth relative pose of the positioning device and the camera device, the mth pose of the electronic screen in the positioning device space, and poses of the positioning device in the positioning device space in the N pieces of shooting data for the mth piece of shooting data group; determining a minimum calibration error from the calibration errors corresponding to the M pieces of shooting data groups; determining the relative pose of the positioning device and the camera device indicated by the minimum calibration error as the target relative pose and determining the pose of the electronic screen in the positioning device space indicated by the minimum calibration error as the target pose.

2. The method of claim 1, wherein, The calibration error corresponding to the mth piece of shooting data group is used to measure the accuracy of the mth relative pose of the positioning device and the camera device and the mth pose of the electronic screen in the positioning device space.

3. The method of claim 2, wherein, The determining the calibration error corresponding to the mth piece of shooting data group based on the mth relative pose of the positioning device and the camera device, the mth pose of the electronic screen in the positioning device space, and the poses of the positioning device in the positioning device space in the N pieces of shooting data comprises: determining an n-th contrast posture of the electronic screen relative to the camera based on the m-th relative posture of the positioning device and the camera, the m-th posture of the electronic screen in the positioning device space, and the posture of the positioning device in the n-th photographing data in the positioning device space, to obtain N contrast postures of the electronic screen relative to the camera; n∈[1, N]; determining the calibration error corresponding to the m-th photographing data set according to the N contrast postures of the electronic screen relative to the camera and the difference between the N reference postures of the electronic screen relative to the camera; wherein the n-th reference posture of the N reference postures of the electronic screen relative to the camera is determined based on the photographing image in the n-th photographing data and the position information of the reference image in the electronic screen space.

4. The method of claim 3, wherein, The position information of the reference image in the electronic screen space includes three-dimensional position information of each corner point in the reference image in the electronic screen space; the photographing image in the n-th photographing data is referred to as an n-th photographing image. Determining the n-th reference posture of the N reference postures of the electronic screen relative to the camera based on the photographing image in the n-th photographing data and the position information of the reference image in the electronic screen space includes: obtaining two-dimensional position information of each corner point in the n-th photographing image in the n-th photographing image; determining the n-th reference posture of the electronic screen relative to the camera according to the position mapping relationship between the three-dimensional position information of each corner point in the reference image in the electronic screen space and the two-dimensional position information of each corner point in the n-th photographing image in the n-th photographing image.

5. The method of claim 3, wherein, The determining the calibration error corresponding to the m-th photographing data set according to the N contrast postures of the electronic screen relative to the camera and the difference between the N reference postures of the electronic screen relative to the camera includes: determining an n-th reference error corresponding to the m-th photographing data set according to the difference between the n-th contrast posture of the N contrast postures of the electronic screen relative to the camera and the n-th reference posture of the N reference postures of the electronic screen relative to the camera, to obtain N reference errors corresponding to the m-th photographing data set; performing error analysis processing on the N reference errors corresponding to the m-th photographing data set to obtain the calibration error corresponding to the m-th photographing data set.

6. The method of claim 5, wherein, The position information of the reference image in the electronic screen space includes three-dimensional position information of each corner point in the reference image in the electronic screen space. The determining the n-th reference error corresponding to the m-th photographing data set according to the difference between the n-th contrast posture of the N contrast postures of the electronic screen relative to the camera and the n-th reference posture of the N reference postures of the electronic screen relative to the camera includes: determining an n-th photographed image corresponding to an n-th reference pose of the electronic screen relative to the camera device from photographed images in the N photographed data according to the n-th reference pose of the electronic screen relative to the camera device; obtaining two-dimensional position information of each corner point in the n-th photographed image in the n-th photographed image; determining mapped two-dimensional position information of each corner point in the reference image in the n-th photographed image according to three-dimensional position information of each corner point in the reference image in the electronic screen space and an n-th comparison pose of the electronic screen relative to the camera device; determining an n-th reference error corresponding to the m-th photographed data group based on a difference between the mapped two-dimensional position information of each corner point in the reference image in the n-th photographed image and the two-dimensional position information of each corner point in the n-th photographed image in the n-th photographed image.

7. The method of claim 5, wherein, The position information of the reference image in the electronic screen space comprises three-dimensional position information of each corner point in the reference image in the electronic screen space. The method further comprises: outputting the minimum calibration error; outputting N reference errors corresponding to a target photographed data group indicated by the minimum calibration error; obtaining the mapped two-dimensional position information of each corner point in the reference image in the n-th photographed image as target two-dimensional position information of each corner point in the n-th photographed image in the reference image in a process of determining the n-th reference error corresponding to the target photographed data group; mapping and displaying the target two-dimensional position information of each corner point in the reference image in the n-th photographed image in the n-th photographed image.

8. The method of claim 1, wherein, The method further comprises: obtaining K test data; any test data comprises a test image and a pose of the positioning device in the positioning device space when the test image is photographed, and K is a positive integer; determining K test reference poses of the electronic screen relative to the camera device based on a test image in k-th test data in the K test data and the position information of the reference image in the electronic screen space, to obtain K test reference poses of the electronic screen relative to the camera device; k ∈ [1, K]; determining K test comparison poses of the electronic screen relative to the camera device based on a target relative pose of the positioning device and the camera device, a target pose of the electronic screen in the positioning device space and the pose of the positioning device in the positioning device space in the k-th test data in the K test data, to obtain K test comparison poses of the electronic screen relative to the camera device; determining k-th test reference error according to a difference between the k-th test comparison pose of the electronic screen relative to the camera device and the k-th test reference pose of the electronic screen relative to the camera device, to obtain K test reference errors; determining k-th test reference error according to a difference between the k-th test comparison pose of the electronic screen relative to the camera device and the k-th test reference pose of the electronic screen relative to the camera device, to obtain K test reference errors; According to the K test reference errors, L test data are selected from the K test data as shooting data and added to the N shooting data, so as to iteratively update the target relative pose and the target pose based on the updated shooting data; wherein the test reference error corresponding to the L test data is greater than other test reference errors in the K test reference errors except the test reference error corresponding to the L test data.

9. The method of claim 8, wherein, The method further comprises: From the N shooting data, other shooting data except each shooting data in the target shooting data group indicated by the target relative pose is deleted to obtain deleted shooting data; The method further comprises: According to the K test reference errors, L test data are selected from the K test data as shooting data and added to the N shooting data, so as to iteratively update the target relative pose and the target pose based on the updated shooting data; wherein the test reference error corresponding to the L test data is greater than other test reference errors in the K test reference errors except the test reference error corresponding to the L test data.

10. The method of claim 1, wherein, The method further comprises: According to the K test reference errors, L test data are selected from the K test data as shooting data and added to the N shooting data, so as to iteratively update the target relative pose and the target pose based on the updated shooting data; wherein the test reference error corresponding to the L test data is greater than other test reference errors in the K test reference errors except the test reference error corresponding to the L test data. The method further comprises: Based on the hth shooting data in the H shooting data and the position information of the reference image in the electronic screen space, an mth relative pose of the positioning device relative to the camera device and an mth pose of the electronic screen in the positioning device space are determined.

11. The method of claim 10, wherein, Based on the hth shooting data in the H shooting data and the position information of the reference image in the electronic screen space, an hth reference pose of the electronic screen relative to the camera device is determined, so as to obtain H reference poses of the electronic screen relative to the camera device, h∈[1, H]; Based on the H shooting data and the position information of the reference image in the electronic screen space, an mth relative pose of the positioning device relative to the camera device and an mth pose of the electronic screen in the positioning device space are determined. Based on the hth shooting data in the H shooting data and the position information of the reference image in the electronic screen space, an hth reference pose of the electronic screen relative to the camera device is determined, so as to obtain H reference poses of the electronic screen relative to the camera device, h∈[1, H]; The hth pose of the positioning device in the positioning device space in the hth shooting data in the H shooting data is referred to as an hth pose of the positioning device in the positioning device space; 12. A pose calibration apparatus, characterized by, The method further comprises: Based on the hth pose of the positioning device in the positioning device space and the mth relative pose of the positioning device relative to the camera device, an hth pose of the camera device in the positioning device space is determined, so as to obtain H poses of the camera device in the positioning device space; Based on the H poses of the camera device in the positioning device space and the H reference poses of the electronic screen relative to the camera device, the mth pose of the electronic screen in the positioning device space is determined. The method further comprises: An acquisition unit is configured to acquire N pieces of shooting data; any piece of shooting data comprises a shooting image and a pose of a positioning device under a positioning device space when the shooting image is obtained; the shooting image is obtained by a camera device bound to the positioning device from any shooting angle of a reference image in an electronic screen; N is a positive integer; A processing unit is configured to construct M groups of shooting data based on the N pieces of shooting data; a number of pieces of shooting data in any group of shooting data is H, H is in a range of [Z, N], Z is a number threshold, and M is a positive integer; The processing unit is further configured to traverse the M groups of shooting data, determine an mth relative pose of the positioning device and the camera device and an mth pose of the electronic screen under the positioning device space based on H pieces of shooting data in a currently traversed mth group of shooting data, the H pieces of shooting data, and position information of the reference image under an electronic screen space; m is in a range of [1, M]; The processing unit is further configured to select a target relative pose of the positioning device and the camera device and a target pose of the electronic screen under the positioning device space from the M relative poses of the positioning device and the camera device and the M poses of the electronic screen under the positioning device space determined based on the M groups of shooting data; the target relative pose and the target pose are used to determine a pose of a virtual camera device relative to a virtual screen; The processing unit is specifically configured to determine a calibration error corresponding to the mth group of shooting data based on the mth relative pose of the positioning device and the camera device, the mth pose of the electronic screen under the positioning device space, and poses of the positioning device under the positioning device space in the N pieces of shooting data for the mth group of shooting data; determine a minimum calibration error from the calibration errors corresponding to the M groups of shooting data; and determine the relative pose of the positioning device and the camera device indicated by the minimum calibration error as the target relative pose and determine the pose of the electronic screen under the positioning device space indicated by the minimum calibration error as the target pose.

13. An electronic device, comprising: The electronic device comprises an input interface and an output interface, and further comprises: a processor configured to execute one or more instructions; and a computer storage medium storing one or more instructions, which are loaded and executed by the processor to implement the pose calibration method according to any one of claims 1-11.

14. A computer storage medium, characterized in that The computer storage medium stores computer program instructions, which are executed by the processor to implement the pose calibration method according to any one of claims 1-11.

15. A computer program product, characterised in that, The computer program product comprises a computer program, which is executed by the processor to implement the pose calibration method according to any one of claims 1-11.

Images