A spatial positioning method and related device

Abstract

The present application discloses a spatial positioning method and related devices, which obtain the initial real focal length of a real camera, and the initial real focal length can determine the virtual object distance between a virtual camera and a virtual object, so as to set the initial virtual position of the virtual camera in the virtual space according to the initial real focal length. For the i-th frame image in the content to be converted shot by the real camera, the i-th real position of the real camera in the i-th frame image and the i-1th real position of the real camera in the i-1th frame image are obtained, and then the difference between the two is determined. According to the difference and the initial virtual position, the i-th virtual position of the virtual camera in the i-th frame image is determined in the virtual space. As a result, there is no need for high-end spatial positioning equipment to transmit the absolute position of the real camera in real time. By continuously accumulating the difference between the positions of the real camera in adjacent frame images on the initial virtual position, the position of the virtual camera in the current frame image can be determined, which reduces the shooting cost.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a space positioning method and related devices. Background Art

[0002] Green screen shooting is generally when a camera shoots real people in front of a blue screen, green screen or other background screen, and then uses a computer or other equipment to cut out the real people, obtain the cutout people, and put the cutout people into the virtual space, thereby integrating the real people into the virtual space. When a real camera shoots the real space, it will move up and down, left and right, etc. In order to better integrate the real people with the virtual space, the virtual camera also needs to make corresponding changes when shooting the virtual space.

[0003] In related technologies, the real space is built in a one-to-one ratio according to the size of the virtual space, and real people are scanned in real time through optical space positioning equipment such as lasers or infrared radars. The real people carry receivers such as lasers or infrared radars to locate their precise position in the real space, thereby determining the movement changes of the real camera and adjusting the position of the virtual camera in the virtual space.

[0004] However, the above method needs to rely on hardware to track real people in real time, and the hardware equipment needs to have the functions of ranging and precise positioning. It has high requirements on the hardware equipment and requires a very large space to accommodate the optical space positioning equipment, resulting in high cost for shooting. Summary of the invention

[0005] In response to the above problems, the present application provides a spatial positioning method and related devices to reduce the cost required for shooting.

[0006] Based on this, the embodiments of the present application disclose the following technical solutions:

[0007] On the one hand, an embodiment of the present application provides a spatial positioning method, the method comprising:

[0008] Get the initial real focal length of the real camera;

[0009] According to the initial real focal length, setting an initial virtual position of the virtual camera in the virtual space;

[0010] For an i-th frame image in the content to be converted captured by the real camera, obtain an i-th real position of the real camera in the i-th frame image and an i-1-th real position of the real camera in the i-1-th frame image, wherein the i-th frame image is one of the N consecutive frames of images included in the content to be converted, and the i-1-th frame image is a previous frame image of the i-th frame image in the N consecutive frames of images;

[0011] The i-th virtual position of the virtual camera in the i-th frame image is determined in the virtual space according to the difference between the i-th real position and the (i-1)-th real position, and the initial virtual position.

[0012] Optionally, the method further includes:

[0013] Obtaining an initial virtual rotation quaternion of the virtual camera in the virtual space, and obtaining an orientation quaternion of the real camera in the i-th frame image;

[0014] The determining, in the virtual space, the i-th virtual position of the virtual camera in the i-th frame image according to the difference between the i-th real position and the i-1-th real position and the initial virtual position comprises:

[0015] The i-th virtual position of the virtual camera in the i-th frame image is determined in the virtual space according to the initial virtual rotation quaternion, the orientation quaternion, the difference between the i-th real position and the i-1-th real position, and the initial virtual position.

[0016] Optionally, the method further includes:

[0017] The matting object is mounted on the virtual camera so that the object distance between the matting object and the virtual camera remains unchanged. The matting object is an object obtained by matting a real object and used to be integrated into the virtual space.

[0018] Optionally, for the i-th frame image, the method further includes:

[0019] Determining a real viewing angle of the real camera according to the initial real focal length, the sensor size of the real camera, and the final imaging width of the real camera;

[0020] Obtaining a virtual viewing angle of the virtual camera;

[0021] If the difference between the virtual viewing wide angle and the real viewing wide angle is greater than a preset threshold, the virtual viewing wide angle is adjusted according to the real viewing wide angle.

[0022] On the other hand, an embodiment of the present application provides a spatial positioning device, the device comprising: an acquisition unit, a setting unit and a determination unit;

[0023] The acquisition unit is used to acquire the initial real focal length of the real camera;

[0024] The setting unit is used to set an initial virtual position of the virtual camera in the virtual space according to the initial real focal length;

[0025] The acquisition unit is further used to acquire, for an i-th frame image in the content to be converted taken by the real camera, an i-th real position of the real camera in the i-th frame image, and an i-1-th real position of the real camera in the i-1-th frame image, wherein the i-th frame image is one of the continuous N frames of images included in the content to be converted, and the i-1-th frame image is a previous frame image of the i-th frame image in the continuous N frames of images;

[0026] The determining unit is used to determine, in the virtual space, an i-th virtual position of the virtual camera in the i-th frame image according to a difference between the i-th real position and the (i-1)-th real position and the initial virtual position.

[0027] Optionally, the acquisition unit is further used to:

[0028] Obtaining an initial virtual rotation quaternion of the virtual camera in the virtual space, and obtaining an orientation quaternion of the real camera in the i-th frame image;

[0029] The determining unit is used for:

[0030] The i-th virtual position of the virtual camera in the i-th frame image is determined in the virtual space according to the initial virtual rotation quaternion, the orientation quaternion, the difference between the i-th real position and the i-1-th real position, and the initial virtual position.

[0031] Optionally, the device further comprises: a mounting unit, configured to:

[0032] The matting object is mounted on the virtual camera so that the object distance between the matting object and the virtual camera remains unchanged. The matting object is an object obtained by matting a real object and used to be integrated into the virtual space.

[0033] Optionally, for the i-th frame image, the device further includes: an adjustment unit, configured to:

[0034] Determining a real viewing angle of the real camera according to the initial real focal length, the sensor size of the real camera, and the final imaging width of the real camera;

[0035] Obtaining a virtual viewing angle of the virtual camera;

[0036] If the difference between the virtual viewing wide angle and the real viewing wide angle is greater than a preset threshold, the virtual viewing wide angle is adjusted according to the real viewing wide angle.

[0037] On the other hand, the present application provides a computer device, the device comprising a processor and a memory:

[0038] The memory is used to store program code and transmit the program code to the processor;

[0039] The processor is used to execute the method described in the above aspects according to the instructions in the program code.

[0040] On the other hand, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store a computer program, and the computer program is used to execute the method described in the above aspects.

[0041] On the other hand, an embodiment of the present application provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method described in the above aspects.

[0042] Compared with the prior art, the advantages of the above technical solution of the present application are:

[0043] The initial real focal length of the real camera is obtained, and the initial real focal length can determine the virtual object distance between the virtual camera and the virtual object, so that the initial virtual position of the virtual camera is set in the virtual space according to the initial real focal length. For the i-th frame image in the content to be converted shot by the real camera, the i-th real position of the real camera in the i-th frame image and the i-1th real position of the real camera in the i-1th frame image are obtained, and then the difference between the two is determined. According to the difference and the initial virtual position, the i-th virtual position of the virtual camera in the i-th frame image is determined in the virtual space. As a result, there is no need for high-end spatial positioning equipment to transmit the absolute position of the real camera in real time. After the initial virtual position of the virtual camera is determined, the position of the virtual camera in the current frame image can be determined by continuously accumulating the difference between the positions of the real camera in adjacent frame images on the initial virtual position. The requirements for hardware equipment are reduced, which not only greatly reduces the hardware cost, but also reduces the shooting cost. Secondly, the absolute position is unbound, and the position of the virtual camera is determined by the relative position, so that more adjustment flexibility is added without completely matching the size of the real space and the virtual space, and the usage scenario is expanded. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0045] Figure 1 A flow chart of a spatial positioning method provided for this application;

[0046] Figure 2 A schematic diagram of a spatial positioning device provided in this application;

[0047] Figure 3 A structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0048] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0049] Integrate real people into virtual space. When the real camera moves and changes while shooting the real space, the virtual camera in the virtual space also needs to change accordingly. For example, when the real camera shoots around the real person, the virtual camera in the virtual scene should also shoot around the cutout person to achieve synchronization. In related technologies, the spatial positioning algorithm used to determine the precise position of the real person in the real space generally needs to rely on external hardware equipment to measure the distance to achieve spatial positioning, and it is necessary to build a real space with basically the same parameters as the virtual space, such as length, width, and height, and use high-end spatial positioning equipment in the real space to accurately determine the absolute position of the real camera and the real person, and correspond the positional relationship between the virtual camera and the cutout person one by one in the virtual space.

[0050] However, this method has the following problems: First, the spatial positioning equipment needs to have the functions of ranging and precise positioning, which has high requirements for hardware equipment, resulting in high purchase costs. Second, the real space needs to accommodate a large number of high-end spatial positioning equipment. For example, high-end spatial positioning equipment needs to be placed in the ceiling and other locations of the real space, which requires a very large venue, such as 300 meters * 400 meters * 3 meters, etc. This also has very high requirements for the venue, and is not conducive to small space positioning solutions, such as e-commerce live broadcasts, news live broadcasts, micro-drama shooting and other small space scenes. Third, the direction and position of the virtual camera and the real camera must also correspond, and the matte character must also stand in the same position as the matte character in the virtual space. This will be very inflexible when the external space is limited, and there are too many restrictions to be commercially available. This leads to high shooting costs and a small scope of application.

[0051] Based on this, the embodiment of the present application provides a spatial positioning method to obtain the initial real focal length of the real camera, which can determine the virtual object distance between the virtual camera and the virtual object, so as to set the initial virtual position of the virtual camera in the virtual space according to the initial real focal length. For the i-th frame image in the content to be converted shot by the real camera, the i-th real position of the real camera in the i-th frame image and the i-1th real position of the real camera in the i-1th frame image are obtained, and then the difference between the two is determined. According to the difference and the initial virtual position, the i-th virtual position of the virtual camera in the i-th frame image is determined in the virtual space. As a result, there is no need for high-end spatial positioning equipment to transmit the absolute position of the real camera in real time. After the initial virtual position of the virtual camera is determined, the position of the virtual camera in the current frame image can be determined by continuously accumulating the difference between the positions of the real cameras in adjacent frame images on the initial virtual position, which reduces the requirements for hardware equipment, not only greatly reduces the hardware cost, but also reduces the shooting cost. Secondly, there is no need for a large real space, which expands the usage scenarios. Finally, the absolute position is untied, and the position of the virtual camera is determined by the relative position, which makes it unnecessary to completely match the size of the real space and the virtual space, and increases the flexibility of adjustment. This reduces the shooting cost and expands the scope of application.

[0052] Combine the following Figure 1 , a spatial positioning method provided by an embodiment of the present application is introduced. Figure 1 , this figure is a flow chart of a spatial positioning method provided in an embodiment of the present application, and the method may include S101-S104.

[0053] S101: Obtaining an initial real focal length of a real camera.

[0054] In actual applications, a shooting environment is built in real space, and real objects such as real people and real objects that need to be cut out are placed at the target real position in the shooting environment. Then, a real camera is placed corresponding to the target real position so that the real camera can capture the real objects, thereby completing the construction of the shooting environment.

[0055] After the shooting environment is built, the real object distance between the real camera and the real object, the initial real focal length of the real camera and other parameters can be determined. The following is an example of the real object distance not changing. As a possible implementation method, in order to ensure that the real object distance does not change, the positions of the real camera and the real person can be ensured not to change.

[0056] It can be understood that the initial real focal length is the basis used when setting the positions of the real camera, real person, virtual camera, and cutout person before the real camera actually starts shooting. If the position of the real camera is not reset, the initial virtual position of the virtual camera will not change.

[0057] As a possible implementation method, through the optical imaging formula, 1 / f=1 / s+1 / u, where f is the focal length, s is the image distance, and u is the object distance. When the real camera does not adjust the lens length, that is, does not change the lens to imaging distance, then the image distance will not change, and the real object distance will not change, so the initial real focal length can be obtained.

[0058] S102: Setting an initial virtual position of the virtual camera in the virtual space according to the initial real focal length.

[0059] The virtual camera does not appear in the virtual space, but simulates the position of the real camera in the virtual space. The audience watches the screen that integrates the matte object into the virtual space according to the shooting angle of the virtual camera. The matte object is a real object placed in the virtual space by matte means.

[0060] In practical applications, before the matte object is merged into the virtual scene, that is, before rendering, the target virtual position of the matte object is determined first, and then the initial virtual position of the virtual camera is determined according to the initial real focal length and the target virtual position.

[0061] It should be noted that the initial virtual position is not a random position, but is set according to the initial real focal length of the real camera, which is explained in detail below.

[0062] In order to simulate the shooting of a real camera, the virtual camera will change with the changes of the real camera, that is, the picture taken by the virtual camera must maintain the same spatial relationship with the picture taken by the real camera. Among them, for a real camera, the real object distance between the real camera and the real object will not change. If you want to change the picture taken by the real camera, you need to change the focal length of the real camera. For a virtual camera, there is no focal length for the virtual camera. It changes the picture taken by the virtual camera by changing the virtual object distance between the virtual camera and the matte object. In other words, the virtual camera achieves that the picture taken by the virtual camera is consistent with the picture taken by the real camera by changing the virtual object distance.

[0063] Therefore, in the process of building a virtual space, the target virtual position of the matting object will be determined according to the design of the virtual space, the virtual object distance between the virtual camera and the matting object will be determined according to the initial real focal length, and the virtual initial position will be determined according to the virtual object distance and the target virtual position.

[0064] S103: For the i-th frame image in the to-be-converted content shot by the real camera, obtain the i-th real position of the real camera in the i-th frame image and the i-1-th real position of the real camera in the i-1-th frame image.

[0065] After the real space and the virtual space are built, the real camera can start shooting to obtain the content to be converted, so as to subsequently convert the content to be converted into content that integrates the real object and the virtual space. It can be understood that the content to be converted includes N consecutive frames of images, where N is a positive integer.

[0066] The following takes the i-th frame image and its previous frame image (i-1-th frame image) of the N frames of images included in the content to be converted as an example to illustrate how the image captured by the virtual camera maintains a consistent spatial relationship with the image captured by the real camera during the conversion process. Where i is an integer less than or equal to N.

[0067] In related technologies, the size of the virtual space must be proportional to the size of the real space, the target real position of the real object in the real space must be consistent with the target virtual position of the matte object in the virtual space, and the real position of the real camera in the i-th frame image must be transmitted to the virtual space in real time through high-end spatial positioning equipment, and then the position of the virtual camera in the i-th frame image is adjusted according to the proportional relationship and the target virtual position of the matte object, and the initial virtual position is updated to ensure that the virtual space and the real space continue to maintain a proportional relationship. However, this method has a high shooting cost and a small scope of application.

[0068] Based on this, the embodiment of the present application unbinds the absolute position in spatial positioning, that is, there is no need to bundle the virtual camera and the real camera positions, and ignore the proportional relationship between the two. After the initial virtual position of the virtual camera is determined, the difference between the position of the real camera in the current frame image (the i-th frame image) and the position of the real camera in the previous frame image (the i-1-th frame image) is continuously accumulated at the initial virtual position to determine the position of the virtual camera in the current frame image.

[0069] It should be noted that, through a computer and other equipment, the position difference of the real camera between adjacent frames in N consecutive image frames can be determined. This application does not make any specific limitation on this, and those skilled in the art can set it according to actual needs.

[0070] S104: Determine, in the virtual space, the i-th virtual position of the virtual camera in the i-th frame image according to the difference between the i-th real position and the i-1-th real position, and the initial virtual position.

[0071] According to the difference between the ith real position and the i-1th real position, the change of the real camera between adjacent frames is determined, and the change is accumulated to the initial virtual position of the virtual camera to obtain the ith virtual position of the virtual camera in the ith frame image. It can be expressed as formula (1)

[0072]

[0073] Among them, resultPos is the i-th virtual position of the virtual camera in the i-th frame image; initPos is the initial virtual position; nextPos is the i-th real position of the real camera in the i-th frame image; prevPos is the i-1th real position of the real camera in the i-1th frame image.

[0074] As a possible implementation, the real camera will not only change upward, downward, left or right, but also rotate. The same is true for rotation, except that the change between the current quaternion and the previous frame quaternion cannot be obtained through difference but through the inverse matrix. Specifically, when rotation occurs, it is also necessary to obtain the initial virtual rotation quaternion of the virtual camera in the virtual space, and obtain the orientation quaternion of the real camera in the i-th frame image. A possible implementation of S104 is as follows:

[0075] The i-th virtual position of the virtual camera in the i-th frame image is determined in the virtual space according to the initial virtual rotation quaternion, the orientation quaternion, the difference between the i-th real position and the i-1-th real position, and the initial virtual position.

[0076] The rotation change caused by the rotation can be expressed as formula (2).

[0077] resultRotation=initRotation×Invert(initRotation)×Rotation (2)

[0078] Among them, resultRotation is the rotation position of the virtual camera in the i-th frame image; initRotation is the initial virtual rotation quaternion. If the direction of the real camera is not reset, the initial virtual rotation quaternion will not change; Invert(initRotation) is the inverse matrix of the initial rotation matrix; Rotation is the orientation quaternion passed in by the real camera in the i-th frame image.

[0079] It should be noted that the orientation quaternion is the rotation information passed in by the real camera, and it is passed in real time. initRotation is the initial virtual rotation quaternion, which is the initial rotation value after the camera direction is reset. After calculating the inverse matrix (invert) of initRotation, it is the change rotation value after the spatial positioning is rotated. Therefore, the orientation quaternion Rotation passed in by the real camera in the i-th frame image is multiplied by the inverse matrix invert (initRotation) of the initial rotation matrix to calculate the rotation difference, which is equal to the rotation change value before the spatial positioning rotation is removed, and equal to the rotation change value of the initial position. Otherwise, the last rotation change value will be included in the Rotation, and it will not change if initRotation is not reset. However, after resetting, initRotation will become the current camera position. Therefore, the above inverse matrix needs to remove the change value of the last rotation to truly obtain the rotation value of the Rotation spatial positioning. Finally, multiply it by the initial rotation value to calculate the rotation position of the virtual camera in the i-th frame image. Among them, matrices and quaternions can basically be understood as vectors that express rotations, except that quaternions take up less space than matrices, reducing the amount of calculation and increasing the conversion speed.

[0080] Thus, the i-th virtual position of the virtual camera in the i-th frame is determined according to the results of formula (1) and formula (2).

[0081] It can be seen from the above technical solution that the initial real focal length of the real camera is obtained, and the initial real focal length can determine the virtual object distance between the virtual camera and the virtual object, so that the initial virtual position of the virtual camera is set in the virtual space according to the initial real focal length. For the i-th frame image in the content to be converted shot by the real camera, the i-th real position of the real camera in the i-th frame image and the i-1th real position of the real camera in the i-1th frame image are obtained, and then the difference between the two is determined. According to the difference and the initial virtual position, the i-th virtual position of the virtual camera in the i-th frame image is determined in the virtual space. Therefore, there is no need for high-end spatial positioning equipment to transmit the absolute position of the real camera in real time. After the initial virtual position of the virtual camera is determined, the position of the virtual camera in the current frame image can be determined by continuously accumulating the difference between the positions of the real camera in adjacent frame images on the initial virtual position, which reduces the requirements for hardware equipment, not only greatly reduces the hardware cost, but also reduces the shooting cost. Secondly, there is no need for a large real space, which expands the usage scenario. Finally, the absolute position is untied, and the position of the virtual camera is determined by the relative position, which makes it unnecessary to completely match the size of the real space and the virtual space, and increases the flexibility of adjustment. This reduces the shooting cost and expands the scope of application.

[0082] As a possible implementation method, if the focal length of the real camera changes, it may result in only part of the object being photographed. For example, the real camera moves forward and backward, causing the real object to exceed the wide angle of view of the real camera lens, and the excess part will be cut off, just like when a user stands too close to the lens when taking a photo, only part of the body can be photographed. Therefore, in order to ensure that the real camera does not partially cut off when photographing real people, the embodiment of the present application proposes to mount the matte object on the virtual camera so that the object distance between the matte object and the virtual camera remains unchanged.

[0083] Specifically, after the matte object (or matte patch) is attached to the virtual camera, no matter how the virtual camera rotates, the matte object always faces the virtual camera (observer), and the real object being photographed will move to the same object distance as the rotation, ensuring that the virtual object distance remains unchanged, and the virtual focal length corresponding to the virtual camera remains unchanged. It can be understood that when the real camera is rotated, the matte patch position moves and rotates around the circle centered on the virtual camera with the virtual camera as the center. In this way, it can be ensured that the real camera rotates left and right to the invisible area of ​​the real object, and the position in the virtual space can also be synchronized.

[0084] The embodiment of the present application does not specifically limit the mounting method, and the following is explained using BillBoard bulletin board technology as an example.

[0085] Among them, the matte object is rendered by using the traditional BillBoard technology, and it is always facing the camera, that is, the traditional BillBoard technology can ensure that the matte object is always facing the virtual camera. In order to achieve mounting, after using the traditional BillBoard technology, the orientation of the matte object is also changed.

[0086] If you want to change the orientation of the matte object, you need to modify not only the rotation but also the movement. That is to say, no matter where the matte object moves to or how the virtual camera rotates, the matte object always maintains a fixed distance from the virtual camera and always faces the virtual camera. Thus, we can get formula (3).

[0087] obj.position=camera.position+distance×camera.rotation×vector3.forward(3)

[0088] Among them, the matte object is represented by obj, the virtual camera is represented by camera, the quaternion of the matte object is represented by obj.rotation, the coordinate position of the matte object is represented by obj.position, the quaternion of the virtual camera is represented by camera.rotatio, the coordinate position of the virtual camera is represented by camera.position, vector3.forward represents the Z-axis direction, that is, Vector(0, 0, 1). camera.rotation×vector3.forward represents the vector of the unit direction of the virtual camera, distance×camera.rotation×vector3.forward represents the vector of the virtual camera direction with the length vector of distance, that is, the direction of the virtual camera. distance is the initial virtual object distance between the virtual camera and the matte object, which can be expressed as:

[0089] distance=Vector3.Distance(camera.position,obj.position),

[0090] Among them, Vector3.Distance is used to calculate the distance between the coordinate position of the virtual camera and the coordinate value of the matte object. The coordinate position of the virtual camera is represented as camera.position, and the coordinate position of the matte object is represented as obj.position.

[0091] Then the direction of the matting object is still the camera, from which we can get formula (4).

[0092] obj.rotation=quaternion.LookRotation(Normalize(camera.position-obj.position)) (4)

[0093] Among them, quaternion.LookRotation calculates the quaternion value according to the vector direction, that is, the rotation value. The coordinate position of the virtual camera is represented by camera.position, and the coordinate position of the matte object is represented by obj.position. Then camera.position-obj.position calculates the direction vector of the virtual camera toward the matte, and Normalize is the normalized vector.

[0094] As a possible implementation method, when moving and rotating the real camera for shooting, it will be found that the matte object in the virtual space will shake. After research, it was found that the key to the shaking problem is that the real focal length of the real camera is different from the virtual focal length of the virtual camera, resulting in different wide angles of view for the two. The different wide angles of view result in different ranges that the real camera and the virtual camera should see when shooting. The different ranges may not be a problem when the camera is not moving, but once it moves, the wide angle of view with a smaller range will obviously move faster than the wide angle of view with a larger range, resulting in different changes in the two cameras and shaking. Therefore, it can be ensured that the virtual focal length of the virtual camera is equal to the real focal length of the real camera.

[0095] However, the virtual camera has no focal length parameter, only the viewing angle parameter. Therefore, the real viewing angle of the real camera can be obtained to adjust the virtual viewing angle of the virtual camera. Specifically, the real viewing angle of the real camera is determined according to the initial real focal length, the sensor size of the real camera and the final imaging width of the real camera, as shown in formula (5); the virtual viewing angle of the virtual camera is obtained; if the difference between the virtual viewing angle and the real viewing angle is greater than a preset threshold, the virtual viewing angle is adjusted according to the real viewing angle. For example, the size of the virtual viewing angle is adjusted to be consistent with the size of the real viewing angle.

[0096] It should be noted that those skilled in the art can set the preset threshold value according to actual needs, such as 10%, etc., and this application does not make any specific limitation on this.

[0097] f=FocalLength / (sensorsize / Width) (5)

[0098] Among them, f represents the real wide angle of view of the real camera, FocalLength is the initial real focal length; sensorsize is the sensor size of the real camera; Width is the final imaging width of the real camera.

[0099] As a possible implementation method, the last drawback of this solution is that the real object cannot move forward and backward, because after the real object moves, the real camera will also crop the picture, and the focal length will change, which will cause problems with the cutout object in the virtual camera. However, to solve this problem, external sensing equipment must be used to sense the real object distance between the real object and the real camera, which will increase the cost. Based on this, the photographer can observe the object in the lens at any time, ensure that the focal length changes as little as possible, and try not to move the real object forward and backward in a large range.

[0100] In addition to the space positioning method provided in the embodiment of the present application, a space positioning device is also provided, such as Figure 2 As shown, it includes: an acquisition unit 201, a setting unit 202 and a determination unit 203;

[0101] The acquisition unit 201 is used to acquire the initial real focal length of the real camera;

[0102] The setting unit 202 is used to set the initial virtual position of the virtual camera in the virtual space according to the initial real focal length;

[0103] The acquisition unit 201 is further configured to acquire, for an i-th frame image in the content to be converted taken by the real camera, an i-th real position of the real camera in the i-th frame image, and an i-1-th real position of the real camera in the i-1-th frame image, wherein the i-th frame image is one of the continuous N frames of images included in the content to be converted, and the i-1-th frame image is a previous frame image of the i-th frame image in the continuous N frames of images;

[0104] The determining unit 203 is used to determine, in the virtual space, an i-th virtual position of the virtual camera in the i-th frame image according to a difference between the i-th real position and the (i-1)-th real position and the initial virtual position.

[0105] As a possible implementation manner, the obtaining unit 201 is further configured to:

[0106] Obtaining an initial virtual rotation quaternion of the virtual camera in the virtual space, and obtaining an orientation quaternion of the real camera in the i-th frame image;

[0107] The determining unit is used for:

[0108] The i-th virtual position of the virtual camera in the i-th frame image is determined in the virtual space according to the initial virtual rotation quaternion, the orientation quaternion, the difference between the i-th real position and the i-1-th real position, and the initial virtual position.

[0109] As a possible implementation, the device further includes: a mounting unit, configured to:

[0110] The matting object is mounted on the virtual camera so that the object distance between the matting object and the virtual camera remains unchanged. The matting object is an object obtained by matting a real object and used to be integrated into the virtual space.

[0111] As a possible implementation manner, for the i-th frame image, the device further includes: an adjustment unit, configured to:

[0112] Determining a real viewing angle of the real camera according to the initial real focal length, the sensor size of the real camera, and the final imaging width of the real camera;

[0113] Obtaining a virtual viewing angle of the virtual camera;

[0114] If the difference between the virtual viewing wide angle and the real viewing wide angle is greater than a preset threshold, the virtual viewing wide angle is adjusted according to the real viewing wide angle.

[0115] It can be seen from the above technical solution that the initial real focal length of the real camera is obtained, and the initial real focal length can determine the virtual object distance between the virtual camera and the virtual object, so that the initial virtual position of the virtual camera is set in the virtual space according to the initial real focal length. For the i-th frame image in the content to be converted shot by the real camera, the i-th real position of the real camera in the i-th frame image and the i-1th real position of the real camera in the i-1th frame image are obtained, and then the difference between the two is determined. According to the difference and the initial virtual position, the i-th virtual position of the virtual camera in the i-th frame image is determined in the virtual space. Therefore, there is no need for high-end spatial positioning equipment to transmit the absolute position of the real camera in real time. After the initial virtual position of the virtual camera is determined, the position of the virtual camera in the current frame image can be determined by continuously accumulating the difference between the positions of the real camera in adjacent frame images on the initial virtual position. The requirements for hardware equipment are reduced, which not only greatly reduces the hardware cost, but also reduces the shooting cost. Secondly, the absolute position is untied and the position of the virtual camera is determined by the relative position, which increases the flexibility of adjustment without having to completely match the size of the real space and the virtual space, expanding the usage scenarios.

[0116] The present application also provides a computer device, see Figure 3 , which shows a structural diagram of a computer device provided in an embodiment of the present application, such as Figure 3 As shown, the device includes a processor 310 and a memory 320:

[0117] The memory 310 is used to store program codes and transmit the program codes to the processor;

[0118] The processor 320 is used to execute any one of the spatial positioning methods provided in the above embodiments according to the instructions in the program code.

[0119] An embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store a computer program, and the computer program is used to execute any one of the spatial positioning methods provided in the above embodiments.

[0120] The embodiment of the present application also provides a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. The processor of the computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the spatial positioning method provided in various optional implementations of the above aspects.

[0121] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the various embodiments can be referred to each other. For the system or device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0122] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0123] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0124] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0125] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A spatial positioning method, characterized in that: The method comprises: Get the initial real focal length of the real camera; Determine the target virtual position of the matting object according to the design of the virtual space, determine the virtual object distance between the virtual camera and the matting object according to the initial real focal length, and determine the virtual initial position according to the virtual object distance and the target virtual position, wherein the matting object is a real object placed in the virtual space by matting means; For an i-th frame image in the content to be converted captured by the real camera, obtain an i-th real position of the real camera in the i-th frame image and an i-1-th real position of the real camera in the i-1-th frame image, wherein the i-th frame image is one of the N consecutive frames of images included in the content to be converted, and the i-1-th frame image is a previous frame image of the i-th frame image in the N consecutive frames of images; According to the difference between the i-th real position and the i-1-th real position, and the initial virtual position, the i-th virtual position of the virtual camera in the i-th frame image is determined in the virtual space, wherein the real object distance between the real camera and the real object does not change.

2. The method according to claim 1, characterized in that The method further comprises: Obtaining an initial virtual rotation quaternion of the virtual camera in the virtual space, and obtaining an orientation quaternion of the real camera in the i-th frame image; The determining, in the virtual space, the i-th virtual position of the virtual camera in the i-th frame image according to the difference between the i-th real position and the i-1-th real position and the initial virtual position comprises: The i-th virtual position of the virtual camera in the i-th frame image is determined in the virtual space according to the initial virtual rotation quaternion, the orientation quaternion, the difference between the i-th real position and the i-1-th real position, and the initial virtual position.

3. The method according to claim 2, characterized in that The method further comprises: The matte object is mounted on the virtual camera so that the object distance between the matte object and the virtual camera remains unchanged. The matte object is an object obtained by matting the real object and used to be integrated into the virtual space.

4. The method according to any one of claims 1 to 3, characterized in that: For the i-th frame image, the method further includes: Determining a real viewing angle of the real camera according to the initial real focal length, the sensor size of the real camera, and the final imaging width of the real camera; Obtaining a virtual viewing angle of the virtual camera; If the difference between the virtual viewing wide angle and the real viewing wide angle is greater than a preset threshold, the virtual viewing wide angle is adjusted according to the real viewing wide angle.

5. A spatial positioning device, characterized in that: The device comprises: an acquisition unit, a setting unit and a determination unit; The acquisition unit is used to acquire the initial real focal length of the real camera; The setting unit is used to determine the target virtual position of the matting object according to the design of the virtual space, determine the virtual object distance between the virtual camera and the matting object according to the initial real focal length, and determine the virtual initial position according to the virtual object distance and the target virtual position, wherein the matting object is a real object placed in the virtual space by matting means; The acquisition unit is further used to acquire, for an i-th frame image in the content to be converted taken by the real camera, an i-th real position of the real camera in the i-th frame image, and an i-1-th real position of the real camera in the i-1-th frame image, wherein the i-th frame image is one of the continuous N frames of images included in the content to be converted, and the i-1-th frame image is a previous frame image of the i-th frame image in the continuous N frames of images; The determination unit is used to determine, in the virtual space, an i-th virtual position of the virtual camera in the i-th frame image based on a difference between the i-th real position and the i-1-th real position, and the initial virtual position, wherein the real object distance between the real camera and the real object does not change.

6. The device according to claim 5, characterized in that The acquisition unit is further used for: Obtaining an initial virtual rotation quaternion of the virtual camera in the virtual space, and obtaining an orientation quaternion of the real camera in the i-th frame image; The determining unit is used for: The i-th virtual position of the virtual camera in the i-th frame image is determined in the virtual space according to the initial virtual rotation quaternion, the orientation quaternion, the difference between the i-th real position and the i-1-th real position, and the initial virtual position.

7. The device according to claim 5, characterized in that The device further comprises: a mounting unit, configured to: The matte object is mounted on the virtual camera so that the object distance between the matte object and the virtual camera remains unchanged. The matte object is an object obtained by matting the real object and used to be integrated into the virtual space.

8. The device according to any one of claims 5 to 7, characterized in that: For the i-th frame image, the device further includes: an adjustment unit, configured to: Determining a real viewing angle of the real camera according to the initial real focal length, the sensor size of the real camera, and the final imaging width of the real camera; Obtaining a virtual viewing angle of the virtual camera; If the difference between the virtual viewing wide angle and the real viewing wide angle is greater than a preset threshold, the virtual viewing wide angle is adjusted according to the real viewing wide angle.

9. A computer device, characterized in that: The device comprises a processor and a memory: The memory is used to store program code and transmit the program code to the processor; The processor is used to execute the method according to any one of claims 1 to 4 according to the instructions in the program code.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store a computer program, and the computer program is used to execute the method according to any one of claims 1 to 4.

Images