A Dynamic Camera Path Planning and Visual Standardization Method

Abstract

The present invention discloses a dynamic camera path planning and vision standardization method, which relates to the field of virtual production technology of Unreal Engine and includes the following steps: S110, adding known target objects to the initial positions; S120, detecting the features and boundaries of the target objects; S130, matching the detected feature points with the known set feature points to analyze whether the attributes of the target objects are defined as organisms; S140, if the attributes of the target objects in S130 are negatively determined as organisms, making the camera move around the target objects along the established trajectory; S150, if the attributes of the target objects in S130 are determined as organisms, judging whether they are dynamic. The automatic positioning camera of the present invention can quickly set and adjust the position and angle of the camera, reducing the workload of manual adjustment and manual setting of keyframe animations. It can quickly generate smooth camera movements and improve development efficiency. The camera in the automated vision development tool makes the use of the entire process more efficient, more intelligent, more standardized and more intuitive.

Description

Technical Field

[0001] The present invention relates to the technical field of virtual production technology of Unreal Engine, and specifically relates to a dynamic camera path planning and visual standardization method based on target tracking for Unreal Engine LookDev. Background Art

[0002] Existing solution: LookDevelopment (abbreviated as LookDev), a term related to computer graphics, sets a unified lighting environment and review standards for digital assets, greatly optimizing the art production process. In the art production of Unreal Engine virtual production, after introducing the LookDev process, the performance of art resources under different lighting conditions is consistent, reducing the modification workload during later integration and improving production efficiency. However, it still has an obvious non-standard factor that accompanies the entire process - the setting and application of the camera. Existing LookDev does not include established cameras and related attributes, nor can it be automatically created. Instead, users need to manually create cameras, which violates the original intention of the unified standard environment of LookDev.

[0003] The main defect lies in that: whether in the real world or in the application field of Unreal Engine virtual production, the use of cameras is also very different. When different external parameter information changes, it will affect the position of the target in the frame, the lighting effect, and the contrast between light and dark. And different internal parameter information will affect the sensitivity, focus, noise, and color balance of the target, and even motion blur. All of the above factors directly change the visual effect in the lens, damaging the unity of the art and review standards. For this reason, we propose a dynamic camera path planning and visual standardization method. Summary of the Invention

[0004] The purpose of the present invention is to solve the problems in the background art. To solve the above problems, the present invention discloses a dynamic camera path planning and visual standardization method, including the following steps:

[0005] S110. Add a known target to the initial position;

[0006] S120. Detect the features and boundaries of the target;

[0007] S130. Match the detected feature points with the known set feature points, and analyze whether the attribute of the target is defined as a creature;

[0008] S140. If the attribute of the target in S130 is negatively determined as a creature, let the camera move around the target along a predetermined trajectory;

[0009] S150. If the attribute of the target object in S130 is determined to be biological, determine whether it is dynamic;

[0010] S160. If it is determined to be dynamic in S150, the camera, as a subclass of the target object, always follows the target object;

[0011] S170. If it is determined to be not dynamic in S150, continue with the operation in S140.

[0012] Preferably, the known 3D model in S110 is assigned to the target object in the template level of the automated vision development tool, and the position and rotation are default.

[0013] Preferably, in S120, the bounding box boundary scale of the 3D model of the target object is obtained, and the numerical values of X, Y, and Z are respectively stored as the cube range X, cube range Y, and cube range Z.

[0014] Preferably, the camera in S140 encapsulates a camera component, and the conventional internal information parameters and external information parameters are set therein, including setting the position and pose of the camera in the world space.

[0015] Preferably, the camera component includes a frustum culling component and a model component. The model component is used to render the target mesh component and its subclasses. The frustum culling component is used to draw the frustum of the camera through the field of view angle, aspect ratio, and the distances from the near plane and the far plane to the camera, and the visible area of the camera can be visually seen in the editor.

[0016] Preferably, the center line of the frustum is the viewing direction of the camera, and the observation target is the target object that the camera is always aligned with. The position of the camera is calculated by observing the position of the objects included in the observation target, and the calculated result is stored back into the observation target.

[0017] Preferably, in S150, in order to keep the position of the observation target relatively stable in the view center, the camera always aligns with the observation target, and the position and pose can be dynamically tracked and adjusted in real time. The camera is bound to the observation target to move and make a tracking motion following the observation target. The observation target, spring arm, camera, and color card are encapsulated together in a skeletal mesh object.

[0018] Preferably, the positions of the spring arm, camera, and color card in the skeletal mesh object are relative coordinates, that is, the relative positions of the child objects with respect to their parent object, and are nested in multiple layers.

[0019] Preferably, in step S140, the camera moves around the target along a predetermined trajectory, which is to move around the Z-axis of the observation target. The value of the Z-axis will cause the camera to continuously rotate over time, and the real-time position of the camera is changed according to the rotation angle. The implemented logic uses the functions of the Unreal Engine.

[0020] Preferably, the real-time position of the color card is set to a fixed position relative to the camera. When the camera moves and the real-time position of the color card is not updated, the position of the color card in the camera will remain unchanged. Then, by converting the frustum into a right triangle and calculating with the cotangent function, the color card can be placed at a suitable position in the image.

[0021] Beneficial effects: This method not only helps artists and developers quickly iterate and optimize the materials, textures, and lighting effects of assets under unified lighting and rendering conditions, but also ensures the consistency and authenticity of the final effect. The automatically positioned camera can quickly set and adjust the position and angle of the camera, reducing the workload of manual adjustment and manual keyframe animation setting. It can quickly generate smooth camera movements, improving development efficiency. The camera in the automated LookDev tool makes the entire process more efficient, intelligent, standardized, and intuitive. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention, and those of ordinary skill in the art can obtain other drawings based on these drawings without creative efforts.

[0023] Figure 1 Shows the flowchart of automatically creating and matching the camera for the target.

[0024] Figure 2 Shows the schematic diagram of the circular motion in S140.

[0025] Figure 3 Shows the relationship diagram between the camera and the lens image.

[0026] Figure 4 Shows the hierarchical diagram encapsulated in the skeletal mesh object class when the camera makes a tracking motion.

[0027] Figure 5 Shows the flowchart of the camera manager updating the viewpoint. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0028] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0029] The present invention provides a technical solution: a dynamic camera path planning and vision standardization method, as Figure 1 , and the basic process is as follows:

[0030] Step S110, add known objects to the initial positions.

[0031] Step S120, detect the features and boundaries of the objects.

[0032] Step S130, match the detected feature points with the pre-set feature points, and analyze whether the attributes of the objects are defined as organisms.

[0033] If not, then in step S140, let the camera move around the object along a pre-determined trajectory, similar to the behavior of the Earth's revolution.

[0034] If so, then in step S150, determine whether it is dynamic.

[0035] If it is dynamic, then in step S160, the camera, as a subclass of the object, always follows the object.

[0036] If it is not dynamic, then continue with step S140.

[0037] In the Unreal Engine, assign the known 3D model in step S110 to the object in the template level of the automated vision development tool, and the default position is (X-axis = 0.000000, Y-axis = 0.000000, Z-axis = 0.000000), and the rotation is (pitch angle = 0.000000, yaw angle = 0.000000, roll angle = 0.000000).

[0038] In step S120, obtain the bounding box (BoundScale) of the 3D model of the object and store the XYZ values as the cube range X, cube range Y, and cube range Z respectively. It is the bounding box of the object rather than the scale of the 3D model itself. This is because the Unreal Engine provides a unified benchmark to simplify calculations, ensure compatibility and consistency, and the default value of the scale is 1. In step S130, analyze whether the attributes of the object are defined as organisms.

[0039] If the target object is not a creature but a static mesh object, then in step S140, the camera moves around the target object along a predefined trajectory, similar to the behavior of the Earth's revolution, such as Figure 2 . The camera in the Unreal Engine encapsulates a camera component. Conventional internal and external information parameters can be set in it, including the position, orientation, and field of view of the camera in world space. The field of view refers to the angular value between the top and bottom boundaries within the frustum, that is, the range of the scene that can be seen. The camera component also includes two other components, one is the draw frustum component, and the other is the model component.

[0040] The model component is used to render the target mesh component and its subclasses, while the draw frustum component draws the camera's frustum based on the field of view angle, aspect ratio, and the distances from the near plane and the far plane to the camera. You can visually see the visible area of the camera in the frustum editor. ( Figure 3 The magenta frame, which only appears in the editor). In addition, the center line of the frustum represents the viewing direction of the camera. The ViewTarget, that is, the object that the camera is always aimed at. Generally, the position of the ViewTarget is used to calculate the position of the camera's viewpoint, and the calculated result is then stored in the viewpoint in the ViewTarget.

[0041] Reference Figure 5 , when the camera manager (PlayerCameraManger) in the Unreal Engine updates the viewpoint, it will call a function (CalcCamera()) of the object for calculating the camera's viewpoint, which is usually used to customize the calculation logic of the camera. The Unreal Engine provides strong support for it in blueprints and C++. It will first check if there is a camera component and then check if another boolean property (bFindCameraComponentWhenViewTarget) is true. If so, it will obtain the position and orientation of the camera component; otherwise, it will obtain the coordinates and orientation of the ViewTarget object to update the viewpoint information. Other properties of the camera, such as the aspect ratio, which is defined as the ratio of the horizontal viewport size to the vertical viewport size. Set it to the common 16:9 and do not change it in real time, which will not affect the actual field of view. Set the default value of the current focal length to 50, the default value of the current aperture to 8, and disable the focus method and post-processing.

[0042] Set the initial position of the camera (X-axis = X1, Y-axis = Y1, Z-axis = Z1). Usually, in the Unreal Engine, the Field of View (FOV) refers to the angular value between the top and bottom boundaries within the frustum, which defines the Vertical Field of View (VFOV) and the Horizontal Field of View (HFOV) of the projection within the camera lens. And now the values of HFOV and VFOV are known. So, Y1 and Z1 can be calculated through Y1 = Cube range Z ÷ (HFOV ÷ 200) and Z1 = Cube range Z ÷ (VFOV ÷ 10), and X1 = 0.

[0043] Set the initial position of the scale: X-axis = X2, Y-axis = 0.000000, Z-axis = 0.000000. The value of the X-axis of the scale can be calculated using the Cube range X, Cube range Y, and the Pythagorean theorem. The rule for automatically generating the height scale of the target object is as follows: When the Cube range Z is less than 100 cm, only the height of the target object is generated. When the Cube range Z is between 100 cm and 1000 cm, the height of the target object and the three-digit integer of the middle value are generated. When the Cube range Z is greater than 1000 cm, the unit cm is converted to m, and the height of the target object and the integer of the middle value are generated.

[0044] Add an empty object subclass to the camera. The initial position of the color card is also the position of the object subclass. Set its initial position as (X-axis = X3, Y-axis = Y3, Z-axis = Z3). Calculate the position of the object subclass using the cotangent in trigonometric functions and pass it to the color card.

[0045] If it is a creature, then in step S150, determine whether it is dynamic. If it is dynamic, the type of the target object is a skeletal mesh object. If animation data needs to be assigned, then in step S160, to keep the position of the observed target relatively stable at the center of the view. The camera always aims at it and can dynamically track and adjust the position and pose in real time. Bind the camera to the target object (that is, as a subclass of the target object) to follow the target object's movement for tracking. Package the target object, spring arm, camera, and color card together in the skeletal mesh object class. Their hierarchical relationship refers to Figure 4 , in Figure 4The position of the camera in it is a relative coordinate, that is, the relative position of the child object relative to its parent object, and there are multiple levels of nesting, and the same is true for the spring arm and the color card. The spring arm between the target object and the camera is mainly used to constrain the position and behavior of the camera. When the camera moves or rotates to the target position, sometimes a certain lag needs to be made to simulate some effects in reality, such as the buffering effect when moving. In addition, sometimes it is also necessary to handle the situation where the camera position and the target are quickly switched (which will work together with interpolation). The main settings for the spring arm are: set the distance between the spring arm and the camera to 0, and then use the class (Pawn class) that controls the camera to control the rotation (because the spring arm itself does not rotate, only resets and rotates its children according to the inherited rotation settings, rather than maintaining its relative rotation according to its parent). If you want to rotate the target based on all settings, you can use the function to obtain the target rotation (GetTargetRotation()) in the Unreal Engine and another very important function to obtain the current view rotation (APawn::GetViewRotation()). This rotation usually represents the rotation direction of the camera, and this direction is usually the rotation direction of the camera, rather than the rotation direction of the class that controls the camera itself. Finally, it is also necessary to enable the camera rotation delay and control the delay speed, which can achieve the smooth movement and following effect of the camera, making the movement of the camera more natural and smooth.

[0046] If it is not dynamic and the type of the target object is still a static mesh object, then continue to step S140.

[0047] In step S140, the camera makes a circular motion around the target object along a predetermined trajectory, which is a motion around the Z-axis of the target object. This value of the Z-axis will achieve the continuous rotation of the camera over time, and change the real-time position of the camera according to the rotation angle. The implemented logic can use the function (RotateVector) in the Unreal Engine to adjust the direction of the vector with a specified rotation. The initial position of the camera - the initial position of the target object + the position of the target object = the real-time position of the camera.

[0048] The real-time position of the color card is the real-time position of the child empty object, and the child empty object is set to a fixed position relative to the camera. When the camera moves and the relative position of the child object is not updated, then its position in the camera will remain unchanged. Then convert the frustum into a right triangle and calculate with the cotangent function, and the color card can be placed at the appropriate position in the picture, and it does not change in the lens in a similar way.

[0049] In the description of this specification, the descriptions referring to the terms "one embodiment", "example", "specific example", etc. mean that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0050] The preferred embodiments of the present invention disclosed above are only used to help illustrate the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to the specific embodiments described. Obviously, many modifications and variations can be made according to the content of this specification. These embodiments are selected and specifically described in this specification in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can well understand and utilize the present invention. The present invention is only limited by the claims and their full scope and equivalents.

Claims

1. A dynamic camera path planning and vision standardization method, characterized in that, It includes the following steps: S110. Add a known target to the initial position; S120. Detect the features and boundaries of the target; S130. Match the detected feature points with the pre - set known feature points and analyze whether the attribute of the target is defined as biological; S140. If the attribute of the target in S130 is negatively determined as biological, let the camera move around the target along a pre - determined trajectory; S150. If the attribute of the target in S130 is determined as biological, judge whether it is dynamic; S160. If it is determined as dynamic in S150, the camera, as a subclass of the target, always follows the target; S170. If it is negatively determined as dynamic in S150, continue with the operation in S140; Designate the known 3D model in S110 to the target in the template level of the automated vision development tool, and rotate it at the default position; Obtain the bounding box boundary scale of the 3D model of the target in S120, and store the X, Y, and Z values as the cube range X, cube range Y, and cube range Z respectively; In S150, in order to keep the position of the observed target relatively stable in the center of the view, the camera always aims at the observed target, and can dynamically track and adjust the position and pose in real - time. Bind the camera to the observed target to move with the observed target for tracking motion, and encapsulate the observed target, spring arm, camera, and color card in a skeletal mesh object.

2. The dynamic camera path planning and vision standardization method according to claim 1, characterized in that, The camera in S140 encapsulates a camera component, and conventional internal information parameters and external information parameters are set therein, including setting the position and pose of the camera in the world space.

3. A dynamic camera path planning and vision standardization method according to claim 2, characterized in that, The camera component includes a frustum culling component and a model component. The model component is used to render the target mesh component and its subclasses. The frustum culling component is used to draw the camera's frustum through the field of view angle, aspect ratio, and the distances from the near plane and far plane to the camera, and visually see the visible area of the camera in the editor.

4. A dynamic camera path planning and vision standardization method according to claim 3, characterized in that, The center line of the frustum is the viewing direction of the camera. The observed target is the target that the camera always aims at. Calculate the position of the camera based on the positions of the objects included in the observed target, and store the calculated result back into the observed target.

5. A dynamic camera path planning and vision standardization method according to claim 1, characterized in that, The positions of the spring arm, camera, and color card in the skeletal mesh object are relative coordinates, that is, the relative positions of the sub - objects relative to their parent objects, and are nested in multiple layers.

6. A method for dynamic camera path planning and vision standardization according to claim 5, characterized in that In step S140, when the camera moves around the target along a pre - determined trajectory, it moves around the Z - axis of the observed target. The value of the Z - axis will achieve continuous rotation of the camera over time, and change the real - time position of the camera according to the rotation angle. The implemented logic applies the functions of the Unreal Engine.

7. A dynamic camera path planning and vision standardization method according to claim 6, characterized in that Set the real - time position of the color card as a fixed position relative to the camera. If the camera moves while the real - time position of the color card is not updated, then the position of the color card in the camera will remain unchanged. Then convert the frustum into a right - angled triangle and use the cotangent function to calculate, and the color card can be placed at the appropriate position in the picture.

Images