Dynamic camera path planning and visual standardization method

Abstract

The invention discloses a dynamic camera path planning and visual standardization method, and relates to the technical field of virtual film production of an unreal engine, and the method comprises the following steps: S110, adding a known target object to an initial position; s120, detecting features and boundaries of the target object; s130, matching the detected feature points with known set feature points, and analyzing whether the attribute of the target object is defined as an organism or not; s140, if the attribute of the target object in the S130 is negative to be a biological object, enabling the camera to perform surrounding motion around the target object according to a set track; and S150, if the attribute of the target object in the S130 is determined to be biological, judging whether the target object is dynamic or not. According to the automatic positioning camera, the position and the angle of the camera can be rapidly set and adjusted, and the workload of manual adjustment and manual setting of key frame animations is reduced. Smooth camera motion can be quickly generated, and the development efficiency is improved. A camera in the automatic vision development tool enables the whole process to be more efficient, more intelligent, more standardized and more visual.

Description

Technical Field

[0001] The present invention relates to the technical field of virtual production technology of Unreal Engine, and particularly relates to a dynamic camera path planning and visual standardization method based on target tracking for Unreal Engine LookDev. Background Art

[0002] Existing solutions: LookDevelopment (abbreviated as LookDev), a term related to computer graphics, sets a unified lighting environment and review standards for digital assets, greatly optimizing the art production process. In the art production of Unreal Engine virtual production, after introducing the LookDev process, the performance of art resources under different lighting conditions is consistent, reducing the modification workload during later integration and improving production efficiency. However, it still has an obvious non-standard factor that accompanies the entire process - the setting and application of the camera. Existing LookDev does not include established cameras and related attributes, nor can it be automatically created, but requires users to manually create cameras, which violates the original intention of the unified standard environment of LookDev.

[0003] The main defect lies in that: whether in the real world or in the application field of Unreal Engine virtual production, the use of the camera is also very different. When different external parameter information changes, it will affect the position of the target in the picture, the lighting effect, and the light and dark contrast. And different internal parameter information will affect the sensitivity, sharpness, noise, and color balance of the target, and even motion blur. All of the above factors directly change the visual effect in the lens, damaging the unity of the art and review standards. For this reason, we propose a dynamic camera path planning and visual standardization method. Summary of the Invention

[0004] The purpose of the present invention is to solve the problems in the background art. To solve the above problems, the present invention discloses a dynamic camera path planning and visual standardization method, including the following steps: S110. Add a known target to the initial position; S120. Detect the features and boundaries of the target; S130. Match the detected feature points with the known set feature points, and analyze whether the attribute of the target is defined as a creature; S140. If the attribute of the target in S130 is negatively determined as a creature, let the camera move around the target along a predetermined trajectory; S150. If the attribute of the target in S130 is determined as a creature, judge whether it is dynamic; S160. If it is determined in S150 that it is dynamic, the camera, as a subclass of the target object, always follows the target object. S170. If it is determined in S150 that it is not dynamic, continue with the operation in S140.

[0005] Preferably, the known 3D model in S110 is assigned to the target object in the automated vision development tool template level, and the position and rotation are default.

[0006] Preferably, in S120, the bounding box boundary scale of the target object's 3D model is obtained, and the XYZ values are respectively stored as the cube range X, cube range Y, and cube range Z.

[0007] Preferably, the camera in S140 encapsulates a camera component, and the conventional internal information parameters and external information parameters are set therein, including setting the position and pose of the camera in the world space.

[0008] Preferably, the camera component includes a frustum culling component and a model component. The model component is used to render the target mesh component and its subclasses. The frustum culling component is used to draw the camera's frustum through the field of view angle, aspect ratio, and the distances from the near plane and far plane to the camera, and the visible area of the camera can be visually seen in the editor.

[0009] Preferably, the center line of the frustum is the viewing direction of the camera. The observation target is the target object that the camera is always aligned with. The position of the camera is calculated by observing the position of the objects included in the observation target, and the calculated result is stored back into the observation target.

[0010] Preferably, in S150, in order to keep the position of the observation target relatively stable in the view center, the camera always aligns with the observation target and can dynamically track and adjust the position and pose in real time. The camera is bound to the observation target to move and do tracking movement following the observation target, and the observation target, spring arm, camera, and color card are encapsulated together in a skeletal mesh object.

[0011] Preferably, the positions of the spring arm, camera, and color card in the skeletal mesh object are relative coordinates, that is, the relative positions of the sub-objects relative to their parent object, and there are multiple levels of nesting.

[0012] Preferably, in step S140, the camera makes a circular motion around the target object along a predetermined trajectory, which is a motion around the Z-axis of the observation target. The value of the Z-axis will achieve continuous rotation of the camera over time, and the real-time position of the camera is changed according to the rotation angle, and the implemented logic applies the functions of the Unreal Engine.

[0013] Preferably, the real-time position of the color card is set to a fixed position relative to the camera. If the camera moves and the real-time position of the color card is not updated, then the position of the color card in the camera will remain unchanged. By converting the frustum into a right triangle and calculating with the cotangent function, the color card can be placed at an appropriate position in the picture.

[0014] Beneficial effects: This method not only helps artists and developers quickly iterate and optimize the materials, textures, and lighting effects of assets under unified lighting and rendering conditions, but also ensures the consistency and authenticity of the final effect. The automatic positioning camera can quickly set and adjust the position and angle of the camera, reducing the workload of manual adjustment and manual keyframe animation setting. It can quickly generate smooth camera movements, improving development efficiency. The camera in the automated LookDev tool makes the entire process more efficient, intelligent, standardized, and intuitive. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0016] Figure 1 Shows the flowchart of automatically creating and matching a camera for a target object; Figure 2 Shows the schematic diagram of the surrounding movement in S140; Figure 3 Shows the relationship diagram between the camera and the lens view; Figure 4 Shows the hierarchical diagram encapsulated in the skeletal mesh object class when the camera makes a tracking movement; Figure 5 Shows the flowchart of the camera manager updating the viewpoint. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0017] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0018] The present invention provides a technical solution: a dynamic camera path planning and vision standardization method, as Figure 1 , the basic process is: Step S110, Add a known target object to the initial position; Step S120, Detect the features and boundaries of the target object; Step S130, Match the detected feature points with the pre - set known feature points and analyze whether the attribute of the target object is defined as biological; If not, then in Step S140, let the camera move around the target object along a pre - determined trajectory, similar to the behavior of the Earth's revolution; If so, then in Step S150, determine whether it is dynamic; If it is dynamic, then in Step S160, the camera, as a subclass of the target object, always follows the target object; If it is not dynamic, then continue with Step S140.

[0019] In the Unreal Engine, assign the known 3D model in Step S110 to the target object in the template level of the automated vision development tool, and by default, the position (X - axis = 0.000000, Y - axis = 0.000000, Z - axis = 0.000000), rotation (pitch angle = 0.000000, yaw angle = 0.000000, roll angle = 0.000000).

[0020] Step S120, Obtain the bounding box (BoundScale) of the target object's 3D model and store the X, Y, and Z values as the cube range X, cube range Y, and cube range Z respectively. It is the bounding box of the target object rather than the scale of the 3D model itself. This is because the Unreal Engine provides a unified benchmark to simplify calculations and ensure compatibility and consistency, and the default value of the scale is 1. Step S130, Analyze whether the attribute of the target object is defined as biological.

[0021] If it is not biological and the type of the target object is a static mesh object, then in Step S140, let the camera move around the target object along a pre - determined trajectory, similar to the behavior of the Earth's revolution as Figure 2 In the Unreal Engine, the camera encapsulates a camera component. Conventional internal information parameters and external information parameters can be set therein, including the position, pose, and field - of - view range of the camera in the world space. The field - of - view range refers to the angular value between the top and bottom boundaries within the frustum, that is, the range of the scene that can be seen. The camera component also includes two other components, one is the frustum culling component, and the other is the model component.

[0022] The model component is used to render the target object's mesh components and their sub - classes, while the frustum culling component draws the camera's frustum based on the field - of - view angle, aspect ratio, and the distances from the near plane and far plane to the camera. One can visually see the visible area of the camera in the camera frustum editor. ( Figure 3The bright pink box (only available in the editor). In addition, the center line of the frustum represents the viewing direction of the camera. The ViewTarget, which is the object that the camera is always aimed at, is generally used to calculate the position of the camera's viewpoint based on the position of the ViewTarget, and then store the calculated result in the viewpoint of the ViewTarget.

[0023] Reference Figure 5 , when the camera manager (PlayerCameraManger) in the Unreal Engine updates the viewpoint, it will call a function (CalcCamera()) of the object used to calculate the camera's viewpoint, which is usually used to customize the calculation logic of the camera. The Unreal Engine provides strong support for it in blueprints and C++. It will first determine whether there is a camera component and then determine whether another boolean property (bFindCameraComponentWhenViewTarget) is true. If so, it will obtain the position and orientation of the camera component; otherwise, it will obtain the coordinates and orientation of the ViewTarget object to update the viewpoint information. Other properties of the camera, such as the aspect ratio, which is defined as the ratio between the horizontal viewport size and the vertical viewport size. Setting it to the relatively common 16:9 and not changing it in real time will not affect the actual field of view. Set the current focal length default value to 50, the current aperture default value to 8, and disable the focusing method and post-processing.

[0024] Set the initial position of the camera (X-axis = X1, Y-axis = Y1, Z-axis = Z1). Usually, in the Unreal Engine, the Field of View (FOV) refers to the angular value between the top and bottom boundaries within the frustum, which defines the vertical projection visible range (Vertical Field of View, VFOV) and the horizontal projection visible range (Horizontal Field of View, HFOV) within the camera lens. And now the values of HFOV and VFOV are known, so Y1 and Z1 can be obtained through the calculations of Y1 = cube range Z ÷ (HFOV ÷ 200) and Z1 = cube range Z ÷ (VFOV ÷ 10), and X1 = 0.

[0025] Set the initial position of the scale: X-axis = X2, Y-axis = 0.000000, Z-axis = 0.000000. The value of the X-axis of the scale can be calculated using the cube range X, the cube range Y, and the Pythagorean theorem. Automatically generate the rules for the height scale of the target object: If the cube range Z is less than 100 cm, only generate the height of the target object. If the cube range Z is between 100 cm and 1000 cm, generate the height of the target object and the three-digit integer of the middle value. If the cube range Z is greater than 1000 cm, convert the unit from centimeters to meters and generate the height of the target object and the integer of the middle value.

[0026] Add an empty object subclass to the camera. The initial position of the color card is also the position of the object subclass. Set its initial position to (X-axis = X3, Y-axis = Y3, Z-axis = Z3). Calculate the position of the object subclass using the cotangent in trigonometric functions and pass it to the color card.

[0027] If it is a creature, then in step S150, determine whether it is dynamic. If it is dynamic, the type of the target object is a skeletal mesh object. If animation data needs to be assigned, then in step S160, to keep the position of the observation target relatively stable in the center of the view. The camera always aims at it and can dynamically track and adjust the position and posture in real time. Bind the camera to the target object (that is, as a subclass of the target object) to follow the target object's movement for tracking. Enclose the target object, the spring arm, the camera, and the color card together in the skeletal mesh object class. Their hierarchical relationship refers to Figure 4 , in Figure 4The position of the camera in it is a relative coordinate, that is, the relative position of the child object relative to its parent object, and there are multiple levels of nesting, and the same is true for the spring arm and the color card. The spring arm between the target object and the camera is mainly used to constrain the position and behavior of the camera. When the camera moves or rotates to the target position, sometimes a certain lag needs to be made to simulate certain effects in reality, such as the buffering effect when moving. In addition, sometimes it is also necessary to handle the situation where the camera position and the target are quickly switched (which will work together with interpolation). The main settings for the spring arm are: set the distance between the spring arm and the camera to 0, and then use the class that controls the camera (Pawn class) to control the rotation (because the spring arm itself does not rotate, it only resets and rotates its children according to the inherited rotation settings, rather than maintaining its relative rotation according to its parent). If you want to rotate the target based on all settings, you can use the function to obtain the target rotation (GetTargetRotation()) in the Unreal Engine and another very important function to obtain the current view rotation (APawn::GetViewRotation()). This rotation usually represents the rotation direction of the camera, and this direction is usually the rotation direction of the camera, rather than the rotation direction of the class that controls the camera itself. Finally, it is also necessary to enable the camera rotation delay and control the delay speed, which can achieve the smooth movement and following effect of the camera, making the movement of the camera more natural and smooth.

[0028] If it is not dynamic and the type of the target object is still a static mesh object, then continue to step S140.

[0029] In step S140, the camera makes a circular motion around the target object along a predetermined trajectory, which is a motion around the Z-axis of the target object. This value of the Z-axis will achieve the continuous rotation of the camera over time, and change the real-time position of the camera according to the rotation angle. The implemented logic can use the function (RotateVector) in the Unreal Engine to adjust the direction of the vector with a specified rotation. The initial position of the camera - the initial position of the target object + the position of the target object = the real-time position of the camera.

[0030] The real-time position of the color card is the real-time position of the subclass empty object, and the subclass empty object is set to a fixed position relative to the camera. If the camera moves and the relative position of the subclass object is not updated, then its position in the camera will remain unchanged. Then, by converting the frustum into a right triangle and calculating with the cotangent function, the color card can be placed at a suitable position in the picture, and a similar method is used in the lens.

[0031] In the description of this specification, the descriptions referring to terms such as "one embodiment", "example", "specific example", etc. mean that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in a suitable manner in any one or more embodiments or examples.

[0032] The preferred embodiments of the present invention disclosed above are only used to help explain the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to the specific embodiments described. Obviously, many modifications and variations can be made according to the content of this specification. These embodiments are selected and specifically described in this specification in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can well understand and utilize the present invention. The present invention is only limited by the claims and their full scope and equivalents.

Claims

1. A dynamic camera path planning and visual standardization method, characterized in that: The following steps are involved: S110, adding a known target object to the initial position; S120, detecting features and boundaries of the target object; S130, matching the detected feature points with known set feature points, and analyzing whether the attributes of the target object are defined as a living thing; S140, if the attribute of the target object in S130 is denied to be a living thing, making the camera orbit around the target object according to a predetermined trajectory; S150, if the attribute of the target object is determined to be a living thing in S130, determine whether it is dynamic; S160, if it is determined to be dynamic in S150, the camera, as a subclass of the target object, always follows the target object; S170: If the determination in S150 is negative that the operation is dynamic, then continue the operation in S140.

2. A dynamic camera path planning and visual standardization method according to claim 1, characterized in that: The three-dimensional model known in S110 is assigned to the target object in the template level of the automated visual development tool, and the position is rotated according to the default setting.

3. A dynamic camera path planning and visual standardization method according to claim 2, characterized in that: In the S120 , the bounding box boundary scale of the target object three-dimensional model is obtained, and the values ​​of XYZ are stored as cube range X, cube range Y and cube range Z respectively.

4. A dynamic camera path planning and visual standardization method according to claim 3, characterized in that: The camera in S140 encapsulates a camera component, in which conventional internal information parameters and external information parameters are set, including setting the position and posture of the camera in the world space.

5. A method for dynamic camera path planning and visual standardization according to claim 4, characterized in that: The camera component includes a drawing frustum component and a model component. The model component is used to render the target object mesh component and its subclasses. The drawing frustum component is used to draw the camera's frustum through the field of view angle, aspect ratio, and the distance from the near plane and far plane to the camera. The camera's visible area can be intuitively seen in the editor.

6. A method for dynamic camera path planning and visual standardization according to claim 5, characterized in that: The center line of the viewing cone is the viewing direction of the camera. The observation target is the target object that the camera is always aimed at. The position of the camera is calculated by observing the position of the objects contained in the target, and the calculated result is stored in the observation target.

7. A method for dynamic camera path planning and visual standardization according to claim 2, characterized in that: In S150, in order to make the position of the observation target relatively stable in the center of the view, the camera is always aimed at the observation target, and can dynamically track and adjust the position and posture in real time. The camera is bound to the observation target to follow the movement of the observation target for tracking movement, and the observation target, spring arm, camera, and color card are encapsulated together in the skeletal mesh object.

8. A method for dynamic camera path planning and visual standardization according to claim 7, characterized in that: The positions of the spring arms, cameras, and color chips within the Skeletal Mesh object are relative coordinates, that is, the relative position of the child object relative to its parent object, and they are nested in multiple layers.

9. A method for dynamic camera path planning and visual standardization according to claim 8, characterized in that: In step S140, the camera performs a circular motion around the target object according to a predetermined trajectory, and moves around the Z axis of the observed target. The value of the Z axis will realize continuous rotation of the camera over time, and the real-time position of the camera is changed according to the rotation angle, and the logic implemented is a function of the Unreal Engine.

10. A method for dynamic camera path planning and visual standardization according to claim 9, characterized in that: The real-time position of the color card is set to a fixed position relative to the camera. If the camera moves but the real-time position of the color card is not updated, the real-time position of the color card in the camera will remain unchanged. Then, the viewing cone is converted into a right triangle and the cotangent function is used to calculate, so that the color card can be placed in the appropriate position of the picture.

Images