Camera capture method and assembly based on unreal engine

Abstract

This invention discloses a camera capture method and components based on Unreal Engine. When a camera capture command is received from a user, viewport capture is performed according to a preset capture frequency based on the command. Each viewport capture executes the following steps: determining the target camera based on the camera capture command and acquiring camera information; calculating the target view matrix and target projection matrix of the rendering target based on the camera information; determining the current scene view state; constructing a rendering view family and configuring the screen percentage and rendering identifier; constructing a rendering view based on the camera information, target view matrix, target projection matrix, and current scene view state, and adding the obtained rendering view to the rendering view family to obtain the target rendering view family; submitting the obtained target rendering view family to the renderer, performing rendering based on the main rendering pipeline, and obtaining the rendered camera capture view. The rendering path of this invention is the same as the main view, effectively avoiding the loss of rendering features or behavioral deviations.

Description

Technical Field

[0001] This invention relates to the field of image processing, and more particularly to a camera capture method and components based on Unreal Engine. Background Technology

[0002] Currently, in Unreal Engine (UE), the common method for capturing scene images is through the SceneCapture2D component combined with TextureRenderTarget2D. While this method can render scene content onto textures, it fails to achieve a good balance between performance and image quality. Especially in scenes requiring high-precision camera views (such as virtual shooting, real-time pre-visualization, and camera monitoring systems), it suffers from flickering, jitter, and brightness drift, resulting in significantly subpar effects.

[0003] Note:

[0004] SceneCapture2D is a 2D scene capture component, which is a built-in component of UE.

[0005] TextureRenderTarget2D is a 2D texture rendering target used to store the rendering result. Summary of the Invention

[0006] This invention addresses the shortcomings of existing scene capture methods, which fail to achieve a good balance between performance and image quality, resulting in captured camera images that are difficult to meet user needs. It provides a camera capture method and components based on Unreal Engine.

[0007] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0008] In a first aspect, the present invention provides a camera capture method based on Unreal Engine, wherein when a camera capture command is received from a user, viewport capture is performed according to a preset capture frequency based on the camera capture command.

[0009] Each viewport capture performs the following steps:

[0010] The target camera is determined based on the camera capture command, and camera information is acquired.

[0011] Based on the camera information, the target view matrix and the first projection matrix of the rendering target are calculated, and the first projection matrix is ​​dynamically corrected based on the aspect ratio of the rendering target to obtain the target projection matrix.

[0012] Determine the current scene view state; when performing viewport capture for the first time based on the camera capture command, allocate a new scene view state as the current scene view state according to the current world's graphics characteristic level; reuse the scene view state in subsequent viewport captures;

[0013] Build the rendering view family and configure the screen percentage and rendering flags;

[0014] A rendering view is constructed based on camera information, the target view matrix, the target projection matrix, and the current scene view state. The resulting rendering view is then added to the rendering view family to obtain the target rendering view family.

[0015] The resulting target rendering view family is submitted to the renderer, which performs rendering based on the main rendering pipeline to obtain the rendered camera capture view.

[0016] The internal calls of the renderer module must ensure correct timing and parameter configuration;

[0017] Improper settings for SceneCaptureSource, EngineShowFlags, or High Dynamic Range (HDR) start flag bIsHDR may cause color deviation.

[0018] Furthermore, directly building and submitting a rendering view family involves complex engineering details and requires proper handling of memory and resource lifecycles, while also taking into account version compatibility issues. Therefore, in practice, domain developers usually prefer to use the existing SceneCapture interface.

[0019] This application ensures that the renderer will inevitably enter the same rendering pipeline branch as the main view when performing rendering by allocating and reusing the scene view state; therefore, the capture process is completely consistent with the main view in the rendering stage, post-processing stage and time-related rendering logic, rather than the independent capture path used by SceneCapture2D.

[0020] This application implements the same rendering path as the main view during the capture process, avoiding the loss of rendering characteristics or behavioral deviations caused by different rendering paths in SceneCapture2D, and ensuring that the capture results are highly consistent with the main game screen in terms of color, brightness curves and final compositing.

[0021] As one possible implementation method:

[0022] A scene view initialization option is pre-established, and camera information, target view matrix, target projection matrix and current scene view state are injected into the scene view initialization option to construct the rendering view.

[0023] As one possible implementation, when constructing the rendered view, post-processing settings are also injected, and the following steps are executed sequentially:

[0024] Start the final post-processing settings to collect all post-processing volume effects that are not bound to the entity object Actor;

[0025] Blend the target camera's own post-processing settings into the current scene view;

[0026] The post-processing effect weight set is checked item by item. For each post-processing effect, a parameterizable shader instance is generated at runtime. The generated shader instance and its corresponding weight information are injected into the final post-processing chain of the rendering view so that it can participate in the post-processing calculation of the current frame.

[0027] Existing technologies only use one type of post-processing method. This application, with its design based on fixed-order merging of post-processing, can solve the problems of missing post-processing of captured images or inconsistent post-processing order, ensuring that the captured images and the main camera maintain consistency in post-processing effects such as brightness / tone mapping, color correction, depth of field, and volumetric fog.

[0028] In practical use, it supports dynamically adding custom post-processing materials (such as color correction, gamma correction, and special effects) and mixing them by weight, making it highly scalable.

[0029] As one possible implementation, when constructing the rendered view, the step of hiding the rendered object based on the rendered component identifier is also included, and the following steps are executed in sequence:

[0030] Based on the user-preset exclusion list, determine the objects to be excluded in the current world;

[0031] Iterate through the primitive components corresponding to each object to be excluded, obtain the unique identifier of each primitive component in the rendering scene, and obtain the corresponding rendering component identifier;

[0032] Configure the view's hidden list based on the obtained rendering component identifier.

[0033] As one possible implementation, the current scene view state is determined in each viewport capture as follows:

[0034] Check if the scene view state is valid. If it is invalid, allocate a new scene view state according to the current world's graphics feature level, and assign the reference of the scene view state to the state interface in the corresponding scene view initialization options so that the subsequently created rendering views can share the same set of time-related rendering state data.

[0035] Existing scene capture technologies recreate the rendering state object every frame, making it impossible to retain historical frame data and thus failing to guarantee time consistency.

[0036] This application solves the problem that existing scene capture methods cannot maintain time-related rendering data by creating a cross-frame rendering state object for the first time during the component's lifecycle and continuously reusing it in subsequent frames. This includes global illumination calculation modules based on multi-frame accumulation, historical reflection buffers, motion blur based on velocity fields, and other rendering features based on temporal continuity. Reusing this state object enables the capture view to correctly read the illumination, reflection, and historical accumulated data of the previous frame during each frame rendering, thereby ensuring that the anti-aliasing algorithm based on historical frames and global illumination calculation maintain stable consistency in the captured image, avoiding distortions such as flickering, jitter, and brightness drift. Furthermore, it ensures that the captured image maintains pixel-level consistency with the main view in terms of illumination, reflection, and post-processing results, significantly reducing visual differences.

[0037] As one possible implementation, the step of dynamically correcting the first projection matrix based on the aspect ratio of the rendering target to obtain the target projection matrix is ​​as follows:

[0038] When the aspect ratio of the target being rendered is the same as the aspect ratio of the corresponding camera viewport, the first projection matrix is ​​directly used as the target projection matrix.

[0039] When the aspect ratio of the corresponding camera viewport is greater than the aspect ratio of the rendering target, calculate the first scaling factor, and scale the first projection matrix in the horizontal direction based on the first scaling factor to obtain the corresponding target projection matrix.

[0040] When the aspect ratio of the corresponding camera viewport is less than that of the rendering target, the second scaling factor is calculated, and the first projection matrix is ​​scaled vertically based on the first scaling factor to obtain the corresponding target projection matrix.

[0041] As one possible implementation, the rendering resolution is determined based on the user-configured screen percentage using dynamic sampling rate interpolation.

[0042] Secondly, the present invention proposes a camera capture component based on Unreal Engine, used to receive camera capture commands issued by a user, and to perform viewport capture according to a preset capture frequency based on the camera capture commands; the component is used to execute the camera capture method described in any of the above-mentioned methods.

[0043] This invention, by adopting the above technical solutions, has significant technical effects:

[0044] This invention, by allocating and reusing scene view states, can avoid differential distortions caused by flickering / jittering / brightness drift, effectively improve the pixel-level consistency between the captured image and the main view, and reduce visual differences;

[0045] This invention, through the design of the entire camera capture method workflow, is able to capture and output advanced rendering features consistent with those of a native camera, and achieves pixel-level consistency and inter-frame consistency that the existing SceneCapture2D cannot achieve. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 This is a flowchart illustrating a camera capture method based on Unreal Engine according to the present invention. Detailed Implementation

[0048] The present invention will be further described in detail below with reference to the embodiments. The following embodiments are explanations of the present invention, but the present invention is not limited to the following embodiments.

[0049] The class names appearing in the following embodiments are example names in Unreal Engine 4 / 5. This invention is not limited to specific class names in a specific version. Any module that implements the same logical function is within the scope of protection of this application.

[0050] Example 1: A camera capture method based on Unreal Engine, wherein when a camera capture command is received from a user, viewport capture is performed according to a preset capture frequency based on the camera capture command;

[0051] like Figure 1 As shown, each viewport capture performs the following steps:

[0052] S100: Determine the target camera based on the camera capture command and acquire camera information;

[0053] The camera capture command is used to instruct the camera component used for image capture, i.e., the target camera;

[0054] The camera information includes camera view information, which is used to indicate the target camera's camera position, rotation direction, field of view, and the core parameters of the frustum that define the visible range.

[0055] S200: Calculate the target view matrix and the first projection matrix of the rendering target based on the camera information, and dynamically correct the first projection matrix based on the aspect ratio of the rendering target to obtain the target projection matrix;

[0056] In practical applications, there are situations where the aspect ratio ArRT of the render target is inconsistent with the aspect ratio ArCam of the target camera's viewport. In such cases, the UE's native system will directly stretch or crop the image, resulting in geometric distortion. To address this issue, this embodiment first determines whether the aspect ratio ArRT of the render target is consistent with the aspect ratio ArCam of the corresponding camera viewport. If they are inconsistent, the capture viewport is expanded by scaling the X-axis component (if ArCam>ArRT) or Y-axis component (if ArCam<ArRT) of the projection matrix, thereby maintaining the original composition and avoiding image stretching or cropping.

[0057] ArCam = Width of camera viewport / Length of camera viewport;

[0058] ArRT = Width of the render target / Length of the render target.

[0059] The specific steps are as follows:

[0060] S210. Calculate the target view matrix and the first projection matrix based on the camera information;

[0061] Given the camera information, those skilled in the art can easily obtain the target view matrix and the first projection matrix based on existing methods for calculating view matrices and projection matrices, which will not be elaborated upon in this embodiment;

[0062] S220, Correction judgment based on the aspect ratio of the rendering target;

[0063] S231. When the aspect ratio of the rendering target is the same as the aspect ratio of the corresponding camera viewport, the first projection matrix is ​​directly used as the target projection matrix.

[0064] S232. When the aspect ratio of the corresponding camera viewport is greater than the aspect ratio of the rendering target, that is, ArCam>ArRT, calculate the first scaling factor ScalingX=ArCam / ArRT, and scale the first projection matrix in the horizontal direction based on the first scaling factor ScalingX to obtain the corresponding target projection matrix.

[0065] S233. When the aspect ratio of the corresponding camera viewport is less than the aspect ratio of the rendering target, that is, ArCam < ArRT, calculate the second scaling factor ScalingY = ArRT / ArCam, and scale the first projection matrix in the vertical direction based on the first scaling factor ScalingY to obtain the corresponding target projection matrix.

[0066] In existing technologies, when resolution mismatch occurs, simple stretching / compression or field-of-view cropping is often used to ensure that the image composition matches the camera's expected perspective. This embodiment compares the aspect ratio of the corresponding camera viewport with the aspect ratio of the rendering target, generates a corresponding scaling factor based on the comparison result, and adjusts the first projection matrix horizontally or vertically based on the obtained scaling factor. This maintains the composition consistent with the real camera without image cropping. Compared with the existing SceneCapture2D, there is no distortion, and the geometric position of the output pixels is consistent with the main camera viewport.

[0067] S300, Determine the current scene view state;

[0068] When a viewport is captured for the first time based on the camera capture command, a new scene view state is assigned as the current scene view state according to the current world's graphics characteristics level.

[0069] The scene view state is reused in subsequent viewport captures.

[0070] SceneViewState is the core state object in Unreal Engine UE that manages temporal rendering data (such as TAA history frames and Lumen cache).

[0071] In this embodiment, the current scene view state is determined in each viewport capture as follows:

[0072] Check if the scene view state is valid. If invalid, allocate a new scene view state based on the current world's feature level. Assign a reference to the scene view state (SceneViewState) to the state interface (SceneViewStateInterface) in the corresponding scene view initialization options (FSceneViewInitOptions). This allows subsequently created rendering views (FSceneView) to share the same set of time-related rendering state data, thereby ensuring the consistency of inter-frame rendering effects, effectively improving pixel-level consistency between the captured image and the main view, reducing visual differences, and avoiding multiple rendering overheads by sharing the scene view state. This is more efficient than the traditional SceneCapture2D.

[0073] SceneViewState is not a disposable object. It internally stores: the historical texture of the previous frame; Lumen lighting cache; exposure adaptive history; motion blur speed history. In the existing SceneCapture2D, it is created or garbage collected every frame, causing all this data to be lost. Each frame is rendered as if it were the first frame, which can cause problems such as flickering, exposure jumps, and unstable reflections.

[0074] This embodiment binds the lifecycle of SceneViewState to the capturing component itself (i.e., the capturing component currently executing the camera capture command), rather than binding it to a single capture action. Instead of simply creating SceneViewState, it manages its lifecycle as a long-term member object of the capturing component: it is allocated during the first capture, continuously reused during multi-frame capture, and prevented from being reclaimed through the GC (garbage collection) reference registration mechanism, and is only released when the component is destroyed or the world is unloaded.

[0075] This management strategy ensures that time-dependent rendering data remains valid across consecutive frames, and is a necessary technical means to achieve stable Lumen, TAA (Temporal Anti-Aliasing), and exposure control.

[0076] In summary, this embodiment, through the design of allocating and reusing the state of the previous scene view, can solve the problem of inconsistency between Lumen global illumination, reflection buffer, motion blur and temporal characteristics in the scene capture. Furthermore, reusing SceneViewState can ensure that the temporal sequence is consistent with historical frame data, thereby ensuring that the time-related rendering of TAA / Lumen (such as light probes and previous frame information) has the correct historical state in the capture view, avoiding the difference distortion of flickering, jittering and brightness drift.

[0077] S400: Build the rendering view family and configure the screen percentage and rendering identifier;

[0078] The rendering view family, FSceneViewFamilyContext, abbreviated as ViewFamily, is the container and context for rendering views in Unreal Engine;

[0079] The RenderView family is used to indicate the format of the RenderTarget, screen percentage, display mode (Lit / Unlit / LightingOnly), HDR, post-processing volume enabled, and other parameters;

[0080] in:

[0081] Lit indicates that the display mode is illuminated (or has lighting rendering), referring to a display mode that includes full dynamic lighting calculations;

[0082] Unlit means unlit (or unlit rendering), which refers to a display mode that does not use dynamic lighting and only displays the material's basic color / self-illumination and other properties;

[0083] LightingOnly means lighting only (or lighting mode only), which means that only the direct / indirect lighting contribution is displayed, and the surface properties of the material are hidden.

[0084] HDR, High Dynamic Range, is a rendering and display technology that supports brightness values ​​beyond the standard range (0-1) and can better represent the contrast between light and dark in the real world.

[0085] S410: Based on dynamic sampling rate interpolation, the rendering resolution is determined according to the screen percentage configured by the user.

[0086] In Unreal Engine, the screen percentage is typically controlled by two parts:

[0087] Render resolution: The resolution at which 3D rendering is actually performed;

[0088] Display resolution: The resolution at which the final output is displayed on the screen.

[0089] This embodiment processes the user-defined screen percentage to obtain the corresponding rendering resolution. The specific steps are as follows:

[0090] S411. Calculate the sampling ratio based on the screen percentage. In this embodiment, the screen percentage ranges from 10% to 100%, which is converted to [0.1, 1.0].

[0091] S412. Calculate the corresponding rendering resolution based on the sampling ratio and the size of the rendering target, specifically:

[0092] X-axis rendering resolution = width of the rendering target * sampling ratio;

[0093] Y-axis rendering resolution = length of the rendering target * sampling ratio.

[0094] In practical applications, the renderer will render the scene at the rendering resolution, and then in the post-processing stage, the image will be upsampled to the display resolution.

[0095] The existing SceneCapture2D only supports passive scaling, not active control, which will lead to TAA sampling errors and loss of sharpness.

[0096] This embodiment, through custom design of screen percentage and resolution conversion, can explicitly control resolution scaling, precisely control pixel sampling rate, TAA / temporal behavior, and post-processing effect intensity, thereby improving the visual clarity, sharpness, and main view of the captured output. Figure 1 To make or controllable adjustments.

[0097] S420, Set and merge rendering identifiers;

[0098] The aforementioned display modes (Lit / Unlit / LightingOnly) and HDR are rendering identifiers;

[0099] In this embodiment, the rendering identifier also includes:

[0100] Enable post-processing volume (ApplyScenePostProcessVolume).

[0101] Enable motion blur.

[0102] ExcludedActorLayers;

[0103] Add rendering identifiers such as custom post-processing materials (WeightedBlendables);

[0104] As one possible implementation method:

[0105] Get the default rendering identifier, then get the first rendering identifier;

[0106] Obtain the rendering identifier corresponding to the main view, and then obtain the second rendering identifier;

[0107] Obtain the user-configured rendering identifier and acquire the third rendering identifier;

[0108] The first rendering identifier, the second rendering identifier, and the third rendering identifier are ranked from low to high. Based on the priority, the first rendering identifier, the second rendering identifier, and the third rendering identifier are merged to obtain the target rendering identifier.

[0109] S500: Construct a rendering view based on camera information, target view matrix, target projection matrix and current scene view state, and add it to the rendering view family to obtain the target rendering view family;

[0110] In this embodiment, a scene view initialization option FSceneViewInitOptions is pre-established, and camera information, target view matrix, target projection matrix and current scene view state are injected into the scene view initialization option FSceneViewInitOptions to construct the rendering view View.

[0111] For example, create a SceneView using newFSceneView(ViewInitOptions) and set options such as bIsOfflineRender (offline rendering start flag) and bIsGameView (whether it is a game view);

[0112] As one possible implementation, post-processing settings injection is also performed when constructing the rendering view. The specific steps are as follows:

[0113] S510, Post-processing settings are merged, sequentially merging the scene post-processing volume (PostProcessVolume), the camera post-processing volume (PostProcess) corresponding to the target camera, and the weighted mixable post-processing items. Specifically:

[0114] S511. Start the final post-processing settings and collect all post-processing volume effects that are not bound to entity objects;

[0115] In this embodiment, the process of building the final post-processing configuration is initiated by collecting parameters of global post-processing regions in the scene that are not bound to specific objects. Specifically, the final post-processing configuration is initialized based on the current viewpoint position, and the parameters of all active global post-processing regions in the scene are written into this configuration. This allows post-processing effects defined in these regions, such as exposure, tone mapping, and depth of field, to participate in the final post-processing calculation of the current frame. Specifically, the current viewpoint position is passed as a parameter to the SceneView function.

[0116] S512: Mix the target camera's own post-processing settings into the current scene view;

[0117] In this embodiment, the post-processing parameters of the target camera are mixed into the rendering view of the current frame according to a preset weight;

[0118] That is, the set of post-processing parameters carried by the target camera is parsed, and the parameters are integrated into the final post-processing configuration of the rendering view according to the corresponding blending weights, so that the captured image is consistent with the target camera in terms of post-processing effects such as tone mapping, exposure, and depth of field; the specific implementation method is to call the OverridePostProcessSettings function in SceneView.

[0119] S513. Check the set of post-processing effect weights item by item, generate a parameterizable shader instance for each post-processing effect at runtime, and inject the generated shader instance and its corresponding weight information into the final post-processing chain of the rendering view so that it can participate in the post-processing calculation of the current frame.

[0120] For example:

[0121] The system iterates through the set of post-processing effects carried by the currently captured component, filtering shader resources used for screen-space post-processing. For each post-processing resource, a configurable shader instance is generated at runtime, and based on its preset priority and blending position, this instance is inserted into the final post-processing execution sequence of the rendered view. This ensures that multi-source post-processing effects participate in the post-processing calculation of the current frame in the correct order. Specifically, the priority and blending position parameters are passed to the SceneView's FinalPostProcessSettings variable.

[0122] in:

[0123] SceneView represents the scene view;

[0124] FinalPostProcessSettings.BlendableManager represents the final postprocessing settings blendable manager.

[0125] This fixed-sequence combination algorithm surpasses the default single-source post-processing logic of UE, achieving complete integration of scene, camera, and custom materials, and ensuring consistency in exposure, depth of field, and tone mapping. This step resolves the issues of missing or inconsistent post-processing of captured images.

[0126] As one possible implementation, when constructing the rendering view, the step of hiding the rendering object based on the rendering component identifier is also included, specifically: S521, determining the objects to be excluded in the current world based on the user-preset exclusion list;

[0127] For example, a dedicated function (such as GetPrimitiveComponentsFromLayers) can be used to iterate through all entity objects (Actors) in the current world; check each entity object to see if its attribute identifiers (including but not limited to hierarchy, label, or type) are in a preset exclusion list, and obtain the objects to be excluded.

[0128] S522. Traverse all primitive components contained in the object to be excluded, and obtain the unique identifier of each primitive component in the rendering scene:

[0129] For the Actor that matches the exclusion layer, i.e. the object to be excluded, we further traverse all the primitive components (i.e., the rendered geometric parts) it contains, obtain the unique identifier PrimitiveComponentId of each primitive component in the rendering scene, and obtain the corresponding rendering component identifier.

[0130] S523. Configure the view hidden list based on the obtained rendering component identifier:

[0131] The resulting render component identifier is added to the HiddenPrimitives set in the scene view initialization options, and such render components will be hidden during the rendering process.

[0132] Those skilled in the art need to address the potential differences in this ID field across different Unreal Engine versions in practical use to ensure compatibility.

[0133] When the renderer processes this view subsequently, it will skip all rendering components included in the hidden primitive list. This method achieves pixel-level precise hiding, avoiding the side effects (such as collisions or logical interruptions) that may result from hiding the entire entity object, while its performance overhead is far lower than that of rendering techniques such as masking or opacity.

[0134] S600: Submit the obtained target rendering view family to the renderer, perform rendering based on the main rendering pipeline, and obtain the rendered camera capture view;

[0135] In this embodiment, based on the encapsulated ViewFamily, i.e., the target rendering view family, a rendering command (`ENQUEUE_RENDER_COMMAND(RenderViewFamilyCommand)`) is generated and the rendering command is submitted to the rendering thread;

[0136] In this embodiment, an FCanvas object (the core class for performing Immediate Mode 2D rendering) is also created and bound to a RenderTargetResource. After the rendering thread finishes execution, the corresponding pixel data is directly written to the RenderTarget. The RenderTarget stores the final captured high-quality camera capture image that is consistent with the native camera view at the pixel level. It can be read by the CPU or used for subsequent processing and supports dynamic updates at runtime.

[0137] For example, a rendering canvas object is constructed for performing drawing operations on an image output buffer, and the canvas object is bound to a texture buffer for receiving rendering results; and a view set containing at least one rendering view is constructed based on preset rendering parameters; then the view set is submitted to the graphics rendering processing module to perform a complete rendering process including geometry processing, lighting calculation and post-processing, thereby writing the rendering results into the aforementioned texture buffer.

[0138] In this embodiment, when constructing the rendering view family FSceneViewFamily, the world main Scene is used, the GameView flag is set, and the main view ViewExtensions are enabled, instead of reusing the scene view state SceneViewState. This ensures that the renderer will inevitably enter the same rendering pipeline branch as the main view when executing the corresponding rendering commands. Therefore, the renderer triggers a complete rendering pipeline that is completely consistent with the main view, including geometry culling, basepass, lighting calculation (including Lumen), shadows, the entire post-processing chain, and the final tone mapping.

[0139] In summary, this embodiment unifies the capture process to the main rendering path through structural constraints, ensuring the effectiveness and timing consistency of all advanced rendering features, rather than the independent capture path used by SceneCapture2D.

[0140] Note: The above methods can be implemented in C++, or encapsulated as BlueprintNodes, or called through other scripting languages ​​(such as Python for UE), or as direct modifications to the engine source code. This manual does not provide detailed limitations on these methods.

[0141] Example 2: A camera capture component based on Unreal Engine, used to execute the camera capture method described in Example 1;

[0142] In this embodiment, the camera capture component (UCameraCaptureComponent2D) is a custom component that inherits from UActorComponent (BlueprintSpawnableComponent). It is used as a plugin / component for users to attach to the Actor that contains the camera component UCameraComponent.

[0143] in:

[0144] UActorComponent is an actor component, belonging to the base class of components that can be attached to Actors in Unreal Engine UE. An Actor represents an entity object.

[0145] BlueprintSpawnableComponent is a metadata specifier for Unreal Engine UE, used to declare that the corresponding component can be dynamically created (generated) in a Blueprint and added to an Actor.

[0146] In practical applications, the target camera (UCameraComponent) is determined based on the camera capture command, and the camera capture component provided in this embodiment is bound to the corresponding target camera to ensure that the captured view is synchronized with the camera parameters.

[0147] As one possible implementation, the main member variables of the camera capture component (UCameraCaptureComponent2D) include:

[0148] UTextureRenderTarget2DRenderTarget: Used to output the target texture (RenderTarget) of the captured image.

[0149] bool bCapture: Whether to enable capture.

[0150] bool bExtendCapturedView: Whether to expand the capture viewport to avoid stretching when the RenderTarget is inconsistent with the camera aspect ratio.

[0151] float ScreenPercentage: Screen percentage control (10%~100%), used to control the sampling resolution.

[0152] bool bApplyScenePostProcessVolume: Whether to apply the post-processing volume of the scenario;

[0153] bool bMotionBlur: Whether to enable motion blur.

[0154] Those skilled in the art can set the main member variables according to actual needs, and this embodiment does not limit them in detail.

[0155] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0156] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0157] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0158] This invention is described with reference to flowchart illustrations and / or block diagrams of the method, terminal device (system), and computer program product according to the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0159] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0160] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0161] It should be noted that:

[0162] The phrase "an embodiment" or "an embodiment" used in this specification means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the invention. Therefore, the phrase "an embodiment" or "an embodiment" appearing in various places throughout the specification does not necessarily refer to the same embodiment.

[0163] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the invention.

[0164] Furthermore, it should be noted that the shapes and names of the parts and components described in the specific embodiments described in this specification may differ. All equivalent or simple variations made to the structure, features, and principles described in this patent concept are included within the protection scope of this patent. Those skilled in the art to which this invention pertains may make various modifications or additions to the described specific embodiments or use similar methods to replace them, as long as they do not depart from the structure of this invention or exceed the scope defined in these claims, they should all fall within the protection scope of this invention.

Claims

1. A camera capture method based on Unreal Engine, wherein when a camera capture command is received from a user, viewport capture is performed according to a preset capture frequency based on the camera capture command; Its features are, Each viewport capture performs the following steps: The target camera is determined based on the camera capture command, and camera information is acquired. The target view matrix and the first projection matrix of the rendering target are calculated based on the camera information, and the first projection matrix is ​​dynamically corrected based on the aspect ratio of the rendering target to obtain the target projection matrix. Determine the current scene view state; When a viewport capture is initially performed based on the camera capture command, a new scene view state is assigned as the current scene view state according to the current world's graphics characteristics level; the scene view state is reused in subsequent viewport captures. Build the rendering view family and configure the screen percentage and rendering flags; A rendering view is constructed based on camera information, the target view matrix, the target projection matrix, and the current scene view state. The resulting rendering view is then added to the rendering view family to obtain the target rendering view family. The resulting target rendering view family is submitted to the renderer, which performs rendering based on the main rendering pipeline to obtain the rendered camera capture view.

2. The camera capture method based on Unreal Engine according to claim 1, characterized in that, A scene view initialization option is pre-established, and camera information, target view matrix, target projection matrix and current scene view state are injected into the scene view initialization option to construct the rendering view.

3. The camera capture method based on Unreal Engine according to claim 2, characterized in that, When constructing the rendered view, perform post-processing settings injection, and execute the following steps in sequence: Start the final post-processing settings to collect all post-processing volume effects that are not bound to entity objects; Blend the target camera's own post-processing settings into the current scene view; The post-processing effect weight set is checked item by item. For each post-processing effect, a parameterizable shader instance is generated at runtime. The generated shader instance and its corresponding weight information are injected into the final post-processing chain of the rendering view so that it can participate in the post-processing calculation of the current frame.

4. The camera capture method based on Unreal Engine according to claim 2 or 3, characterized in that, When constructing the rendered view, the process also includes a step of hiding the rendered object based on the rendered component identifier, which involves performing the following steps in sequence: Based on the user-preset exclusion list, determine the objects to be excluded in the current world; Iterate through the primitive components corresponding to each object to be excluded, obtain the unique identifier of each primitive component in the rendering scene, and obtain the corresponding rendering component identifier; Configure the view's hidden list based on the obtained rendering component identifier.

5. The camera capture method based on Unreal Engine according to claim 2, characterized in that, The current scene view state is determined in each viewport capture as follows: Check if the scene view state is valid. If it is invalid, allocate a new scene view state according to the current world's graphics feature level, and assign the reference of the scene view state to the state interface in the corresponding scene view initialization options so that the subsequently created rendering views can share the same set of time-related rendering state data.

6. The camera capture method based on Unreal Engine according to claim 1 or 2, characterized in that, The steps for dynamically correcting the first projection matrix based on the aspect ratio of the rendering target to obtain the target projection matrix are as follows: When the aspect ratio of the target being rendered is the same as the aspect ratio of the corresponding camera viewport, the first projection matrix is ​​directly used as the target projection matrix. When the aspect ratio of the corresponding camera viewport is greater than the aspect ratio of the rendering target, calculate the first scaling factor, and scale the first projection matrix in the horizontal direction based on the first scaling factor to obtain the corresponding target projection matrix. When the aspect ratio of the corresponding camera viewport is less than that of the rendering target, the second scaling factor is calculated, and the first projection matrix is ​​scaled vertically based on the first scaling factor to obtain the corresponding target projection matrix.

7. The camera capture method based on Unreal Engine according to claim 1 or 2, characterized in that, The rendering resolution is determined based on the user-configured percentage of the screen using dynamic sampling rate interpolation.

8. A camera capture component based on Unreal Engine, used to receive camera capture commands issued by a user, and to perform viewport capture according to a preset capture frequency based on the camera capture commands; characterized in that, Used to perform the camera capture method according to any one of claims 1 to 7.

Images