Camera path drawing method and rendering interaction system based on camera path drawing
The camera motion path is drawn through third-order Bezier curve fitting, and the user interaction parameters are combined to generate diverse rendered images, solving the problems of high acquisition costs of real robots and unpredictable paths, and achieving efficient and diverse image acquisition and rendering effects.
Patent Information
- Application Number
- CN202111578343.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-22
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2041-12-22
AI Technical Summary
In the prior art, using real robots to acquire images of various perspectives in real-world scene environments has high cost and unpredictable path problems, and the diversity of image acquisition cannot be achieved. The existing rendering method is not suitable for camera tracks of general scenes. Path planning.
The third-order Bezier curve fitting method is used to draw the camera motion path based on the key points input by the user, and render it in combination with the relevant parameters input by the user interaction to generate a variety of rendered images, real-time through the communication connection between the client and the server. Display and render.
It realizes rapid and accurate acquisition of camera motion paths, generates rendered images of various perspectives that meet users' expectations, reduces costs, and improves the diversity and universality of image acquisition. Users can intuitively feel the camera motion trajectory and customize the keys. Points generate camera path.
Smart Images

Figure CN114237436B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of graphics rendering, and in particular relates to a camera path rendering method and a rendering interaction system based on the camera path rendering. Background Art
[0002] With the rapid development of big data and artificial intelligence, building various models based on big data analysis often requires images from various viewpoints of a scene. Currently, the main method for collecting images from various viewpoints is to use real robots in real-world environments. However, this method has the following disadvantages:
[0003] (1) The manufacturing cost of real robots is high, which leads to high image acquisition costs; (2) When real robots collect data in real scene environments, there is no expected collection path, and repeated experiments are required to obtain a collection path with a better perspective, thereby obtaining better images; (3) When using real robots to collect data in real scene environments, they are also restricted by the real environment and cannot achieve diversity in image acquisition.
[0004] In view of the above-mentioned shortcomings of using real robots to collect data in real scene environments, the existing technology can adopt the method of image rendering of three-dimensional scene models to obtain a large number of rendered images from various perspectives. However, there will be problems in the rendering path design during the rendering process to obtain rendered images from various perspectives that meet user expectations.
[0005] Patent document CN111445556 A discloses a method for rendering dynamic driving trajectories. The method constructs a trajectory model by configuring trajectory parameters and pattern parameters, and performs rendering based on the trajectory model. Since this method is suitable for dynamic driving, the trajectory model is constructed based on the rut part, the safety area part, and the blank part, and is not visualized. Therefore, it is not suitable for camera trajectory path planning and image rendering in general scenarios.
[0006] Patent document CN110609622A discloses a method for realizing multi-person interaction by combining 3D and virtual reality technologies. In this interaction method, the 3D host and the virtual reality host respectively determine the corresponding position and motion trajectory of each experiencer in the same virtual scene based on the positioning data of each experiencer, so as to determine the display content from the corresponding perspective of each experiencer and render it. This method can obtain renderings of various required perspectives and paths through virtual reality technology, but the sampling cost is high and it is not suitable for camera trajectory path planning and image rendering in general scenes. Summary of the Invention
[0007] In view of the above, a first object of the present invention is to provide a camera path drawing method, which can quickly and accurately draw a camera motion path based on a Bezier curve according to key points input by a user.
[0008] The second object of the present invention is to provide a rendering interaction system based on camera path drawing, which constructs the camera motion path through key points input by the user, and performs rendering in combination with relevant parameters input by the user interaction to generate a variety of rendered images.
[0009] To achieve the first object of the invention, an embodiment provides a camera trajectory drawing method, comprising the following steps:
[0010] Get the key points of user input;
[0011] When there are two key points, the line connecting the two key points is used as the camera trajectory;
[0012] When there are more than two key points, a third-order Bezier curve is fitted for every two keys according to the timestamp order of the key points to obtain the path curve between every two keys and passing through the key points. At the same time, the second-order continuity of the two adjacent path curves at the common key point is ensured to obtain the entire camera path.
[0013] In one embodiment, when a third-order Bezier curve is used to fit m-1 path curves to m key points, for two adjacent path curves, the first-order derivative and second-order derivative of the first path curve at the end point are required to be equal to the first-order derivative and second-order derivative of the second path curve at the starting point. At the same time, the second-order derivative of the first path curve of the entire camera path formed by the m-1 path curves must be 0 at the starting point, and the second-order derivative of the m-1th path curve must be 0 at the end point.
[0014] Combining the two-dimensional coordinates of m key points and the third-order Bezier curve expression corresponding to each path curve, each path curve is determined through matrix solution, and then the entire camera path is obtained.
[0015] To achieve the second object of the invention, an embodiment provides a rendering interaction system based on camera path drawing, comprising a client and a server connected in communication, wherein the client receives user input information, including key points with input timestamps, camera motion parameters, and image quantity-related parameters, and transmits the input information to the server;
[0016] The server side draws the entire camera path based on the received key points according to the camera path drawing method;
[0017] The image rendering engine on the server side performs scene rendering along the entire camera path according to the camera motion parameters and parameters related to the number of images to generate a rendered image;
[0018] When receiving the key points input by the user, the client displays the camera path formed by the key points in real time.
[0019] In one embodiment, the camera motion parameters include camera motion speed, camera motion height, and camera motion pitch angle; the image quantity related parameters include: total image frame number and rendering frame rate;
[0020] When the parameter related to the number of received images is the total number of image frames, and it is greater than the number of key points, the total number of image frames is evenly distributed to each path curve, under the premise of ensuring that each key point corresponds to a frame of image. The number of image frames corresponding to each path curve is then evenly divided to obtain the camera two-dimensional coordinates corresponding to each frame of image. Combined with the camera's height during use and the mapping relationship from three-dimensional coordinates to two-dimensional coordinates, the camera three-dimensional coordinates corresponding to each frame of image are determined. Then, the image rendering engine performs scene rendering based on the camera's three-dimensional coordinates and the pitch angle of the camera during movement to obtain a rendered image.
[0021] In one embodiment, when the total number of received image frames is less than or equal to the number of key points, the three-dimensional coordinates of the camera corresponding to each key point are determined based on the two-dimensional coordinates of each key point, combined with the height of the camera during operation and the mapping relationship between the three-dimensional coordinates and the two-dimensional coordinates. Then, the image rendering engine performs scene rendering based on the three-dimensional coordinates of the camera corresponding to each key point and the pitch angle of the camera during movement to obtain a rendered image.
[0022] In one embodiment, when the parameter related to the number of received images is the rendering frame rate, the time required for each frame of image is calculated based on the rendering frame rate, and the camera two-dimensional coordinates corresponding to each frame of image according to the camera movement speed are confirmed in real time along the entire camera path. The camera three-dimensional coordinates corresponding to each frame of image are determined in combination with the camera height during use and the mapping relationship between three-dimensional coordinates and two-dimensional coordinates. When the image rendering engine renders based on the camera three-dimensional coordinates, starting from the first frame of image, the length of the trajectory of the camera movement is accumulated. When the accumulated trajectory length is less than the length of the entire camera path, the camera continues to move along the camera path; when the accumulated trajectory length is equal to the length of the entire camera path, the current frame rendered image is retained and the camera stops moving; when the accumulated trajectory length is greater than the length of the entire camera path, the current frame rendered image is discarded and the camera stops moving.
[0023] In one embodiment, when the distance between the camera two-dimensional coordinate corresponding to each frame image and the adjacent key point along the path is less than a preset threshold according to the camera motion speed, the camera two-dimensional coordinate is updated to the adjacent key point.
[0024] In one embodiment, the camera path drawing-based rendering interaction system further includes: the server side determines the length of the entire camera path by discretely integrating each segment of the third-order Bezier curve corresponding to the entire camera path.
[0025] In one embodiment, the client provides the user with an input information editing interface and provides functions of inserting, deleting, and modifying key points of the camera path.
[0026] In one embodiment, the camera path determination method formed by the key points displayed in real time by the client includes: using a third-order Bezier curve for fitting, during fitting, taking three consecutive key points as a fitting unit, and then determining the two middle control points of the third-order Bezier curve based on the three key points in each fitting unit and the principle of similar triangles, and then, combining the first and last two key points of each fitting unit to calculate and determine the third-order Bezier curve, that is, the corresponding fitting path.
[0027] Compared with the prior art, the present invention has the following beneficial effects:
[0028] The camera path drawing method provided in the embodiment uses third-order Bezier curve fitting to perform segmented fitting of the camera path. At the same time, to ensure that the fitted camera path passes through key points and each segment of the camera path is connected at a common key point, constraints are set. Based on the constraints and combined with the coordinates of the key points, the third-order Bezier curve is fitted and solved, so that the camera motion path can be obtained quickly and accurately.
[0029] The rendering interaction system based on camera path drawing provided in the embodiment uses the third-order Bezier curve fitting to determine the camera path, and combines the camera motion parameters and image quantity related parameters input by the user through the client to perform scene rendering along the entire camera path to generate a rendered image. In this way, rendered images of each perspective that meet the user's expectations are obtained. At the same time, the client also displays the camera path formed by the key points in real time, allowing the user to more intuitively feel the trajectory of the camera movement along the key points. Furthermore, the camera path is generated by the user based on the user-defined key points, is not restricted by other conditions, and has strong universality. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0031] Figure 1 is a flowchart of a camera path drawing method provided by an embodiment;
[0032] Figure 2 Schematic diagram of a Bezier curve and its control points provided by an embodiment;
[0033] Figure 3 This is a schematic diagram of fitting a camera path in a home decoration scene provided by an embodiment;
[0034] Figure 41 is a schematic structural diagram of a rendering interaction system based on camera path drawing provided by an embodiment;
[0035] Figure 5 This is a schematic diagram of the principle of calculating control points based on the triangle similarity principle provided by an embodiment. DETAILED DESCRIPTION
[0036] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and do not limit the scope of protection of the present invention.
[0037] Example 1
[0038] In order to quickly fit a camera path that meets the user's expectations based on key points drawn by the user, an embodiment provides a camera path drawing method. Figure 1 Flowchart of the camera path drawing method provided by the embodiment. Figure 1 As shown, the embodiment provides a camera path drawing method, characterized by comprising the following steps:
[0039] Step 1: Get the key points input by the user.
[0040] Step 2: When there are two key points, the line connecting the two key points is used as the camera trajectory.
[0041] Step 3: When there are more than two key points, perform a third-order Bezier curve fitting on every two key points in the order of their timestamps to obtain the path curve between each two key points and passing through the key points. At the same time, ensure that the two adjacent path curves are second-order continuous at the common key point to obtain the entire camera path.
[0042] In this embodiment, a key point refers to a trajectory point used to fit the camera path. This key point must lie on the fitted camera path. When rendering an image according to the camera path, the key point's position is rendered and drawn to produce a keyframe image. When drawing the camera path, a camera path is not formed for a single key point input. If two key points are input, the line connecting the two key points is used as the fitted camera trajectory.
[0043] When the input is greater than two key points, a multi-segment Bezier curve is used to fit the camera path. The traditional Bezier curve fitting method treats multiple points as control points of the Bezier curve, but this approach has two major limitations in user experience: First, the Bezier curve does not pass through most control points, as shown below: Figure 2As shown. The displayed points are control points, and the curves are Bezier curves generated by fitting under the control points. Obviously, the key points selected by the user on the visual interface are the trajectory points that the path is expected to pass through, rather than estimating the position of the path control points based on the expected path. On the other hand, the Bezier curves controlled by multiple control points do not have the characteristics of local editing and modification. This means that when the user finds that a certain section of the camera path drawn does not meet expectations and wants to modify it, each key point changed will affect the entire camera path. Therefore, when the Bezier curve is used to fit the camera path in the embodiment, the fitting process is defined to meet the following fitting conditions: multiple Bezier curves passing through the key points drawn by the user are required to be smoothly spliced to fit the entire motion path.
[0044] To achieve these fitting conditions, the control points of the Bezier curve need to be inferred from the points it passes through. At the same time, multiple Bezier curve segments must be smooth at the key points where they are joined. This is achieved by using every two adjacent key points as the starting and ending points of a fitting curve, and fitting each fitting curve using a third-order Bezier curve. Adjacent third-order Bezier curves must be second-order continuous at their shared endpoints (i.e., shared key points).
[0045] If a Bezier curve has n control points, the expression of the Bezier curve is: Among them, P i is the i-th control point, Then the expression of the third-order Bezier curve of each segment can be written as:
[0046] P(t)=(1-t) 3 P0+3t(1-t) 2 P1+3t 2 (1-t)P2+t 3 P3,t∈[0,1]
[0047] When the user inputs m key points, it corresponds to m-1 segments of fitting path curves, and the two key points of each segment of the path curve correspond to the control points P0 and P3 of the third-order Bezier curve. Then the problem is transformed into finding the control points P0 and P2 on each segment of the path curve so that the adjacent curves are second-order continuous at the shared key points. That is, assuming P j (t), P j+1 (t) are two adjacent segments on the m-1 segment path curve, then the second-order continuity condition should be satisfied: the previous segment path curve P j (t) at the end point P j The first and second derivatives of (t=1) and the latter path curve P j+1 (t) At the starting point P j+1The first and second derivatives of (t=0) are equal, and the expression is:
[0048] The problem is thus transformed into: for m-1 path curves, there are 2(m-1) control points to be determined. The above second-order continuity condition gives 2(m-2) expressions that the m-2 control points must satisfy. Therefore, 2(m-1) unknown variables need to be determined, and two more expressions need to be added. Therefore, when solving the problem, two boundary conditions are added: the second-order derivative of the first path curve of the entire camera path formed by the m-1 path curves is 0 at the starting point, and the second-order derivative of the m-1th path curve is 0 at the end point. The expression is
[0049] Under the constraints of the above second-order continuity conditions and boundary conditions, the expressions of m third-order Bezier curves formed by the coordinates of m key points are combined, and the expressions of m-1 Bezier curves with respect to the parameter t are obtained by matrix solution. The sample of the fitting curve is as follows: Figure 3 As shown in the figure, the white dots are the key points specified by the user and the curve is the final fitted camera path.
[0050] Example 2
[0051] Example 2 provides a rendering interaction system based on camera path drawing. Through this rendering interaction system, users can directly draw the key points of the expected camera motion path in the various complex scenes presented in the visual interface, and adjust the camera state parameters at the initial stage and during the motion process, so as to obtain high-quality first-person perspective renderings during camera motion with less time and labor cost.
[0052] Figure 4 Schematic diagram of the structure of the rendering interaction system based on camera path drawing provided by the embodiment. Figure 4 As shown, the rendering interaction system provided by the embodiment includes a client and a server communicating with the client. The client receives input information input by the user and transmits it to the server, wherein the input information includes key points with input timestamps, camera motion parameters, and image number-related parameters. The server draws the entire camera path based on the received key points according to the above-mentioned camera path drawing method; then the image rendering engine on the server performs scene rendering along the entire camera path according to the camera motion parameters and image number-related parameters to generate a rendered image. When receiving the key points input by the user, the client also displays the camera path formed by the key points in real time.
[0053] The client provides a visual interface for users to interact with. Users can select a top-down view of the scene and draw the actual camera path within it. By controlling key points, a smooth camera path is generated. The client also provides functions for inserting, deleting, and modifying key points in the camera path. These functions allow users to insert, delete, and modify key points in the camera path.
[0054] In the embodiment, from the perspective of camera path drawing, the drawn camera path is divided into three key point operations, namely, drawing the path starting point, drawing the path middle point, and drawing the path end point.
[0055] For path starting point drawing, when the mouse is in the hot zone of the visual operation interface (the hot zone is the interface area for camera path drawing and operation), a bubble reminder will appear on the mouse: "Click to set the path starting point". At this time, by clicking the left mouse button, the path starting point can be determined according to the latitude and longitude of the current mouse position, which is also the first key point of the camera path.
[0056] When drawing a path midpoint and determining the starting point, there are two possible actions: A) Continue to the next step, completing the drawing of the key points of the subsequent camera path; B) Undo the current operation, returning to the previous step. Therefore, the follow bubble on the mouse changes to "Left click to add, right click to undo" to prompt the user to perform the relevant operation.
[0057] When the user clicks the left mouse button, the hotspot determines a second keypoint based on the mouse's latitude and longitude at the time of the second click, and a camera path is formed from these two keypoints. A camera path with only two keypoints completes the most basic camera path: a straight line. To draw a more complex camera estimate, you need to continue adding keypoints to the hotspot. The hotspot will update the camera path in real time based on these additional keypoints.
[0058] When the user right-clicks, the hotspot will undo the current operation and return to the previous shot. If the previous step was drawing the path starting point, the hotspot will return to the corresponding shot. For details on the shot, refer to the instructions for drawing the path starting point. During the entire camera path drawing process, the user can click the left mouse button again at any time to draw a new key point based on the undo shot.
[0059] In addition to the operations of adding and undoing key points, the embodiment also provides the ability to insert, delete, and modify key points. For inserting key points, a camera path with 10 key points is preset. When you need to insert another path control point between the first and second key points, just hover the mouse over the path segment between the first and second key points. The interface will then prompt: "Click & drag to add a new point." Simply click anywhere on the camera path to insert that location as a key point, and the camera path will change in real time based on the newly added key points.
[0060] To delete or modify track keypoints, a preset camera path with 10 keypoints is already created. To modify the second keypoint while retaining the others, hover your mouse over the relevant point. A prompt will appear: "Drag to modify, click to remove." Click the left mouse button to delete the relevant point. Hold down the left mouse button and drag; the relevant point will move according to the mouse position until you release it.
[0061] For path endpoint drawing, define all non-starting key points as endpoints during the trajectory drawing process. Since the camera path is stored locally in real time, no additional saving is required. When the camera path has reached the required point, select Exit to complete the current camera path drawing.
[0062] When drawing a camera path through the client, the client displays the camera path formed by the key points in real time. Considering the camera path fitting calculation overhead and data transmission overhead, the camera path displayed in real time by the client is calculated based on the principle of triangle similarity, allowing users to more intuitively experience the trajectory of the camera along the key points.
[0063] In the embodiment, different path fitting solutions are adopted according to the number of key points. When there is only one key point, it is a single key point; when there are two key points, the two key points are connected into a straight line; when there are three or more key points, a third-order Bezier curve is adopted for fitting. When fitting, three consecutive key points are used as a fitting unit, and then the two middle control points of the third-order Bezier curve are determined based on the three key points in each fitting unit and the principle of similar triangles. Then, the third-order Bezier curve, i.e., the corresponding fitting path, is calculated and determined in combination with the two key points at the beginning and end of each fitting unit.
[0064] In the Bézier curve diagram, when t is 0 and 1, the two starting and ending control points are tangent. Key points can serve as the starting and ending control points of the Bézier curve, respectively. Because the tangents are tangent, each Bézier curve segment is tangent at a common point, and adjacent curves are second-order continuous. Therefore, the trajectory is transformed into a continuous multi-segment Bézier curve. Considering the number of control points, the overall trajectory is integrated into a combination of multiple quadratic and cubic Bézier curves.
[0065] Because the key points of the trajectory act as the starting and ending control points of the Bezier curve, such as Figure 5 The key points A, B, and C in the image are calculated manually. The intermediate control points P1 and P2 need to be calculated manually. In the embodiment, similar triangles are used for calculation. The specific calculation process is:
[0066] If the coordinates of key points A, B and C are known, find the coordinates of control points P1 and P2. The trajectory is assumed to be Figure 5 As shown, and can be based on Figure 5 The right triangle drawn in [1] leads to the following conclusion: △AOC is similar to △P1NB and △BMP2, and the similarity ratios are set to S1 and S2 respectively. Since the coordinates of key points A, B, and C are known, the length of the line segment can be calculated based on the similarity ratio, and then the coordinates of control points P1 and P2 can be obtained. Here, the similarity ratio used to calculate control point P1 is calculated as S1 = K*(AB / (AB+BC)) (K is a constant, and the value is 0.5 based on the actual situation). In this way, the path curve can be fitted using a third-order Bezier curve based on key points A and C and control points P1 and P2.
[0067] In the embodiment, a path direction needs to be added at the end of the camera path to indicate the direction of camera movement. The tangent direction determined by the last control point and the last key point of the last Bezier curve of the fitted camera path is used as the path direction.
[0068] After the camera path is drawn, the information configuration is completed through the client, including the selection and input of the camera motion parameters and the parameters related to the number of images. The server can then render and output the scene rendering image based on the key points, camera motion parameters, and the parameters related to the number of images.
[0069] In an embodiment, the camera motion parameters include the camera motion speed (m / s), the camera motion height, and the camera motion pitch angle; the image quantity related parameters include the total number of image frames rendered by the camera, frameCount, and the rendering frame rate fps during the camera motion.
[0070] Based on the above different image quantity-related parameters, the rendering interaction system provides users with two modes for specifying the number of camera rendering outputs, namely the output mode based on frameCount and the output mode based on speed+fps. The former directly outputs images according to the user-specified frameCount, while the latter calculates the total number of image frames based on speed+fps and the camera path. When frameCount is set, the output mode based on frameCount is preferred.
[0071] In the interactive rendering system provided in this embodiment, camera motion parameters (speed, height, pitch) support three modes: a user-defined constant value through the client; a random initialization with the parameter remaining constant during rendering; and a random initialization with the parameter randomly varying from the initial value during rendering. If a constant value exists, the server renders based on it; otherwise, the latter two modes are automatically used.
[0072] In this embodiment, after receiving the keypoints input by the client, the server uses the above-mentioned camera path drawing method to draw the entire camera path. If the input is a single keypoint and no camera path exists, then when the frameCount-based rendering mode is selected, multiple renderings are performed at the keypoint using the initial camera position keyPoints and posture to obtain a rendered image with the specified frameCount. If the input is two keypoints, the line connecting the two keypoints is used as the drawing path for the entire camera path. If the input is greater than two keypoints, the entire camera path is determined using Bezier curve fitting. For details, please refer to the camera path drawing method provided in Example 1 above.
[0073] After drawing the entire camera path, the server uses the image rendering engine to select the corresponding output mode according to the camera motion parameters and image quantity related parameters, and renders the scene along the entire camera path to generate a rendered image.
[0074] When the output mode based on frameCount is selected and the input frameCount is greater than the number of key points, the frameCount is evenly distributed to each path curve interval, under the premise of ensuring that the key point corresponds to a frame of image, and the number of image frames corresponding to each path curve interval is further evenly divided, that is, the time [0,1] corresponding to each path curve interval is evenly divided to interpolate to obtain multiple time points, and then the data values of the path curve expression at multiple time points are obtained. According to the data value, the camera two-dimensional coordinates corresponding to the rendered image at each time point can be obtained. Combined with the camera height when in use and the mapping relationship between three-dimensional coordinates and two-dimensional coordinates, the camera three-dimensional coordinates corresponding to each frame of image are determined. Then, the image rendering engine renders the scene according to the camera three-dimensional coordinates and the pitch angle during camera movement to obtain a rendered image.
[0075] When the output mode based on frameCount is selected and the input frameCount is less than or equal to the number of key points, the image is rendered at each key point. That is, the three-dimensional coordinates of the camera corresponding to each key point are determined according to the two-dimensional coordinates of each key point, combined with the height of the camera when in use and the mapping relationship between three-dimensional coordinates and two-dimensional coordinates. Then, the image rendering engine renders the scene according to the three-dimensional coordinates of the camera corresponding to each key point and the pitch angle of the camera during movement. When the camera moves to the key point reached by frameCount, the rendering is stopped, which reasonably simulates the real scene.
[0076] When the output mode based on speed+fps is selected, the time required for each frame of image is calculated according to fps, and the distance between the two frames is determined by combining the speed between the two adjacent frames. The camera's two-dimensional coordinates corresponding to each frame of image are confirmed in real time along the entire camera path. The camera's three-dimensional coordinates corresponding to each frame of image are determined by combining the camera's height during use and the mapping relationship between three-dimensional coordinates and two-dimensional coordinates. When the image rendering engine renders according to the camera's three-dimensional coordinates, starting from the first frame of image, the length of the trajectory traversed by the camera's movement is accumulated. When the accumulated trajectory length is less than the length of the entire camera path, the camera continues to move along the camera path; when the accumulated trajectory length is equal to the length of the entire camera path, the current frame rendered image is retained and the camera stops moving; when the accumulated trajectory length is greater than the length of the entire camera path, the current frame rendered image is discarded and the camera stops moving.
[0077] Therefore, in the speed+fps output mode, the entire camera path length needs to be estimated. When the number of keypoints is two, the camera path is fitted with a straight line, and the entire camera path length is the distance between the two keypoints. When the number of keypoints is greater than two, the entire camera path length is determined by discrete integration of each third-order Bezier curve segment corresponding to the entire camera path. The specific process includes: performing discrete integration on each Bezier curve segment (the discrete granularity is adjustable as a parameter), interpolating each Bezier curve segment as many times as possible within CPU resource performance, and using adjacent interpolated values as the endpoints of a line segment. In this way, each Bezier curve segment can be approximated with as many line segments as possible. The distance between each line segment can be calculated based on the interpolated points, so the length of each Bezier curve segment can be estimated. The length of the entire trajectory can be obtained by summing the lengths of all Bezier curve segments.
[0078] In an embodiment, when the distance between the camera two-dimensional coordinate corresponding to each frame image and the adjacent key point along the path is less than a preset threshold according to the camera motion speed, the camera two-dimensional coordinate is updated to the adjacent key point.
[0079] In the interactive system provided by the embodiment, the user can see the RGB rendering of a top-down perspective of one or more scenes in a visual interface, and set the key points of the trajectory on the top view. The client transmits the key frame information to the server. The server determines the path fitting algorithm to be used based on the number of key frames, and combines the other parameter configurations of the camera movement process selected by the user, and finally gives a picture that meets the user's expectations, which is rendered by the camera during the movement of the custom trajectory. In this way, the complex calculation formula is configured in a visual way, which lowers the threshold for using the software. By setting a small number of trajectory key points, a complete camera motion trajectory can be drawn, which simplifies the trajectory drawing process and improves operational efficiency. At the same time, the drawing results on the server side can achieve the trajectory data that meets the delivery production standards, ensuring the high precision of the output results. Using the top view of the scene as the background for design makes the shooting settings closer to the production environment.
[0080] The specific implementation methods described above provide a detailed description of the technical solutions and beneficial effects of the present invention. It should be understood that the above is only the most preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, supplements and equivalent substitutions made within the scope of the principles of the present invention should be included in the scope of protection of the present invention.
Claims
1. A rendering interaction system based on camera path drawing, comprising a client and a server connected in communication, characterized in that: The client receives user input information, including key points with input timestamps, camera motion parameters, and image quantity related parameters, and transmits the input information to the server; The server draws the entire camera path based on the received key points, including: when there are two key points, the line connecting the two key points is used as the drawn camera trajectory; when there are more than two key points, a third-order Bezier curve is fitted for every two key points according to the timestamp order of the key points to obtain a path curve between every two key points and passing through the key points, and at the same time, ensure that two adjacent path curves are second-order continuous at the common key point to obtain the entire camera path; The image rendering engine on the server side performs scene rendering along the entire camera path according to the camera motion parameters and parameters related to the number of images to generate a rendered image, wherein the camera motion parameters include the camera motion speed, the camera motion height, and the camera motion pitch angle; the parameters related to the number of images include: the total number of image frames and the rendering frame rate; When the parameter related to the number of received images is the total number of image frames, and it is greater than the number of key points, the total number of image frames is evenly distributed to each path curve on the premise that one frame of image corresponds to each key point, and the number of image frames corresponding to each path curve is evenly divided again to obtain the camera two-dimensional coordinates corresponding to each frame of image, and the three-dimensional coordinates of the camera corresponding to each frame of image are determined in combination with the height of the camera when in use and the mapping relationship between the three-dimensional coordinates and the two-dimensional coordinates. Then, the image rendering engine performs scene rendering according to the camera three-dimensional coordinates and the pitch angle of the camera when in motion to obtain a rendered image; When receiving the key points input by the user, the client displays the camera path formed by the key points in real time.
2. The rendering interaction system based on camera path drawing according to claim 1, characterized in that: When the total number of image frames received is less than or equal to the number of key points, the image is rendered at each key point, that is, the three-dimensional coordinates of the camera corresponding to each key point are determined based on the two-dimensional coordinates of each key point, combined with the height of the camera when in use and the mapping relationship from three-dimensional coordinates to two-dimensional coordinates. Then, the image rendering engine performs scene rendering based on the three-dimensional coordinates of the camera corresponding to each key point and the pitch angle of the camera when in motion to obtain a rendered image.
3. The rendering interaction system based on camera path drawing according to claim 1, characterized in that: When the parameter related to the number of received images is the rendering frame rate, the time required for each frame of image is calculated based on the rendering frame rate, and the camera two-dimensional coordinates corresponding to each frame of image according to the camera movement speed are confirmed in real time along the entire camera path. The camera three-dimensional coordinates corresponding to each frame of image are determined in combination with the camera height when in use and the mapping relationship between three-dimensional coordinates and two-dimensional coordinates. When the image rendering engine renders according to the camera three-dimensional coordinates, starting from the first frame of image, the length of the trajectory traversed by the camera movement is accumulated. When the accumulated trajectory length is less than the length of the entire camera path, the camera continues to move along the camera path; when the accumulated trajectory length is equal to the length of the entire camera path, the current frame rendered image is retained and the camera stops moving; when the accumulated trajectory length is greater than the length of the entire camera path, the current frame rendered image is discarded and the camera stops moving.
4. The rendering interaction system based on camera path drawing according to claim 3, characterized in that: When the distance between the camera two-dimensional coordinates corresponding to each frame image according to the camera motion speed and the adjacent key points along the path is less than a preset threshold, the camera two-dimensional coordinates are updated to the adjacent key points.
5. The rendering interaction system based on camera path drawing according to claim 1, characterized in that: When a third-order Bezier curve is used to fit m-1 path curves to m key points, for two adjacent path curves, the first-order derivative and second-order derivative of the previous path curve at the end point are required to be equal to the first-order derivative and second-order derivative of the next path curve at the starting point. At the same time, the second-order derivative of the first path curve of the entire camera path formed by the m-1 path curves is required to be 0 at the starting point, and the second-order derivative of the m-1 path curve at the end point is required to be 0. Combining the two-dimensional coordinates of the m key points and the third-order Bezier curve expression corresponding to each path curve, each path curve is determined by matrix solution, and then the entire camera path is obtained.
6. The rendering interaction system based on camera path drawing according to claim 1, characterized in that: Also includes: The server determines the length of the entire camera path by discretely integrating each segment of the third-order Bezier curve corresponding to the entire camera path.
7. The rendering interaction system based on camera path drawing according to claim 1, characterized in that: The client provides the user with an input information editing interface, and provides the functions of inserting, deleting and modifying key points of the camera path.
8. The rendering interaction system based on camera path drawing according to claim 1, characterized in that: The camera path determination method formed by the key points displayed in real time by the client includes: using a third-order Bezier curve for fitting, during fitting, taking three consecutive key points as a fitting unit, and then determining two intermediate control points of the third-order Bezier curve based on the three key points in each fitting unit and the principle of similar triangles, and then, combining the first and last two key points of each fitting unit to calculate and determine the third-order Bezier curve, that is, the corresponding fitting path.
