Three-dimensional model video frame generation method and three-dimensional model rendering mode determination method and device

Abstract

The invention relates to the technical field of three-dimensional model processing, provides a three-dimensional model video frame generation method and a three-dimensional model rendering mode determination method and device, and can reduce the complexity of generating video frames by a server in a server rendering mode. A server side of the application can execute the following steps: sampling a cursor movement track corresponding to model operation based on a user side to obtain a two-dimensional plane position sequence of a cursor; according to the two-dimensional plane position sequence and the mapping relation between the two-dimensional plane coordinate system and the three-dimensional space coordinate system, a three-dimensional space position sequence of the cursor is obtained; performing projection transformation on the target three-dimensional model according to the three-dimensional space position sequence to obtain a mapping image sequence of the target three-dimensional model in a two-dimensional plane; and carrying out superposition processing on the mapping image sequence to obtain a video frame sequence, corresponding to the model operation, of the target three-dimensional model.

Description

Technical Field

[0001] The present application relates to the technical field of three-dimensional model processing, and in particular to a method and device for generating a three-dimensional model video frame, a method and device for determining a three-dimensional model rendering mode, a computer device, a storage medium and a computer program product. Background Art

[0002] The 3D model rendering method includes server-side rendering, which mainly places complex logic such as graphics rendering on the server. The video frames of the 3D model generated by the server are fed back to the user end, and the user end can play and display them.

[0003] However, users will operate the 3D model, such as rotating the 3D model to view other parts of the 3D model. In this interactive scenario, how to reduce the complexity of the server-side generated video frames in the server-side rendering mode to improve the real-time interactive response is a technical problem that needs to be solved. Summary of the invention

[0004] Based on this, it is necessary to provide a method and device for generating three-dimensional model video frames, a method and device for determining a three-dimensional model rendering method, a computer device, a storage medium and a computer program product to address the above technical problems.

[0005] The present application provides a method for generating a three-dimensional model video frame, the steps of the method are performed by a server, and the method comprises the following steps:

[0006] Based on the sampling processing of the cursor movement trajectory corresponding to the model operation by the user end, a two-dimensional plane position sequence of the cursor is obtained;

[0007] Obtaining the three-dimensional space position sequence of the cursor according to the two-dimensional plane position sequence and the mapping relationship between the two-dimensional plane coordinate system and the three-dimensional space coordinate system;

[0008] According to the three-dimensional space position sequence, a target three-dimensional model is projected and transformed to obtain a mapping image sequence of the target three-dimensional model in a two-dimensional plane;

[0009] The mapping image sequence is overlaid to obtain a video frame sequence of the target three-dimensional model corresponding to the model operation.

[0010] In one embodiment, according to the three-dimensional space position sequence, a projection transformation is performed on the target three-dimensional model to obtain a mapping image sequence of the target three-dimensional model in a two-dimensional plane, including:

[0011] For each three-dimensional space position in the three-dimensional space position sequence, starting from the point light source of the three-dimensional space position, projecting the target three-dimensional model to obtain a mapping image corresponding to the three-dimensional space position;

[0012] The mapping image sequence is obtained according to the mapping images corresponding to each three-dimensional space position.

[0013] In one embodiment, the mapping image sequence is overlaid to obtain a video frame sequence of the target three-dimensional model corresponding to the model operation, including:

[0014] Superimposing at least two consecutive frames of mapping images in the mapping image sequence to obtain a synthetic video frame sequence;

[0015] The synthesized video frame sequence is inserted into the mapped image sequence to obtain the video frame sequence.

[0016] In one embodiment, inserting the synthesized video frame sequence into the mapped image sequence to obtain the video frame sequence includes:

[0017] For the first type of synthesized video frame in the synthesized video frame sequence, the following steps are performed: determining two consecutive mapping images used to superimpose the first type of synthesized video frame, and determining an insertion position of the first type of synthesized video frame at a position between the two consecutive mapping images;

[0018] For the second type of composite video frames in the composite video frame sequence, the following steps are performed: determining the three consecutive mapping images used to form the second type of composite video frames by superimposing, and determining the insertion position of the second type of composite video frames at the position between the first two mapping images of the three consecutive mapping images;

[0019] The synthetic video frame sequence is inserted into the mapping image sequence according to respective insertion positions of the first type of synthetic video frames and the second type of synthetic video frames to obtain the video frame sequence.

[0020] In one embodiment, the method further comprises:

[0021] Obtaining the page size of the model display page of the user terminal;

[0022] Obtaining a scaling transformation matrix between a two-dimensional plane coordinate system and a three-dimensional space coordinate system according to the page size and a model size of the target three-dimensional model;

[0023] According to the scaling transformation matrix, a mapping relationship between the two-dimensional plane coordinate system and the three-dimensional space coordinate system is obtained.

[0024] In one embodiment, the method further comprises:

[0025] Acquire a two-dimensional plane coordinate system related to the model display page of the user terminal;

[0026] Obtaining a non-scaling transformation matrix between the two-dimensional plane coordinate system and the three-dimensional space coordinate system according to the positions of the reference points in the two-dimensional plane coordinate system and the three-dimensional space coordinate system, respectively; the non-scaling transformation matrix corresponds to at least one of rotation and translation;

[0027] According to the non-scaling transformation matrix, a mapping relationship between the two-dimensional plane coordinate system and the three-dimensional space coordinate system is obtained.

[0028] The present application provides a device for generating a three-dimensional model video frame, the device is provided on a server, and the device includes the following modules:

[0029] A cursor position acquisition module, used for obtaining a two-dimensional plane position sequence of the cursor based on the sampling processing of the cursor movement trajectory corresponding to the model operation by the user end;

[0030] A position conversion module, used for obtaining the three-dimensional space position sequence of the cursor according to the two-dimensional plane position sequence and the mapping relationship between the two-dimensional plane coordinate system and the three-dimensional space coordinate system;

[0031] A mapping image acquisition module, used to perform a projection transformation on the target three-dimensional model according to the three-dimensional space position sequence to obtain a mapping image sequence of the target three-dimensional model in a two-dimensional plane;

[0032] The video frame sequence processing module is used to perform superposition processing on the mapping image sequence to obtain a video frame sequence of the target three-dimensional model corresponding to the model operation.

[0033] The present application provides a method for determining a 3D model rendering mode, wherein the steps of the method are performed by a server, and the method comprises the following steps:

[0034] Obtaining the hardware configuration information of the user terminal and the model identifier of the target three-dimensional model;

[0035] Determining the standard hardware configuration information of the target three-dimensional model according to the correspondence between the model identifier and the preset model identifier and the standard hardware configuration information;

[0036] If the hardware configuration information does not meet the standard hardware configuration information, the rendering mode of the target three-dimensional model is determined to be a server-side rendering mode; the server-side rendering mode includes the steps of any embodiment of the above-mentioned three-dimensional model video frame generation method.

[0037] In one embodiment, the method further comprises:

[0038] If the hardware configuration information satisfies the standard hardware configuration information, determining that the rendering mode of the target three-dimensional model is a user-side rendering mode;

[0039] Sending a user-side rendering instruction to the user side; the user-side rendering instruction is used to instruct the user side to request the target three-dimensional model of a first precision from the cache server and to request the target three-dimensional model of a second precision from the local side; the first precision is lower than the second precision.

[0040] The present application provides a device for determining a 3D model rendering mode, the device being arranged on a server, and comprising the following modules:

[0041] An information acquisition module is used to acquire the hardware configuration information of the user terminal and the model identification of the target three-dimensional model;

[0042] A standard hardware configuration determination module, used to determine the standard hardware configuration information of the target three-dimensional model according to the correspondence between the model identifier and the preset model identifier and the standard hardware configuration information;

[0043] A rendering mode determination module is used to determine that the rendering mode of the target three-dimensional model is a server-side rendering mode if the hardware configuration information does not meet the standard hardware configuration information; the server-side rendering mode includes the steps of any embodiment of the above-mentioned three-dimensional model video frame generation method.

[0044] The present application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the above method.

[0045] The present application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to execute the above method.

[0046] The present application provides a computer program product, on which a computer program is stored, and the computer program is executed by a processor to perform the above method.

[0047] In the method for generating three-dimensional model video frames provided by the present application, based on the sampling processing of the cursor movement trajectory corresponding to the model operation by the user end, a two-dimensional plane position sequence of the cursor is obtained; according to the two-dimensional plane position sequence and the mapping relationship between the two-dimensional plane coordinate system and the three-dimensional space coordinate system, a three-dimensional space position sequence of the cursor is obtained; according to the three-dimensional space position sequence, a projection transformation is performed on the target three-dimensional model to obtain a mapping image sequence of the target three-dimensional model in the two-dimensional plane; the mapping image sequence is overlaid to obtain a video frame sequence of the target three-dimensional model corresponding to the model operation; the time required for the process of mapping the image by spatial transformation is longer than the time required for the image overlay operation. Therefore, the solution provided by the present application can reduce the number of times video frames are generated by spatial transformation while ensuring a certain number of video frames, simplify the complexity of generating video frames on the server side, and improve the real-time performance of interactive response. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0049] Figure 1 A diagram of an application environment of a method for generating a three-dimensional model video frame in an embodiment;

[0050] Figure 2 A schematic diagram of a flow chart of a method for generating a 3D model video frame in an embodiment;

[0051] FIG3 (a) is a schematic diagram of a cursor movement trajectory in one embodiment;

[0052] FIG3( b ) is a schematic diagram of sampling of a cursor movement trajectory in one embodiment;

[0053] Figure 4 A schematic diagram of mapping image superposition in one embodiment;

[0054] FIG5( a ) is a schematic diagram of selecting a mapping image for superposition by window in one embodiment;

[0055] FIG5( b ) is a schematic diagram of a video frame sequence in one embodiment;

[0056] Figure 6 A schematic diagram of a flow chart of a method for determining a 3D model rendering method in one embodiment;

[0057] Figure 7 A schematic diagram of determining a rendering method of a three-dimensional model in one embodiment;

[0058] Figure 8 A schematic diagram of a process flow of a user-side rendering method in one embodiment;

[0059] Fig. 9 A schematic diagram of a process of a server-side rendering method in one embodiment;

[0060] Fig.10 is a structural block diagram of a device for generating a three-dimensional model video frame in one embodiment;

[0061] Fig.11 It is a structural block diagram of a device for determining a 3D model rendering method in one embodiment;

[0062] Fig.12 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0063] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0064] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments.

[0065] The method for generating a three-dimensional model video frame and the method for determining a three-dimensional model rendering method provided in this application can be applied to a digital twin system built based on a B / S (Browser / Server) model. This type of system is introduced below.

[0066] Digital twin technology is a cutting-edge technology with a surreal concept, which aims to build a digital "clone" for objects in real life. The technology includes two key elements: one is the requirement for rapid data collection and real-time transmission; the other is the requirement to establish a three-dimensional model that can accurately reflect the characteristics of the subject. Thanks to the rapid development and application of big data technology and 5G communication technology, data collection and real-time transmission capabilities have been greatly improved. As for model construction, there are now many three-dimensional model construction tools. Based on the constructed three-dimensional model and the collected real data, a digital twin system can be built through relevant technical means.

[0067] The digital twin system is built based on the C / S (Client / Server) model, and users need to install the application software before use. This type of system has the advantage of high reliability. However, the volume of the three-dimensional model is mostly large, and it will take up a lot of client storage space after packaging and installation. In addition, the rendering of high-precision models also has high requirements for the configuration of the computer equipment used by the user (hereinafter referred to as the user end). The promotion of the concept of cloud computing has led to more and more applications starting to go to the cloud, and building digital twin systems based on the B / S model has gradually become mainstream.

[0068] There are two ways to render 3D models in the B / S mode: user-side rendering and server-side rendering.

[0069] When the performance of the client cannot meet the minimum hardware configuration requirements for client-side rendering, server-side rendering can be used to render using the server-side hardware resources. Server-side rendering is performed by playing video streams. By obtaining the cursor position information when the user manipulates the model display page on the client side, combined with the relevant algorithms on the server side, a video frame sequence is dynamically generated and returned to the model display page on the client side for playback and display.

[0070] In order to increase the complexity of the server-side video frame generation and improve the real-time interactive response, the present application provides a method for generating a three-dimensional model video frame, the method comprising steps that can be executed by the server, and the server can perform data transmission with the user end, such as Figure 1 shown.

[0071] The method includes Figure 2 Steps shown:

[0072] Step S201 , based on the sampling processing of the cursor movement trajectory corresponding to the model operation by the user end, a two-dimensional plane position sequence of the cursor is obtained.

[0073] The 3D model to be rendered may be referred to as a target 3D model.

[0074] The user end can initialize a streaming media player instance in a specified area of ​​the page (the instance is mainly used to play videos and does not require tools such as a progress bar), thereby generating a corresponding model display page. The model display page can be a Web (World Wide Web) page created by the user end's browser.

[0075] After generating the model display page, the user end can add a transparent canvas on the top layer, and the transparent canvas can record the cursor position.

[0076] When the user does not perform any model operation on the target 3D model, the cursor remains at the same position and does not move, and the target 3D model is in a static state; at this time, the server can add a front view image of the target 3D model in a static state to the streaming media communication link at a set time interval (such as 40ms); the user can obtain the front view image from the streaming media communication link and display the front view image on the model display page to ensure the normal display of the target 3D model in a static state.

[0077] When the user performs model operations on the target 3D model, the model operations are achieved by dragging the cursor, so the cursor will move, and the cursor position on the transparent canvas constitutes the cursor movement trajectory, as shown in Figure 3 (a). The user end can sample the cursor movement trajectory at a set sampling time interval (such as 100ms), thereby obtaining the cursor position sequence on the transparent canvas, as shown in Figure 3 (b); since the transparent canvas is a canvas superimposed on the model display page, the transparent canvas, like the model display page, belongs to a two-dimensional plane, so the cursor position sequence on the transparent canvas can be called the cursor's two-dimensional plane position sequence.

[0078] The client can send the collected two-dimensional plane position sequence of the cursor to the server through the WebSocket channel. The client can send the two-dimensional plane position of the cursor to the server through the WebSocket channel every time it collects a two-dimensional plane position of the cursor. The client can also send multiple collected two-dimensional plane positions to the server through the WebSocket channel after multiple collections. In this way, the server can obtain the two-dimensional plane position sequence of the cursor. WebSocket is a full-duplex communication protocol based on TCP, which allows two-way communication between the client and the server on a single TCP connection. The full name of TCP in English is Transmission Control Protocol, and its Chinese name is Transmission Control Protocol.

[0079] Step S202 , obtaining a three-dimensional space position sequence of the cursor according to the two-dimensional plane position sequence and the mapping relationship between the two-dimensional plane coordinate system and the three-dimensional space coordinate system.

[0080] The server can obtain the cursor position sequence in three-dimensional space (which can be called three-dimensional space position sequence) based on the two-dimensional plane position sequence of the cursor transmitted by the user end and the mapping relationship between the two-dimensional plane coordinate system and the three-dimensional space coordinate system set in advance.

[0081] Step S203 , performing a projection transformation on the target three-dimensional model according to the three-dimensional space position sequence, to obtain a mapping image sequence of the target three-dimensional model in a two-dimensional plane.

[0082] After the server obtains the cursor's three-dimensional spatial position sequence, it can perform a projection transformation on the target three-dimensional model according to each three-dimensional spatial position in the three-dimensional spatial position sequence, thereby obtaining a mapping image of the target three-dimensional model corresponding to the three-dimensional spatial position in a two-dimensional plane. According to the mapping image corresponding to each three-dimensional spatial position, a mapping image sequence is obtained.

[0083] If a complete model operation of the user takes 1 second, then based on the sampling time interval of 100ms, the server only needs to generate 10 mapping images during the entire process.

[0084] Step S204: performing superposition processing on the mapped image sequence to obtain a video frame sequence of the target three-dimensional model corresponding to the model operation.

[0085] The server can use the mapping image as one of the video frames, and can also superimpose the mapping images, and the superimposed image is also used as one of the video frames. Due to the persistence characteristics of the human eye, objects will produce a "smear" phenomenon during rapid and continuous movement. This application can use this characteristic to superimpose at least two consecutive mapping images in the mapping image sequence to generate an image with a "smear" effect. Figure 4 In the example shown, the first mapping image and the second mapping image are two consecutive frames of images. The first mapping image and the second mapping image are superimposed to generate an image with a "smear" effect. Then, the image with the "smear" effect can be inserted between the two consecutive frames of mapping images to obtain a video frame sequence of the target 3D model, which presents the movement of the target 3D model, and the presented movement matches the aforementioned model operation.

[0086] In the above-mentioned method for generating three-dimensional model video frames, based on the sampling processing of the cursor movement trajectory corresponding to the model operation by the user end, a two-dimensional plane position sequence of the cursor is obtained; according to the two-dimensional plane position sequence and the mapping relationship between the two-dimensional plane coordinate system and the three-dimensional space coordinate system, the three-dimensional space position sequence of the cursor is obtained; according to the three-dimensional space position sequence, a projection transformation is performed on the target three-dimensional model to obtain a mapping image sequence of the target three-dimensional model in the two-dimensional plane; the mapping image sequence is superimposed to obtain a video frame sequence of the target three-dimensional model corresponding to the model operation; the time required for the process of mapping the image by spatial transformation is longer than the time required for the image superposition operation. Therefore, the solution provided by the present application can reduce the number of times video frames are generated by spatial transformation while ensuring a certain number of video frames, simplify the complexity of generating video frames on the server side, and improve the real-time performance of interactive response.

[0087] In one embodiment, a projection transformation is performed on the target three-dimensional model according to a three-dimensional space position sequence to obtain a mapping image sequence of the target three-dimensional model in a two-dimensional plane, which may specifically include: for each three-dimensional space position in the three-dimensional space position sequence, starting from the point light source of the three-dimensional space position, the target three-dimensional model is projected to obtain a mapping image corresponding to the three-dimensional space position; based on the mapping images corresponding to each three-dimensional space position, a mapping image sequence is obtained.

[0088] The present application uses each three-dimensional space position as a point light source, emits light from the point light source at the three-dimensional space position, and projects the target three-dimensional model, thereby obtaining a mapping image of the target three-dimensional model on a two-dimensional plane corresponding to the three-dimensional space position. In this way, a mapping image of the target three-dimensional model on a two-dimensional plane corresponding to each three-dimensional space position can be obtained, thereby forming a mapping image sequence.

[0089] In this embodiment, considering that the user will focus on the cursor when operating the target three-dimensional model through the cursor, the target three-dimensional model is projected from the point light source of the three-dimensional space position of the cursor, so as to obtain a mapping image that is more in line with the user's observation perspective.

[0090] In one embodiment, the mapping image sequence is overlaid to obtain a video frame sequence of the target three-dimensional model corresponding to the model operation, which may specifically include: overlaying at least two consecutive mapping image frames in the mapping image sequence to obtain a composite video frame sequence; inserting the composite video frame sequence into the mapping image sequence to obtain a video frame sequence.

[0091] Exemplarily, the mapping images in the mapping image sequence include {N1, N2, N3, N4}, where N1 and N2 belong to two consecutive frames of mapping images, N2 and N3 belong to two consecutive frames of mapping images, and N3 and N4 belong to two consecutive frames of mapping images. In this embodiment, at least two consecutive frames of mapping images can be superimposed according to a certain weight to obtain a plurality of synthetic video frames, and the plurality of synthetic video frames constitute a synthetic video frame sequence.

[0092] In this embodiment, considering that the offset position of the target three-dimensional model is not obvious in a short time, at least two consecutive mapping image frames are superimposed to obtain a synthetic video frame sequence. The synthetic video frames in the synthetic video frame sequence are pictures with a "smear" effect, which can make the movement of the target three-dimensional model presented in the video frame sequence smoother and in line with the actual movement situation.

[0093] In one embodiment, inserting a synthetic video frame sequence into a mapped image sequence to obtain a video frame sequence includes:

[0094] For the first type of synthetic video frames in the synthetic video frame sequence, the following steps are performed: determine the two consecutive mapping images used to superimpose the first type of synthetic video frames, and determine the insertion position of the first type of synthetic video frames at the position between the two consecutive mapping images; for the second type of synthetic video frames in the synthetic video frame sequence, the following steps are performed: determine the three consecutive mapping images used to superimpose the second type of synthetic video frames, and determine the insertion position of the second type of synthetic video frames at the position between the first two mapping images of the three consecutive mapping images; insert the synthetic video frame sequence into the mapping image sequence according to the respective insertion positions of the first type of synthetic video frames and the second type of synthetic video frames to obtain a video frame sequence.

[0095] For example, two consecutive frames of mapped images may be superimposed according to a certain weight to obtain a corresponding composite video frame, and three consecutive frames of mapped images may be superimposed according to a certain weight to obtain a corresponding composite video frame. In order to distinguish, the composite video frame superimposed by two consecutive frames of mapped images is called a first type of composite video frame, and the composite video frame superimposed by three consecutive frames of mapped images is called a second type of composite video frame.

[0096] The first type of synthetic video frame G obtained by superimposing N1 and N2 1_2 Take this as an example:

[0097] G 1_2 It is obtained by superimposing N1 and N2 according to a certain weight. G can be determined at the position between N1 and N2. 1_2 Furthermore, the position of G between N1 and N2 can be determined according to the weights assigned to N1 and N2. 1_2 Insertion position.

[0098] For example, if the weights assigned to N1 and N2 are the same (for example, both are 0.5), then G 1_2 The closeness to N1 is equal to G 1_2 The proximity to N2, therefore, can determine G 1_2 The insertion position is the middle position between N1 and N2.

[0099] For another example, if the weights assigned to N1 and N2 are different, and the weight assigned to N1 is less than the weight assigned to N2, then G 1_2 The proximity to N1 is less than G 1_2 The proximity to N2, therefore, can determine G 1_2 The insertion position is between N1 and N2 and close to N2.

[0100] According to the above method, N2 and N3 can be superimposed to obtain the first type of synthetic video frame G 2_3 . G 2_3The insertion position of can be the middle position between N2 and N3, or the position between N2 and N3 and close to N2, or the position between N2 and N3 and close to N3.

[0101] According to the above method, N3 and N4 can be superimposed to obtain the first type of synthetic video frame G 3_4 . G 3_4 The insertion position of may be an intermediate position between N3 and N4, or a position between N3 and N4 and close to N3, or a position between N3 and N4 and close to N4.

[0102] The first type of synthetic video frame G 1_2 , G 2_3 and G 3_4 After inserting the mapping image sequence {N1, N2, N3, N4}, we get {N1, G 1_2 , N2, G 2_3 , N3, G 3_4 , N4}.

[0103] The second type of synthetic video frame P obtained by superimposing N1, N2 and N3 1_2_3 Take this as an example:

[0104] P 1_2_3 It is obtained by superimposing N1, N2 and N3 according to a certain weight; N1, N2 and N3 belong to three consecutive frames of mapping images, among which N1 and N2 belong to the first two frames of mapping images, and P can be determined at the position between N1 and N2 1_2_3 Insertion position.

[0105] The second type of synthetic video frame P obtained by superimposing N2, N3 and N4 2_3_4 Take this as an example:

[0106] P 2_3_4 It is obtained by superimposing N2, N3 and N4 according to a certain weight; N2, N3 and N4 belong to three consecutive frames of mapping images, among which N2 and N3 belong to the first two frames of mapping images, and P can be determined at the position between N2 and N3 2_3_4 Insertion position.

[0107] Furthermore, if G 2_3 The insertion position is the middle position between N2 and N3, which can be adjusted according to the weights assigned to N2 and N3 and G 2_3 The insertion position of P is determined between N2 and N3. 2_3_4 For example, if the weight assigned to N2 is greater than the weight assigned to N3, then N2 and G 2_3 The position between 2_3_4 The insertion position; if the weight assigned to N2 is less than the weight assigned to N3, then G 2_3The position between P and N3 is determined 2_3_4 If the weight assigned to N2 is equal to the weight assigned to N3, then N2 and G 2_3 between G 2_3 and N3, randomly select P 2_3_4 Insertion position.

[0108] If G 1_2 =0.5×N1+0.5×N2, G 2_3 =0.5×N2+0.5×N3, G 3_4 =0.5×N3+0.5×N3,P 1_2_3 =0.25×N1+0.5×N2+0.25×N3, P 2_3_4 =0.25×N2+0.5×N3+0.25×N4, then the video frame sequence formed after insertion is {N1, G 1_2 , P 1_2_3 , N2, G 2_3 , P 2_3_4 , N3, G 3_4 , N4}.

[0109] In this embodiment, for the first type of synthetic video frames synthesized based on two consecutive frames of mapping images, the insertion position of the first type of synthetic video frames is determined at the position between the two consecutive frames of mapping images used for synthesis, so that the movement of the target three-dimensional model presented by the video frame sequence is smoother; for the second type of synthetic video frames synthesized based on three consecutive frames of mapping images, the insertion position of the first type of synthetic video frames is determined at the position between the first two frames of mapping images. Compared with the first type of synthetic video frames, the second type of synthetic video frames carry more information of multiple frames of mapping images, and therefore, the movement smoothness of the target three-dimensional model presented by the video frame sequence can be further improved.

[0110] In some scenarios, the server may set a window group, which includes a first window and a second window, wherein the first window is used to select two consecutive frames of mapping images for superposition, and the second window is used to select three consecutive frames of mapping images for superposition, wherein the starting position of the first window sliding may be a position corresponding to N2, and the starting position of the second window sliding may be a position corresponding to N1, as shown in FIG5(a).

[0111] Referring again to FIG5(a), in the scenario where the mapping images are generated sequentially, after the server generates N1, N2, and N3 sequentially, it uses the window group shown in FIG5(a) to perform the first slide, selects N2 and N3 and superimposes them with weights of 0.5 and 0.5 to obtain G 2_3 , select N1, N2 and N3 and superimpose them with weights of 0.25, 0.5 and 0.25 to obtain P 1_2_3, thus obtaining the video frame sequence corresponding to the first sliding {N1, P 1_2_3 , N2, G 2_3 , N3}. The server appends all the video frames in the video frame sequence corresponding to the first slide to the streaming communication link, so that the user end can obtain {N1, P 1_2_3 , N2, G 2_3 , N3}, as shown in Figure 5 (b), after the server completes the append corresponding to the first slide, it can release the first frame (that is, N1) in the second window during the first slide and the corresponding synthetic video frame (that is, G 2_3 and P 1_2_3 ) to free up some cache resources.

[0112] Next, the server generates N4. At this time, the control window group slides for the second time, selects N3 and N4 and superimposes them with weights of 0.5 and 0.5 to obtain G 3_4 , select N2, N3 and N4 and superimpose them with weights of 0.25, 0.5 and 0.25 to obtain P 2_3_4 , thus obtaining the video frame sequence corresponding to the second slide {N2, G 2_3 , P 2_3_4 , N3, G 3_4 , N4}, and P 2_3_4 , G 3_4 and N4 are added to the streaming media communication link, so that the user end can obtain P 2_3_4 , G 3_4 After the server completes the append corresponding to the second slide, it can release the first frame (that is, N2) in the second window during the second slide and the corresponding synthesized video frame (P 2_3_4 and G 3_4 ) to free up some cache resources.

[0113] When the same projection transformation method is used to generate mapping images for the same three-dimensional model, each mapping image has the same pixel resolution and does not require alignment processing when superimposed.

[0114] According to the above method, 4 video frames (40fps) can be output every 100ms, which is higher than the minimum frame rate (24fps) that the human eye can distinguish when the video is obviously stuttered, and can meet the requirements of smooth playback. At the same time, the above method reduces the number of projection transformation operations, and the real-time performance of the video can also be guaranteed.

[0115] When the user pauses the model operation, the server can fetch the last mapping image currently cached, and append the mapping image to the streaming media communication link at a frequency of once every 40 ms.

[0116] In one embodiment, the method for generating a three-dimensional model video frame provided by the present application also includes: obtaining the page size of the model display page of the user end; obtaining a scaling transformation matrix between the two-dimensional plane coordinate system and the three-dimensional space coordinate system based on the page size and the model size of the target three-dimensional model; and obtaining a mapping relationship between the two-dimensional plane coordinate system and the three-dimensional space coordinate system based on the scaling transformation matrix.

[0117] Considering that the model size of the target three-dimensional model stored on the server is fixed, while the page size of the model display page on the user side is variable, in order to make the target three-dimensional model presented in the video frame match the model display page, the user side can collect the page size of the model display page at a set interval and send the page size to the server; the server obtains the scaling transformation matrix between the two-dimensional plane coordinate system and the three-dimensional space coordinate system based on the page size and the model size of the target three-dimensional model; based on the scaling transformation matrix, the mapping relationship between the two-dimensional plane coordinate system and the three-dimensional space coordinate system is obtained.

[0118] In this embodiment, a scaling transformation matrix between the two-dimensional plane coordinate system and the three-dimensional space coordinate system is obtained based on the page size of the model display page and the model size of the target three-dimensional model, thereby obtaining a mapping relationship, so that the target three-dimensional model presented in the video frame is more matched with the model display page, avoiding the mismatch caused by the change of the page size of the model display page.

[0119] In one embodiment, the method for generating a three-dimensional model video frame provided by the present application also includes: obtaining a two-dimensional plane coordinate system related to the model display page of the user terminal; obtaining a non-scaling transformation matrix between the two-dimensional plane coordinate system and the three-dimensional space coordinate system according to the positions of the reference points in the two-dimensional plane coordinate system and the three-dimensional space coordinate system respectively; the non-scaling transformation matrix corresponds to at least one of rotation and translation; according to the non-scaling transformation matrix, obtaining a mapping relationship between the two-dimensional plane coordinate system and the three-dimensional space coordinate system.

[0120] The upper left corner of the model display page can be preset as the origin of the two-dimensional plane coordinate system, and the width and height directions of the model display page are the x and y axes respectively, so the two-dimensional plane coordinate system is related to the model display page. According to the position of the reference point in the two-dimensional plane coordinate system and the position of the reference point in the three-dimensional space coordinate system, a non-scaling transformation matrix between the two-dimensional plane coordinate system and the three-dimensional space coordinate system can be obtained, such as a rotation transformation matrix and a translation transformation matrix, and the mapping relationship between the two-dimensional plane coordinate system and the three-dimensional space coordinate system is obtained according to the non-scaling transformation matrix.

[0121] In this embodiment, since the two-dimensional plane coordinate system is related to the model display page, the obtained mapping relationship can be used in subsequent cursor position conversion to avoid errors. In addition, based on the position of the reference point in the two-dimensional plane coordinate system and the position of the reference point in the three-dimensional space coordinate system, at least one of the rotation transformation matrix and the translation transformation matrix can be obtained to improve the accuracy of the cursor position conversion.

[0122] The present application also provides a method for determining a 3D model rendering method, by which it is possible to determine whether to use a server-side rendering method. If so, the server is triggered to execute the steps described in the above-mentioned 3D model video frame generation method embodiment. The steps of the method for determining a 3D model rendering method can be executed by the server, including: Figure 6 The steps shown are:

[0123] Step S601, obtaining the hardware configuration information of the user terminal and the model identifier of the target three-dimensional model.

[0124] When the page is initialized, the client can obtain the hardware configuration information of the local machine (such as CPU information and graphics card information); the client can also send the model identifier of the target 3D model to the server. The full name of CPU is Central Processing Unit, and its Chinese name is ‌Central Processing Unit.

[0125] Step S602: determining the standard hardware configuration information of the target three-dimensional model according to the correspondence between the model identifier and the preset model identifier and the standard hardware configuration information.

[0126] After receiving the model identifier of the target three-dimensional model, the server can determine the standard hardware configuration information of the target three-dimensional model from the correspondence between the model identifier and the standard hardware configuration information stored in the database.

[0127] Step S603: if the hardware configuration information does not meet the standard hardware configuration information, determining that the rendering mode of the target three-dimensional model is a server-side rendering mode.

[0128] After determining the standard hardware configuration information of the target three-dimensional model, the server can determine whether the hardware configuration information of the user side meets the standard hardware configuration information. If not, the rendering mode of the target three-dimensional model is determined to be the server rendering mode, and the server rendering mode includes the steps introduced in the above-mentioned embodiment of the method for generating three-dimensional model video frames.

[0129] The server can send a server rendering instruction to the client. The client responds to the server rendering instruction and establishes a streaming communication link with the server for transmitting video frames. After the streaming communication link is successfully established, the client can also establish a WebSocket channel with the server for transmitting the two-dimensional plane position of the cursor.

[0130] In this embodiment, the rendering mode of the target three-dimensional model is determined according to the hardware configuration information of the user end, such as the user end rendering mode or the server end rendering mode. Compared with fixing a certain rendering mode, this embodiment is more flexible.

[0131] In one embodiment, the method for determining the rendering mode of a three-dimensional model provided in the present application also includes: if the hardware configuration information meets the standard hardware configuration information, determining that the rendering mode of the target three-dimensional model is a user-end rendering mode; sending a user-end rendering instruction to the user end; the user-end rendering instruction is used to instruct the user end to request a target three-dimensional model of a first precision from a cache server and to request a target three-dimensional model of a second precision from the local end; the first precision is lower than the second precision.

[0132] If the hardware configuration information of the user terminal meets the standard hardware configuration information of the target three-dimensional model, the rendering mode of the target three-dimensional model is determined to be the user terminal rendering mode. At this time, the server can send a user terminal rendering instruction to the user terminal. In response to the user terminal rendering instruction, the user terminal can first request a low-precision target three-dimensional model (that is, a target three-dimensional model of the first precision) from the cache server. After the user terminal obtains the low-precision target three-dimensional model, it can display the low-precision target three-dimensional model and request a high-precision target three-dimensional model (that is, a target three-dimensional model of the second precision) from the server. In the process of displaying the low-precision target three-dimensional model, the user terminal can cache the pose information of the low-precision target three-dimensional model after each model operation. When receiving the high-precision target three-dimensional model, the high-precision target three-dimensional model is displayed according to the pose information after the latest cached model operation, and the low-precision target three-dimensional model is not displayed.

[0133] In this embodiment, the user terminal preferentially loads and renders the low-precision target 3D model, and replaces the low-precision target 3D model after the high-precision target 3D model is requested and loaded, thereby reducing the page loading time. In addition, the user terminal displays the high-precision target 3D model according to the pose information of the latest cached model after the operation is completed, so as to complete the seamless switching from the low-precision target 3D model to the high-precision target 3D model, and significantly improve the accuracy and texture of the target 3D model.

[0134] In order to better understand the above method, an application example of the method for determining the 3D model rendering method and the method for generating the 3D model video frame of the present application is described in detail below.

[0135] This application example involves a user end, a cache server and a server end, wherein the user end may include a model display page.

[0136] Figure 7 The process flow for determining the rendering mode is shown in FIG. Figure 7, the user can access the model display page through the browser. When the model display page is accessed, the user end can obtain the hardware configuration information of the local machine, and send the hardware configuration information of the local machine and the model identifier of the target three-dimensional model to the server end. After receiving the model identifier of the target three-dimensional model, the server end can send the model identifier of the target three-dimensional model to the database. The database determines the model identifier that matches the model identifier of the target three-dimensional model in the correspondence between the stored model identifiers and the standard hardware configuration information, and uses the standard hardware configuration information corresponding to the model identifier as the standard hardware configuration information of the target three-dimensional model and feeds it back to the server end. The server end compares the standard hardware configuration information of the target three-dimensional model with the hardware configuration information sent by the user end, determines the rendering method based on the comparison result, and feeds the rendering method back to the user end.

[0137] Figure 8 The processing flow of the user-side rendering method is shown. If the rendering method is the user-side rendering method, refer to Figure 8 , the user end can request a low-precision target 3D model from the cache server. The cache server sends the low-precision target 3D model to the user end. The user end can initialize an instance of the low-precision target 3D model to display the low-precision target 3D model. The user end can also send a request to the server, which is used to obtain a high-precision target 3D model. After sending the request, the user end can start caching the pose information of the low-precision target 3D model after each model operation. When the user end receives the high-precision target 3D model fed back by the server, it can initialize an instance of the high-precision target 3D model according to the pose information after the latest cached model operation is completed to display the high-precision target 3D model, and destroy the instance of the low-precision target 3D model.

[0138] Fig. 9 The processing flow of the server-side rendering method is shown. If the rendering method is server-side rendering, refer to Fig. 9 , the client and the server establish a streaming media communication link for transmitting video frames. After the streaming media communication link is successfully established, the client and the server establish a WebSocket channel for transmitting the two-dimensional plane position of the cursor. After the WebSocket channel is successfully established, the server can add the front view of the target model in a static state to the streaming media communication link at a frequency of 40ms, thereby transmitting the video stream to the client. The client displays the target three-dimensional model based on the front view of the target model in a static state in the video stream.

[0139] The user operates the target 3D model by dragging the cursor. The client can read the position of the cursor on the transparent canvas every 100ms, obtain the 2D plane position of the cursor, and send the 2D plane position of the cursor to the server.

[0140] The server generates corresponding video frames according to the two-dimensional plane position of the cursor and appends them to the streaming communication link for display by the user.

[0141] In this application example, the rendering method of the target three-dimensional model (user-side rendering method or server-side rendering method) is determined according to the hardware configuration information of the user end. Compared with a fixed rendering method, this embodiment is more flexible. In the user-side rendering method, the user end preferentially loads and renders the low-precision target three-dimensional model, and replaces the low-precision target three-dimensional model when the high-precision target three-dimensional model is requested and loaded, so as to reduce the page loading time. In addition, the user end displays the high-precision target three-dimensional model according to the pose information after the operation of the latest cached model is completed, so as to complete the seamless switching from the low-precision target three-dimensional model to the high-precision target three-dimensional model, and significantly improve the accuracy and texture of the target three-dimensional model. The server-side rendering method can reduce the number of times video frames are generated through spatial transformation while ensuring a certain number of video frames, simplify the complexity of the server-side generation of video frames, and improve the real-time nature of the interactive response.

[0142] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0143] Based on the same inventive concept, the embodiment of the present application also provides a 3D model video frame generation device for implementing the above-mentioned 3D model video frame generation method. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above-mentioned method, so the specific limitations in the embodiments of the device for generating one or more 3D model video frames provided below can refer to the limitations of the 3D model video frame generation method in the above text, and will not be repeated here.

[0144] In one embodiment, Fig.10 As shown, a device for generating a three-dimensional model video frame is provided, the device is arranged on a server, and the device includes the following modules:

[0145] The cursor position acquisition module 1001 is used to obtain a two-dimensional plane position sequence of the cursor based on the sampling processing of the cursor movement trajectory corresponding to the model operation by the user end;

[0146] A position conversion module 1002 is used to obtain the three-dimensional space position sequence of the cursor according to the two-dimensional plane position sequence and the mapping relationship between the two-dimensional plane coordinate system and the three-dimensional space coordinate system;

[0147] A mapping image acquisition module 1003 is used to perform a projection transformation on the target three-dimensional model according to the three-dimensional space position sequence to obtain a mapping image sequence of the target three-dimensional model in a two-dimensional plane;

[0148] The video frame sequence processing module 1004 is used to perform superposition processing on the mapping image sequence to obtain a video frame sequence of the target three-dimensional model corresponding to the model operation.

[0149] In one embodiment, the mapping image acquisition module 1003 is also used to: for each three-dimensional spatial position in the three-dimensional spatial position sequence, starting from the point light source of the three-dimensional spatial position, project the target three-dimensional model to obtain a mapping image corresponding to the three-dimensional spatial position; and obtain the mapping image sequence based on the mapping images corresponding to each three-dimensional spatial position.

[0150] In one embodiment, the video frame sequence processing module 1004 is further used to: superimpose at least two consecutive mapping image frames in the mapping image sequence to obtain a composite video frame sequence; and insert the composite video frame sequence into the mapping image sequence to obtain the video frame sequence.

[0151] In one embodiment, the video frame sequence processing module 1004 is also used to: for a first type of synthetic video frames in the synthetic video frame sequence, perform the following steps: determine two consecutive mapping images used to superimpose to form the first type of synthetic video frames, and determine the insertion position of the first type of synthetic video frames at the position between the two consecutive mapping images; for a second type of synthetic video frames in the synthetic video frame sequence, perform the following steps: determine three consecutive mapping images used to superimpose to form the second type of synthetic video frames, and determine the insertion position of the second type of synthetic video frames at the position between the first two mapping images of the three consecutive mapping images; insert the synthetic video frame sequence into the mapping image sequence according to the respective insertion positions of the first type of synthetic video frames and the second type of synthetic video frames to obtain the video frame sequence.

[0152] In one embodiment, the device also includes a mapping relationship acquisition module, which is used to: obtain the page size of the model display page of the user terminal; obtain a scaling transformation matrix between the two-dimensional plane coordinate system and the three-dimensional space coordinate system based on the page size and the model size of the target three-dimensional model; and obtain a mapping relationship between the two-dimensional plane coordinate system and the three-dimensional space coordinate system based on the scaling transformation matrix.

[0153] In one embodiment, the device also includes a mapping relationship acquisition module, which is used to: acquire a two-dimensional plane coordinate system related to the model display page of the user terminal; obtain a non-scaling transformation matrix between the two-dimensional plane coordinate system and the three-dimensional space coordinate system according to the positions of the reference points in the two-dimensional plane coordinate system and the three-dimensional space coordinate system respectively; the non-scaling transformation matrix corresponds to at least one of rotation and translation; based on the non-scaling transformation matrix, obtain the mapping relationship between the two-dimensional plane coordinate system and the three-dimensional space coordinate system.

[0154] Each module in the above-mentioned 3D model video frame generation device can be implemented in whole or in part by software, hardware or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0155] Based on the same inventive concept, the embodiment of the present application also provides a device for determining a 3D model rendering mode for implementing the method for determining a 3D model rendering mode involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the embodiments of one or more devices for determining a 3D model rendering mode provided below can refer to the limitations of the method for determining a 3D model rendering mode above, and will not be repeated here.

[0156] In one embodiment, Fig.11 As shown, a device for determining a 3D model rendering mode is provided, the device is arranged on a server, and the device includes the following modules:

[0157] The information acquisition module 1101 is used to acquire the hardware configuration information of the user terminal and the model identification of the target three-dimensional model;

[0158] A standard hardware configuration determination module 1102 is used to determine the standard hardware configuration information of the target three-dimensional model according to the correspondence between the model identifier and the preset model identifier and the standard hardware configuration information;

[0159] The rendering mode determination module 1103 is used to determine that the rendering mode of the target 3D model is a server-side rendering mode if the hardware configuration information does not meet the standard hardware configuration information; the server-side rendering mode includes the steps of any embodiment of the above-mentioned 3D model video frame generation method.

[0160] In one embodiment, the device also includes a user-side rendering processing module, which is used to: if the hardware configuration information meets the standard hardware configuration information, determine that the rendering mode of the target three-dimensional model is the user-side rendering mode; send a user-side rendering instruction to the user terminal; the user-side rendering instruction is used to instruct the user terminal to request the target three-dimensional model of the first precision from the cache server and to request the target three-dimensional model of the second precision from the local terminal; the first precision is lower than the second precision.

[0161] Each module in the above-mentioned device for determining the rendering mode of a three-dimensional model can be implemented in whole or in part by software, hardware, or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each of the above modules.

[0162] In an exemplary embodiment, a computer device is provided. The server of the above embodiment can be implemented by the computer device. The internal structure diagram of the computer device can be as follows: Fig.12 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data involved in the above method. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for generating a three-dimensional model video frame or a method for determining the three-dimensional model rendering method is implemented.

[0163] Those skilled in the art will understand that Fig.12The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0164] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps in the above-mentioned various method embodiments when executing the computer program.

[0165] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0166] In one embodiment, a computer program product is provided, on which a computer program is stored, and the computer program is used by a processor to execute the steps in the above-mentioned various method embodiments.

[0167] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0168] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0169] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0170] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for generating a three-dimensional model video frame, characterized in that: The steps of the method are performed by the server, and the method includes the following steps: Based on the sampling processing of the cursor movement trajectory corresponding to the model operation by the user end, a two-dimensional plane position sequence of the cursor is obtained; Obtaining the three-dimensional space position sequence of the cursor according to the two-dimensional plane position sequence and the mapping relationship between the two-dimensional plane coordinate system and the three-dimensional space coordinate system; According to the three-dimensional space position sequence, a target three-dimensional model is projected and transformed to obtain a mapping image sequence of the target three-dimensional model in a two-dimensional plane; The mapping image sequence is overlaid to obtain a video frame sequence of the target three-dimensional model corresponding to the model operation.

2. The method according to claim 1, characterized in that According to the three-dimensional space position sequence, a target three-dimensional model is projected and transformed to obtain a mapping image sequence of the target three-dimensional model in a two-dimensional plane, including: For each three-dimensional space position in the three-dimensional space position sequence, starting from the point light source of the three-dimensional space position, projecting the target three-dimensional model to obtain a mapping image corresponding to the three-dimensional space position; The mapping image sequence is obtained according to the mapping images corresponding to each three-dimensional space position.

3. The method according to claim 1, characterized in that The mapping image sequence is overlaid to obtain a video frame sequence of the target three-dimensional model corresponding to the model operation, including: Superimposing at least two consecutive frames of mapping images in the mapping image sequence to obtain a synthetic video frame sequence; The synthesized video frame sequence is inserted into the mapped image sequence to obtain the video frame sequence.

4. The method according to claim 3, characterized in that Inserting the synthesized video frame sequence into the mapped image sequence to obtain the video frame sequence comprises: For the first type of synthesized video frame in the synthesized video frame sequence, the following steps are performed: determining two consecutive mapping images used to superimpose the first type of synthesized video frame, and determining an insertion position of the first type of synthesized video frame at a position between the two consecutive mapping images; For the second type of composite video frames in the composite video frame sequence, the following steps are performed: determining the three consecutive mapping images used to form the second type of composite video frames by superimposing, and determining the insertion position of the second type of composite video frames at the position between the first two mapping images of the three consecutive mapping images; The synthetic video frame sequence is inserted into the mapping image sequence according to respective insertion positions of the first type of synthetic video frames and the second type of synthetic video frames to obtain the video frame sequence.

5. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: Obtaining the page size of the model display page of the user terminal; Obtaining a scaling transformation matrix between a two-dimensional plane coordinate system and a three-dimensional space coordinate system according to the page size and a model size of the target three-dimensional model; According to the scaling transformation matrix, a mapping relationship between the two-dimensional plane coordinate system and the three-dimensional space coordinate system is obtained.

6. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: Acquire a two-dimensional plane coordinate system related to the model display page of the user terminal; Obtaining a non-scaling transformation matrix between the two-dimensional plane coordinate system and the three-dimensional space coordinate system according to the positions of the reference points in the two-dimensional plane coordinate system and the three-dimensional space coordinate system, respectively; the non-scaling transformation matrix corresponds to at least one of rotation and translation; According to the non-scaling transformation matrix, a mapping relationship between the two-dimensional plane coordinate system and the three-dimensional space coordinate system is obtained.

7. A method for determining a 3D model rendering method, characterized in that: The steps of the method are performed by the server, and the method includes the following steps: Obtaining the hardware configuration information of the user terminal and the model identifier of the target three-dimensional model; Determining the standard hardware configuration information of the target three-dimensional model according to the correspondence between the model identifier and the preset model identifier and the standard hardware configuration information; If the hardware configuration information does not meet the standard hardware configuration information, the rendering mode of the target three-dimensional model is determined to be a server-side rendering mode; the server-side rendering mode includes the steps of the method as described in any one of claims 1 to 6.

8. The method according to claim 7, characterized in that The method further comprises: If the hardware configuration information satisfies the standard hardware configuration information, determining that the rendering mode of the target three-dimensional model is a user-side rendering mode; Sending a user-side rendering instruction to the user side; the user-side rendering instruction is used to instruct the user side to request the target three-dimensional model of a first precision from the cache server and to request the target three-dimensional model of a second precision from the local side; the first precision is lower than the second precision.

9. A device for generating a three-dimensional model video frame, characterized in that: The device is provided on the server side, and comprises the following modules: A cursor position acquisition module, used for obtaining a two-dimensional plane position sequence of the cursor based on the sampling processing of the cursor movement trajectory corresponding to the model operation by the user end; A position conversion module, used for obtaining the three-dimensional space position sequence of the cursor according to the two-dimensional plane position sequence and the mapping relationship between the two-dimensional plane coordinate system and the three-dimensional space coordinate system; A mapping image acquisition module, used to perform a projection transformation on the target three-dimensional model according to the three-dimensional space position sequence to obtain a mapping image sequence of the target three-dimensional model in a two-dimensional plane; The video frame sequence processing module is used to perform superposition processing on the mapping image sequence to obtain a video frame sequence of the target three-dimensional model corresponding to the model operation.

10. A device for determining a 3D model rendering method, characterized in that: The device is provided on the server side, and comprises the following modules: An information acquisition module is used to acquire the hardware configuration information of the user terminal and the model identification of the target three-dimensional model; A standard hardware configuration determination module, used to determine the standard hardware configuration information of the target three-dimensional model according to the correspondence between the model identifier and the preset model identifier and the standard hardware configuration information; A rendering mode determination module is used to determine that the rendering mode of the target three-dimensional model is a server-side rendering mode if the hardware configuration information does not meet the standard hardware configuration information; the server-side rendering mode includes the steps of the method as described in any one of claims 1 to 6.

Images