A three-dimensional model construction method, device and storage medium

Abstract

The present application provides a three-dimensional model construction method, device and storage medium, which relate to the technical field of three-dimensional model, and are used to solve the technical problem that three-dimensional models cannot be flexibly constructed. The method includes: after obtaining the regional position information of the area to be rendered under the initial three-dimensional model, and the video data of the scene corresponding to the initial three-dimensional model, the target texture position information matching the regional position information of the area to be rendered can be determined from the texture position information of multiple video frame data included in the video data, and the target texel in the target video frame data to which the target texture position information belongs is obtained, and the target texel is rendered into the area to be rendered to obtain a target three-dimensional model including the video data of the area to be rendered. Wherein, each video frame data includes multiple texture position information and multiple texels corresponding to the multiple texture position information one by one. The present application improves the flexibility of constructing three-dimensional models.

Description

Technical Field

[0001] The present application belongs to the technical field of three-dimensional models, and in particular, relates to a three-dimensional model construction method, device and storage medium. Background Art

[0002] As 3D model construction technology becomes increasingly sophisticated, a fusion technology of 3D models and video data has emerged in related technologies. This technology can dynamically display the content of the video data in the 3D model by restoring the video data to the 3D model.

[0003] With the development of related technologies, some users have put forward more refined and diversified requirements for dynamically displaying video data in 3D models. For example, the traffic control department hopes to focus on displaying road-related content in the 3D model, and the airport management department hopes to focus on displaying waterway-related content in the 3D model. However, related technologies are difficult to meet the needs of these users. Summary of the invention

[0004] The present application provides a three-dimensional model construction method, device and storage medium, which are used to solve the problem that a three-dimensional model cannot be flexibly constructed.

[0005] In order to achieve the above purpose, this application adopts the following technical solutions:

[0006] In a first aspect, a three-dimensional model construction method is provided, comprising: after obtaining the regional position information of the area to be rendered under the initial three-dimensional model and the video data of the scene corresponding to the initial three-dimensional model, target texture position information matching the regional position information of the area to be rendered can be determined from the texture position information of multiple video frame data included in the video data, and the target texel in the target video frame data to which the target texture position information belongs is obtained, and the target texel is rendered into the area to be rendered, so as to obtain the target three-dimensional model including the video data of the area to be rendered. Each video frame data includes multiple texture position information and multiple texels corresponding to the multiple texture position information one by one.

[0007] Optionally, the method of determining target texture position information matching the region position information of the region to be rendered from the texture position information of the plurality of video frame data specifically comprises: determining a first coordinate of the region position information of the region to be rendered in a preset coordinate system. Determining a second coordinate of the texture position information of the plurality of video frame data in the preset coordinate system. From the second coordinates, selecting the same coordinate as the first coordinate as the target coordinate. Determining the texture position information corresponding to the target coordinate as the target texture position information.

[0008] Optionally, the region position information includes the region coordinates of the region to be rendered in the local coordinate system of the initial three-dimensional model. The method for determining the first coordinates of the region position information of the region to be rendered in the preset coordinate system specifically includes: determining a first matrix set based on the positional relationship between the local coordinate system and the preset coordinate system. The first matrix set is used to convert the coordinates in the local coordinate system into the coordinates in the preset coordinate system. The first matrix set includes: a first model matrix, a first view matrix and a first projection matrix. Based on the first matrix set, the region coordinates are converted into the first coordinates.

[0009] Optionally, the video data also includes: the focal length and attitude angle of the shooting device that shoots the video data. The multiple texture position information includes the texture coordinates of the multiple texels in the camera coordinate system of the shooting device. The method for determining the second coordinates of the texture position information of multiple video frame data in the preset coordinate system specifically includes: determining the second matrix set based on the focal length, attitude angle, the positional relationship between the camera coordinate system and the preset coordinate system. The second matrix set is used to convert the coordinates in the camera coordinate system into the coordinates in the preset coordinate system. The second matrix set includes: a second model matrix, a second view matrix and a second projection matrix. Based on the second matrix set, the texture coordinates are converted into the second coordinates.

[0010] In a second aspect, a three-dimensional model construction device is provided, including: an acquisition unit and a determination unit. The acquisition unit is used to acquire the regional position information of the area to be rendered under the initial three-dimensional model. The acquisition unit is also used to acquire video data of the scene corresponding to the initial three-dimensional model. The video data includes a plurality of video frame data. Each video frame data includes a plurality of texture position information and a plurality of texels corresponding to the plurality of texture position information one by one. The determination unit is used to determine the target texture position information that matches the regional position information of the area to be rendered from the texture position information of the plurality of video frame data. The acquisition unit is also used to acquire the target texel in the target video frame data from the target video frame data to which the target texture position information belongs, and render the target texel into the area to be rendered, so as to obtain the target three-dimensional model including the video data of the area to be rendered.

[0011] Optionally, the determination unit is specifically used to: determine a first coordinate of the area position information of the area to be rendered in a preset coordinate system. Determine a second coordinate of the texture position information of the plurality of video frame data in the preset coordinate system. From the second coordinates, select a coordinate that is the same as the first coordinate to determine as the target coordinate. Determine the texture position information corresponding to the target coordinate as the target texture position information.

[0012] Optionally, the region position information includes the region coordinates of the region to be rendered in the local coordinate system of the initial three-dimensional model. The determination unit is specifically used to: determine a first matrix set based on a positional relationship between the local coordinate system and the preset coordinate system. The first matrix set is used to convert coordinates in the local coordinate system into coordinates in the preset coordinate system. The first matrix set includes: a first model matrix, a first view matrix, and a first projection matrix. Based on the first matrix set, the region coordinates are converted into first coordinates.

[0013] Optionally, the video data also includes: the focal length and attitude angle of the shooting device that shoots the video data. The multiple texture position information includes the texture coordinates of the multiple texels in the camera coordinate system of the shooting device. The determination unit is specifically used to: determine the second matrix set based on the focal length, attitude angle, the positional relationship between the camera coordinate system and the preset coordinate system. The second matrix set is used to convert the coordinates in the camera coordinate system into the coordinates in the preset coordinate system. The second matrix set includes: a second model matrix, a second view matrix, and a second projection matrix. Based on the second matrix set, the texture coordinates are converted into second coordinates.

[0014] In a third aspect, a three-dimensional model construction device is provided, comprising a memory and a processor; the memory is used to store computer execution instructions, and the processor is connected to the memory through a bus; when the three-dimensional model construction device is running, the processor executes the computer execution instructions stored in the memory, so that the three-dimensional model construction device performs the three-dimensional model construction method as in the first aspect.

[0015] The three-dimensional model building device can be a network device, or a part of a network device, such as a chip system in a network device. The chip system is used to support the network device to implement the functions involved in the first aspect and any possible implementation thereof, for example, receiving, determining, and diverting the data and / or information involved in the above-mentioned data processing method. The chip system includes a chip, and may also include other discrete devices or circuit structures.

[0016] In a fourth aspect, a computer-readable storage medium is provided, characterized in that the computer-readable storage medium includes computer execution instructions, and when the computer execution instructions are run on a computer, the computer executes the three-dimensional model construction method as in the first aspect.

[0017] It should be noted that the above-mentioned computer instructions may be stored in whole or in part on a first computer-readable storage medium. The first computer-readable storage medium may be packaged together with the processor of the 3D model building device, or may be packaged separately from the processor of the 3D model building device, and this application does not limit this.

[0018] In this application, the name of the above-mentioned 3D model building device does not limit the device or functional module itself. In actual implementation, these devices or functional modules may appear with other names. As long as the functions of each device or functional module are similar to those of this application, they belong to the scope of this application and its equivalent technologies.

[0019] These and other aspects of the present application will become more apparent from the following description.

[0020] The technical solution provided by this application brings at least the following beneficial effects:

[0021] Based on any of the above aspects, in the present application, after obtaining the regional position information of the area to be rendered under the initial three-dimensional model and the video data of the scene corresponding to the initial three-dimensional model, since the target texture position information matching the regional position information of the area to be rendered can be determined from the texture position information of the multiple video frame data included in the video data, the target texel in the target video frame data can be obtained from the target video frame data to which the target texture position information belongs, and the target texel is rendered into the area to be rendered, so as to obtain the target three-dimensional model including the video data of the area to be rendered. Among them, each video frame data includes multiple texture position information and multiple texels corresponding to the multiple texture position information one by one. In this way, compared with the prior art that directly superimposes all the video data on the three-dimensional model for dynamic display, the present application can make the target three-dimensional model dynamically display the user-specified area (i.e., the area to be rendered) in the video data, thereby improving the flexibility of constructing the three-dimensional model. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 A schematic diagram of the structure of a three-dimensional model building system provided in an embodiment of the present application;

[0023] Figure 2 A schematic diagram of a hardware structure of a communication device provided in an embodiment of the present application;

[0024] Figure 3 A schematic diagram of another hardware structure of a communication device provided in an embodiment of the present application;

[0025] Figure 4 A schematic diagram of a three-dimensional model construction method provided in an embodiment of the present application Figure 1 ;

[0026] Figure 5 An interface display diagram of an area to be rendered provided in an embodiment of the present application;

[0027] Figure 6 A schematic diagram of an interface of video frame data provided in an embodiment of the present application;

[0028] Figure 7 A schematic diagram of a three-dimensional model construction method provided in an embodiment of the present application Figure 2 ;

[0029] Figure 8 A schematic diagram of a three-dimensional model construction method provided in an embodiment of the present application Figure 3 ;

[0030] Fig. 9 A schematic diagram of a three-dimensional model construction method provided in an embodiment of the present application Figure 4 ;

[0031] Fig.10 A schematic diagram of the structure of a three-dimensional model building device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0033] It should be noted that, in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0034] In order to facilitate a clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, words such as "first" and "second" are used to distinguish between identical or similar items with substantially the same functions and effects. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and order of execution.

[0035] In addition, the terms "including" and "having" in the embodiments of the present application and the accompanying drawings are not exclusive. For example, a process, method, system, product or device including a series of steps or modules is not limited to the listed steps or modules, and may also include steps or modules that are not listed.

[0036] As described in the background technology, with the development of relevant technologies, some users have put forward more refined and diversified requirements for dynamically displaying video data in three-dimensional models. For example, the traffic control department hopes to focus on displaying road-related content in the three-dimensional model, and the airport management department hopes to focus on displaying waterway-related content in the three-dimensional model. However, relevant technologies are difficult to meet the needs of these users.

[0037] In view of the above problems, an embodiment of the present application provides a three-dimensional model construction method. After obtaining the regional position information of the area to be rendered under the initial three-dimensional model and the video data of the scene corresponding to the initial three-dimensional model, since the target texture position information matching the regional position information of the area to be rendered can be determined from the texture position information of the multiple video frame data included in the video data, the target texel in the target video frame data to which the target texture position information belongs can be obtained, and the target texel is rendered into the area to be rendered, so as to obtain the target three-dimensional model including the video data of the area to be rendered. Wherein, each video frame data includes multiple texture position information and multiple texels corresponding to the multiple texture position information one by one.

[0038] In this way, compared with the prior art that directly superimposes all video data on the three-dimensional model for dynamic display, the present application can enable the target three-dimensional model to dynamically display the user-specified area in the video data (i.e., the area to be rendered), thereby improving the flexibility of constructing the three-dimensional model.

[0039] The three-dimensional model construction method is suitable for a three-dimensional model construction system. Figure 1 FIG. 1 shows a structure of the three-dimensional model building system 100. Figure 1 As shown, the 3D model construction system 100 includes: a 3D model construction device 101 and a server 102. The 3D model construction device 101 and the server 102 are connected in communication.

[0040] Optional, Figure 1 The three-dimensional model building device 101 may be a functional module in the server 102, or may be an electronic device independently provided from the server 102 and used for building a three-dimensional model.

[0041] Optionally, when the three-dimensional model construction device 101 is an electronic device that is independently set up from the server 102 and is used to construct a three-dimensional model, the electronic device can be a terminal. The terminal is a device that provides voice and / or data connectivity to a user, a handheld device with a wireless connection function, or other processing devices connected to a wireless modem. The wireless terminal can communicate with one or more core networks via a radio access network (RAN). The wireless terminal can be a mobile terminal, such as a computer with a mobile terminal, or a portable, pocket-sized, handheld, or computer-built-in mobile device that exchanges language and / or data with a wireless access network, such as a mobile phone, a tablet computer, a laptop computer, a netbook, or a personal digital assistant (PDA). The embodiments of the present application do not impose any restrictions on this.

[0042] Optionally, when the 3D model building device 101 is an electronic device that is independently provided with the server 102 and is used to build the 3D model, the electronic device may also be a server. When the electronic device is a server, the electronic device may be a functional module inside the server 102 for building the 3D model, or may be an independent server that is independently provided with the server 102, and this application does not specifically limit this.

[0043] For ease of understanding, this application is mainly explained by taking the three-dimensional model building device 101 and the server 102 as independent settings.

[0044] It is easy to understand that when the 3D model construction device 101 is a functional module inside the server 102, the communication method between the 3D model construction device 101 and the server 102 is the communication between the internal modules of the server. In this case, the communication process between the two is the same as "the communication process between the two when the 3D model construction device 101 and the server 102 are independent of each other".

[0045] The server 102 is used to collect and provide the 3D model building device 101 with the area position information of the area to be rendered under the initial 3D model or the video data of the scene corresponding to the initial 3D model.

[0046] Optionally, the server 102 may be a single server, or may be a server cluster composed of multiple servers. In some implementations, the server cluster may also be a distributed cluster. The present application embodiment does not impose any limitation on this.

[0047] The basic hardware structures of the 3D model building device 101 and the server 102 in the 3D model building system 100 are similar, both of which include Figure 2 or Figure 3The components included in the communication device shown are described below. Figure 2 and Figure 3 Taking the communication device shown as an example, the hardware structure of the three-dimensional model building device 101 and the server 102 is introduced.

[0048] like Figure 2 , which is a schematic diagram of a hardware structure of a communication device provided in an embodiment of the present application. The communication device includes a processor 21, a memory 22, a communication interface 23, and a bus 24. The processor 21, the memory 22, and the communication interface 23 can be connected via a bus 24.

[0049] The processor 21 is the control center of the communication device, which can be a processor or a general term for multiple processing elements. For example, the processor 21 can be a general-purpose central processing unit (CPU) or other general-purpose processors. Among them, the general-purpose processor can be a microprocessor or any conventional processor.

[0050] As an embodiment, the processor 21 may include one or more CPUs, such as Figure 2 CPU0 and CPU1 are shown in the figure.

[0051] The memory 22 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, an electrically erasable programmable read-only memory (EEPROM), a disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0052] In a possible implementation, the memory 22 may exist independently of the processor 21, and the memory 22 may be connected to the processor 21 via a bus 24 to store instructions or program codes. When the processor 21 calls and executes the instructions or program codes stored in the memory 22, the three-dimensional model construction method provided in the following embodiments of the present application can be implemented.

[0053] In the embodiment of the present application, the software programs stored in the memory 22 of the 3D model construction device 101 and the server 102 are different, so the functions implemented by the 3D model construction device 101 and the server 102 are different. The functions performed by each device will be described in conjunction with the following flowchart.

[0054] In another possible implementation, the memory 22 may also be integrated with the processor 21 .

[0055] The communication interface 23 is used for the communication device to connect with other devices through a communication network, and the communication network may be Ethernet, wireless access network, wireless local area network (WLAN), etc. The communication interface 23 may include a receiving unit for receiving data and a sending unit for sending data.

[0056] The bus 24 may be an industry standard architecture (ISA) bus, a peripheral component interconnect (PCI) bus, or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 2 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0057] It should be pointed out that Figure 2 The structure shown in the figure does not constitute a limitation on the communication device. Figure 2 In addition to the components shown, the communication device may include more or fewer components than shown, or combine certain components, or arrange the components differently.

[0058] Figure 3 FIG. 2 shows another hardware structure of the communication device in the embodiment of the present application. Figure 3 As shown, the communication device may include a processor 31 and a communication interface 32. The processor 31 is coupled to the communication interface 32.

[0059] The functions of the processor 31 may refer to the description of the processor 21. In addition, the processor 31 also has a storage function and can function as the memory 22 described above.

[0060] The communication interface 32 is used to provide data to the processor 31. The communication interface 32 may be an internal interface of the communication device, or an external interface of the communication device (equivalent to the communication interface 23).

[0061] It should be pointed out that Figure 2 (or Figure 3 ) does not constitute a limitation on the communication device, except Figure 2 (or Figure 3 ) In addition to the components shown in the figure, the communication device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0062] like Figure 4 The figure is a flow chart of a three-dimensional model construction method provided in an embodiment of the present application. The three-dimensional model construction method is applied to a three-dimensional model construction device, and the three-dimensional model construction device belongs to Figure 1 The three-dimensional model building system shown is composed of a three-dimensional model building device and a server. The three-dimensional model building method includes: S401-S404.

[0063] S401. The three-dimensional model building device obtains the area position information of the area to be rendered under the initial three-dimensional model.

[0064] Specifically, when it is necessary to dynamically display the area to be rendered in the initial three-dimensional model, the three-dimensional model construction device can obtain the area position information of the area to be rendered in the initial three-dimensional model from the server.

[0065] For example, Figure 5 As shown, the area to be rendered can be Figure 5 The area marked by the dotted box.

[0066] Optionally, the region position information includes region coordinates of the region to be rendered in a local coordinate system of the initial three-dimensional model.

[0067] Optionally, the server may pre-collect and store the regional position information of the area to be rendered under the initial three-dimensional model. When it is necessary to obtain the regional position information of the area to be rendered under the initial three-dimensional model, the three-dimensional model construction device may send a request message for obtaining the regional position information of the area to be rendered to the server. In response to the request message, the server may read the regional position information of the area to be rendered, and send the regional position information of the area to be rendered to the three-dimensional model construction device. Accordingly, the three-dimensional model construction device may receive the regional position information of the area to be rendered sent by the server.

[0068] Optionally, the three-dimensional model construction device may first determine the positioning data of each vertex of the area to be rendered in the initial three-dimensional model, and determine the area position information of the area to be rendered under the initial three-dimensional model based on the acquired positioning data of each vertex.

[0069] It can be understood that the three-dimensional model construction device can determine the regional coordinates of the vertex in the local coordinate system of the initial three-dimensional model based on the elevation data and longitude and latitude of the vertex, and then determine the regional position information of the area to be rendered in the initial three-dimensional model based on the regional coordinates of each vertex.

[0070] Optionally, when the 3D model building device is connected to a computer external input device (such as a mouse), the user can click the mouse to Figure 5 As shown in , each vertex of the area to be rendered under the initial three-dimensional model is determined to determine the area to be rendered. Accordingly, the three-dimensional model construction device can obtain the positioning data of each vertex clicked by the mouse. The positioning data includes data such as elevation data, longitude and latitude, and pixel value of each vertex.

[0071] Optionally, when the three-dimensional model building device determines the positioning data of each vertex of the area to be rendered, it can also automatically trigger the mouse to click on each vertex of the area to be rendered through a timing task to obtain the positioning data of each vertex of the area to be rendered.

[0072] S402: The three-dimensional model building device obtains video data of a scene corresponding to the initial three-dimensional model.

[0073] Specifically, in order to render the area to be rendered under the initial three-dimensional model, the three-dimensional model construction device can obtain video data of the scene corresponding to the initial three-dimensional model from the server.

[0074] The video data includes a plurality of video frame data. Each video frame data includes a plurality of texture position information and a plurality of texels corresponding to the plurality of texture position information. The plurality of texture position information includes texture coordinates of the plurality of texels in a camera coordinate system of a shooting device.

[0075] Optionally, the method for the 3D model building device to obtain video data of the scene corresponding to the initial 3D model can refer to the method in S401 for the 3D model building device to obtain the area position information of the area to be rendered under the initial 3D model, which will not be repeated here.

[0076] Optionally, the 3D model building device may also obtain video data of a scene corresponding to the initial 3D model through a new attribute video tag of the front-end hypertext markup language 5 (HTML5) technology.

[0077] Optionally, after acquiring the video data, the 3D model building device may determine multiple texture position information and multiple texels of each video frame data in the video data through an open source or non-open source web geographic information system (WebGis) tool.

[0078] Exemplarily, the 3D model construction device may send each point in the video frame data to a rasterization module of a graphics processing unit (GPU) based on a vertex shader of a web graphics library (WebGL) for rasterization processing.

[0079] Next, the rasterization module can form multiple triangles based on the positional relationship between the points, and then use the multiple triangles to form various faces, thereby obtaining multiple texture position information and multiple texels of the video frame data. For example, the initial video frame data can be as follows Figure 6 As shown in (a) in the figure, the video frame data after rasterization can be Figure 6 As shown in (b) in .

[0080] Optionally, the video data may be video data captured by a capturing device (such as a camera, etc.) within any preset time period.

[0081] Optionally, the format of the video data may be a moving picture experts group 4 (MPEG-4) format or a Windows Media Video (WMV) format.

[0082] S403: The 3D model construction apparatus determines target texture position information that matches the area position information of the area to be rendered from the texture position information of the plurality of video frame data.

[0083] Specifically, after obtaining the regional position information of the area to be rendered under the initial three-dimensional model and the video data of the scene corresponding to the initial three-dimensional model, in order to determine the texture pixels matching the area to be rendered to complete the rendering, the three-dimensional model construction device can convert the regional position information of the area to be rendered and the video data of the scene corresponding to the initial three-dimensional model into coordinate information under the same coordinate system.

[0084] Next, the 3D model construction device determines, in the same coordinate system, target texture position information matching the area position information of the area to be rendered from the texture position information of the plurality of video frame data.

[0085] S404. The three-dimensional model construction device obtains the target texel in the target video frame data from the target video frame data to which the target texture position information belongs, and renders the target texel into the area to be rendered to obtain a target three-dimensional model including the video data of the area to be rendered.

[0086] Specifically, after determining the target texture position information that matches the area position information of the area to be rendered, the three-dimensional model construction device can determine at least one target texture position information included in each video frame data in sequence according to the playback order of the video data, and obtain at least one target texel corresponding to the at least one target texture position information.

[0087] Next, the three-dimensional model construction device may render the at least one target texel into the area to be rendered to determine a three-dimensional model after rendering the video frame data.

[0088] After determining multiple 3D models rendered from multiple video frame data, the 3D model construction device may sort the determined multiple 3D models according to the playback order of the video data to obtain a target 3D model including the video data of the area to be rendered.

[0089] In one achievable approach, combining Figure 4 ,like Figure 7 As shown, in the above S403, the method for the 3D model construction device to determine the target texture position information matching the area position information of the area to be rendered from the texture position information of multiple video frame data specifically includes: S501-S504.

[0090] S501. A three-dimensional model building apparatus determines a first coordinate of area position information of a to-be-rendered area in a preset coordinate system.

[0091] Specifically, in order to match the target texture position information corresponding to the region position information of the area to be rendered in the preset coordinate system, the 3D model building device may determine the first coordinate of each region position information in the area to be rendered in the preset coordinate system.

[0092] Optionally, the preset coordinate system may be a world coordinate system or a coordinate system preset manually.

[0093] Optionally, when determining the first coordinates of the region position information of the region to be rendered in the preset coordinate system, the 3D model building device may convert the region coordinates in the region position information into the first coordinates in the preset coordinate system based on existing matrix transformation technology (such as model view matrix, model view projection matrix, etc.). This embodiment of the present application is not limited to this.

[0094] S502: The three-dimensional model building apparatus determines second coordinates of texture position information of a plurality of video frame data in a preset coordinate system.

[0095] Specifically, in order to match the target texture position information corresponding to the area position information of the area to be rendered in the preset coordinate system, the three-dimensional model construction device can determine the second coordinate of each texture position information of each video frame data in the preset coordinate system in multiple video frame data.

[0096] Optionally, when determining the second coordinates of the texture position information of the plurality of video frame data in the preset coordinate system, the 3D model construction device may convert the texture coordinates included in the texture position information in each video frame data into the first coordinates in the preset coordinate system based on the existing matrix transformation technology (such as the model view matrix, the model view projection matrix, etc.). This embodiment of the present application is not limited to this.

[0097] S503: The three-dimensional model building device selects coordinates that are the same as the first coordinates from the second coordinates and determines them as target coordinates.

[0098] Specifically, after determining the first coordinate corresponding to the regional position information, the three-dimensional model building device may select the coordinate that is the same as the first coordinate from the second coordinate and determine it as the target coordinate.

[0099] Exemplarily, the first coordinate is preset to (2, 5, 8), and when there is a second coordinate with the coordinate of (2, 5, 8), the second coordinate is determined as the target coordinate.

[0100] S504: The three-dimensional model building device determines the texture position information corresponding to the target coordinates as the target texture position information.

[0101] Specifically, after determining the target coordinates, the three-dimensional model construction device may determine the texture position information corresponding to the target coordinates in each target video frame data as the target texture position information, so as to determine the target texel.

[0102] In one achievable approach, combining Figure 7 ,like Figure 8 As shown, in the above S501, the method for the three-dimensional model building device to determine the first coordinate of the area position information of the area to be rendered in the preset coordinate system includes: S601-S602.

[0103] S601. The three-dimensional model building apparatus determines a first matrix set based on a positional relationship between a local coordinate system and a preset coordinate system.

[0104] Specifically, in order to convert the regional coordinates in the regional position information into the first coordinates in the preset coordinate system, the three-dimensional model construction device can determine the first matrix set based on the positional relationship between the local coordinate system of the initial three-dimensional model and the preset coordinate system.

[0105] The first matrix set is used to convert the coordinates in the local coordinate system into the coordinates in the preset coordinate system. The first matrix set includes: a first model matrix, a first view matrix and a first projection matrix.

[0106] Optionally, the positional relationship can be used to represent the coordinate point in the local coordinate system, and the required translation distance, rotation method, etc. can be converted to the preset coordinate system.

[0107] S602: The three-dimensional model building device converts the region coordinates into first coordinates based on the first matrix set.

[0108] Specifically, after determining the first matrix set, the three-dimensional model construction device may, based on the first matrix set, sequentially multiply the region coordinates with the first model matrix, the first view matrix, and the first projection matrix, thereby converting the region coordinates into the first coordinates.

[0109] In one achievable approach, combining Figure 7 ,like Fig. 9 As shown, in the above S502, the video data may also include the focal length and attitude angle of the shooting device shooting the video data. The method for the 3D model building device to determine the second coordinate of the texture position information of multiple video frame data in a preset coordinate system specifically includes: S701-S702.

[0110] S701. The three-dimensional model building device determines a second matrix set based on the focal length, the attitude angle, and the positional relationship between the camera coordinate system and the preset coordinate system.

[0111] Specifically, in order to convert the texture coordinates in the texture position information into the second coordinates in the preset coordinate system, the three-dimensional model construction device can determine the second matrix set based on the focal length, attitude angle, and positional relationship between the camera coordinate system and the preset coordinate system when the shooting device is shooting video data.

[0112] The second matrix set is used to convert the coordinates in the camera coordinate system into the coordinates in the preset coordinate system. The second matrix set includes: a second model matrix, a second view matrix and a second projection matrix.

[0113] Optionally, when determining the second matrix set, the three-dimensional model construction device may determine the positional relationship between the texture coordinates and the camera coordinate system based on the focal length and attitude angle of the shooting device when shooting the video data.

[0114] Next, the 3D model construction device may determine the second matrix set required for converting the texture coordinates to the preset coordinate system according to the positional relationship between the texture coordinates and the camera coordinate system, and the positional relationship between the camera coordinate system and the preset coordinate system.

[0115] S702: The three-dimensional model building device converts the texture coordinates into second coordinates based on the second matrix set.

[0116] Specifically, after determining the second matrix set, the three-dimensional model construction device may, based on the second matrix set, sequentially multiply the texture coordinates with the second model matrix, the second view matrix, and the second projection matrix to convert them into second coordinates.

[0117] In the present application, after obtaining the regional position information of the area to be rendered under the initial three-dimensional model and the video data of the scene corresponding to the initial three-dimensional model, the three-dimensional model construction device can determine the target texture position information matching the regional position information of the area to be rendered from the texture position information of the multiple video frame data included in the video data. Therefore, the target texel in the target video frame data to which the target texture position information belongs can be obtained, and the target texel is rendered into the area to be rendered, so as to obtain the target three-dimensional model including the video data of the area to be rendered. Each video frame data includes multiple texture position information and multiple texels corresponding to the multiple texture position information.

[0118] In this way, compared with the prior art that directly superimposes all video data on the three-dimensional model for dynamic display, the present application can enable the target three-dimensional model to dynamically display the user-specified area in the video data (i.e., the area to be rendered), thereby improving the flexibility of constructing the three-dimensional model.

[0119] The above mainly introduces the solution provided by the embodiment of the present application from the perspective of the method. In order to realize the above functions, it includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0120] The embodiment of the present application can divide the functional modules of the three-dimensional model construction device according to the above method example. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware or in the form of software functional modules. Optionally, the division of modules in the embodiment of the present application is schematic and is only a logical functional division. There may be other division methods in actual implementation.

[0121] like Fig.10As shown, it is a schematic diagram of the structure of a three-dimensional model building device provided in an embodiment of the present application. The three-dimensional model building device can be used to perform the following Figures 4 to 9 The three-dimensional model construction method shown. Fig.10 The three-dimensional model building device shown includes: an acquisition unit 801 and a determination unit 802 .

[0122] The acquisition unit 801 is used to acquire the area position information of the area to be rendered under the initial three-dimensional model. Figure 4 , the acquisition unit 801 can be used to execute S401.

[0123] The acquisition unit 801 is also used to acquire video data of the scene corresponding to the initial 3D model. The video data includes a plurality of video frame data. Each video frame data includes a plurality of texture position information and a plurality of texels corresponding to the plurality of texture position information. Figure 4 , the acquisition unit 801 can be used to execute S402.

[0124] The determination unit 802 is configured to determine target texture position information that matches the region position information of the region to be rendered from the texture position information of the plurality of video frame data. Figure 4 , the determination unit 802 can be used to execute S403.

[0125] The acquisition unit 801 is further configured to acquire a target pixel in the target video frame data from the target video frame data to which the target texture position information belongs, and render the target pixel into the area to be rendered, so as to obtain a target 3D model including the video data of the area to be rendered. Figure 4 , the acquisition unit 801 can be used to execute S404.

[0126] Optionally, the determining unit 802 is specifically configured to:

[0127] Determine the first coordinate of the area position information of the area to be rendered in the preset coordinate system. For example, Figure 7 , the determining unit 802 can be used to execute S501.

[0128] Determine the second coordinates of the texture position information of the plurality of video frame data in the preset coordinate system. For example, in combination with Figure 7 , the determination unit 802 can be used to execute S502.

[0129] From the second coordinates, select the coordinates that are the same as the first coordinates and determine them as the target coordinates. Figure 7 , the determining unit 802 can be used to execute S503.

[0130] The texture position information corresponding to the target coordinates is determined as the target texture position information. Figure 7 , the determining unit 802 can be used to execute S504.

[0131] Optionally, the region position information includes the region coordinates of the region to be rendered in the local coordinate system of the initial three-dimensional model. The determination unit 802 is specifically configured to:

[0132] Based on the positional relationship between the local coordinate system and the preset coordinate system, a first matrix set is determined. The first matrix set is used to convert the coordinates in the local coordinate system into the coordinates in the preset coordinate system. The first matrix set includes: a first model matrix, a first view matrix, and a first projection matrix. For example, in combination Figure 8 , the determining unit 802 can be used to execute S601.

[0133] Based on the first matrix set, the region coordinates are converted to the first coordinates. For example, combining Figure 8 , the determining unit 802 can be used to execute S602.

[0134] Optionally, the video data further includes: a focal length and an attitude angle of a shooting device for shooting the video data. The plurality of texture position information includes texture coordinates of the plurality of texels in a camera coordinate system of the shooting device. The determination unit 802 is specifically configured to:

[0135] Based on the focal length, the attitude angle, and the positional relationship between the camera coordinate system and the preset coordinate system, a second matrix set is determined. The second matrix set is used to convert the coordinates in the camera coordinate system into the coordinates in the preset coordinate system. The second matrix set includes: a second model matrix, a second view matrix, and a second projection matrix. For example, in combination Fig. 9 , the determining unit 802 can be used to execute S701.

[0136] Based on the second matrix set, the texture coordinates are transformed into second coordinates. For example, combining Fig. 9 , the determination unit 802 can be used to execute S702.

[0137] Those skilled in the art should be aware that in one or more of the above examples, the functions described in this application can be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. Computer-readable media include computer-readable storage media and communication media, wherein communication media include any media that facilitates the transmission of computer programs from one place to another. Storage media can be any available media that can be accessed by general or special-purpose computers.

[0138] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0139] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the modules or units is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the coupling or direct coupling or communication connection between each other shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms. The unit described as a separate component may or may not be physically separated, and the component displayed as a unit may be a physical unit or multiple physical units, that is, it may be located in one place, or it may be distributed in multiple different places. Some or all of the units can be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0140] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by any technician familiar with the technical field within the technical scope disclosed in the present application should be included in the protection scope of the present application.

Claims

1. A three-dimensional model construction method, characterized in that: include: Obtaining the area position information of the area to be rendered under the initial three-dimensional model; The region position information includes the region coordinates of the region to be rendered in the local coordinate system of the initial three-dimensional model; Acquire video data of a scene corresponding to the initial three-dimensional model; the video data includes a plurality of video frame data; each video frame data includes a plurality of texture position information and a plurality of texels corresponding to the plurality of texture position information one by one; Determining a first matrix set based on a positional relationship between the local coordinate system and a preset coordinate system; The first matrix set is used to convert the coordinates in the local coordinate system into the coordinates in the preset coordinate system; The first matrix set includes: a first model matrix, a first view matrix and a first projection matrix; Based on the first matrix set, converting the region coordinates into first coordinates in the preset coordinate system; Determine a second coordinate of the texture position information of the plurality of video frame data in the preset coordinate system; From the second coordinates, select the coordinates that are the same as the first coordinates and determine them as the target coordinates; Determine the texture position information corresponding to the target coordinates as target texture position information that matches the area position information of the area to be rendered; From the target video frame data to which the target texture position information belongs, a target texel in the target video frame data is obtained, and the target texel is rendered into the area to be rendered to obtain a target three-dimensional model including the video data of the area to be rendered.

2. The three-dimensional model construction method according to claim 1, characterized in that: The video data further includes: a focal length and an attitude angle of a shooting device that shoots the video data; the plurality of texture position information includes texture coordinates of the plurality of texels in a camera coordinate system of the shooting device; and the second coordinates of the texture position information of the plurality of video frame data in the preset coordinate system are determined, including: Based on the focal length, the attitude angle, and the positional relationship between the camera coordinate system and the preset coordinate system, a second matrix set is determined; the second matrix set is used to convert the coordinates in the camera coordinate system into the coordinates in the preset coordinate system; the second matrix set includes: a second model matrix, a second view matrix, and a second projection matrix; The texture coordinates are converted to the second coordinates based on the second set of matrices.

3. A three-dimensional model construction device, characterized in that: include: Get unit and determine unit; The acquisition unit is used to acquire the area position information of the area to be rendered under the initial three-dimensional model; The region position information includes the region coordinates of the region to be rendered in the local coordinate system of the initial three-dimensional model; The acquisition unit is further used to acquire video data of a scene corresponding to the initial three-dimensional model; the video data includes a plurality of video frame data; each video frame data includes a plurality of texture position information and a plurality of texels corresponding to the plurality of texture position information one by one; The determining unit is used to determine a first matrix set based on a positional relationship between the local coordinate system and a preset coordinate system; The first matrix set is used to convert the coordinates in the local coordinate system into the coordinates in the preset coordinate system; The first matrix set includes: a first model matrix, a first view matrix and a first projection matrix; The determining unit is further configured to convert the region coordinates into first coordinates in the preset coordinate system based on the first matrix set; The determining unit is further used to determine a second coordinate of the texture position information of the plurality of video frame data in the preset coordinate system; The determining unit is further configured to select a coordinate that is the same as the first coordinate from the second coordinates and determine it as a target coordinate; The determining unit is further used to determine the texture position information corresponding to the target coordinates as the target texture position information matching the area position information of the area to be rendered; The acquisition unit is also used to acquire target texels in the target video frame data from the target video frame data to which the target texture position information belongs, and render the target texels into the area to be rendered to obtain a target three-dimensional model including the video data of the area to be rendered.

4. The three-dimensional model construction device according to claim 3, characterized in that: The video data further includes: a focal length and a posture angle of a shooting device shooting the video data; the plurality of texture position information includes texture coordinates of the plurality of texels in a camera coordinate system of the shooting device; the determination unit is specifically used to: Based on the focal length, the attitude angle, and the positional relationship between the camera coordinate system and the preset coordinate system, a second matrix set is determined; the second matrix set is used to convert the coordinates in the camera coordinate system into the coordinates in the preset coordinate system; the second matrix set includes: a second model matrix, a second view matrix, and a second projection matrix; The texture coordinates are converted to the second coordinates based on the second set of matrices.

5. A three-dimensional model construction device, characterized in that: It comprises a memory and a processor; the memory is used to store computer-executable instructions, and the processor is connected to the memory through a bus; when the three-dimensional model building device is running, the processor executes the computer-executable instructions stored in the memory, so that the three-dimensional model building device executes the three-dimensional model building method as described in claim 1 or 2.

6. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes computer-executable instructions, and when the computer-executable instructions are executed on a computer, the computer is enabled to execute the three-dimensional model construction method according to claim 1 or 2.

Images